Best Practices for Extracting/Reading Data

The following shows the different methods for extracting or reading data.

 

Important Note:

When a time series for second and/or millisecond data has multiplicity set to multiple, then more than one value per time stamp can be stored. In this case, use get records API methods to extract the data. Do not use query execution methods for data extraction.

 

XMIM Software

The XMIM software is very powerful for modeling and evaluating trade opportunities. The user builds queries using a near-English query language. Users can either type in the query language or use the GUI interface to build their queries. Queries can have custom events with date and time conditions and can incorporate studies and market events (e.g., create moving averages, highest high, lowest low etc.) XMIM has many built-in report options for analyzing the data results.

The user will get the most out of the software when they are comfortable with the MIM query language. For users that are unfamiliar with the MIM query language, there are features in the software that will help them get started. The XMIM software (release 6.0+) has the following features to help the novice user:

  • Query Wizard - no prior query language knowledge required. Create queries using market-driven events.

  • How Do I Say - teaches the query language

  • Templates - assists with complex queries

Recommendation: If you have a large amount of data points to read then use the BMIM scripting language to browse the schema.

With the XMIM software you can:

  • observe/model/test/simulate trading

  • build simple to complex queries for risk management, back-testing and cross-market analysis

  • browse the data schema

  • search the data schema and filter and filter your search

  • build a notification system that alerts the trader when an event occurs

  • use let statements: The LET statement allows the user to assign properties to variables and is defined in the query above the query block. This variable may be used to range the query over different symbols or values, used in place of complex query statements, or query data from a file. The variable may be used as a short-hand substitute that can be used again and again throughout the query.

  • work with rollovers

  • use date conditions, custom events, market events and studies

  • use templates, How Do I Say feature, and Query Wizard to get started building queries

  • save favorite queries in a hot list for easy access. (My Queries)

  • chart the results

  • table the results with many report options (e.g., For the Trade Statistic report you will see number of trades, number of closed trades, number of winning trades etc.)

MIMIC Software

This MIMIC software is very good for creating seasonal studies, charting results, and working with formulas. The software has integrated unit conversions and is excellent for working with rollovers and futures data. This software package works well for the novice as well as the advanced user. The MIMIC software can run standalone or inside of Excel and the data results (charts or tables) are easily posted into Excel spreadsheets.

With the MIMIC software you can:

  • browse the data schema

  • search the data schema and filter your search

  • bring data table results and charts into Excel spreadsheets

  • create charts that have trendlines, legends and may be zoomed

  • run the software from within Excel

  • generate forward curves

  • perform units of measure and currency conversion

  • create seasonality studies

  • create custom contracts and edit your rollover policies

  • create formula expressions

  • have a favorites hot list

  • create new values using formulas and store them for use

Recommendation: Use the XMIM software when working with millisecond data. Currently, millisecond data cannot be extracted with the MIMIC software.

MIM Excel Add-in Software

The MIM Excel Add-in software is a very straight forward product that is good for pulling out data into spreadsheets. Note that both MIMIC and XMIM can output to Excel and have more built-in functionality.

You can only create charts of forward curves in the MIM Excel Add-in. Recommendation: Use MIMIC when you want to bring charts into Excel. Note that both XMIM and MIMIC can output data results into Excel.

With the MIM Excel Add-in you can:

  • browse and search the data schema

  • perform basic queries and pull-out data into spreadsheets

  • create queries and select query data directly from an Excel spreadsheet for use in the query

  • create forward curve studies where multiple contracts can be charted over several years on a single chart.

  • pull options information into a spreadsheet

  • set how rollover contracts are evaluated. Set how as well as when a contract will roll.

  • perform units of measure and currency conversions

BMIM Scripting Language

The BMIM print_schema command is useful for browsing the schema hierarchy and the relation and relcol properties.

Application Programming Interfaces (APIs)

This method is best when you need your own custom solution. The following lists the APIs offered:

Database Catalog

The Database Catalog is available as a link from the LIM Web site (www.lim.com). Use the Database Catalog to view all the data offered by LIM.