Tick Data’s historical intraday Cash Index data includes:
- Tick-by-tick data for 60+ indices from around the world – See List of Available Cash Indices
- Time stamped to the millisecond (HH:MM:SS.000) since Jul 2011
- Data delivered in delimited text files for easy integration (e.g. R, MATLAB®, MongoDB®, Kdb+, etc.)
- Build custom time series via TickWrite® GUI or TickAPI®
- Daily updates available to keep your data current
Our Data Collection Process
Tick Data strongly believes that one cannot rely on a single data feed collected in real-time to build a complete, error-free archive of data for research purposes. Systems fail and back-up routines do not always ensure an uninterrupted data stream. Likewise, transmission platforms, such as the Internet or dedicated digital lines, are not 100% reliable, resulting in lost data packets and gaps.
So we base our cash index data on archives built using a primary, secondary and tertiary data collection infrastructure. Having three (3) isolated, redundant, geographically diverse archives means real-time data transmission errors and omissions are not a factor in our data collection. This proprietary data archival system compares the three archives to create a single, robust data series for each symbol, ensuring that any problems that may occur in one data archive do not make their way into our data sets.
Data Set Generation and Filtering
Despite all of the care we take in obtaining our raw tick-by-tick data, omissions and errors persist even in data sourced directly from exchanges. Therefore, all data passes through our validation process; a suite of extraction, filtering, verification, and reporting programs developed in-house for the sole purpose of producing clean, robust data. Tick Data adds significant value to the data we offer through extensive data cleaning and verification. Algorithmic data filters are employed to identify bad prints, decimal errors, transposition errors, and other data irregularities. These filters take advantage of the fact that since we are not producing data in real-time, we have the ability to look at the tick following a suspected bad tick before we decide whether or not the tick is valid. We have developed a number of filters that identify a suspect tick and hold it until the following tick confirms its validity. The filters are proprietary, and are based upon recent tick volatility, moving standard deviation windows, and time of day.
It is important to note that each cash index is different. Some indices update only once per minute, while others are disseminated every few seconds. While all of our cash index data is visually verified, due to the nature of the data, we cannot employ our proprietary, automated filtering techniques to clean the data algorithmically.