Before setting up a data feed, it helps to understand how the data feed engine works. The data feed engine downloads data via data feed plug-ins and stores the downloaded data in memory. When new data arrives, the data feed engine notifies the rest of the program so it can retrieve this data and display it to you, the user. Periodically (adjustable via the application settings window) this data is saved to your hard drive. What this means is that there is a delay between when data is downloaded and when you can edit and view it via the ‘Tickers’ window. The big advantage is you can run many different data feeds for many different markets without adversely affecting the performance of the rest of the program. This also means that if you edit the settings of a data feed or edit the ticker data on the hard drive, there is a small delay before the changes will take effect as the data feed engine needs to shut down and reload changes from the hard disk.

The data feed engine can run in two different modes ‘Normal’ and ‘Smart’.

To understand these two different modes we need to categorize the different data that can be downloaded via a data feed plugin. EOD data and stock information is classified as historical. Real-time and tick data is classified as real-time.

Normal Mode

In ‘Normal’ mode the data feed engine will download the maximum amount of real-time and historical data for all tickers in the database regardless of whether they have trades or not. For a small number of stocks this is fine but if you have a large number of stocks in many different markets, this can be too much data for even the best designed program to handle.

Smart Mode

In ‘Smart’ mode the data feed engine will download all historical data available for all tickers in the database the same as in normal mode. The difference between ‘Normal’ and ‘Smart’ mode occurs when downloading real-time data. Real-time data is downloaded for tickers that have trades and only for the period where the trade was open. This is the default and recommended mode.

The maximum amount of data that will be downloaded can be adjusted via the settings window. The default is 25 years of EOD, 30 days of real-time and 1 day of tick. Be aware that even a few days of tick data can easily amount to a million bars of data.

Setting up and activating a new data feed is easy! To set up a new data feed simply open the ‘Data Feeds’ window.

Next click ‘New’.

In the ‘Data Feed’ drop down box select the data feed plugin you wish to use. Once selected this should enable/disable options which are applicable to the plugin. You can set any plugin specific settings by clicking ‘Plugin Settings’ (if applicable).

Is Manual? – This option specifies whether you want the plug-in to run manually or automatically when required. When a plugin is setup to run automatically, it will run depending on a number of factors. If it is a historical data feed plugin, it will run when the program is first launched. It will also run when a new ticker is added or when a new trade is added. If it is a real-time data feed plugin, it will run periodically based on the ‘Update Interval’. When a plugin is setup to run manually, it will only run when you click the run button which can be found next to the data feed in the main navigation panel.

Update Interval – Specifies how often (in seconds) you want the data feed to run. This option is only applicable to real time and tick data feeds. Please note that certain free data providers, such as Yahoo, will ban users that request data too often. So be careful not to pick a too short an interval. If the throttle requests option is enabled, the update interval that you have set may not happen.

Target Market – Specifies the market this data feed applies to.

Max Threads – Specifies how many threads to use concurrently to fetch data.

Maximum Errors – Specifies how many errors a plug-in can generate before it is shut down. If you have created a backup data feed, then the data feed engine will switch to it when the maximum number of errors is reached for the current data feed.

Throttle Requests – Most data providers have a limit as to how much data you can request within a given time frame. For example, if you set the ‘Update Interval’ to 1 sec and have 10 stocks to update, this will be equal to 10 requests per second to the data provider. If on the other hand you had say 200 stocks to update, this would be 200 requests per second which may be too much. By enabling this option you can limit exactly how many requests will be made to the data provider within a given time frame regardless of how many stocks need to be updated.

Maximum Number Of Requests – Maximum number of data requests that can be made.

Time Span In Seconds – The time span within which the maximum number of requests can be made.

Next, click ‘Save’. Your newly created data feed should appear in the data feeds window and, once successfully loaded, it should also appear in the main navigation panel located in the main view. The small button located next to the data feed name will run the data feed. See image bellow: