Setting Up Your Own Algorithmic Trading Server: A How-To Guide

How to have your own complete algorithmic trading development environment that pulls data and backtests a strategy

Proprietary trading firms have been known to spend millions on the closely guarded tech powering their organizations. This article shows you how to use an open source alternative, called LEAN (developed by QuantConnect), that provides all the same capabilities as top Wall Street institutions.

Getting the right infrastructure in place is critical for algorithmic trading to ensure the success of your strategies. With live capital on the line, you want to be sure you are running a robust, well-tested stack. LEAN has been developed and reviewed by a global community of engineers, which allows you to focus on the fun part, developing algorithms!

Installing LEAN

Since LEAN is open source, installation is as simple as cloning the master repo on GitHub or downloading it as a .zip file and unpacking it in your desired directory. To install via git command, navigate to the directory you want to unpack it in and run the clone command:

git clone https://github.com/QuantConnect/Lean.git

Using Visual Studio for local testing and development is recommended, but not required. If you don’t already have Visual Studio installed on your machine and want to use it, you can download and install the free version either for Mac or Windows. Linux users need to install Mono, which is covered below.

Windows

To finish the installation for Windows:

1. Open QuantConnect.Lean.sln in Visual Studio
2. Build the Solution by clicking Build Menu -> Build Solution (this should trigger the NuGet package restore, which downloads all the required dependencies)
3. Press F5 to run

NuGet packages not being restored is the most common build issue. By default Visual Studio includes NuGet, but if your installation of Visual Studio (or your IDE) cannot find DLL references, install NuGet, run NuGet on the Solution, and rebuild the Solution again.

If you plan on using Python, you need to:

1. Install Python 3.6 using the Windows x86–64 MSI installer from python.org or Anaconda for Windows installer.
2. When asked to select the features to be installed, make sure you select “Add python.exe to Path”
3. Install pandas and other dependencies

Mac

To finish installation on Mac:

1. Open QuantConnect.Lean.sln in Visual Studio. Visual Studio automatically starts to restore the NuGet packages. If not, in the menu bar, click Project > Restore NuGet Packages.
2. In the menu bar, click Run > Start Debugging to finish the installation.

If you plan on using Python, you will need to:

1. Use the macOS x86–64 package installer from Anaconda and follow “Installing on macOS” instructions from Anaconda documentation page.
2. Install pandas and its dependencies.

Note: If you encounter the “System.DllNotFoundException: python3.6m” runtime error when running Python algorithms on macOS:

1. Find libpython3.6m.dylib in your Python installation folder. If you installed Python with Anaconda, it may be found at

/Users/{your_user_name}/anaconda3/lib/libpython3.6m.dylib

2. Open Lean/Launcher/bin/Debug/Python.Runtime.dll.config, add the following text and save:

Linux

Linux users will need to:

1. Install Mono

If you get this error on the last command:

Unable to locate package referenceassemblies-pcl

run the following command (it works on the current version of Ubuntu — 17.10):

2. Install NuGet

sudo apt-get update && sudo apt-get install -y nuget

3. Restore NuGet packages then compile:

msbuild QuantConnect.Lean.sln

If you get: “Error initializing task Fsc: Not registered task Fsc.”

sudo apt-get upgrade mono-complete

If you get: “XX not found” -> Make sure NuGet ran successfully, and re-run if necessary.

1. Run the compiled .exe file:

LEAN supports Python in Linux as well. By default, miniconda is installed in the user’s home directory ($HOME):

Note: There is a known issue with python 3.6.5 that prevents Pythonnet installation, please upgrade python to version 3.6.6:

conda install -y python=3.6.6

Quick Algorithm Test

Once LEAN is fully installed, and the Solution is built, open the config.json file in QuantConnect.Lean.Launcher and change the algorithm-type-name to BasicTemplateAlgorithm to run a test algorithm. Run this by selecting F5.

To run the Python version of the BasicTemplateAlgorithm, change the algorithm language and location in config.json to “Python” and “../../../Algorithm.Python/BasicTemplateAlgorithm.py”, respectively. Then, build the Solution again and run the algorithm by selecting F5. Running BasicTemplateAlgoriothm.py produces a matching output.

LEAN Data Format

LEAN is built to read its financial data from flat files on disk. All files are CSV or JSON and compressed into zip format.

Folder Structure

Data files are separated and nested in a few predictable layers:

• Tick, Second and Minute Financial Data: /data/securityType/marketName/resolution/ticker/date_tradeType.zip
• Hour, Daily Financial Data: /data/securityType/marketName/resolution/ticker.zip

The market property is used to separate different tradable assets with the same ticker. E.g., many brokerages support EURUSD, all with slightly different prices.

Core Data Types

LEAN has a few core data types which are represented in all the asset classes supported. Below are links to their implementation in LEAN.

• TradeBar — TradeBar represents trade ticks of assets consolidated for a period. TradeBar file format is slightly different for high resolution (second, minute) and low resolution (daily, hour).
• QuoteBar — QuoteBar represents top of book quote data consolidated over a given time interval (bid and ask bar).
• Tick — Tick data represents an individual record of trades (“trade ticks”) or quote updates (“quote tick”) for an asset.

Adding Data

Now that you have LEAN installed and have a basic understanding of how to run an algorithm, you need to import data. LEAN allows you to download data from numerous sources in a LEAN-compatible format.

1. Using the LEAN ToolBox (found in Lean/ToolBox directory), you can download data from sources such as IEX by using the ToolBox Data Downloaders. To download data from IEX, open a command window and set your directory for “…/Lean/ToolBox/bin/Debug”. From here, you can run the command

To find information on how to use the ToolBox for data downloads in general. To download daily data for Netflix from 2018–01–01 to 2018–12–31, run the command:

QuantConnect.ToolBox.exe --help

The terminal lets you know the directory path it is saved to (the default path assigned in config.json in the case above), and now this data is ready to be used in your LEAN algorithms.

2. Using the ApiDataProvider, you can access any data stored in your online QuantConnect data library. To enable this, change the “data-provider” field in config.json to “QuantConnect.Lean.Engine.DataFeeds.ApiDataProvider”, and the “job-user-id” and “api-access-token” fields to your QuantConnect account ID and API token, found on the web page under the My Account tab.

Continuous Data Download

You can set up a downloader to run daily, keeping all of your data up-to-date. IEX posts daily data at 8:30 pm New York/Eastern time, so scheduling a download for or after that provides you with up-to-date data.

Windows

1. Write a batch (.bat) file to be executed daily. The code below provides a template, where the user defines the appropriate path and start date argument.

2. Open Task Scheduler and click Create Task. Give it a name and a description and open the Trigger tab. Click New to create a new trigger, which you can configure to run the batch script at the desired time of day and a specified frequency.

3. Open the Actions tab and click New. Use the Browse button or enter the path of the batch file to be executed. The batch file executes daily, and the data is updated accordingly.

Linux

1. Write a shell (.sh) script to be executed daily. The code below provides a template, where the user defines the appropriate path and start date argument.

2. Create a crontab by opening the terminal and entering

crontab -e

3. Select your editor, and then enter the arguments for the crontab to execute the script daily (at 9pm in this case) and update the data accordingly.

0 21 * * * /bin/bash {USER PATH TO SHELL SCRIPT}

Mac

1. Write a shell script to be executed daily, such as the following:

2. Scripts in Mac’s launchd are triggered by job definitions which are .plist files stored in specific directories. The XML file below gives the job a name, the script to be launched, and determines when to execute the script.

3. Save this file at ~/Library/LaunchAgents.

4. Once you’ve created the script and saved the agent into the right place, you’ll need to load it into launchctl. Loading is done automatically on future logins.

5. Loading a job puts it into the launchd queue, and the job will run at the time specified in its launch conditions.

Coding an Algorithm

Once LEAN is installed and configured, and you have the data that you need, you can run algorithms! LEAN has an extensive library of demonstration algorithms and examples of other algorithms online. The LEAN Share Class Mean-Reversion Algorithm provides an easy yet powerful example of a LEAN algorithm (you can view the full Python code here and C# code here).

This algorithm trades different classes of shares issued by the same company. It is not uncommon for companies to issue different share classes, whether to allow investors access at a different price point or to offer different levels of voting rights. These dual share classes have the same underlying factors driving their price movement, and in an ideal world, are priced the same.

Dual share classes are priced correctly in a perfect market. However, if the two classes both trade with sufficient liquidity (Class A shares are common stock and other classes may not be in demand for investors), then there exists an arbitrage opportunity if mispricing occurs.

To build an algorithm to implement this strategy, we follow the steps below.

1. Chose Viacom Class A and Class B shares since both are sufficiently liquid and affordable. This data can be downloaded using the IEX Data Downloader example above. We anticipate any mispricing to be short term, so generally expect the mean reversion to happen quickly and the margin from the arbitrage to be small. Therefore active trading and high-resolution data are required.

2. Build an Alpha Model, which is a class that interacts with the rest of the algorithm. The Alpha Model is where the data gets passed into, and our trading logic executes. Our goal is to design it such that it uses historical trends to try and identify the overpriced and underpriced share classes at any given time. Then, the algorithm establishes a nearly dollar-neutral position by buying the underpriced share class and shorting the overpriced share class. Once the prices revert to their anticipated mean, the positions liquidate.

3. Create a 10-period Simple Moving Average Indicator (using minute resolution data) to keep track of the historical value of the difference in price changes and a Rolling Window to keep track of the two most recent values of this position. Since we expect mispricing to happen frequently and to correct itself quickly, a 10-period SMA Indicator allows us to track the recent historical movement of our position and quickly examine short-term mispricing.

LEAN has an extensive library of Indicators, and a Rolling Window is an object unique to LEAN that is similar to an array but can store either decimal values, TradeBars, or QuoteBars.

4. Establish our initial long and short position sizes with CalculateOrderQuantity and set the weighting for each to be 50%. After this, and once the SMA Indicator is warmed up, we calculate the value of the long/short position, which should be almost 0. Note that long VIA and short VIAB is only a starting point, and we could run the algorithm with the same results if we initially went short VIA and long VIAB.

5. If the portfolio is empty, each time a new data slice comes through the Alpha Model checks if the current position value is above or below its SMA value. If it’s above, then we expect that the shares in the long position are overpriced and the shares in the short position are underpriced, and we emit a Down Insight for VIA and an Up Insight for VIAB. These insights capture the anticipation that VIAB will increase in price and VIA will decrease (we group these Insights since the trigger for emission is the same for both). If the position value is below the SMA, then we take the opposite position in VIA and VIAB.

(In an Alpha Model, we want to emit Insights that describe the direction and magnitude of the anticipated change in each stock’s price. The Portfolio Selection and Execution models use the Insights to create weighting schemes and place trades.)

6. We hold this position until, using the Rolling Window, we see that the long/short position value has crossed back over its SMA value.

7. If this occurs, we liquidate our positions and take our arbitrage profits.

An Equal Weighting Portfolio Construction Model ensures we maintain equal weighting in each share class, and the Immediate Execution Model provides quick execution of our desired trades so that the arbitrage opportunity doesn’t disappear before we can take advantage of it.

To add the algorithm to the VS Solution, right-click on the QuantConnect.Algorithm.Python project in the Solution Explorer and click Add -> Add Existing Item and select the algorithm that we saved earlier. You can do the equivalent in C# by adding the algorithm in the QuantConnect.Algorithm.CSharp project. Then, click File -> Save All and rebuild the Solution. Once rebuilt, change the algorithm-type-name in config.json to be ShareClassMeanReversionAlgorithm, the same name as the algorithm class, and the algorithm-location path to “../../../Algorithm.Python/ShareClassMeanReversionAlgorithm.py”.

Now the algorithm is ready to run, just hit F5!

(Note: we used minute-data from 4/15/19 to 5/15/19. IEX minute-data is only available for the trailing 30-days, so individual backtest results may differ)

Setting Up Your Own Algorithmic Trading Server: A How-To Guide was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.