Software Infrastructure

I am at a point where I need to decide on a design for my trading platform including the software and tools that will support it. The diagram below illustrates at a high level the proposed design.

The design is made up of a number of components that will collectively manage and automate trading strategies.

First we have the Stock Ticks DB Manager. This will interface with the Online Broker Gateway to obtain real-time stock data and record them into the database. To enhance performance it will use memcached to cache the most frequent records and expire them as it deems necessary.

There will be one or more Strategy Managers that each implement a particular trading algorithm. These managers will communicate with the Stock Ticks DB Manager and Risk Manager to monitor and execute entry and exit trades.

The Risk Manager will interface with the Portfolio DB Manager to determine current risk level. The Risk Manager will also trade positions on behalf of the Strategy Managers via the Portfolio DB Manager which maintains an interface to the Online Brokers Gateway.

The Risk Manager will also be responsible for ensuring capital is allocated relative to the performance of the portfolio, individual strategies performance and risk ratings. In other words, it will be the clearing house for all trading decisions regardless of the number of strategies in place.

Each of the components will communicate via a common bus architecture which in this case will be implemented using message queues. Message queues have several advantages the most important being they will allow me to utilise different software infrastructure and languages if I find my original choices are not optimised to address a particular requirement.

Finally, there will be a Notification Manager. This component will subscribe to message queues that I am interested in and keep me updated on the trading platforms real-time activities. It will also manage certain queues that will be used by certain processes to seek my acknowledgment before proceeding further down the path. For example, the Risk Manager may seek my acknowledgment before implementing a trade of a particular value. My acknowledgment may be in the form of a reply to an Instant Message or Email that in turn accepts or rejects the request.

Here is a list of the initial software choices I have made to support the platform.

Of course these choices are for all intents and purposes untested in the context of running a trading platform. As I progress along this journey I may encounter issues that lead me to seek alternatives. If this situation arises I will endeavour to explain my rational for making the switch. However, it is obvious now from the choices I have made that I wish to remain within my comfort zone. I can rationalise this decision on the basis that I have more to learn concerning the fundamentals of quantitative trading strategies; and I need not add to that workload when it comes to the implementation.

Language: Perl

Perl is my strongest language. Whilst I can read and write other languages (C, Python, Java) they are not languages I use on a day to day basis and therefore developing in these languages is going to slow me down. Furthermore, there are a number of Perl modules I will leverage to expedite development. These include statistical methods available here and bindings to quantlib. Other than financial methods, I will rely on database, messaging, notification and HTTP interfaces to assist with data management, communications and web site (in this case the brokers) control.

As a side note the quantlib bindings for Perl just finished compiling. Good to see things work as advertised :)

Installing /usr/local/lib/perl/5.10.1/auto/QuantLib/QuantLib.bs
Installing /usr/local/lib/perl/5.10.1/auto/QuantLib/QuantLib.so
Installing /usr/local/lib/perl/5.10.1/QuantLib.pm

Database: Postgresql

I have chosen Postgresql as my relational database platform. This is a database I am very familiar with. I have used it on numerous projects and it has proven to be a reliable performer. It may not be the fastest RDBMS but then I do not require the fastest.

Communications Bus: RabbitMQ

My trading platform is going to be made up of multiple programs and processes that will need to communicate and coordinate activities. To address this requirement I am going to rely on RabbitMQ. This software implements a comprehensive messaging and queueing system (the glue) which will be used to satisfy the requirements of a robust communication bus. This particular implementation includes good support for Perl which is paramount. I did look at another contender Spread which also seems pretty popular however the available Perl interfaces are quite limiting. Spread has better support for the Python language. Both support message persistence which is one of my main requirements as I want the trading platform to be able to resume where it left off if infrastructure fails momentarily.

Optimisation: memcached

My platform will utilise memory to optimise the retrieval of frequently accessed objects (e.g. time series and stock prices). This will be implemented using memcached - a distributed memory object caching system.

One thing I have not mentioned is all my choices are open source or free to use for non-commercial use. In making these choices I did not assess commercial alternatives.

Q. For those reading this I would be interested to know what works for you now and what didn't work - bearing in mind my intent is to create an automated system.