Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1
Technology Edge in Algo Trading
Sameer Kumar
14th
March, 2015
FINBRIDGE 2015
2
It's the Latency, Stupid
http://rescomp.stanford.edu/~cheshire/rants/Latency.html
Well known and referenced article “a n...
3
Misnomer – Bad Terminology
Would you say that a Boeing 747 is three times "faster" than a Boeing 737? Of
course not. The...
4
Traditional Trading System Architecture
• Traditionally a trading system would consist of
– A system to read data from t...
5
Traditional Trading System Architecture
• A system to read data from the market
Workshop on Algorithmic & High Frequency...
6
Traditional Trading System Architecture
• A storehouse of historical data
– Which could also be directly purchased from ...
7
Traditional Trading System Architecture
• A subset of the database is queried and stored locally for operational uses
Wo...
8
Traditional Trading System Architecture
• The trader’s tool would then analyze current data against patterns
discovered ...
9
Traditional Trading System Architecture
• The trader’s tool would then generate orders which will be forwarded to
the or...
10
Traditional Trading System Architecture
• The order manager would then route the orders to the exchange
Workshop on Alg...
11
Traditional Trading System Architecture
• The data warehouse would also probably store records of orders sent out
Works...
12
Traditional Trading System Architecture
• The whole system could be broken down to three components
Workshop on Algorit...
13
Traditional Trading System Architecture
• The exchange (and other data sources) – i.e. the external world
Workshop on A...
14
Traditional Trading System Architecture
• The server – which is mostly a data store
Workshop on Algorithmic & High Freq...
15
Traditional Trading System Architecture
• The applications in the trader’s pc which do all the processing
Workshop on A...
16
Traditional Trading System Architecture
• If data operations are simple… operational data store can be in application l...
17
Automated Trading System Architecture
• With the advent of DMA & automated trading, the following changes in
architectu...
18
Automated Trading System Architecture
• To generate orders quickly, market events are now handled in the server
instead...
19
Automated Trading System Architecture
• Complex mathematical operations are handled in a dedicated calculation
block in...
20
Automated Trading System Architecture
• The role of the application layer has reduced drastically – (i) an input
screen...
21
Automated Trading System Architecture
• The role of the application layer has reduced drastically – (ii) a monitor of
p...
22
Automated Trading System Architecture
• The role of the application layer has reduced drastically – (iii) preliminary
f...
23
Automated Trading System Architecture
• RMS is now automated and checked by the OMS before an order is
generated
Worksh...
24
Automated Trading System Architecture
• Because RMS is automated, a second level of monitoring is necessary – an
overal...
25
Automated Trading System Architecture
• With Multiple destinations being connected to the automated systems,
standardiz...
26
Automated Trading System Architecture
• This also necessitated adding data normalizer block in the market data
adaptors...
27
Automated Trading System Architecture
• Moreover, an Order Router had to be added to the OMS to route orders
from the s...
28
Automated Trading System Architecture
• Regulatory requirements have complicated storage requirements –
requiring stora...
29
Automated Trading System Architecture
• And obviously it should be able to read information from third party
vendors as...
30
Automated Trading System Architecture
• Moreover, the CEP engine has its own storage requirements of event
history to i...
31
Automated Trading System Architecture
• Complex functionality third party applications have to be tightly
integrated wi...
32
Automated Trading System Architecture
• Independent data management tools to verify the sanity of the
information have ...
33
Automated Trading System Architecture
• To be able to back-test strategies, two components are required: (i) replay
of ...
34
Automated Trading System Architecture
• To be able to back-test strategies, two components are required: (ii) a
simulat...
35
Introduction to low latency
• Technology – State of Art
• Approach to latency improvement
• Latest in Low Latency -appr...
36
Why aim for Low Latency or Lowest?
• It may be necessary to lower latency just to remain competitive
• The strategy dem...
37
Latencies – Strategy wise
Citihub, 2009
38
Current Scenario
• Synchronized Markets – Correlated
• Volumes and Trades have increased
• Orders per Trade has increas...
39
Performance Degradation with Throughput
http://www.ibm.com/developerworks/websphere/library/techarticles/0706_lou/0706_...
40
Causes of Degradation
• Massive headroom to allow for bursts typically 10:1 (like bridges and dams)
• Can be triggered ...
41
Parallel computing
• Amdahl's law, also
known as Amdahl's
argument,is used to
find the maximum
expected improvement
to ...
42
Network Effects
43
Latency by Distance
44
Spread Networks
Estimated roundtrip time for an ordinary cable is 14.5 milliseconds, giving users of Spread
Networks a ...
45
Microbursts
46
How to ask bandwidth question
“For this feed, how much bandwidth is needed to protect
99.99% of packets from loss with ...
47
Indian context
• The scenario is generally broken down into colocation ( 1+ Gbps) and non-
colocation ( 20+ Mbps).
• Ex...
48
Latency Breakdown
• Latency can be broken down into the following components
• L = P + N + S + I + AP
– P is Propagatio...
49
Technology Mix
50
Tips
• Servers – Fastest Cores, Cache,
• Operating Systems – RT kernels
• Fastest Network Infra (Switches, Routers )
• ...
51
Sample Solution Path
52
Thank you!
To Learn Automated Trading
Email: contact@quantinsti.com
Connect With Us:
SINGAPORE
11 Collyer Quay,
#10-10,...
Upcoming SlideShare
Loading in …5
×

Technology Edge in Algo Trading: Traditional Vs Automated Trading System Architecture

3,269 views

Published on

This presentation was delivered by QuantInsti director and co-founder Mr Sameer Kumar at a workshop on 'Technology Edge in Algorithmic Trading' organized at Finbridge Expo 2015, Mumbai on 14th March 2015.

This event was India’s first ever integrated financial service providers’ exhibition that brought together top mutual funds, insurance companies, leading brokers, banks or financial institutions, fund managers, decision makers along with thousands of retail financial market participants, distributors, investors and vendors.

In this workshop Mr Sameer Kumar has explained how latest technology has changed trading environment and how you can increase your profits by taking advantage of it. He thoroughly discussed how tradition trading systems evolved into automated trading systems over past few years. Following are the topics that are covered in this presentation,

1) Significance of Latency
2) Traditional Trading System Architecture
3) Automated Trading System Architecture
4) Low Latency
5) Current Scenario
6) Performance Degradation with Throughput
7) Causes of Degradation
8) Parallel computing
9) Network Effects
10) Latency by Distance
11) Spread Networks
12) Microbursts
13) Latency Breakdown
14) Technology Mix
15) Tips
16) Sample Solution Path

This presentation will help you understand the traditional and automated trading system architecture, it will explain you the significance of latency in an automated trading environment and few tips along with a sample solution path.

Published in: Economy & Finance
  • A professional Paper writing services can alleviate your stress in writing a successful paper and take the pressure off you to hand it in on time. Check out, please ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • If you have any problems with writing, feel free to ask our writers for help! ⇒ HelpWriting.net ⇐ is ready to help with any kind of academic writing!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Technology Edge in Algo Trading: Traditional Vs Automated Trading System Architecture

  1. 1. 1 Technology Edge in Algo Trading Sameer Kumar 14th March, 2015 FINBRIDGE 2015
  2. 2. 2 It's the Latency, Stupid http://rescomp.stanford.edu/~cheshire/rants/Latency.html Well known and referenced article “a network link with low bandwidth can be made better with money, but network link with bad latency cannot be helped” This was the scene in 1996, when bandwidth was the constraint. Speeds were in Kbps. Cheshire later become Wizard at Apple. Pioneering Zeroconf
  3. 3. 3 Misnomer – Bad Terminology Would you say that a Boeing 747 is three times "faster" than a Boeing 737? Of course not. They both cruise at around 500 miles per hour. The difference is that the 747 carries 500 passengers where as the 737 only carries 150. The Boeing 747 is three times bigger than the Boeing 737, not faster. Now, if you wanted to go from New York to London, the Boeing 747 is not going to get you there three times faster. It will take just as long as the 737. In fact, if you were really in a hurry to get to London quickly, you'd take Concorde, which cruises around 1350 miles per hour. It only seats 100 passengers though, so it's actually the smallest of the three. Size and speed are not the same thing. On the other hand, If you had to transport 1500 people and you only had one aeroplane to do it, the 747 could do it in three trips where the 737 would take ten, so you might say the Boeing 747 can transport large numbers of people three times faster than a Boeing 737, but you would never say that a Boeing 747 is three times faster than a Boeing 737.
  4. 4. 4 Traditional Trading System Architecture • Traditionally a trading system would consist of – A system to read data from the market – A storehouse of historical data – A tool to analyse historical data – A system where the trader can input his trading decisions – A system to route orders to the exchange
  5. 5. 5 Traditional Trading System Architecture • A system to read data from the market Workshop on Algorithmic & High Frequency Trading Market Data Exchange
  6. 6. 6 Traditional Trading System Architecture • A storehouse of historical data – Which could also be directly purchased from third party data vendor directly Workshop on Algorithmic & High Frequency Trading Market Data Exchange Data Warehouse / Storehouse of historical data Data Vendor
  7. 7. 7 Traditional Trading System Architecture • A subset of the database is queried and stored locally for operational uses Workshop on Algorithmic & High Frequency Trading Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor
  8. 8. 8 Traditional Trading System Architecture • The trader’s tool would then analyze current data against patterns discovered in the operational data store Workshop on Algorithmic & High Frequency Trading Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor
  9. 9. 9 Traditional Trading System Architecture • The trader’s tool would then generate orders which will be forwarded to the order management tool Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders
  10. 10. 10 Traditional Trading System Architecture • The order manager would then route the orders to the exchange Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders
  11. 11. 11 Traditional Trading System Architecture • The data warehouse would also probably store records of orders sent out Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders
  12. 12. 12 Traditional Trading System Architecture • The whole system could be broken down to three components Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders
  13. 13. 13 Traditional Trading System Architecture • The exchange (and other data sources) – i.e. the external world Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders Exchange
  14. 14. 14 Traditional Trading System Architecture • The server – which is mostly a data store Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders Server Exchange
  15. 15. 15 Traditional Trading System Architecture • The applications in the trader’s pc which do all the processing Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operational Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders Application Server Exchange
  16. 16. 16 Traditional Trading System Architecture • If data operations are simple… operational data store can be in application layer (trader’s pc) Workshop on Algorithmic & High Frequency Trading Order Manager Market Data Operatio nal Data Store Exchange Data Warehouse / Storehouse of historical data Data Vendor Trader’s tool Main Centre of operations – analyzing market data wrt to historical data in operational data store and generating orders Application Server Exchange
  17. 17. 17 Automated Trading System Architecture • With the advent of DMA & automated trading, the following changes in architecture took place: – Latency between Event Occurrence & Order Generation had to be reduced to an order of milliseconds and lower. – Order Management had to be made more robust to handle generation of thousands of orders in a second – Risk Management had to be done in real time and without human intervention Workshop on Algorithmic & High Frequency Trading
  18. 18. 18 Automated Trading System Architecture • To generate orders quickly, market events are now handled in the server instead of the application - in the CEP module Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange
  19. 19. 19 Automated Trading System Architecture • Complex mathematical operations are handled in a dedicated calculation block in the server block (e.g. Options Greeks calculations) Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Maths Calc
  20. 20. 20 Automated Trading System Architecture • The role of the application layer has reduced drastically – (i) an input screen for strategy settings Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI Maths Calc
  21. 21. 21 Automated Trading System Architecture • The role of the application layer has reduced drastically – (ii) a monitor of position & orders Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Maths Calc
  22. 22. 22 Automated Trading System Architecture • The role of the application layer has reduced drastically – (iii) preliminary fat finger RMS checker Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc
  23. 23. 23 Automated Trading System Architecture • RMS is now automated and checked by the OMS before an order is generated Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS
  24. 24. 24 Automated Trading System Architecture • Because RMS is automated, a second level of monitoring is necessary – an overall global position monitor Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor
  25. 25. 25 Automated Trading System Architecture • With Multiple destinations being connected to the automated systems, standardized protocols (FIX) became the norm Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X
  26. 26. 26 Automated Trading System Architecture • This also necessitated adding data normalizer block in the market data adaptors to convert data from multiple exchanges into a standard format Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer
  27. 27. 27 Automated Trading System Architecture • Moreover, an Order Router had to be added to the OMS to route orders from the same OMS to multiple exchanges Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router
  28. 28. 28 Automated Trading System Architecture • Regulatory requirements have complicated storage requirements – requiring storage of trade information in addition to market data Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store
  29. 29. 29 Automated Trading System Architecture • And obviously it should be able to read information from third party vendors as well Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store Data Vendor
  30. 30. 30 Automated Trading System Architecture • Moreover, the CEP engine has its own storage requirements of event history to identify future opportunities Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store Event History Data Vendor
  31. 31. 31 Automated Trading System Architecture • Complex functionality third party applications have to be tightly integrated with all the blocks Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store Event History Adaptor for third party apps – R, Matlab, etc Data Vendor
  32. 32. 32 Automated Trading System Architecture • Independent data management tools to verify the sanity of the information have to configured Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store Event History Adaptor for third party apps – R, Matlab, etc Data Retrieval Data Vendor
  33. 33. 33 Automated Trading System Architecture • To be able to back-test strategies, two components are required: (i) replay of stored data Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store Event History Adaptor for third party apps – R, Matlab, etc Data Retrieval Data Vendor Replay of stored data
  34. 34. 34 Automated Trading System Architecture • To be able to back-test strategies, two components are required: (ii) a simulator destination instead of an actual exchange Workshop on Algorithmic & High Frequency Trading Application Order Manager Market Data Complex Event Processing engine Exchange 1 Storage Application Server Exchange Strategy Settings UI State Mgmt (PnL + Position) Order / Execution Monitor Within application RMS Maths Calc RMS Admin Monitor Exchange 2 F I X F I X Data Normalizer Order Router Back office record MktData Store Event History Adaptor for third party apps – R, Matlab, etc Data Retrieval Data Vendor Replay of stored data Simulator exchange
  35. 35. 35 Introduction to low latency • Technology – State of Art • Approach to latency improvement • Latest in Low Latency -approaches and technologies being deployed to achieve low latency
  36. 36. 36 Why aim for Low Latency or Lowest? • It may be necessary to lower latency just to remain competitive • The strategy demands low latency, perhaps. • It may be desirable to improve latency to stop getting picked off by competitors • With introduction of Colocations and increasing focus in remaining fastest in the market, significant capital is invested. However it can all go waste, if correct technology is not identified and implemented. • The issue is that latency is difficult to quantify. As a result the value of latency improvement, though easily understood, is extremely difficult to quantify • Lower latency systems cost a lot more to build and deploy. Hence the objective should be to find the right balance between investment and return on investment in low latency
  37. 37. 37 Latencies – Strategy wise Citihub, 2009
  38. 38. 38 Current Scenario • Synchronized Markets – Correlated • Volumes and Trades have increased • Orders per Trade has increased drastically • Synchronized Trading Patterns • Need to increase throughput and lower latency simultaneously • CPUs not getting faster – since 2007. Only cores are being added, not the frequency. Moore’s Law is failing
  39. 39. 39 Performance Degradation with Throughput http://www.ibm.com/developerworks/websphere/library/techarticles/0706_lou/0706_lou.html
  40. 40. 40 Causes of Degradation • Massive headroom to allow for bursts typically 10:1 (like bridges and dams) • Can be triggered by a shortage of any one of it’s resources: – CPU cycles – Memory – I/O channel capacity – Disk transfers – Network transfers • Shortage of one can trigger another e.g. shortage of memory causing CPU Typical system response time against throughput and I/O thrashing • Distributed deployments add an order of magnitude more complexity
  41. 41. 41 Parallel computing • Amdahl's law, also known as Amdahl's argument,is used to find the maximum expected improvement to an overall system when only part of the system is improved. It is often used in parallel computing to predict the theoretical maximum speedup using multiple processors.
  42. 42. 42 Network Effects
  43. 43. 43 Latency by Distance
  44. 44. 44 Spread Networks Estimated roundtrip time for an ordinary cable is 14.5 milliseconds, giving users of Spread Networks a slight advantage. • In October 2012, Spread Networks announced latency improvements, bringing the estimated roundtrip time from 13.1 milliseconds to 12.98 milliseconds. • Some companies, such as McKay Brothers and Tradeworx, are using air-based transmission to offer lower estimated roundtrip times (9 milliseconds and 8.5 milliseconds respectively) that are very close to the theoretical minimum possible (about 7.5-8 milliseconds).
  45. 45. 45 Microbursts
  46. 46. 46 How to ask bandwidth question “For this feed, how much bandwidth is needed to protect 99.99% of packets from loss with no more than 100 microseconds of latency to be experienced during the busy 1 second of the trading day”
  47. 47. 47 Indian context • The scenario is generally broken down into colocation ( 1+ Gbps) and non- colocation ( 20+ Mbps). • Exchanges provides Tick-by-tick which may require taking separate lines.
  48. 48. 48 Latency Breakdown • Latency can be broken down into the following components • L = P + N + S + I + AP – P is Propagation time - sending the bits along the wire, speed of light constrained – N is Network packet processing – routing, switching and protection – S is Serialization time - pulling the bits on/off the wire – I is Interrupt handling time – receiving the packet on a server – AP is Application Processing time
  49. 49. 49 Technology Mix
  50. 50. 50 Tips • Servers – Fastest Cores, Cache, • Operating Systems – RT kernels • Fastest Network Infra (Switches, Routers ) • Retune the TCP stack (e.g. i/o buffer size ) • Program Runtime – isolcpus, stack bypass • Solid State Drives ( SSD ) • Latency Tuning – TCP_NODELAY, sendfile(2), Lock Free Codes
  51. 51. 51 Sample Solution Path
  52. 52. 52 Thank you! To Learn Automated Trading Email: contact@quantinsti.com Connect With Us: SINGAPORE 11 Collyer Quay, #10-10, The Arcade, Singapore - 049317 Phone: +65-6221-3654 INDIA A-309, Boomerang, Chandivali Farm Road, Powai, Mumbai - 400 072 Phone: +91-022-61691400

×