Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Legal Informatics w/ AWS CloudSearch &High-Performance Financial Market Apps                         AWS Michigan Meetup  ...
Objectives ‣   Introduce Quantitative Trading ‣   Present a case study on AWS usage in Quantitative Trading System     Eva...
About Us        http://www.solidlogic.com                                    3
Solid Logic Technology develops innovative customtechnology solutions across a variety of industriesusing leading software...
About usOur expertise                                              Industry experienceInfrastructure and cloud computing  ...
Solid Logic Management Team‣       Eric Detterman, CEO and Co-Founder    •        Professional Experience         -       ...
Case Study:Proprietary Trading Simulation                     http://www.solidlogic.com                                   ...
Case Study: Proprietary Trading Simulation Quantitative Trading and Investment Systems: ‣ (Loose) Definition:   •   Rules-...
Case Study: Proprietary Trading Simulation        Hypothetical Example                                 http://www.solidlog...
Case Study: Proprietary Trading Simulation Challenge:                                         Scope: ‣       Characterize ...
Case Study: Proprietary Trading Simulation Potential solutions: ‣ Run on existing hardware – wait for results ‣ Physical o...
Trading Simulation: ArchitectureThis was our initial version – Not overly elegant, but works very wellwith minimal effort ...
Trading Simulation: Test Process     Trading     System   Source Code    (Git Repo)                                       ...
Trading Simulation: Overview Technology Solution:                      Compute Instance (x16): ‣       Built an optimized ...
Trading Simulation: AMI Creation Process ‣ Use standard Ubuntu Server 12.04.1 LTS for Cluster Instances   AMI x64 (ami-eb7...
Trading Simulation: Test Execution ‣ For each test instance…..   •    ssh -X -i /home/ericd/.aws/first/name.pem ubuntu@IP ...
Trading Simulation: Initial Test Results ‣ Result sets saved to S3 buckets using S3cmd   •   Approximately 6000 result set...
Trading Simulation: Output‣ Run Time: •   Cloud: 45 hours •   Single-seat: 1-2 months •   Order of magnitude improvement i...
Trading Simulation: Economics                                      Co-            On-         Spot                      On...
Trading Simulation: Next Steps Potential Improvements: ‣ Develop improved cloud infrastructure management tools   • Alloca...
Thank you    Eric Detterman                            Michael Bommarito    CEO, Co-Founder                               ...
Upcoming SlideShare
Loading in …5
×

Finance Trading in The Cloud - AWS Michigan Meetup

4,993 views

Published on

This presentation, given at AWS Michigan Meetup on 10-09-2012 provides an overview of how we used Amazon Web Services to conduct a quantitative trading system simulation on Amazon Web Services (AWS). We demonstrate an improvement in processing time of an order of magnitude and cost savings of greater than 99% compared to a traditional, in-house physical infrastructure.

Published in: Technology
  • Be the first to comment

Finance Trading in The Cloud - AWS Michigan Meetup

  1. 1. Legal Informatics w/ AWS CloudSearch &High-Performance Financial Market Apps AWS Michigan Meetup October 9th, 2012 http://www.solidlogic.com 1
  2. 2. Objectives ‣ Introduce Quantitative Trading ‣ Present a case study on AWS usage in Quantitative Trading System Evaluation. ‣ Discuss potential improvements upon our presented architecture. http://www.solidlogic.com 2
  3. 3. About Us http://www.solidlogic.com 3
  4. 4. Solid Logic Technology develops innovative customtechnology solutions across a variety of industriesusing leading software, infrastructure andbusiness practices. http://www.solidlogic.com 4
  5. 5. About usOur expertise Industry experienceInfrastructure and cloud computing ‣ Financial and legal services‣ Scalable, programmatic infrastructure ‣ Logistics management ‣ Automotive‣ Strategic data center design ‣ Defense and homeland security‣ VMware architecture and management ‣ Consumer sales and service‣ Multi-cloud development and deployment‣ Scalable web infrastructure with CDN ‣ Academic and scientific research‣ Security and compliance methods and implementationSoftware development Company Information‣ Analytical solutions - simulation, optimization, big ‣ Founded in 2011 data, natural language processing, quant. finance ‣ Entirely mobile company‣ Enterprise content management, workflow solutions, system integration ‣ Develop both internal projects (IP) and‣ Oracle Transportation Management client software solutions‣ Database technology (Oracle, Vertica, Postgres, Cassandra, etc.)‣ Web application and website development http://www.solidlogic.com 5
  6. 6. Solid Logic Management Team‣ Eric Detterman, CEO and Co-Founder • Professional Experience - Legal IT Business Analyst, Lean Startup, Cloud Computing, Processing Engineering and Consulting - Researched and developed core investment strategies for Birmingham, MI RIA - Currently in production and managing > $20M, AUM growth > 50% annually - Proprietary trading (equities, futures, options), web and software development • Education: B.S. Economics – Oakland University‣ Michael Bommarito, CIO and Partner • Relevant Experience - “Big data” consultant, Oracle ERP architect, Linux cluster administrator. - Software developer - NYC-based quantitative hedge fund - Consultant - multiple quantitative hedge funds • Education : M.S.E Financial Engineering, M.S. Political Science, B.S. Mathematics – University of Michigan‣ Ronald Redmer, Board Member and Lead Technical Advisor • Relevant Experience - CIO, National Default Exchange (NDeX), a business unit of The Dolan Company (NYSE:DM) - CEO defense supplier company, Airport systems software, CEO auto testing company, Affina – software dev mgr, EDS - tech lead http://www.solidlogic.com 6
  7. 7. Case Study:Proprietary Trading Simulation http://www.solidlogic.com 7
  8. 8. Case Study: Proprietary Trading Simulation Quantitative Trading and Investment Systems: ‣ (Loose) Definition: • Rules-based mathematical ‘model’ created by testing and validating a hypothesis about how a tradable market acts or optimizing parameters to create an equation to describe the market. • The goal is to outperform the broad market (S&P 500) or some benchmark after costs. ‣ Example Strategy: • Investment universe = ~50 Fidelity Mutual Funds • Strategy #1: Invest in the top six ranked mutual funds based on proprietary momentum (p0 > p-1) based ranking algorithm. Analyze and rank fund universe every 45 days and re-allocate. • Strategy #2: Invest in the top six ranked mutual funds based on proprietary mean reversion (p0 > p-1) based ranking algorithm. Analyze and rank fund universe every 45 days and re-allocate. http://www.solidlogic.com 8
  9. 9. Case Study: Proprietary Trading Simulation Hypothetical Example http://www.solidlogic.com 9
  10. 10. Case Study: Proprietary Trading Simulation Challenge: Scope: ‣ Characterize the performance and ‣ Assets 62 sensitivity of an equity trading ‣ Tests/asset 96 system across input parameters and market conditions ‣ Total tests 5,952 ‣ Optimize parameters based on profit and risk measures Test Information ‣ Estimated runtime is unacceptable ‣ Mean components/asset 395 on local workstation (>1 month) ‣ Primary bottlenecks are in dense ‣ Points/component 3,135 linear algebra operations ‣ Points/test 1,238,325 • Spectral decomposition (ARPACK) Pairwise comparison of higher-order Total elements 7,370,510,400 • distribution moments (M-M arithmetic) ‣ http://www.solidlogic.com 10
  11. 11. Case Study: Proprietary Trading Simulation Potential solutions: ‣ Run on existing hardware – wait for results ‣ Physical or virtualized servers with supporting job schedulers – requires hardware, software, and specialized labor ‣ Setup cloud infrastructure to process work – requires software and specialized labor http://www.solidlogic.com 11
  12. 12. Trading Simulation: ArchitectureThis was our initial version – Not overly elegant, but works very wellwith minimal effort to setup. Easy to improve upon. Strategy Test Trading System Results Source Code and Custom (S3 Buckets) Config Data Created (Git Repo) AMIs (x16) Availability Zone US East Region http://www.solidlogic.com 12
  13. 13. Trading Simulation: Test Process Trading System Source Code (Git Repo) Strategy Test Results (S3 Custom Buckets) Created AMIs (x16) Availability Zone US East Region Local Development Environment http://www.solidlogic.com 13
  14. 14. Trading Simulation: Overview Technology Solution: Compute Instance (x16): ‣ Built an optimized simulation ‣ 88 Elastic Compute Units (ECU) environment as virtual image ‣ 2x Xeon E5-2670s-16 cores (AWS EC2 AMI) ‣ 60.5GB RAM ‣ Provisioned and configured ‣ 10GbE, dual NIC centralized storage (AWS S3) ‣ 3+TB instance scratch • Experiment configuration • Simulation input Total Compute Resources: • Simulation output ‣ 1408 ECUs • Post-processed results ‣ 512 concurrent threads (HT) ‣ Fully automated deployment of ‣ 968GB RAM simulation to instances through master source control system (1 ECU~=5GFlops) (git) http://www.solidlogic.com 14
  15. 15. Trading Simulation: AMI Creation Process ‣ Use standard Ubuntu Server 12.04.1 LTS for Cluster Instances AMI x64 (ami-eb7bcf82) • cc2.8xLarge – 88ECUs, 16 cores, 60.5GB RAM ‣ Install git, s3cmd, PostgreSQL JDBC drivers ‣ Install and configure test environment and all dependencies ‣ Create new AMI based on the above http://www.solidlogic.com 15
  16. 16. Trading Simulation: Test Execution ‣ For each test instance….. • ssh -X -i /home/ericd/.aws/first/name.pem ubuntu@IP • cd /home/ubuntu/testcode/tradingsystemsales • git pull • cd /usr/local/testcode//bin • sudo ./testcode -nodesktop • parameterSweepSingleNode(Yes,Yes, homeubuntutestcodetradingsystemsalesmodelsdailyAdaptiveStateSpaceSPYdatamasterlist.mat, /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/data/etfs, csv, , /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/data/etfs, 1, homeubuntutestcodetradingsystemsalesmodelsdailyAdaptiveStateSpaceSPYdataETFsToTest.csv, /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/data/etfs, mat, /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/src, No, No) • parameterSweepSingleNode(No,Yes, homeubuntutestcodetradingsystemsalesmodelsdailyAdaptiveStateSpaceSPYdatamasterlist.mat, /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/data/etfs, csv, , /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/data/etfs, 1, homeubuntutestcodetradingsystemsalesmodelsdailyAdaptiveStateSpaceSPYdataETFsToTest.csv, /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/data/etfs, mat, /home/ubuntu/testcode/tradingsystemsales/models/daily/AdaptiveStateSpaceSPY/src, No, No) http://www.solidlogic.com 16
  17. 17. Trading Simulation: Initial Test Results ‣ Result sets saved to S3 buckets using S3cmd • Approximately 6000 result sets http://www.solidlogic.com 17
  18. 18. Trading Simulation: Output‣ Run Time: • Cloud: 45 hours • Single-seat: 1-2 months • Order of magnitude improvement in time! http://www.solidlogic.com 18
  19. 19. Trading Simulation: Economics Co- On- Spot On-Site Location Demand Pricing Cost Model DetailsServer Hardware / ‣ Cost estimates using assumptions $69,045 $69,045 $1,647 $193Instance Usage and calculations in Cost ComparisonNetwork Hardware 13,809 13,809 - - WorksheetHardware Maint. 24,856 24,856 - - ‣ Costs represent one year annualized costs. Assumes a useful life of threeOperating System - - - - years for purchased equipmentPower and Cooling 9,907 - - - ‣ 1= Cost savings using On-Site asData Center baselineConstruction / Co- 8,618 65,136 - -Location Expense ‣ 2= On-Site and Co-Location assumeAdmin. / Remote 100% usage 105,000 240 - -Hands Support ‣ 3= Based on actual 686 machineData Transfer 1 4 1 1 hours usedTotal $231,237 $173,091 $1,647 $193Cost Savings1 N/A 25.12% 99.29% 99.92%$ / Compute Hr.2,3 $26.40 $19.76 $2.40 $0.28 http://www.solidlogic.com 19
  20. 20. Trading Simulation: Next Steps Potential Improvements: ‣ Develop improved cloud infrastructure management tools • Allocation of work across instances • Allow user defined completion time and programmatically scale compute resources to work towards goal • Spread work across unused internal and available external compute resources http://www.solidlogic.com 20
  21. 21. Thank you Eric Detterman Michael Bommarito CEO, Co-Founder CIO, Partner Eric.Detterman@solidlogic.com Michael.Bommarito@solidlogic.com Direct: (248) 792 – 8001 Direct: (646) 450 – 3387 (248) 792 – 8000 www.solidlogic.com 330 East Maple Rd. #231 Birmingham, MI 48009 21

×