How AQR Capital Uses AWS to Research New Investment Signals
•
5 likes•2,351 views
AQR Capital is using AWS Batch for high-performance computing to develop investment signals, enabling AQR researchers to package and submit a job to evaluate a signal without concern over compute resources, cost, security, or timing. The intelligent use of Amazon EC2 instances and Spot by AWS Batch has enabled AQR to process more than 75 years’ worth of compute workload at a very low cost. In this session, learn how to use AWS Batch and containers for high-performance computing work to manage, schedule, or scale underlying EC2 instances.
More Related Content
What's hot
What's hot (20)
Similar to How AQR Capital Uses AWS to Research New Investment Signals
Similar to How AQR Capital Uses AWS to Research New Investment Signals (20)
More from Amazon Web Services
More from Amazon Web Services (20)
How AQR Capital Uses AWS to Research New Investment Signals
- 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Michael Raposa Head of Platform Engineering, AQR Capital John Kain Head of Business and Market Development, Capital Markets, AWS FSI303 How AQR Capital Leverages AWS to Research New Investment Signals
- 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Grid computing trends in Financial Services Widespread deployment Leading edge Traditional (Data Synapse/ Platform Symphony) Spark/Container Orchestration & Schedulers Serverless Current platform of choice AWS Batch AWS Lambda
- 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Introducing AWS Batch AWS Batch Managed No software to install or servers to manage Integrated with AWS Natively integrated with the AWS platform Cost-optimized Automatically provisions compute resources
- 4. How AQR Capital Leverages AWS to Research New Investment Signals Michael Raposa July 17, 2018 Head of Platform Engineering, AQR Not intended for the sale or marketing of financial products or services.
- 5. Agenda • About AQR • Business Problem • Solution Summary • Lessons Learned • Takeaways 5
- 6. Our Firm AQR is a global investment management firm built at the intersection of financial theory and practical application. We strive to deliver superior, long-term results for our clients by looking past market noise to identify and isolate what matters most, and by developing ideas that stand up to rigorous testing. Our focus on practical insights and analysis has made us leaders in alternative and traditional strategies since 1998. At a Glance • AQR takes a systematic, research-driven approach to managing alternative and traditional strategies • We apply quantitative tools to process fundamental information and manage risk • Our clients include institutional investors, such as pension funds, defined contribution plans, insurance companies, endowments, foundations, family offices and sovereign wealth funds, as well as RIAs, private banks and financial advisors • The firm has 36 principals and 914 employees; over half of employees hold advanced degrees • AQR is based in Greenwich, Connecticut, with offices in Boston, Chicago, Hong Kong, London, Los Angeles, and Sydney • Approximately $225 billion in assets under management as of March 31, 2018* *Approximate as of 3/31/2018, includes assets managed by AQR and its advisory affiliates. 6
- 7. Problem Statement Background • Quantitative Asset Manager • Investment Decisions based on numerical models and systematic trading • Researchers develops models and “back test” ideas over many years • Never-ending appetite for more data and non-obvious data sets 7 Source: AQR. For illustrative purposes only.
- 8. Researcher Workflow Idea Gather Data Build Model Backtest Analyze Result 8 Source: AQR. For illustrative purposes only.
- 9. Problem Statement Background • Quantitative Asset Manager • Investment Decisions based on numerical models and systematic trading • Researchers develops models and “back test” ideas over many years • Never-ending appetite for more data and non-obvious data sets Problem • On premise compute grid can’t keep up with demand • CAPEX locked into grid • Researchers wait for grid resources • Researchers need job results as quickly as possible • New experimental use cases, e.g. GPU, require significant time and money upfront investment 9 Source: AQR. For illustrative purposes only.
- 10. Design Considerations 1. Scalable both in compute and memory 2. Fast without long queue times 3. Don’t want to manage a Job Scheduler, e.g. Condor or Sun Grid Engine 4. Easy to use 5. Secure 10
- 11. Solution Summary Burstability Leverage building block application services, e.g. S3, EC2, DynamoDB Reduce costs – Spot Leverage Cloud 11
- 12. Solution Summary Based on Sun Grid Engine Submit jobs via CLI or API – Unlimited compute at fingertips Short feedback loop – immediate results to researcher Backend grid matches researcher workstation – No “It works in DEV but not in the GRID” Seamless Interface 12
- 13. Solution Summary Get out of researchers way – Automate everything “Unlimited” compute Short start times – No “queues” Fast 13
- 14. Solution Summary Encryption Everywhere Leverage AWS Security Tools Security Cloud Engineering problem – Not researcher’s Secure 14
- 19. Solution Summary – Job Submission (CLI) 19
- 20. Solution Summary – Job Submission (CLI) 20
- 21. Solution Summary – Job Submission (CLI) 21
- 22. Solution Summary – Job Submission (CLI) 22
- 23. Solution Summary – Job Submission (CLI) 24
- 24. Solution Summary – Job Submission (CLI) 25
- 25. Solution Summary – Job Submission (API) Step 1 29
- 26. Solution Summary – Job Submission (API) Step 2 30
- 27. Solution Summary – Job Submission (API) Step 3 31
- 28. Solution Summary – Job Submission (API) Step 4 32
- 29. Solution Summary – DAGs 33 Client Job Child jobs called by other jobs
- 30. Lessons Learned - General Use Spot Use as many instance types as you can Use as many AZs as you can Drive the Lowest Cost - $15/1000 vCPU/Hour 34 Spot Instance Availability Zones Instance Families
- 31. Lessons Learned - General ECS Logs Job Output Host Logs …etc. Log Everything! 35
- 32. Lessons Learned - General Job Run Time Job Start-up Times Job Cost by User vCPU Consumption by User High Priority Queue Consumption by User Monitor Everything! 36
- 35. Lessons Learned As we added more Users… 1. TooManyRequestsException 1. DescribeJobs API Call Failing 2. Providing Governance “Guard Rails” 39
- 36. Lessons Learned – Event Based Pipeline 40
- 37. Lessons Learned – Event Based Pipeline 41
- 38. Lessons Learned – Event Based Pipeline 42
- 39. Lessons Learned – Event Based Pipeline 43
- 40. Lessons Learned As we added more compute… 1. TooManyRequestsException … API calls in the container 2. Job State storage woes 3. Job Start times 4. Job Costs 44
- 41. Lessons Learned – API in Containers 45 Parameter Store ECS container Get Secret
- 42. Lessons Learned – API in Containers At Scale 46 Exception: Too many requestsParameter Store ECS container ECS container ECS container ECS container ECS container
- 43. Lessons Learned – API in Containers At Scale 47 ECS container ECS container ECS container ECS container ECS container Amazon S3
- 44. What is Shared Job State Shared Memory Across Containers Job Assignment & Orchestration Job Input and Output 48
- 45. Lessons Learned – Job State Backend 49 Amazon EFS
- 46. Lessons Learned – Job State Backend 50 Amazon EFS Redis
- 47. Lessons Learned – Job State Backend 51 Amazon EFS Amazon S3 Redis
- 48. Lessons Learned – Job Start Times 52
- 49. Lessons Learned – Set Limits 53
- 50. Takeaways Spot Multi-AZ Log Everything Monitor Everything Follow Best Practices 54 Spot Instance Availability Zones
- 51. Takeaways Eliminate API Calls in your containers. Only use services that scale, e.g. DynamoDB Switch to Event/Message Based Status from Poll Based Choose a job state backend that fits your use case Watch for Scale Issues 55
- 52. Takeaways Write a light-weight wrapper around AWS Batch for non-technical users Have a “Governator” function to ensure compliance Make it Easy and Secure 56
- 53. Takeaways Packer and bake AMIs Pre-warm the cluster during active times of the day Give yourself an SLA – 75% of jobs start in 10 mins and 90% in 15 mins Reduce Your Start Times 57
- 54. Takeaways Alarm for large long running jobs -> $$$ Set limits on your Compute Environment Control Runaway Costs 58
- 55. Q&A
- 57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you!