Evolution of AWS infrastructure for ML: from Zero to Hero
•
0 likes•234 views
Real experience of building and evolution of Machine Learning model using AWS ecosystem – from from scratch to fully-fledged production solution generating 20M predictions per day just in 2 month.
![About me
{
“First name”: “Taras”,
“Last name”: “Slipets”,
“Occupation”: “Senior Software Engineer”,
“Company”: “Flixbus”,
“Interests”: [
“Software Engineering”,
“Music”,
“Skiing”
]
}
2](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Evolution of AWS infrastructure for ML: from Zero to Hero
Similar to Evolution of AWS infrastructure for ML: from Zero to Hero (20)
More from Taras Slipets
More from Taras Slipets (11)
Recently uploaded
Recently uploaded (20)
Evolution of AWS infrastructure for ML: from Zero to Hero
- 1. Evolution of AWS infrastructure for ML from Zero to Hero
- 2. About me { “First name”: “Taras”, “Last name”: “Slipets”, “Occupation”: “Senior Software Engineer”, “Company”: “Flixbus”, “Interests”: [ “Software Engineering”, “Music”, “Skiing” ] } 2
- 3. Disclaimer: my own personal experience 3
- 4. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 4
- 5. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 5
- 7. Heterogeneous data sources ● Purchase history ● Website activities ● etc... 7
- 8. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 8
- 10. Speed up Data Science experiments 10
- 11. Git + SageMaker Notebook Instances 11 AWS SageMaker
- 12. Key takeaways ● K.I.S.S. ● Concentrate efforts on business value, not on technologies stack 12
- 13. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 13
- 14. Split up pipeline into clear steps 14
- 15. Input data collection at scale 15
- 16. Features extraction at scale 16 Lambda Redshift Lambda S3Glue Athena
- 17. Model training and inference 17 AWS SageMaker S3S3 CRON + Bash
- 18. We used a VERY big machine 18 AWS SageMaker S3S3 CRON + Bash
- 19. Key takeaways ● Clearly define what you actually need - there might be service ready for use in cloud provider ● Cloud resources are cheaper than long-time manual infrastructure setup and optimisation 19
- 20. MVP in 1 month 20
- 21. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 21
- 22. Containerization 22 ECR CI / CD Pipelines
- 24. Costs reduction 24 CRON ECS 24 / 7 On Demand 44%
- 25. Key takeaways ● Get actual practical numbers ● Actual bottlenecks might be completely different from your preliminar gut feelings ● Resolve low-hanging fruits first 25
- 26. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 26
- 28. Aligned reactive execution 28 ECS Step Function
- 29. Monitoring and metrics 29 CloudWatch QuickSight
- 31. Key takeaways ● Think about entire e2e flow observability ● Start with simple, but business-relevant KPIs ● Separate business and technical metrics ● Offload knowledge of data engineers into code 31
- 32. Agenda ● Problem statement ● Infrastructure start point ● Infrastructure evolution ○ First automations ○ Containerisation and cost reduction ○ Reactive approach ● Future improvements ● Summary and Q&A 32
- 33. Future improvements 33 ● AWS SageMaker ecosystem usage for model training and inference ● Features extraction optimisation via AWS Athena and Glue ● Custom AWS CloudWatch metrics for pipeline execution monitoring ● Make model training and inference independent ● Enable real-time inference beside batch processing ● ...
- 34. Summary ● Start simple, bring business value ● Cloud resources are cheap and fast to setup ● Preliminary optimisation based on guessing is bull$&it ● Fully-fledged infrastructure automation should go after you have MVP, not before ● Have measurable and observable metrics 34
- 35. Q&A 35