Download as PDF, PPTX
Uploaded bySadik Bakiu
AB Testing in Machine Learning.pdf
The document discusses A/B testing in machine learning operations (MLOps). It introduces A/B testing, Kubernetes, Helm, and Seldon-core. A/B testing determines which of two variants performs better by objectively measuring a chosen metric. Kubernetes is a container orchestrator that manages pods and services. Helm is a package manager for Kubernetes that allows deployment of charts with versions. Seldon-core deploys and monitors machine learning models on Kubernetes through containerized microservices with built-in inference, metrics, and monitoring capabilities.
AB Testing in Machine Learning.pdf
- 1. 1 A/B Testing in MLOps SadikBakiu, Data Max
- 2. 2 Agenda What is A/Btesting 1 What is Helm 3 What is seldon 4 Hands On 5 What is Kubernetes 2
- 3. 3 Sadik Bakiu S AD I K B A K I U Co-Founder & Co-CEO, Data Max Graduated 2016, MSc, RWTH Aachen University Working with data for 10+ years now Find me in LinkedIn: linkedin.com/in/sbakiu Email: sadik@data-max.io
- 4.
- 5. 6 A/B Testing A /B T E S T I N G Determine which variant performs better Used in advertisement a lot Important: Objectively measure the results Pick a metric Compares two variants of one variable
- 6.
- 7. 8 Kubernetes K U BE R N E T E S Controller Pods Containers Services Container Orchestrator
- 8.
- 9. 10 Helm H E LM Like Pip Has Charts Charts have versions Package manager for Kubernetes
- 10.
- 11. 12 Seldon-core S E LD O N C O R E Deploy and monitor ML models on Kubernetes Create containerized microservices Out-of-the-box inference server Metrics and monitoring Python app server tailored to ML
- 12.
- 13. 14 Streamlit S T RE A M L I T All in Python Easy to use Build data apps
- 14. What will we bedoing 6
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21. 26 A/B testing testsversions of a variable MLOps tooling is mature now With microservice architecture it is easy to start A/B testing has limits Summary
- 22.