A classifier labeling Van Gogh drawings as invoices and a chatbot insulting users on Twitter are only two examples of Machine Learning (ML) models, which went wild as soon as they hit production. Although evaluated on a test set, in the face of unseen data ML models oftentimes behave in an unpredictable way. Depending on the application, such a model may lead to decreasing revenue, bad reputation or even a threat to the health of people. To ensure a stable rollout of new models into production, we have to promote ML models to first class citiziens in the Continuous Delivery pipeline. Kubernetes and Kafka are two great tools to support the rollout of new machine learning models in a (semi-) automated way. I will show a pipeline built with these tools, which will lead to more confidence in your deployments and happier users.

1. 1Digital Product School by UnternehmerTUM

2. 2Digital Product School by UnternehmerTUM

3. 3Digital Product School by UnternehmerTUM

4. 4Digital Product School by UnternehmerTUM
Test vs. Live Data

5. 5Digital Product School by UnternehmerTUM
Time between Deployment...

6. 6Digital Product School by UnternehmerTUM
… and Feedback

7. 7Digital Product School by UnternehmerTUM
Reproducibility

8. Ship your Machine Learning Application
---
Stockerl Michael

9. 9Digital Product School by UnternehmerTUM
UnternehmerTUM

10. 10Digital Product School by UnternehmerTUM
How we release a new model

11. 11Digital Product School by UnternehmerTUM
Preprocessing Step
trng_doctype_raw
PREPROCESS
trng_doctype_4915414
test_doctype_4915414
PREPRO
CESS

12. 12Digital Product School by UnternehmerTUM
Training Step
trng_doctype_raw
PREPROCESS
trng_doctype_4915414 CANDIDATE
TRAIN
test_doctype_4915414
PREPRO
CESS

13. 13Digital Product School by UnternehmerTUM
Create Baseline on Test Data
trng_doctype_raw
PREPROCESS
trng_doctype_4915414
test_doctype_4915414
PREPRO
CESS
EVALUATE
doctype_model_live

14. 14Digital Product School by UnternehmerTUM
Compare New And Old Model
trng_doctype_raw
PREPROCESS
trng_doctype_4915414
test_doctype_4915414
PREPRO
CESS
EVALUATE
doctype_model_7749864

15. 15Digital Product School by UnternehmerTUM
Publish New Model
trng_doctype_raw
PREPROCESS
trng_doctype_4915414
test_doctype_4915414
PREPRO
CESS
doctype_model_7749864GO
O
D
PUBLISH

16. 16Digital Product School by UnternehmerTUM
Test the Model in the Real World

17. 17Digital Product School by UnternehmerTUM
Service Build Step
Merge Branch Service Image
MODEL
RUN TESTS
BUILD

18. 18Digital Product School by UnternehmerTUM
Service Build Step
Merge Branch Service Image
MODEL
RUN TESTS
BUILD PUBLISH

19. 19Digital Product School by UnternehmerTUM
Production Overview
doctype_classifier_deployment
1
incoming_4915414
2 3
live_doctypes

20. 20Digital Product School by UnternehmerTUM
Canary Deployment
doctype_classifier_deployment
1
incoming_4915414
2 3
live_doctypes
doctype_classifier_canary
1
canary_doctypes

21. 21Digital Product School by UnternehmerTUM
Find Different Classifications
live_doctypes
canary_doctypes
spotchecker_input
JOIN

22. 22Digital Product School by UnternehmerTUM
Select the correct document type:
Spot Checking
INVOICE
CONTRACT
OTHER
INVOICE
CONTRACT
OTHER
INVOICE
CONTRACT
OTHER

23. 23Digital Product School by UnternehmerTUM
Build up Test Data

24. 24Digital Product School by UnternehmerTUM
Which one in Production?

25. 25Digital Product School by UnternehmerTUM
Deploy

26. 26Digital Product School by UnternehmerTUM
Limitations

27. Questions?
Michael Stockerl
Head of Software Engineering - UnternehmerTUM
Email: stockerl@unternehmertum.de
Twitter: @stockerlm

28. 28Digital Product School by UnternehmerTUM
References
Pictures:
• https://de.wikipedia.org
• pxhere.com