"Developing a complicated ensemble model with hundreds of features fetched from a bunch of different sources? Give me two! Showing great metrics to the stakeholders and already discussing how it will hit a home run in production? Why not! And then getting stuck for months trying to deploy the model and fighting with data inconsistency and bugs? Sounds familiar? This talk will focus on providing guidelines on how to build your model development process keeping in mind the deployment phase to come later on." I presented this talk at PyCon & PyData DE 2019 in Berlin and Data Science fwdays 2019 in Kyiv.

1. APPLYING DEPLOYMENT
ORIENTED MINDSET FOR
BUILDING MACHINE LEARNING
MODELS
Marianna Diachuk

2. ➔ Why am I telling you this?
➔ What I mean by deployment?
➔ Deployment problems
➔ Model development process
➔ What deployment oriented mindset gives you
2

3. - I developed and deployed to production
7 scoring and antifraud ensemble models
- I leaded a small but proud team of 2 data
scientists and 1 data engineer
- I’m a mother of 3 dragons ducks
3

4. WHAT WHY
4

5. 5

6. 6

7. ● Model response inconsistency
7
research environment development environment

8. ● Model response inconsistency
8
research environment development environment
dataset

9. ● Features inconsistency
9
research environment development environment
dataset

10. ● Model response inconsistency
● Impossibility to implement features
calculations
● Features inconsistency
● Model is not scalable
and so on and so on…
1
0

11. 1
1

12. Research process
Agile process
1
2

13. Remember
about
deployment.
1
3

14. How fastshould our
modelrespond?
Are
there
any
lim
itations
for
deploym
ent?
Can we fetch
the data
from
DB
quickly?
Candevelopers
help
uswith
deploym
ent?
Should we
worry about
features
calculation
time?
1
4

15. your model
the
system
model
object
features
calculation
fetching
raw data
1
5

16. BUSINESS
UNDERSTANDIN
G
STAGE
Pay attention to:
● Model response time
● Feature calculation time
● Database response
● Human resources
availability
1
6

17. DATA
UNDERSTANDIN
G
STAGE
You can:
● Limit data sources
1
7

18. 1 request per 1 min.
1 application per <1 min.
1
8

19. DATA
UNDERSTANDIN
G
STAGE
You can:
● Limit data sources
● Work closely with
colleagues
1
9

20. Ouch…
It was one of the
top features :(
We
removed
one field
from the
application
form.
2
0

21. MODELLING You can:
● Refactor your code
2
1

22. 1. Write your feature calculation
4. Improve your code efficiency
2. Test the feature
3. Improve your code readability
2
2

23. 2
3

24. MODELLING
You can:
● Refactor your code
● Test model response
2
4

25. • ExecuteTime from jupyter_contrib_nbextensions
• time module
• %%time
2
5

26. Keep track of
versions.
Always!
but not like this
2
6

27. 2
7

28. Document everything!
So your future self and your
teammates would be happy.
2
8

29. ➔ keep clean
Jupyter notebooks
➔ keep all changes
in code
2
9

30. ● Easier to estimate tasks
● Less risks in the late phases -> less postponed deadlines
● Better to understand responsibilities distribution
3
0

31. THANKS FOR YOUR
ATTENTION
AND HAPPY DATA
SCIENCING!

32. marianna-diachuk
@dark_matter88_
@mariaannadiachuk
DarkMatter88

33. Q & A SESSION