Accelerating development velocity of production ML systems with Docker by Kinnary Jangla The rise of microservices has allowed ML systems to grow in complexity but has also introduced new challenges when things inevitably go wrong. Most companies provide isolated development environments for engineers to work within. While a necessity once a team reaches even a small size, this same organizational choice introduces potentially frustrating dependencies when those individual environments inevitably drift. Kinnary Jangla explains how Pinterest dockerized the services powering its home feed to accelerate development and decrease operational complexity and outlines the benefits Pinterest gained from this change that may be applicable to other microservice-based ML systems. This project was initially motivated by challenges arising from the difficulty of testing individual changes in a reproducible way. Without standardized environments, predeployment testing often yielded nonrepresentative results, causing downtime and confusion for those responsible for keeping the service up. The Docker solution that was eventually deployed prepackages all dependencies found in each microservice, allowing developers to quickly set up large portions of the home feed stack and always test on the current team-wide configs. This architecture has enabled the team to debug latency issues, expand its testing suite to include connecting to simulated databases, and more quickly do development on our thrift APIs. Kinnary shares tips and tricks for dockerizing a large-scale legacy production service and discusses how an architectural change like this can change how an ML team works.

2. May 24th, 2018Kinnary Jangla
Accelerating
development
velocityofproduction
MLsystemsusing
docker

3. Pinterest is the visual discovery
engine.
Mission

4. A visual bookmark
saved from the
internet by a user.
Pin

5. Boards

6. ● Most engaging surface.
● Drives ~40-50% of Pinterest’s user engagement
and ads revenue.
● First product powered by machine learning.
● Core problem: predict the engagement
likelihood between a user and a pin.
Homefeed

7. As of Sep 14, 2017
100b+ Pins
75%+ Sign-ups
50%+ of users
are outside of the U.S.

8. Machine learning @ Pinterest
Interest-basedfeed
Recommendations
Search
Ads
VisualDiscovery
1
2
3
4
5

9. Monolithto
Microservices

10. Monolith

11. Microservices

12. Agenda
1
2
3
4
5

13. Smartfeed
Search
Pinnability
Picked-For-You

14. Challenges
withMicroservices

15. End-to-end Model Building
● Training Data Generation
● Model Training
● Offline Evaluation
● Online Experiment

16. Modeltraining
train
models
evaluate
models
A/B
test rollout
generate
hypotheses

17. Candidate Generator
Candidate Generator
Candidate Generator
Ranking
Model
Candidates
Candidates
Candidates
Policy
Layer
Followed Topics
Followed Users
Recommendations
Homefeed
How does Homefeed work?

18. Services During Model Serving

19. Debugging

20. Solutions?

21. Whatis…

22. 1. Bundle applications with its runtime environment
2. Runs on its own network interface
3. Similar to a virtual machine

26. DockerfileDockerfile

27. Connectingservices

28. DockerCompose

29. DockerfileDocker-composefile

30. Demo
Anythingthatcangowrongwillgowrong,EdwardMurphy

31. Docker commands
Inspect
Attach
Exec
Pause,Stop,Remove
Top
1
2
3
4
5

32. SoftwareEngineeringDaily
Podcast
Dockerformicroservices,
Apress

33. Q&A
kinnary@pinterest.com

34. © Copyright, All Rights Reserved, Pinterest Inc. 2018