1. Uber’s Journey into
Microservices
Emily Reinhold, Software Engineer, Uber
JUNE 15, 2016

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
uber-darwin

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. Our Monolith
Background

5. ‣“API”
‣Started January 2011
Fun Facts

6. Lines of Code over Time
0K
75K
150K
225K
300K
2011 2012 2013 2014 2015 2016
Unique Authors over Time
0
150
300
450
600
2011 2012 2013 2014 2015 2016
Start of
migration.
Yay!
API Trends

7. ‣“API”
‣Started January 2011
‣Access to core data models
Fun Facts

8. Core Data Models
Users Trips Payments
Products Cities Documents
Exchange
Rates
Vehicles Promos
The Uber Monolith

9. ‣Postgresql
‣Python 2.7
‣HTTP/JSON
‣uWSGI
Technical Details

10. Request Flow
Matching
Service
haproxy API box nginx uWSGI
process
Postgres
Downstream
Services
Third Parties
API

11. ‣Minimal overhead for cross-
domain features
Benefits

12. “Cross Domain”
Users Trips Payments
Products Cities Documents
Exchange
Rates
Vehicles Promos
The Uber Monolith

13. ‣Minimal overhead for cross-
domain features
‣Reuse code across domains
‣Broad, extensible interface
Benefits

14. ‣Honks
‣Halloween 2014
‣Postgresql Master Failure
Historic Outages
Image courtesy of Anita Hart. Licensed under CC-BY-SA 2.0.
https://flic.kr/p/8PG15b

15. Postgresql Master Failure

16. ‣Honks
‣Halloween 2014
‣Postgresql Master Failure
‣Kafka Outage
Historic Outages

17. Kafka Outage

18. ‣Deploy burden
‣Single Postgres Master
Technical Bottlenecks
Image courtesy of Paul Schadler. Licensed under CC-BY 2.0.
https://flic.kr/p/aDoGkP

19. ‣Rapid hiring
‣Slow test suite
‣Toe stepping, merge conflicts
Developability

20. ‣Growth of business
‣Growth of features
‣Growth of organization
Scalability 3 Ways

21. In Microservices
Our Future

22. ‣Horizontally scalable
Benefits

23. Scalability
Trips Users Promos

24. ‣Horizontally scalable
‣Improved onboarding
‣Clear ownership
Benefits

25. Ownership
No hiding!
You’re on call!

26. ‣Horizontally scalable
‣Improved onboarding
‣Clear ownership
‣Failure isolation
Benefits

27. Granular Failures
Old Way
API
client
show
Payments
Promos
User tags
Edge Service
New Way
Edge Service
Payments Promos Users
?

28. Lessons Learned

29. ‣Drastically changing tech
stack
Limit Scope

30. Tech Stack
That’s
me!
eng.uber.com/building-tincup/

31. Data Migration
1. Develop new
schema
2. Implement
converter
3. Dual write 4. Backfill
6. Move read
queries
5. Validation
?
7. Remove old
writes

32. ‣Drastically changing tech
stack
‣Infrastructure not ready
‣ Understaffed
‣ Lacked tooling
Limit Scope

33. Things Happen
Image courtesy of Kim Scarborough. Licensed under CC-BY-SA 2.0.
https://flic.kr/p/23HUAK

34. ‣Drastically changing tech
stack
‣Infrastructure not ready
‣ Understaffed
‣ Lacked tooling
‣Understanding timelines
Limit Scope

35. ‣Monolith’s consumers
directly impacted
Aligning with Consumers

36. Consumers
API
Pricing
Matching
Signup
Exchange rates
Cities
Users
Trips
Trips
Exchange
Rates
Cities
Users

37. ‣Monolith’s consumers directly impacted
‣Importance of migrating not well
communicated
‣Constant interruption
Aligning with Consumers

38. ‣For service developers and
migrating consumers
‣Tool to split up request
Build Tools Initially

39. Request Splitter
API
Cities
Currencies
Users
Trips
Old Way
requesting trip
info, with
relationships
on user,
currency, city
New Way
API
Cities
Currencies
Users
Trips
Tool
requesting trip
info, with
relationships
on user,
currency, city
request user
request city
request currency
request trip

40. ‣For service developers and migrating
consumers
‣Tool to split up request
‣Tool to validate new response
‣Tool to gain visibility into consumers
Build Tools Initially

41. ‣Most difficult aspect of migration
‣Silently call new system
Consumer Migration Tips

42. Silent Swap
API
Matching
Signup
Exchange ratesCities
Users Trips
Users

43. ‣Most difficult aspect of migration
‣Silently call new system
‣Most successful migration kept
interface the same
Consumer Migration Tips

44. ‣Tendency to avoid changes
‣Tackling core (User) directly
required refactor
Love your Monolith

45. ‣Likely to impact other teams
‣Teams shift focus
Migration Blockers

46. Migration Successes

47. ‣Shared accomplishment
‣Relationships formed
‣New understanding
Collaboration

48. ‣New tech stack: more efficient
‣No more uWSGI!
Hardware efficiency

49. uWSGI Utilization

50. CPU Usage

51. ‣New tech stack: more efficient
‣No more uWSGI!
‣Calls asynchronous
Hardware efficiency

52. ‣How our systems work
‣Design for stability
‣Evenly spread ownership
Deeper Understanding

53. First & Last Name
Email: emilyr@uber.com
Engineering Blog: eng.uber.com
Twitter: @UberEng
Thank you

55. Watch the video with slide synchronization on
InfoQ.com!
https://www.infoq.com/presentations/uber-
darwin