Most large-scale web companies have evolved their system architecture from a monolithic application and monolithic database to a set of loosely coupled microservices. Using examples from Google, eBay, and other large-scale sites, this talk outlines the pros and cons of these different stages of evolution, and makes practical suggestions about when and how other organizations should consider migrating to microservices. It continues with some more advanced implications of a microservices architecture, including SLAs, cost-allocation, and vendor-customer relationships within the organization. It concludes by exploring a set of common service anti-patterns.

1. From the Monolith to
Microservices
Lessons from Google and eBay
Randy Shoup
@randyshoup
linkedin.com/in/randyshoup

2. Architecture
Evolution
• eBay
• 5th generation today
• Monolithic Perl  Monolithic C++  Java  microservices
• Twitter
• 3rd generation today
• Monolithic Rails  JS / Rails / Scala  microservices
• Amazon
• Nth generation today
• Monolithic C++  Java / Scala  microservices

3. Architecture
Evolution
• The Monolith
• Ecosystem of Services
• Building and Operating a Service
• Service Anti-Patterns

4. Architecture
Evolution
• The Monolith
• Ecosystem of Services
• Building and Operating a Service
• Service Anti-Patterns

5. The Monolithic
Architecture
2-3 monolithic tiers
• {JS, iOS, Android}
• {PHP, Ruby, Python}
• {MySQL, Mongo}
Presentation
Application
Database

6. The Monolithic
Application
Simple at first
In-process latencies
Single codebase, deploy unit
Resource-efficient at small scale
Pros
Coordination overhead as team
grows
Poor enforcement of modularity
Poor scaling (vertical only)
All-or-nothing deploy (downtime,
failures)
Long build times
Cons

7. The Monolithic
Database
Simple at first
Join queries are easy
Single schema, deployment
Resource-efficient at small scale
Pros
Coupling over time
Poor scaling and redundancy (all-or-
nothing, vertical only)
Difficult to tune properly
All-or-nothing schema management
Cons

8. “If you don’t end up
regretting your early
technology decisions, you
probably over-engineered”
-- me

9. Architecture
Evolution
• The Monolith
• Ecosystem of Services
• Building and Operating a Service
• Service Anti-Patterns

10. Microservices
• Single-purpose
• Simple, well-defined interface
• Modular and independent
• Fullest expression of encapsulation and modularity
• Isolated persistence (!)
A
C D E
B

11. Microservices
Each unit is simple
Independent scaling and
performance
Independent testing and
deployment
Can optimally tune performance
(caching, replication, etc.)
Pros
Many cooperating units
Many small repos
Requires more sophisticated tooling
and dependency management
Network latencies
Cons

12. Ecosystem
of Services
• Hundreds to thousands of
independent services
• Many layers of dependencies,
no strict tiers
• Graph of relationships, not a
hierarchy
C
B
A E
F
G
D

13. Google
Service Layering
• Cloud Datastore: NoSQL service
o Highly scalable and resilient
o Strong transactional consistency
o SQL-like rich query capabilities
• Megastore: geo-scale structured
database
o Multi-row transactions
o Synchronous cross-datacenter replication
• Bigtable: cluster-level structured storage
o (row, column, timestamp) -> cell contents
• Colossus: next-generation clustered file
system
o Block distribution and replication
• Borg: cluster management infrastructure
o Task scheduling, machine assignment
Cloud Datastore
Megastore
Bigtable
Colossus
Borg

14. Evolution,
not Intelligent Design
• No centralized, top-down design of the system
• Variation and Natural selection
o Create / extract new services when needed to solve a problem
o Deprecate services when no longer used
o Services justify their existence through usage
• Appearance of clean layering is an emergent
property

15. Architecture without an
Architect?
• No “Architect” title / role
• (+) No central approval for technology decisions
o Most technology decisions made locally instead of globally
o Better decisions in the field

16. Standardization
• Standardized communication
o Network protocols
o Data formats
o Interface schema / specification
• Standardized infrastructure
o Source control
o Configuration management
o Cluster management
o Monitoring, alerting, diagnosing, etc.

17. Standards become standards by
being better than the alternatives!

18. Service
Independence
• No standardization of service internals
o Programming languages
o Frameworks
o Persistence mechanisms

19. In a mature ecosystem of services,
we standardize the arcs of the
graph, not the nodes!

20. Creating
New Services
• Spinning out a new service
o Almost always built for particular use-case first
o If successful and appropriate, form a team and generalize for multiple
use-cases
• Pragmatism wins
• Examples
o Google File System
o Bigtable
o Megastore
o Google App Engine
o Gmail

21. Deprecating
Old Services
• What if a service is a failure?
o Repurpose technologies for other uses
o Redeploy people to other teams
• Examples
o Google Wave -> Google Apps
o Multiple generations of core services

22. “Every service at Google is either
deprecated or not ready yet.”
-- Google engineering proverb

23. Architecture
Evolution
• The Monolith
• Ecosystem of Services
• Building and Operating a Service
• Service Anti-Patterns

24. Goals of a
Service Owner
• Meet the needs of my clients …
• Functionality
• Quality
• Performance
• Stability and reliability
• Constant improvement over time
• … at minimum cost and effort
• Leverage common tools and infrastructure
• Leverage other services
• Automate building, deploying, and operating my service
• Optimize for efficient use of resources

25. Responsibilities of a
Service Owner
• End-to-end Ownership
o Team owns service from design to deployment to retirement
o No separate maintenance or sustaining engineering team
o DevOps philosophy of “You build it, you run it”
• Autonomy and Accountability
o Freedom to choose technology, methodology, working environment
o Responsibility for the results of those choices

26. Service-Service
Relationships
• Vendor – Customer Relationship
o Friendly and cooperative, but structured
o Clear ownership and division of responsibility
o Customer can choose to use service or not (!)
• Service-Level Agreement (SLA)
o Promise of service levels by the provider
o Customer needs to be able to rely on the service, like a utility

27. Service-Service
Relationships
• Charging and Cost Allocation
o Charge customers for *usage* of the service
o Aligns economic incentives of customer and provider
o Motivates both sides to optimize for efficiency
o (+) Pre- / post-allocation at Google

28. Architecture
Evolution
• The Monolith
• Ecosystem of Services
• Building and Operating a Service
• Service Anti-Patterns

29. Service
Anti-Patterns
• The “Mega-Service”
o Overbroad area of responsibility is difficult to reason about, change
o Leads to more upstream / downstream dependencies
• Shared persistence
o Breaks encapsulation, encourages “backdoor” interface violations
o Unhealthy and near-invisible coupling of services
o (-) Initial eBay SOA efforts

30. Service
Anti-Patterns
• “Leaky Abstraction” Service
o Interface reflects provider’s model of the interaction, not the consumer’s
model
o Consumer’s model is more aligned with the domain, simpler, more
abstract
o Leaking provider’s model in the interface constrains evolution of the
implementation

31. Thank You!
• @randyshoup
• linkedin.com/in/randyshoup
• Slides will be at slideshare.net/randyshoup