Sildes of an internal talk given at Twitter similar to a previous webinar for Redhat with the same title. Speeding up development is a key concern, cloud and technology improvements like Docker speed up key steps that make continuous delivery possible. Breaking up the work into many separate microservices and datastores with stable APIs allows teams to make progress independently so that the organization scales. Monolithic apps are preferred for small projects, built by small teams and when very low latency and high efficiency is the primary requirement. Monitoring microservices is currently a challenge with solutions starting to emerge.

1. Microservices
The Good, Bad and Ugly
Adrian Cockcroft @adrianco
Technology Fellow - Battery Ventures
June 2015

2. When microservices are good
and when you don’t need them
What new problems they bring

3. What does @adrianco do?
@adrianco
Technology Due
Diligence on Deals
Presentations at
Conferences
Presentations at
Companies
Technical Advice
for Portfolio
Companies
Program
Committee for
Conferences
Networking with
Interesting PeopleTinkering with
Technologies
Maintain
Relationship with
Cloud Vendors

5. Key Goal of the CIO?

6. Key Goal of the CIO?
Align IT with the business

7. Key Goal of the CIO?
Align IT with the business
Note: Netflix doesn’t have a CIO…

8. Key Goal of the CIO?
Align IT with the business
Note: Netflix doesn’t have a CIO…
Product related IT is embedded in the business - totally aligned

9. Value Chain Mapping
Simon Wardley http://blog.gardeviance.org/2014/11/how-to-get-to-strategy-in-ten-steps.html
Related tools and training http://www.wardleymaps.com/

10. Value Chain Mapping
Simon Wardley http://blog.gardeviance.org/2014/11/how-to-get-to-strategy-in-ten-steps.html
Related tools and training http://www.wardleymaps.com/
Your unique product - Agile

11. Value Chain Mapping
Simon Wardley http://blog.gardeviance.org/2014/11/how-to-get-to-strategy-in-ten-steps.html
Related tools and training http://www.wardleymaps.com/
Your unique product - Agile Best of breed as a Service - Lean

12. Value Chain Mapping
Simon Wardley http://blog.gardeviance.org/2014/11/how-to-get-to-strategy-in-ten-steps.html
Related tools and training http://www.wardleymaps.com/
Your unique product - Agile
Undifferentiated
utility suppliers - 6sigma
Best of breed as a Service - Lean

13. Product
Development
Processes

14. Assumption:
Process prevents
problems

15. Organizations build up
slow complex “Scar
tissue” processes

16. Observe
Orient
Decide
Act Continuous
Delivery

17. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Measure
Customers
Continuous
Delivery

18. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
INNOVATION
Measure
Customers
Continuous
Delivery

19. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
Model
Hypotheses
INNOVATION
Measure
Customers
Continuous
Delivery

20. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
Model
Hypotheses
BIG DATA
INNOVATION
Measure
Customers
Continuous
Delivery

21. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
JFDI
Plan Response
Share Plans
Model
Hypotheses
BIG DATA
INNOVATION
Measure
Customers
Continuous
Delivery

22. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
JFDI
Plan Response
Share Plans
Model
Hypotheses
BIG DATA
INNOVATION
CULTURE
Measure
Customers
Continuous
Delivery

23. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
JFDI
Plan Response
Share Plans
Incremental
Features
Automatic
Deploy
Launch AB
Test
Model
Hypotheses
BIG DATA
INNOVATION
CULTURE
Measure
Customers
Continuous
Delivery

24. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
JFDI
Plan Response
Share Plans
Incremental
Features
Automatic
Deploy
Launch AB
Test
Model
Hypotheses
BIG DATA
INNOVATION
CULTURE
CLOUD
Measure
Customers
Continuous
Delivery

25. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
JFDI
Plan Response
Share Plans
Incremental
Features
Automatic
Deploy
Launch AB
Test
Model
Hypotheses
BIG DATA
INNOVATION
CULTURE
CLOUD
Measure
Customers
Continuous
Delivery

26. Observe
Orient
Decide
Act
Land grab
opportunity Competitive
Move
Customer Pain
Point
Analysis
JFDI
Plan Response
Share Plans
Incremental
Features
Automatic
Deploy
Launch AB
Test
Model
Hypotheses
BIG DATA
INNOVATION
CULTURE
CLOUD
Measure
Customers
Continuous
Delivery

27. Breaking Down the SILOs

28. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr

29. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Monolithic Delivery
Product Team Using Monolithic Delivery

30. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Product Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery

31. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Platform TeamProduct Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery

32. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Platform Team
A
P
I
Product Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery

33. Breaking Down the SILOs
QA DBA
Sys
Adm
Net
Adm
SAN
Adm
DevUX
Prod
Mgr
Product Team Using Microservices
Product Team Using Monolithic Delivery
Platform Team
DevOps is a Re-Org!
A
P
I
Product Team Using Microservices
Product Team Using Microservices
Product Team Using Monolithic Delivery

34. Release Plan
Developer
Developer
Developer
Developer
Developer
QA Release
Integration
Ops Replace Old
With New
Release
Monolithic service updates
Works well with a small number
of developers and a single
language like php, java or ruby

35. Release Plan
Developer
Developer
Developer
Developer
Developer
QA Release
Integration
Ops Replace Old
With New
Release
Bugs
Monolithic service updates
Works well with a small number
of developers and a single
language like php, java or ruby

36. Release Plan
Developer
Developer
Developer
Developer
Developer
QA Release
Integration
Ops Replace Old
With New
Release
Bugs
Bugs
Monolithic service updates
Works well with a small number
of developers and a single
language like php, java or ruby

37. Use monolithic apps for small teams,
simple systems and when you must,
to optimize for efficiency and latency

38. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components

39. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components

40. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Bugs
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components

41. Developer
Developer
Developer
Developer
Developer
Old Release Still
Running
Release Plan
Release Plan
Release Plan
Release Plan
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Bugs
Deploy
Feature to
Production
Immutable microservice deployment
scales, is faster with large teams and
diverse platform components

42. Configure
Configure
Developer
Developer
Developer
Release Plan
Release Plan
Release Plan
Deploy
Standardized
Services
Standardized portable container
deployment saves time and effort
https://hub.docker.com

43. Configure
Configure
Developer
Developer
Developer
Release Plan
Release Plan
Release Plan
Deploy
Standardized
Services
Deploy
Feature to
Production
Deploy
Feature to
Production
Deploy
Feature to
Production
Bugs
Deploy
Feature to
Production
Standardized portable container
deployment saves time and effort
https://hub.docker.com

44. Developer Developer
Run What You Wrote
Developer Developer

45. Developer Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer

46. Developer Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Monitoring
Tools

47. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Monitoring
Tools

48. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Site
Reliability
Monitoring
Tools
Availability
Metrics
99.95% customer
success rate

49. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Manager Manager
Site
Reliability
Monitoring
Tools
Availability
Metrics
99.95% customer
success rate

50. DeveloperDeveloper Developer
Run What You Wrote
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Micro
service
Developer Developer
Manager Manager
VP
Engineering
Site
Reliability
Monitoring
Tools
Availability
Metrics
99.95% customer
success rate

51. Change One Thing
at a Time!

52. What Happened?
Rate of change
increased
Cost and size and
risk of change
reduced

53. Low Cost of Change Using Docker
Developers
• Compile/Build
• Seconds
Extend container
• Package dependencies
• Seconds
PaaS deploy Container
• Docker startup
• Seconds

54. Low Cost of Change Using Docker
Fast tooling supports continuous delivery of many tiny changes
Developers
• Compile/Build
• Seconds
Extend container
• Package dependencies
• Seconds
PaaS deploy Container
• Docker startup
• Seconds

55. Disruptor:
Continuous Delivery with
Containerized Microservices

56. Microservices

57. A Microservice Definition
Loosely coupled service oriented
architecture with bounded contexts

58. A Microservice Definition
Loosely coupled service oriented
architecture with bounded contexts
If every service has to be
updated at the same time
it’s not loosely coupled

59. A Microservice Definition
Loosely coupled service oriented
architecture with bounded contexts
If every service has to be
updated at the same time
it’s not loosely coupled
If you have to know too much about surrounding
services you don’t have a bounded context. See the
Domain Driven Design book by Eric Evans.

60. Coupling Concerns
http://en.wikipedia.org/wiki/Conway's_law
●Conway’s Law - organizational coupling
●Centralized Database Schemas
●Enterprise Service Bus - centralized message queues
●Inflexible Protocol Versioning

61. Non-Destructive Production Updates
● “Immutable Code” Service Pattern
● Existing services are unchanged, old code remains in service
● New code deploys as a new service group
● No impact to production until traffic routing changes
● A|B Tests, Feature Flags and Version Routing control traffic
● First users in the test cell are the developer and test engineers
● A cohort of users is added looking for measurable improvement
● Finally make default for everyone, keeping old code for a while

62. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years

63. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks

64. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Container Deployments
• Deploy in seconds
• Live for minutes/hours

65. Speeding Up Deployments
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Container Deployments
• Deploy in seconds
• Live for minutes/hours
AWS Lambda Events
• Respond in milliseconds
• Live for seconds

66. Speeding Up Deployments
Measuring CPU usage once a minute makes no sense for containers…
Coping with rate of change is the first challenge for monitoring tools.
Datacenter Snowflakes
• Deploy in months
• Live for years
Virtualized and Cloud
• Deploy in minutes
• Live for weeks
Container Deployments
• Deploy in seconds
• Live for minutes/hours
AWS Lambda Events
• Respond in milliseconds
• Live for seconds

67. Inspiration

68. http://www.infoq.com/presentations/scale-gilt
http://www.slideshare.net/mcculloughsean/itier-breaking-up-the-monolith-philly-ete
http://www.infoq.com/presentations/Twitter-Timeline-Scalability
http://www.infoq.com/presentations/twitter-soa
http://www.infoq.com/presentations/Zipkin
https://speakerdeck.com/mattheath/scaling-micro-services-in-go-highload-plus-plus-2014
State of the Art in Web Scale
Microservice Architectures
AWS Re:Invent : Asgard to Zuul https://www.youtube.com/watch?v=p7ysHhs5hl0
Resiliency at Massive Scale https://www.youtube.com/watch?v=ZfYJHtVL1_w
Microservice Architecture https://www.youtube.com/watch?v=CriDUYtfrjs

69. Microservice Concerns
ConfigurationTooling Discovery Routing Observability
Development: Languages and Container
Operational: Orchestration and Deployment Infrastructure
Datastores

70. Microservices
Edda
Archaius
Configuration
Asgard
Aminator
Tooling
Eureka
Prana
Discovery
Denominator
Zuul, Netty
Ribbon 2.0
Routing
Hystrix
Pytheus
SALP
Observability
Java, Groovy, Scala, Clojure, Python, Node.js with AMI and Docker Containers
Manual Orchestration with Asgard and deployment on AWS or Eucalyptus
Ephemeral datastores using Dynomite, Memcached, Astyanax, Staash, Priam, Cassandra

71. Microservices
Edda
Archaius
Configuration
Asgard
Aminator
Tooling
Eureka
Prana
Discovery
Denominator
Zuul, Netty
Ribbon 2.0
Routing
Hystrix
Pytheus
SALP
Observability
Java, Groovy, Scala, Clojure, Python, Node.js with AMI and Docker Containers
Manual Orchestration with Asgard and deployment on AWS or Eucalyptus
Ephemeral datastores using Dynomite, Memcached, Astyanax, Staash, Priam, Cassandra
Focus on global distribution, high scale and availability

73. NetflixOSS Components
• See
“Zero
to
Docker”
to
try
out
Ne1lixOSS
in
seconds…
• Security
and
access
management
setup
-­‐
Security
Monkey
• Account
Management:
Asgard
to
deploy
&
Ice
for
cost
monitoring
• Build
Tools:
Aminator
to
automate
baking
AMIs
• Service
Registry
and
Searchable
Account
History:
Eureka
&
Edda
• ConfiguraOon
Management:
Archaius
dynamic
property
system
• Data
storage:
Cassandra,
Astyanax,
Priam,
EVCache,
SURO
logging
to
KaUa/ElasOc
• Dynamic
traffic
rouOng:
Denominator,
Zuul,
Ribbon,
Karyon
• Availability:
Simian
Army
(Chaos
Monkey),
Hystrix,
Turbine,
Atlas
monitoring
• Developer
producOvity:
Blitz4J,
GCViz,
Pytheas
dashboards,
RxJava
• Big
Data:
Genie
for
Hadoop
PaaS,
LipsOck
visualizer
for
Pig,
Pigpen
for
Clojure
• Apps
to
get
started:
SpringCloud,
RSS
Reader,
ACME
Air,
FluxCapacitor

75. Twitter Microservices
Decider
ConfigurationTooling
Finagle
Zookeeper
Discovery
Finagle
Netty
Routing
Zipkin
Observability
Scala with JVM Container
Orchestration using Aurora deployment in datacenters using Mesos
Custom Cassandra-like datastore: Manhattan

76. Twitter Microservices
Decider
ConfigurationTooling
Finagle
Zookeeper
Discovery
Finagle
Netty
Routing
Zipkin
Observability
Scala with JVM Container
Orchestration using Aurora deployment in datacenters using Mesos
Custom Cassandra-like datastore: Manhattan
Focus on efficient datacenter deployment at scale

79. Challenges for
Microservices

80. Managing Scale

81. A Possible Hierarchy
Continents
Regions
Zones
Services
Versions
Containers
Instances
How Many?
3 to 5
2-4 per Continent
1-5 per Region
100’s per Zone
Many per Service
1000’s per Version
10,000’s
It’s much more challenging
than just a large number of
machines

82. Flow

83. Some tools can show
the request flow
across a few services

84. But interesting
architectures have a
lot of microservices!
Flow visualization is
a big challenge.
See http://www.slideshare.net/LappleApple/gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture

85. Failures

86. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones

87. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones

88. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones
Zone partition/failure
What should you do?
What should monitors show?

89. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones
Zone partition/failure
What should you do?
What should monitors show?
By design, everything works
with 2 of 3 zones running.
This is not an outage, inform
but don’t touch anything!
Halt deployments perhaps?

90. ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash Data
Access Layer
Priam Cassandra
Datastore
Simple NetflixOSS
style microservices
architecture on three
AWS Availability Zones
Zone partition/failure
What should you do?
What should monitors show?
By design, everything works
with 2 of 3 zones running.
This is not an outage, inform
but don’t touch anything!
Halt deployments perhaps?
Challenge: understand and
communicate common
microservice failure patterns.

91. Testing

92. Testing monitoring tools at scale
gets expensive quickly…

93. Simulation

94. Simulated Microservices
Model and visualize microservices
Simulate interesting architectures
Generate large scale configurations
Eventually stress test real tools
See github.com/adrianco/spigo
Simulate Protocol Interactions in Go
Visualize with D3
ELB Load Balancer
Zuul API Proxy
Karyon
Business
Logic
Staash
Data
Access
Layer
Priam Cassandra
Datastore
Three
Availability
Zones

95. Definition of an architecture
{
"arch": "netflixoss",
"description":"A very simple Netflix service. See http://netflix.github.io/ to decode the package names",
"version": "arch-0.0",
"victim": "homepage",
"services": [
{ "name": "cassSubscriber", "package": "priamCassandra", "count": 6, "regions": 1, "dependencies": ["cassSubscriber", "eureka"]},
{ "name": "evcacheSubscriber","package": "store", "count": 3, "regions": 1, "dependencies": []},
{ "name": "subscriber", "package": "staash", "count": 6, "regions": 1, "dependencies": ["cassSubscriber", "evcacheSubscriber"]}
{ "name": "login", "package": "karyon", "count": 18, "regions": 1, "dependencies": ["subscriber"]},
{ "name": "homepage", "package": "karyon", "count": 24, "regions": 1, "dependencies": ["subscriber"]},
{ "name": "wwwproxy", "package": "zuul", "count": 6, "regions": 1, "dependencies": ["login", "homepage"]},
{ "name": "www-elb", "package": "elb", "count": 0, "regions": 1, "dependencies": ["wwwproxy"]},
{ "name": "www", "package": "denominator", "count": 0, "regions": 0, "dependencies": ["www-elb"]}
]
}
Header includes
chaos monkey victim
New tier
name
Tier
package
Region
count: 1
Node
count
List of tier
dependencies

96. How does Spigo work?
Creates and animates microservices
Single Go program on this laptop
Simulates 100,000+ instances
Dynamic Go channels replace tcp/ip
“gotocol” messages simulate http etc.
About 250,000 messages/sec
Supports social network architecture
LAMP or NetflixOSS architectures

97. Spigo Nanoservice Structure
func Start(listener chan gotocol.Message) {
...
for {
select {
case msg := <-listener:
switch msg.Imposition {
case gotocol.Hello: // get named by parent
...
case gotocol.NameDrop: // someone new to talk to
...
case gotocol.GetRequest: // upstream request handler
...
case gotocol.GetResponse:// downstream response handler
...
case gotocol.Goodbye: // tell parent I’m going away now
gotocol.Message{gotocol.Goodbye, nil, time.Now(), name}.GoSend(parent)
return
}
case <-eurekaTicker.C: // poll the service registry
...
}
}
}

98. Why Build Spigo?
Generate test microservice configurations at scale
Stress monitoring tools display capabilities
Eventually (i.e. not implemented yet)
Dynamically vary configuration: autoscale, code push
Chaos monkey for microservice, zone, region failures
D3 websocket dynamic browser interface

99. Conference Driven Development…
OSCON Microservices Workshop

100. What’s Next?

101. Developer Concerns
Agile, Lean & Rugged
(Faster, Cheaper and Safer)
See GOTO London Sept 2015

102. Forward Thinking

103. Forward Thinking

104. Any Questions?
Disclosure: some of the companies mentioned may be Battery Ventures Portfolio Companies
See www.battery.com for a list of portfolio investments
● Battery Ventures http://www.battery.com
● Adrian’s Tweets @adrianco and Blog http://perfcap.blogspot.com
● Slideshare http://slideshare.com/adriancockcroft
● Monitorama Opening Keynote Portland OR - May 7
th
, 2014
● GOTO Chicago Opening Keynote May 20
th
, 2014
● Qcon New York – Speed and Scale - June 11
th
, 2014
● Structure - Cloud Trends - San Francisco - June 19th, 2014
● GOTO Copenhagen/Aarhus – Fast Delivery - Denmark – Sept 25
th
, 2014
● DevOps Enterprise Summit - San Francisco - Oct 21-23rd, 2014 #DOES14
● GOTO Berlin - Migrating to Microservices - Germany - Nov 6th, 2014
● AWS Re:Invent - Cloud Native Cost Optimization - Las Vegas - November 14th, 2014
● O’Reilly Software Architecture Conference - Fast Delivery, Monitoring Challenge - Boston March 16th 2015

105. Security
Visit http://www.battery.com/our-companies/ for a full list of all portfolio companies in which all Battery Funds have invested.
Palo Alto Networks
Enterprise IT
Operations &
Management
Big DataCompute
Networking
Storage