The Seven (More) DEADLY SINS OF Microservices
@opencredo
Daniel Bryant
@danielbryantuk
Previously, AT Devoxx UK & QCON NYC 2015...
30/01/2017 @danielbryantuk
https://www.infoq.com/presentations/7-sins-microservices
The Seven (more) Deadly Sins of Microservices
1. LUST - Using the (Unevaluated) latest and greatest tech…
2. GLUTTONY - Communication lock-in
3. GREED - What'S Mine is mine (within the organisation)…
4. SLOTH - Getting lazy with NFRs
5. WRATH - Blowing up when bad things happen
6. ENVY - The shared single domain (and data store) fallacy
7. PRIDE - testing in the world of transience
30/01/2017 @danielbryantuk
@danielbryantuk
• Chief Scientist at OpenCredo, CTO at SpectoLabs
• Agile, architecture, CI/CD, Programmable infrastructure
• Java, Go, JS, microservices, cloud, containers
• Continuous delivery of value through effective technology and teams
31/01/2017 @danielbryantuk
bit.ly/2jWDSF7
1. Lust - Using THE LATEST and Greatest Tech…
30/01/2017 @danielbryantuk
New technology is great... Until it isn'T
30/01/2017 @danielbryantuk
developers with new tech be like
F*cking new technology...
Credit to Michael Hausenblas
This	has	been	us	
many	times!
Evaluation is a key skill...
30/01/2017 @danielbryantuk
Evaluation - are Microservices A good fit?
• “our 'mode TWO' apps are Microservices”
– Middle-management latch on to Buzzword
– New app evolution limited by existing system
– Lipstick on the pig
• Not understanding architecture principles
– Not building around business Functionality
– Creating Mini-monoliths (no twelve factors)
• No Well-defined DevOps / SRE / Ops
– Deployment/ops free-for-all
30/01/2017 @danielbryantuk
Evaluation of tech - The’Spine Model
• Effective conversations make for effective
collaboration
• It's a TOOL Problem
– As a species, we have always been Tool users
and makers.
– We use _____ to get our work done
• People get stuck in a dilemma where equally
plausible options are available
• “Going up the Spine” breaks deadlock http://spinemodel.info/explanation/introduction/
Evaluation of tech - Fitness functions
• Great for evaluation and documentation
– Platforms / Language
– Middleware
– Data stores
• Microservices as an Evolutionary Architecture
– Neal Ford and Rebecca Parsons
30/01/2017 @danielbryantuk
AN example: To containerise, or not to containerise?
(dockaH, dockah, dockah... Dockah?)
30/01/2017 @danielbryantuk
Strategy #fail
30/01/2017 @danielbryantuk
Architecture/ops: Expectations versus reality
30/01/2017 @danielbryantuk
“DevOps”
Choices: Beware of Confirmation bias
30/01/2017 @danielbryantuk
https://thehftguy.wordpress.com/2016/11/01/docker-in-production-an-history-of-failure/ http://patrobinson.github.io/2016/11/05/docker-in-production/
Evaluation - It'S easy to be tricked
30/01/2017 @danielbryantuk
Evaluation - beware of bias and heuristics
30/01/2017 @danielbryantuk
2. GLUTTONY - Communication lock-in
30/01/2017 @danielbryantuk
Rpc - not the devil in disguise
• Don'T rule out RPC (e.g. grpc)
– Sometimes the contract (and speed) are beneficial
– Human readability of JSON can be over-rated
• Sometime events are better
– Asynchronous (AP vs CP)
– durable message queues are interesting (But, No one size fits all)
30/01/2017 @danielbryantuk
The ESB is dead - long live the esb!
30/01/2017 @danielbryantuk
The ESB is dead - long live the esb!
30/01/2017 @danielbryantuk
The ESB is dead - long live the esb!
30/01/2017 @danielbryantuk
• Is this an ESB?
• Or an API gateway?
The ESB is dead - long live the API Gateway!
30/01/2017 @danielbryantuk
• Watch for the API Gateway morphing
into an Enterprise service bus
– Loose coupling is vital
• But let me be clear...
– The API Gateway pattern is awesome
– Centralise cross-cutting concerns
– Prevent wheel-reinvention (plugins)
– Check out kong, apigee, Mulesoft etc
3. GREED - What'S mine is mine... (within the organisation)…
30/01/2017 @danielbryantuk
Previously...
• Conway'S Law
• Microservices are about people, as much as they are tech
– Maybe more
– Particularly in a migration / transformation
30/01/2017 @danielbryantuk
We hear this a lot...
“We’ve decided to reform our teams around squads, chapters and guilds”
• Beware of cargo-culting
– Repeat three times “We are not spotify”
• Understand the practices, principles, values etc
30/01/2017 @danielbryantuk
4. SLOTH - Getting Lazy with NFRs
30/01/2017 @danielbryantuk
Getting lazy with non-Functional Requirements
“The driving technical requirements for a system should be identified early
to ensure they are properly handled in subsequent design”
Aidan Casey
Guiding principles for evolutionary architecture
30/01/2017 @danielbryantuk
Getting lazy with non-Functional Requirements
• The 'ilities' Can be (often) be an afterthought
– Availability, Scalability, auditability, testability etc
• Agile/Lean: Delay decisions to the ‘last responsible moment’
– NewsFlash - Sometimes this is up-front
• It can be costly (or prohibitive) to adapt late in the project
– Microservices don'T make this easier (sometimes more difficult)
30/01/2017 @danielbryantuk
Getting lazy with NFRs - security
30/01/2017 @danielbryantuk
www.slideshare.net/spnewman/appsec-microservices-velocity-2016 www.infoq.com/news/2016/08/secure-docker-microservices
Testing NFRs in the build pipeline
• Performance and Load testing
– Gatling / jmeter
– Flood.io
• Security testing
– Findsecbugs / OWASP Dependency check
– Bdd-security (OWASP ZAP) / Arachni
– Gauntlt / Serverspec
– Docker Bench for Security / Clair
30/01/2017 @danielbryantuk
5. WRATH - Blowing up when bad things happen
30/01/2017 @danielbryantuk
Previously - Bring in Michael Nygard (Or some monkeys)
30/01/2017 @danielbryantuk
When bad things happen, people are always involved
30/01/2017 @danielbryantuk	|	@oakinger
People Pain point - How does Devops fit into this?
• http://web.devopstopologies.com/
• @	matthewpskelton
• @beerops and @sigje
• Google SRE
30/01/2017 @danielbryantuk
Devops - the 'fullstack engineer' myth
“I'M sorry, but if you'RE not designing the computer chips and
writing the website, then I don'T wanna hear from you”
Charity Majors (@mipsytipsy), CraftConf 2016
http://www.ustream.tv/recorded/86181845
30/01/2017 @danielbryantuk
Devops - define responsibilities
• Do you really want to build an
entire microservices platform?
• Focus on what matters
– Ci/CD
– Mechanical sympathy
– Logging
– Monitoring
30/01/2017 @danielbryantuk
Worth considering: Open source PaaS
30/01/2017 @danielbryantuk
6. ENVY - The shared SINGLE domain (and Data Store) fallacy
30/01/2017 @danielbryantuk
Previously - One Model to Rule Them All...
• One model…
– Breaks encapsulation
– Introduces coupling
• Know your DDD
– Entities
– Value Objects
– Aggregates and Roots
30/01/2017 @danielbryantuk
Context mapping (static) & event storming (dynamic)
30/01/2017 @danielbryantuk	|	@spoole167 40
www.infoq.com/articles/ddd-contextmapping
ziobrando.blogspot.co.uk/2013/11/introducing-event-storming.html
Choose (and use) data stores appropriately
• RDBMS
– Valuable for structured data
• Cassandra is Awesome
– but don'T treat it like an RDBMS!
• Don'T build a graph with RDBMS
– Use neo4j, Titan etc
• Beware of operational overhead
30/01/2017 @danielbryantuk
7. PRIDE - testing in the world of transience
30/01/2017 @danielbryantuk
Previously...
• Local verification
– Consumer-Driven contracts
• End-to-end
– BDD-style critical path
• Remember the test pyramid
30/01/2017 @danielbryantuk
martinfowler.com/articles/microservice-testing/
Service virtualisation / API simulation
• Virtualise request/response of services
– Unavailable
– Expensive to run
– Fragile/brittle
– Non-deterministic
– Cannot simulate failures
https://dzone.com/articles/continuously-delivering-soa
30/01/2017 @danielbryantuk
Service virtualisation
• Classics
– CA service virtualization
– Parasoft virtualize
– HPE service virtualization
– IBM Test Virtualization server
• New (open source) kids on the block
– Hoverfly
– Wiremock
– VCR/Betamax
– Mountebank
– mirage
30/01/2017 @danielbryantuk
Hoverfly
• Lightweight Service virtualisation
– Open source (Apache 2.0)
– Go-based / single binary
– Written by @Spectolabs
• Flexible API simulation
– HTTP / HTTPS
– Highly performant
30/01/2017 @danielbryantuk
30/01/2017 @danielbryantuk
• Middleware
• Remove	PII
• Rate	limit
• Add	headers
• Middleware
• Fault	injection
• Chaos	monkey
Right, Let'S Wrap this up...
30/01/2017 @danielbryantuk
The Seven (more) Deadly Sins of Microservices
1. LUST - Using the (Unevaluated) latest and greatest tech…
2. GLUTTONY - Communication lock-in
3. GREED - What'S Mine is mine (within the organisation)…
4. SLOTH - Getting lazy with NFRs
5. WRATH - Blowing up when bad things happen
6. ENVY - The shared single domain (and data store) fallacy
7. PRIDE - testing in the world of transience
30/01/2017 @danielbryantuk
The Seven (more) Deadly Sins of Microservices
1. LUST - Using the (Unevaluated) latest and greatest tech…
2. GLUTTONY - Communication Lock-in
3. GREED - What'S Mine is mine (within the organisation)…
4. SLOTH - Getting lazy with NFRs
5. WRATH - Blowing up when bad things happen
6. ENVY - The shared single domain (and data store) fallacy
7. PRIDE - testing in the world of transience
30/01/2017 @danielbryantuk
Bedtime reading
30/01/2017 @danielbryantuk
THANKS...
www.opencredo.com / justcloudit.io
muservicesweekly.com
(Credit to Tareq Abedrabbo for inspiration/guidance)
01/02/2017 @danielbryantuk

OOP/MM 2017: "Seven (More) Deadly Sins of Microservices"

  • 1.
    The Seven (More)DEADLY SINS OF Microservices @opencredo Daniel Bryant @danielbryantuk
  • 2.
    Previously, AT DevoxxUK & QCON NYC 2015... 30/01/2017 @danielbryantuk https://www.infoq.com/presentations/7-sins-microservices
  • 3.
    The Seven (more)Deadly Sins of Microservices 1. LUST - Using the (Unevaluated) latest and greatest tech… 2. GLUTTONY - Communication lock-in 3. GREED - What'S Mine is mine (within the organisation)… 4. SLOTH - Getting lazy with NFRs 5. WRATH - Blowing up when bad things happen 6. ENVY - The shared single domain (and data store) fallacy 7. PRIDE - testing in the world of transience 30/01/2017 @danielbryantuk
  • 4.
    @danielbryantuk • Chief Scientistat OpenCredo, CTO at SpectoLabs • Agile, architecture, CI/CD, Programmable infrastructure • Java, Go, JS, microservices, cloud, containers • Continuous delivery of value through effective technology and teams 31/01/2017 @danielbryantuk bit.ly/2jWDSF7
  • 5.
    1. Lust -Using THE LATEST and Greatest Tech… 30/01/2017 @danielbryantuk
  • 6.
    New technology isgreat... Until it isn'T 30/01/2017 @danielbryantuk developers with new tech be like F*cking new technology... Credit to Michael Hausenblas This has been us many times!
  • 7.
    Evaluation is akey skill... 30/01/2017 @danielbryantuk
  • 8.
    Evaluation - areMicroservices A good fit? • “our 'mode TWO' apps are Microservices” – Middle-management latch on to Buzzword – New app evolution limited by existing system – Lipstick on the pig • Not understanding architecture principles – Not building around business Functionality – Creating Mini-monoliths (no twelve factors) • No Well-defined DevOps / SRE / Ops – Deployment/ops free-for-all 30/01/2017 @danielbryantuk
  • 9.
    Evaluation of tech- The’Spine Model • Effective conversations make for effective collaboration • It's a TOOL Problem – As a species, we have always been Tool users and makers. – We use _____ to get our work done • People get stuck in a dilemma where equally plausible options are available • “Going up the Spine” breaks deadlock http://spinemodel.info/explanation/introduction/
  • 10.
    Evaluation of tech- Fitness functions • Great for evaluation and documentation – Platforms / Language – Middleware – Data stores • Microservices as an Evolutionary Architecture – Neal Ford and Rebecca Parsons 30/01/2017 @danielbryantuk
  • 11.
    AN example: Tocontainerise, or not to containerise? (dockaH, dockah, dockah... Dockah?) 30/01/2017 @danielbryantuk
  • 12.
  • 13.
    Architecture/ops: Expectations versusreality 30/01/2017 @danielbryantuk “DevOps”
  • 14.
    Choices: Beware ofConfirmation bias 30/01/2017 @danielbryantuk https://thehftguy.wordpress.com/2016/11/01/docker-in-production-an-history-of-failure/ http://patrobinson.github.io/2016/11/05/docker-in-production/
  • 15.
    Evaluation - It'Seasy to be tricked 30/01/2017 @danielbryantuk
  • 16.
    Evaluation - bewareof bias and heuristics 30/01/2017 @danielbryantuk
  • 17.
    2. GLUTTONY -Communication lock-in 30/01/2017 @danielbryantuk
  • 18.
    Rpc - notthe devil in disguise • Don'T rule out RPC (e.g. grpc) – Sometimes the contract (and speed) are beneficial – Human readability of JSON can be over-rated • Sometime events are better – Asynchronous (AP vs CP) – durable message queues are interesting (But, No one size fits all) 30/01/2017 @danielbryantuk
  • 19.
    The ESB isdead - long live the esb! 30/01/2017 @danielbryantuk
  • 20.
    The ESB isdead - long live the esb! 30/01/2017 @danielbryantuk
  • 21.
    The ESB isdead - long live the esb! 30/01/2017 @danielbryantuk • Is this an ESB? • Or an API gateway?
  • 22.
    The ESB isdead - long live the API Gateway! 30/01/2017 @danielbryantuk • Watch for the API Gateway morphing into an Enterprise service bus – Loose coupling is vital • But let me be clear... – The API Gateway pattern is awesome – Centralise cross-cutting concerns – Prevent wheel-reinvention (plugins) – Check out kong, apigee, Mulesoft etc
  • 23.
    3. GREED -What'S mine is mine... (within the organisation)… 30/01/2017 @danielbryantuk
  • 24.
    Previously... • Conway'S Law •Microservices are about people, as much as they are tech – Maybe more – Particularly in a migration / transformation 30/01/2017 @danielbryantuk
  • 25.
    We hear thisa lot... “We’ve decided to reform our teams around squads, chapters and guilds” • Beware of cargo-culting – Repeat three times “We are not spotify” • Understand the practices, principles, values etc 30/01/2017 @danielbryantuk
  • 26.
    4. SLOTH -Getting Lazy with NFRs 30/01/2017 @danielbryantuk
  • 27.
    Getting lazy withnon-Functional Requirements “The driving technical requirements for a system should be identified early to ensure they are properly handled in subsequent design” Aidan Casey Guiding principles for evolutionary architecture 30/01/2017 @danielbryantuk
  • 28.
    Getting lazy withnon-Functional Requirements • The 'ilities' Can be (often) be an afterthought – Availability, Scalability, auditability, testability etc • Agile/Lean: Delay decisions to the ‘last responsible moment’ – NewsFlash - Sometimes this is up-front • It can be costly (or prohibitive) to adapt late in the project – Microservices don'T make this easier (sometimes more difficult) 30/01/2017 @danielbryantuk
  • 29.
    Getting lazy withNFRs - security 30/01/2017 @danielbryantuk www.slideshare.net/spnewman/appsec-microservices-velocity-2016 www.infoq.com/news/2016/08/secure-docker-microservices
  • 30.
    Testing NFRs inthe build pipeline • Performance and Load testing – Gatling / jmeter – Flood.io • Security testing – Findsecbugs / OWASP Dependency check – Bdd-security (OWASP ZAP) / Arachni – Gauntlt / Serverspec – Docker Bench for Security / Clair 30/01/2017 @danielbryantuk
  • 31.
    5. WRATH -Blowing up when bad things happen 30/01/2017 @danielbryantuk
  • 32.
    Previously - Bringin Michael Nygard (Or some monkeys) 30/01/2017 @danielbryantuk
  • 33.
    When bad thingshappen, people are always involved 30/01/2017 @danielbryantuk | @oakinger
  • 34.
    People Pain point- How does Devops fit into this? • http://web.devopstopologies.com/ • @ matthewpskelton • @beerops and @sigje • Google SRE 30/01/2017 @danielbryantuk
  • 35.
    Devops - the'fullstack engineer' myth “I'M sorry, but if you'RE not designing the computer chips and writing the website, then I don'T wanna hear from you” Charity Majors (@mipsytipsy), CraftConf 2016 http://www.ustream.tv/recorded/86181845 30/01/2017 @danielbryantuk
  • 36.
    Devops - defineresponsibilities • Do you really want to build an entire microservices platform? • Focus on what matters – Ci/CD – Mechanical sympathy – Logging – Monitoring 30/01/2017 @danielbryantuk
  • 37.
    Worth considering: Opensource PaaS 30/01/2017 @danielbryantuk
  • 38.
    6. ENVY -The shared SINGLE domain (and Data Store) fallacy 30/01/2017 @danielbryantuk
  • 39.
    Previously - OneModel to Rule Them All... • One model… – Breaks encapsulation – Introduces coupling • Know your DDD – Entities – Value Objects – Aggregates and Roots 30/01/2017 @danielbryantuk
  • 40.
    Context mapping (static)& event storming (dynamic) 30/01/2017 @danielbryantuk | @spoole167 40 www.infoq.com/articles/ddd-contextmapping ziobrando.blogspot.co.uk/2013/11/introducing-event-storming.html
  • 41.
    Choose (and use)data stores appropriately • RDBMS – Valuable for structured data • Cassandra is Awesome – but don'T treat it like an RDBMS! • Don'T build a graph with RDBMS – Use neo4j, Titan etc • Beware of operational overhead 30/01/2017 @danielbryantuk
  • 42.
    7. PRIDE -testing in the world of transience 30/01/2017 @danielbryantuk
  • 43.
    Previously... • Local verification –Consumer-Driven contracts • End-to-end – BDD-style critical path • Remember the test pyramid 30/01/2017 @danielbryantuk martinfowler.com/articles/microservice-testing/
  • 44.
    Service virtualisation /API simulation • Virtualise request/response of services – Unavailable – Expensive to run – Fragile/brittle – Non-deterministic – Cannot simulate failures https://dzone.com/articles/continuously-delivering-soa 30/01/2017 @danielbryantuk
  • 45.
    Service virtualisation • Classics –CA service virtualization – Parasoft virtualize – HPE service virtualization – IBM Test Virtualization server • New (open source) kids on the block – Hoverfly – Wiremock – VCR/Betamax – Mountebank – mirage 30/01/2017 @danielbryantuk
  • 46.
    Hoverfly • Lightweight Servicevirtualisation – Open source (Apache 2.0) – Go-based / single binary – Written by @Spectolabs • Flexible API simulation – HTTP / HTTPS – Highly performant 30/01/2017 @danielbryantuk
  • 47.
    30/01/2017 @danielbryantuk • Middleware •Remove PII • Rate limit • Add headers • Middleware • Fault injection • Chaos monkey
  • 48.
    Right, Let'S Wrapthis up... 30/01/2017 @danielbryantuk
  • 49.
    The Seven (more)Deadly Sins of Microservices 1. LUST - Using the (Unevaluated) latest and greatest tech… 2. GLUTTONY - Communication lock-in 3. GREED - What'S Mine is mine (within the organisation)… 4. SLOTH - Getting lazy with NFRs 5. WRATH - Blowing up when bad things happen 6. ENVY - The shared single domain (and data store) fallacy 7. PRIDE - testing in the world of transience 30/01/2017 @danielbryantuk
  • 50.
    The Seven (more)Deadly Sins of Microservices 1. LUST - Using the (Unevaluated) latest and greatest tech… 2. GLUTTONY - Communication Lock-in 3. GREED - What'S Mine is mine (within the organisation)… 4. SLOTH - Getting lazy with NFRs 5. WRATH - Blowing up when bad things happen 6. ENVY - The shared single domain (and data store) fallacy 7. PRIDE - testing in the world of transience 30/01/2017 @danielbryantuk
  • 51.
  • 52.
    THANKS... www.opencredo.com / justcloudit.io muservicesweekly.com (Creditto Tareq Abedrabbo for inspiration/guidance) 01/02/2017 @danielbryantuk