Reactive Microservices Roadshow | 28.09.2016 | Christian Deger
Highway to heaven
Building microservices in the cloud
Christian Deger
Chief Architect
cdeger@autoscout24.com
@cdeger
2,4 Million Vehicles
Microservices in the cloud adoption?
2000 Servers
2 Data Centers
MTBF optimized
Dev and Ops Silos
Development
“Change”
Operations
“Stability”
New
CEO
Talent?
Do you attract
21st Century
What does a
tech company
look like?
Great Design
Universally Connected
Mobile First
Instant Business Value
Massive Data Insight
Highly Available
good, but not great
Hmm, we are
Reboot
everything
Project
Tatsu
.NET / Windows to JVM / Linux
Monolith to Microservices
Data center to AWS
Devs + Ops to Collaboration culture
Involve product people
Major JVM Languages
No traction in major
internet companies
Major JVM Languages
Major JVM Languages
No traction in major
internet companies
Not accepted by C#
developers
Major JVM Languages
No traction in major
internet companies
Not accepted by C#
developers
Attracts talent
Is a starting point
Why Microservices?
Speed
Independent deployable
Fast local decisionsAutonomous teams
Strong boundaries
Loosely coupled Technology diversity
Scale the organization
same
direction
STRATEGIC
GOALS
Goals of the business side
ARCHITECTURAL
PRINCIPLES
High-Level Principles
DESIGN AND DELIVERY
PRINCIPLES
Tactical measures
REDUCE TIME TO MARKET
Establish fast feedback loops to learn, validate and
improve. Remove friction, hand-offs and undifferentiated
work.
MOBILE FIRST
Start small and use device capabilities.
SUPPORT DATA-DRIVEN DECISIONS
Provide relevant metrics and data for user and market
insights. Validate hypothesis for problems worth solving.
YOU BUILT IT, YOU RUN IT
The team is responsible for shaping, building, running and
maintaining its products. Fast feedback from live and
customers helps us to continuously improve.
ORGANIZED AROUND BUSINESS CAPABILITIES
Build teams around products not projects. Follow the
domain and respect bounded contexts. Make boundaries
explicit. Inverse Conway Maneuver.
LOOSELY COUPLED
By default avoid sharing and tight coupling.
No integration database. Don’t create the next monolith.
MACRO AND MICRO ARCHITECTURE
Clear separation. Autonomous micro services within the
rules and constraints of the macro architecture.
AWS FIRST
Favor AWS platform service over managed service,
over self-hosted OSS, over self built solutions.
DATA-DRIVEN / METRIC-DRIVEN
Collect business and operational metrics. Analyze, alert
and act on them.
ELIMINATE ACCIDENTAL COMPLEXITY
Strive to keep it simple. Don’t over-engineer.
Focus on necessary domain complexity.
AUTONOMOUS TEAMS
Make fast local decisions. Be responsible. Know your
boundaries. Share findings.
INFRASTRUCTURE AS CODE
Automate everything: Reproducible, traceable, auditable
and tested. Immutable servers.
CROSS-FUNCTIONAL TEAMS
Engineers from all backgrounds work together in
collaborative teams as engineers and share
responsibilities. No silos.
BE BOLD
Go into production early. Value monitoring over tests.
Fail fast, recover and learn. Optimize for MTTR not MTBF.
SECURITY, COMPLIANCE AND DATA PRIVACY
Build with least privilege and data privacy in mind.
Know your threat model. Limit blast radius.
COST EFFICIENCY
Run your segment in the right balance of cost and value.
ONE SCOUT IT
Foster collaboration. Harmonize and standardize tools.
Pull common capabilities into decoupled platform services.
Version 2.0
Icons made by Freepik from www.flaticon.com are licensed under CC BY 3.0
BEST TALENT
Autonomy, Purpose and Mastery: We know why we do
things, we decide how to approach them and deliberately
practice our skills.
Build
MeasureLearn
Autonomous teams
business capabilities
organized around
collaboration culture
You build it,
you run it.
Monitoring is the new testing
How (not) to share
shared nothing as default
loosely coupled
fast local decisions
voluntary adoption
exception: macro concerns
Follow the
trail
Continuous
Delivery
Application code in
one repository per
service.
CI
Deployment package
as artifact.
CD
Deliver package to
servers
Delivery Pipeline – Data Center
Application code and
infrastructure
specification in one
repository per
service.
CI
Deployment package
and infrastructure
declaration as artifact.
CD
1. Create or update
service infrastructure.
2. New instances pull
down package and
start application.
Delivery Pipeline – AWS
Cattle,
not pets
Separate
code deployment
feature release
from
No
staging
environment
SQS + S3
Kinesis + S3
Kinesis + DynamoDB
SQS + DynamoDB
Proxy + DynamoDB
DynamoDB
Evolution
Unlimited Infrastructure with APIs
Migration strategy
PageSpeed Module
css (page+fragment)
js (page+fragment)
ngx_pagespeed
css (page)
js (page)
css (fragment)
js (fragment)
Event Streaming
Event Sourcing
one way data highway
and data pumps
Commit to Production
20 Minutes Cycle Time
New Service
1 Day Service Bootstrapping
3 Days Frontend
4 Days Backend
015 Teams
025 Lambda Functions
200 Repositories
040 Microservices
009 Systems
Status Quo
Picture Credits
Tatsu Sign by Martin Lewison from The Hague, Zuid-Holland, The Netherlands under CC BY-SA 2.0
Martin Fowler by Webysther Nunes under CC BY-SA 4.0
Werner Vogels by Guido van Nispen under CC BY 2.0
"HotWheels - '69 Ford Torino Talladega“ by Leap Kye, licensed under CC BY-ND 2.0
Differences between Traditional vs Next Generation by Simon Wardley under CC BY-SA 3.0
Enterprise IT Adoption Cycle by Simon Wardley under CC BY-SA 3.0
And the future is private by Simon Wardley under CC BY-SA 3.0
Leosvel et Diosmani by Ludovic Péron under CC BY-SA 3.0
Spare wheel by Brian Snelson under CC BY 2.0
Wandergeselle by Sigismund von Dobschütz under CC BY-SA 3.0
Wheel clamps Texas by Richard Anderson from Denton, United States (Boots.) under CC BY-SA 2.0
Sharing Sucks (4536747557) by eyeliam from Portland, United States under CC BY 2.0
Traffic Jam by Doo Ho Kim under CC BY-SA 2.0
Puzzling by Bernd Gessler (Own work) CC BY-SA 3.0
Amazon16 by Neil Palmer/CIAT under CC BY-SA 2.0
Pizza by Jakob Dettner, Rainer Zenz under CC BY-SA 2.0 de
Bezos’ Iconic Laugh by Steve Jurvetson under CC BY 2.0

Reactive Microservices Roadshow Berlin

  • 1.
    Reactive Microservices Roadshow| 28.09.2016 | Christian Deger Highway to heaven Building microservices in the cloud
  • 2.
  • 3.
  • 4.
    Microservices in thecloud adoption?
  • 5.
    2000 Servers 2 DataCenters MTBF optimized
  • 6.
    Dev and OpsSilos Development “Change” Operations “Stability”
  • 8.
  • 9.
  • 11.
    21st Century What doesa tech company look like?
  • 13.
    Great Design Universally Connected MobileFirst Instant Business Value Massive Data Insight Highly Available
  • 14.
    good, but notgreat Hmm, we are
  • 15.
  • 16.
  • 17.
    .NET / Windowsto JVM / Linux Monolith to Microservices Data center to AWS Devs + Ops to Collaboration culture Involve product people
  • 18.
  • 19.
    No traction inmajor internet companies Major JVM Languages
  • 20.
    Major JVM Languages Notraction in major internet companies Not accepted by C# developers
  • 21.
    Major JVM Languages Notraction in major internet companies Not accepted by C# developers Attracts talent Is a starting point
  • 22.
    Why Microservices? Speed Independent deployable Fastlocal decisionsAutonomous teams Strong boundaries Loosely coupled Technology diversity Scale the organization
  • 23.
  • 24.
    STRATEGIC GOALS Goals of thebusiness side ARCHITECTURAL PRINCIPLES High-Level Principles DESIGN AND DELIVERY PRINCIPLES Tactical measures REDUCE TIME TO MARKET Establish fast feedback loops to learn, validate and improve. Remove friction, hand-offs and undifferentiated work. MOBILE FIRST Start small and use device capabilities. SUPPORT DATA-DRIVEN DECISIONS Provide relevant metrics and data for user and market insights. Validate hypothesis for problems worth solving. YOU BUILT IT, YOU RUN IT The team is responsible for shaping, building, running and maintaining its products. Fast feedback from live and customers helps us to continuously improve. ORGANIZED AROUND BUSINESS CAPABILITIES Build teams around products not projects. Follow the domain and respect bounded contexts. Make boundaries explicit. Inverse Conway Maneuver. LOOSELY COUPLED By default avoid sharing and tight coupling. No integration database. Don’t create the next monolith. MACRO AND MICRO ARCHITECTURE Clear separation. Autonomous micro services within the rules and constraints of the macro architecture. AWS FIRST Favor AWS platform service over managed service, over self-hosted OSS, over self built solutions. DATA-DRIVEN / METRIC-DRIVEN Collect business and operational metrics. Analyze, alert and act on them. ELIMINATE ACCIDENTAL COMPLEXITY Strive to keep it simple. Don’t over-engineer. Focus on necessary domain complexity. AUTONOMOUS TEAMS Make fast local decisions. Be responsible. Know your boundaries. Share findings. INFRASTRUCTURE AS CODE Automate everything: Reproducible, traceable, auditable and tested. Immutable servers. CROSS-FUNCTIONAL TEAMS Engineers from all backgrounds work together in collaborative teams as engineers and share responsibilities. No silos. BE BOLD Go into production early. Value monitoring over tests. Fail fast, recover and learn. Optimize for MTTR not MTBF. SECURITY, COMPLIANCE AND DATA PRIVACY Build with least privilege and data privacy in mind. Know your threat model. Limit blast radius. COST EFFICIENCY Run your segment in the right balance of cost and value. ONE SCOUT IT Foster collaboration. Harmonize and standardize tools. Pull common capabilities into decoupled platform services. Version 2.0 Icons made by Freepik from www.flaticon.com are licensed under CC BY 3.0 BEST TALENT Autonomy, Purpose and Mastery: We know why we do things, we decide how to approach them and deliberately practice our skills.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
    Monitoring is thenew testing
  • 30.
    How (not) toshare shared nothing as default loosely coupled fast local decisions voluntary adoption exception: macro concerns
  • 31.
  • 32.
  • 33.
    Application code in onerepository per service. CI Deployment package as artifact. CD Deliver package to servers Delivery Pipeline – Data Center
  • 34.
    Application code and infrastructure specificationin one repository per service. CI Deployment package and infrastructure declaration as artifact. CD 1. Create or update service infrastructure. 2. New instances pull down package and start application. Delivery Pipeline – AWS
  • 36.
  • 37.
  • 38.
  • 39.
    SQS + S3 Kinesis+ S3 Kinesis + DynamoDB SQS + DynamoDB Proxy + DynamoDB DynamoDB Evolution
  • 40.
  • 41.
  • 43.
    PageSpeed Module css (page+fragment) js(page+fragment) ngx_pagespeed css (page) js (page) css (fragment) js (fragment)
  • 44.
  • 45.
    Event Sourcing one waydata highway and data pumps
  • 46.
    Commit to Production 20Minutes Cycle Time New Service 1 Day Service Bootstrapping 3 Days Frontend 4 Days Backend
  • 47.
    015 Teams 025 LambdaFunctions 200 Repositories 040 Microservices 009 Systems Status Quo
  • 49.
    Picture Credits Tatsu Signby Martin Lewison from The Hague, Zuid-Holland, The Netherlands under CC BY-SA 2.0 Martin Fowler by Webysther Nunes under CC BY-SA 4.0 Werner Vogels by Guido van Nispen under CC BY 2.0 "HotWheels - '69 Ford Torino Talladega“ by Leap Kye, licensed under CC BY-ND 2.0 Differences between Traditional vs Next Generation by Simon Wardley under CC BY-SA 3.0 Enterprise IT Adoption Cycle by Simon Wardley under CC BY-SA 3.0 And the future is private by Simon Wardley under CC BY-SA 3.0 Leosvel et Diosmani by Ludovic Péron under CC BY-SA 3.0 Spare wheel by Brian Snelson under CC BY 2.0 Wandergeselle by Sigismund von Dobschütz under CC BY-SA 3.0 Wheel clamps Texas by Richard Anderson from Denton, United States (Boots.) under CC BY-SA 2.0 Sharing Sucks (4536747557) by eyeliam from Portland, United States under CC BY 2.0 Traffic Jam by Doo Ho Kim under CC BY-SA 2.0 Puzzling by Bernd Gessler (Own work) CC BY-SA 3.0 Amazon16 by Neil Palmer/CIAT under CC BY-SA 2.0 Pizza by Jakob Dettner, Rainer Zenz under CC BY-SA 2.0 de Bezos’ Iconic Laugh by Steve Jurvetson under CC BY 2.0