Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Radical Agility
with Autonomous Teams and Microservices in the Cloud
DevOps Con 2015 - Berlin - Germany
ARCHITECTURE
LEADERSHIP
ABOUT US
RADICAL AGILITY
INFRASTRUCTURE
AGENDA
Jan Löffler
● Head of Platform Engineering
● Twitter: @jlsoft2
● jan.loeffler@zalando.de
ABOUT US
Henning Jacobs
● STUPS Hacker
● Twitter: @try_except_
● henning.jacobs@zalando.de
ABOUT US
15 countries
3 fulfillment centers
15+ million active customers
2.2+ billion € revenue 2014
130+ million visits per month
...
ENVIRONMENT
DATACENTER ENVIRONMENT
DataCenter I
Gütersloh, Germany
DataCenter II
Berlin, Germany
DataCenter III
Berlin, Germany
Global...
THE LOST
HIGHWAY
CLOUD PROJECTS
2013/14 2014
Pequod
2013
Noah’s ARKzCloud
TOPIC 1
WHERE
TO GO
THIS NEEDS TO STOP
Doing it yourself is not the
most sensible thing.
Amazon invested already
thousands...
RADICAL
AGILITY
GOAL
DELIVER AMAZING
PRODUCTS
EFFICIENTLY AT
SCALE, AND
FEELING GREAT
ABOUT IT.
3 PRINCIPLES
PURPOSE
AUTONOMY
MASTERY
LEADERSHIP
FROM
CONTROL &
COMMAND
TO PURPOSE
AND TRUST
NEW LEADERSHIP
DELIVERY
LEAD
PEOPLE
LEAD
GLOBAL
REGRESSION
INNOVATION LAB
TECH
SERVICE
PRODUCT
OWNER
PRODUCT
SPECIALIST
DELIVERY
LEAD
PEOPLE
LEAD
BUSINESS
ASSURANC...
STEERING
ALIGNING
TARGETS VIA
OKR: WORK
TOWARDS
COMPANY
PURPOSE
ARCHITECTURE
AN
ARCHITECTURE
FOR
INNOVATION
API FIRST
REST
SAAS
MICRO
SERVICES
CLOUD
INFRASTRUCTURE
ROBUST
PLATFORM
SERVICES TO
ENABLE
AUTONOMY
STUPS
STUPS To Unleash Penguin Swarms
AWS
STUPS
DOCKER
DEPLOY
SSH
ACCESS
AUDIT
REPORTS
FULL AWS
ACCESS
A PLATFORM ON TOP OF AMAZON WEB SERVICES
AUTONOMY AND COMPLIANCE
STUPS offers
maximum freedom for developers
while enabling
near-real-time audit compliance
for eve...
One AWS account per Team
Deployment with Docker
Managed SSH Access
REST/OAuth 2.0 mandatory
Supports Traceability of Chang...
Public Internet
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”ELB ELB
EC2
Instance
EC2
InstanceEC2
InstanceEC2
...
DEPLOYMENT
IMMUTABLE STACKS
AWS
DEPLOYMENT WITH SENZA
Senza CLI
Pier One
docker pull
docker push
Taupage
SENZA: DEFINITION YAML
SENZA: BOOTSTRAP NEW CLOUD FORMATION STACK
SENZA: MANAGE STACKS
LOGGING
APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR
SSH ACCESS
SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER
MONITORING
TODO: Screenshot
ZMON
ZMON APPLIANCE
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”
EC2
Instance
EC2
InstanceEC2
Instance
EC2
Instanc...
HYSTRIX TURBINE
OAUTH
OAUTH: APPLICATION REGISTRATION IN YOUR TURN
OAUTH: CREDENTIAL DISTRIBUTION VIA S3 BUCKETS
AWS
YOUR TURN
get access
token
Taupage
Kio Mint
OAuth
Provider
store
passwor...
STUPS Frontpage
http://stups.io
STUPS Documentation
http://docs.stups.io
GitHub Repositories
https://github.com/zalando-st...
QUESTIONS?
BACKUP
● ELB for
inbound traffic
● NAT instances
for outbound
● HTTPS Only
● Internal subnets
for app instances
DMZ DMZ DMZ
inter...
Pier One Docker Reg.
build
approve
EC2 Instances
Docker
Container
Application “myapp”
issue_management: Jira
Application V...
ZALANDO TECH CONSTITUTION PT. 1
ZALANDO TECH CONSTITUTION PT. 2
ENGINEER
DELIVERS CUTTING EDGE SOFTWARE PRODUCTS END
TO END
MAINTAINS AND OPERATES THE SOFTWARE PRODUCTS
IS ACCOUNTABLE FO...
PRODUCER
ACCOUNTABLE FOR NON-CORE ENGINEERING TASKS
NECESSARY FOR SUCCESSFUL E2E DELIVERY AND
OPERATION
ORGANIZES TEAM EXT...
DELIVERY LEAD
DELIVER
TRUST BASED LEADERSHIP
UNDERSTAND THE BUSINESS
DRIVE PURPOSE
SUPPORT AUTONOMY
DELIVERY LEAD
DEVELOPS VISION AND ENSURES HIGH QUALITY
CUTTING EDGE PRODUCT DELIVERY
BUILDS POWERFUL TEAMS
ENABLES TEAMS A...
PEOPLE LEAD
ENABLES PEOPLE TO CONTINUOUSLY GROW AND
DEVELOP THEIR CAREER THROUGH ROTATIONAL TOUR
OF MASTERY
ALIGNS COMPANY...
PEOPLE LEAD
COACH
SUPPORT
SUPPORT AUTONOMY
DRIVER OF MASTERY
TOURS OF MASTERY
LOTS OF DIRECTS
What we've built at Zalando is complex. Supporting – profitably – a publicly traded e-commerce company that does
business ...
Radical Agility with Autonomous Teams and Microservices in the Cloud
Radical Agility with Autonomous Teams and Microservices in the Cloud
Radical Agility with Autonomous Teams and Microservices in the Cloud
Upcoming SlideShare
Loading in …5
×

Radical Agility with Autonomous Teams and Microservices in the Cloud

15,461 views

Published on

A talk by software engineers Jan Löffler and Henning Jacobs on Zalando's adoption of microservices, cloud computing and autonomous teams. Zalando is Europe's largest online fashion platform, doing business in 15 countries with more than 15 million users. Visit tech.zalando.com for more information about Zalando's technology, open source projects and opportunities.

Published in: Technology

Radical Agility with Autonomous Teams and Microservices in the Cloud

  1. 1. Radical Agility with Autonomous Teams and Microservices in the Cloud DevOps Con 2015 - Berlin - Germany
  2. 2. ARCHITECTURE LEADERSHIP ABOUT US RADICAL AGILITY INFRASTRUCTURE AGENDA
  3. 3. Jan Löffler ● Head of Platform Engineering ● Twitter: @jlsoft2 ● jan.loeffler@zalando.de ABOUT US
  4. 4. Henning Jacobs ● STUPS Hacker ● Twitter: @try_except_ ● henning.jacobs@zalando.de ABOUT US
  5. 5. 15 countries 3 fulfillment centers 15+ million active customers 2.2+ billion € revenue 2014 130+ million visits per month 8.000+ employees ONE of EUROPE’S LARGEST ONLINE FASHION RETAILERS Visit us: tech.zalando.com
  6. 6. ENVIRONMENT
  7. 7. DATACENTER ENVIRONMENT DataCenter I Gütersloh, Germany DataCenter II Berlin, Germany DataCenter III Berlin, Germany Global Traffic Management AWS Dublin, Ireland
  8. 8. THE LOST HIGHWAY
  9. 9. CLOUD PROJECTS 2013/14 2014 Pequod 2013 Noah’s ARKzCloud
  10. 10. TOPIC 1 WHERE TO GO THIS NEEDS TO STOP Doing it yourself is not the most sensible thing. Amazon invested already thousands of engineering hours… we must utilize this. (Eric Bowman)
  11. 11. RADICAL AGILITY
  12. 12. GOAL DELIVER AMAZING PRODUCTS EFFICIENTLY AT SCALE, AND FEELING GREAT ABOUT IT.
  13. 13. 3 PRINCIPLES
  14. 14. PURPOSE
  15. 15. AUTONOMY
  16. 16. MASTERY
  17. 17. LEADERSHIP FROM CONTROL & COMMAND TO PURPOSE AND TRUST
  18. 18. NEW LEADERSHIP DELIVERY LEAD PEOPLE LEAD
  19. 19. GLOBAL REGRESSION INNOVATION LAB TECH SERVICE PRODUCT OWNER PRODUCT SPECIALIST DELIVERY LEAD PEOPLE LEAD BUSINESS ASSURANCE PRODUCT OVERARCHING ADMIN & SUPPORT CONTROLLING EXECUTIVE SUPPORT COMPLIANCE RISK , SECURITY & STRATEGY ONBOARDING & TECHADEMY AGILE COACHING PROJECT MANGEMENT ENGINEERING PRODUCTIVITY DELIVERY LEAD PEOPLE LEAD DELIVERY
  20. 20. STEERING ALIGNING TARGETS VIA OKR: WORK TOWARDS COMPANY PURPOSE
  21. 21. ARCHITECTURE AN ARCHITECTURE FOR INNOVATION
  22. 22. API FIRST
  23. 23. REST
  24. 24. SAAS
  25. 25. MICRO SERVICES
  26. 26. CLOUD
  27. 27. INFRASTRUCTURE ROBUST PLATFORM SERVICES TO ENABLE AUTONOMY
  28. 28. STUPS STUPS To Unleash Penguin Swarms
  29. 29. AWS STUPS DOCKER DEPLOY SSH ACCESS AUDIT REPORTS FULL AWS ACCESS A PLATFORM ON TOP OF AMAZON WEB SERVICES
  30. 30. AUTONOMY AND COMPLIANCE STUPS offers maximum freedom for developers while enabling near-real-time audit compliance for every single application.
  31. 31. One AWS account per Team Deployment with Docker Managed SSH Access REST/OAuth 2.0 mandatory Supports Traceability of Changes STUPS IN A NUTSHELL
  32. 32. Public Internet *.foo.example.org *.bar.example.org Team “Foo” Team “Bar”ELB ELB EC2 Instance EC2 InstanceEC2 InstanceEC2 Instance EC2 InstanceEC2 Instance Data Center LB EC2 InstanceEC2 InstanceLegacy Instances ISOLATED AWS ACCOUNTS
  33. 33. DEPLOYMENT
  34. 34. IMMUTABLE STACKS
  35. 35. AWS DEPLOYMENT WITH SENZA Senza CLI Pier One docker pull docker push Taupage
  36. 36. SENZA: DEFINITION YAML
  37. 37. SENZA: BOOTSTRAP NEW CLOUD FORMATION STACK
  38. 38. SENZA: MANAGE STACKS
  39. 39. LOGGING
  40. 40. APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR
  41. 41. SSH ACCESS
  42. 42. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER
  43. 43. MONITORING
  44. 44. TODO: Screenshot ZMON
  45. 45. ZMON APPLIANCE *.foo.example.org *.bar.example.org Team “Foo” Team “Bar” EC2 Instance EC2 InstanceEC2 Instance EC2 Instance ZMON Appliance ZMON Appliance KairosDB EC2 Instance EC2 Instance ZMON Controller ELB ELB
  46. 46. HYSTRIX TURBINE
  47. 47. OAUTH
  48. 48. OAUTH: APPLICATION REGISTRATION IN YOUR TURN
  49. 49. OAUTH: CREDENTIAL DISTRIBUTION VIA S3 BUCKETS AWS YOUR TURN get access token Taupage Kio Mint OAuth Provider store passwords get password S3 rotate passwords
  50. 50. STUPS Frontpage http://stups.io STUPS Documentation http://docs.stups.io GitHub Repositories https://github.com/zalando-stups Trying out Senza and Taupage http://docs.stups.io/en/latest/user-guide/standalone-deployment.html LINKS
  51. 51. QUESTIONS?
  52. 52. BACKUP
  53. 53. ● ELB for inbound traffic ● NAT instances for outbound ● HTTPS Only ● Internal subnets for app instances DMZ DMZ DMZ internalinternal eu-west-1a eu-west-1b eu-west-1c ELB EC2 internal EC2 NAT STUPS: AWS ACCOUNT VPC SETUP
  54. 54. Pier One Docker Reg. build approve EC2 Instances Docker Container Application “myapp” issue_management: Jira Application Version “1.0” artifact: docker/myart:1.0 Taupage AMI Ticket System Kio Application Registry Ticket System SCM Image “docker/myart:1.0” commit: afb123Issue “ABC-123” spec: [...] Commit “afb123” msg: ABC-123.. ✓ specs approved ✓ artifact tested ✓ artifact approved STUPS: TRACEABILITY
  55. 55. ZALANDO TECH CONSTITUTION PT. 1
  56. 56. ZALANDO TECH CONSTITUTION PT. 2
  57. 57. ENGINEER DELIVERS CUTTING EDGE SOFTWARE PRODUCTS END TO END MAINTAINS AND OPERATES THE SOFTWARE PRODUCTS IS ACCOUNTABLE FOR HIGH QUALITY OF SOFTWARE PRODUCTS AND THEIR SPECIFICATION
  58. 58. PRODUCER ACCOUNTABLE FOR NON-CORE ENGINEERING TASKS NECESSARY FOR SUCCESSFUL E2E DELIVERY AND OPERATION ORGANIZES TEAM EXTERNAL PROJECT DEPENDENCIES, EXTERNAL DELIVERABLES AND ROADMAPS TAKES CARE AND GETS RID OF IMPEDIMENTS
  59. 59. DELIVERY LEAD DELIVER TRUST BASED LEADERSHIP UNDERSTAND THE BUSINESS DRIVE PURPOSE SUPPORT AUTONOMY
  60. 60. DELIVERY LEAD DEVELOPS VISION AND ENSURES HIGH QUALITY CUTTING EDGE PRODUCT DELIVERY BUILDS POWERFUL TEAMS ENABLES TEAMS AND CHALLENGES DECISIONS OVERSEES ALL TECHNICAL TOPICS
  61. 61. PEOPLE LEAD ENABLES PEOPLE TO CONTINUOUSLY GROW AND DEVELOP THEIR CAREER THROUGH ROTATIONAL TOUR OF MASTERY ALIGNS COMPANY AND TEAM PURPOSE WITH PEOPLE’ S DRIVERS CONTRIBUTES TO ALL PEOPLE RELATED TOOLS AND PROCESSES AND ENSURES THAT THEY ARE STATE OF THE ART
  62. 62. PEOPLE LEAD COACH SUPPORT SUPPORT AUTONOMY DRIVER OF MASTERY TOURS OF MASTERY LOTS OF DIRECTS
  63. 63. What we've built at Zalando is complex. Supporting – profitably – a publicly traded e-commerce company that does business in 15 diverse European markets, with more than 15 million active users who all speak different languages, use different payment methods, prefer different shipping methods, and have different product tastes, has required nonstop innovation. Until recently we've focused on building a unified, comprehensive retail system, quickly, that solves just our problems. But to truly fight against complexity – particularly the accidental complexity that slows down our development process – we have adopted a microservices architecture. And when it comes to DevOps, we’ve gone a step beyond the "You build it, you run it" motto--working in autonomous teams with DevOps treated as a "first-class entity.” In this talk, Henning Jacobs (Software Architect) and Jan Löffler (Head of Platform Engineering) will share their experience implementing “Radical Agility” from a DevOps perspective. “Radical Agility” is the Zalando technology team’ s multi-pronged approach to managing the complexity that results from building an architecture of massive size. Henning and Jan will focus on how microservices enable Zalando’s engineers to move faster and build systems that scale, at scale, and avoid dependencies. They will show how microservices, in conjunction with a cloud infrastructure, support teams as they try strive for autonomy. Finally, they will draw upon their experiences to show how this all works in practice, and discuss what is organizationally and architecturally necessary to make DevOps a top priority for all members of your tech organization. ABSTRACT

×