Successfully reported this slideshow.

STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framework for Scale

2

Share

Loading in …3
×
1 of 49
1 of 49

STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framework for Scale

2

Share

Download to read offline

This talk shed lights on how Zalando, Europe’s leading eCommerce platform for fashion, has developed an architecture and set of open source cloud tools that scales as its technology team (100 autonomous engineering teams and counting) grows and evolves. It also describes the challenges regarding compliance and innovation that we face, and how we built our cloud infrastructure on top of AWS and Docker to manage these challenges.

Talk given by Henning Jacobs at the WHD.local event in Frankfurt on September 10th 2015.

This talk shed lights on how Zalando, Europe’s leading eCommerce platform for fashion, has developed an architecture and set of open source cloud tools that scales as its technology team (100 autonomous engineering teams and counting) grows and evolves. It also describes the challenges regarding compliance and innovation that we face, and how we built our cloud infrastructure on top of AWS and Docker to manage these challenges.

Talk given by Henning Jacobs at the WHD.local event in Frankfurt on September 10th 2015.

More Related Content

More from Henning Jacobs

Related Books

Free with a 30 day trial from Scribd

See all

STUPS by Zalando @WHD.local Frankfurt: STUPS.io - an Open Source Cloud Framework for Scale

  1. 1. STUPS.io an Open Source Cloud Framework for Scale henning.jacobs@zalando.de / @try_except_ WHD.local Frankfurt, 2015-09-10 background image based on https://commons.wikimedia.org/wiki/File:CERN_Server_03.jpg by Florian Hirzinger - www.fh-ap.com
  2. 2. Henning Jacobs ● STUPS Hacker ● Twitter: @try_except_ ● henning.jacobs@zalando.de
  3. 3. 15 countries 3 fulfillment centers 15+ million active customers 2.2+ billion € revenue 2014 130+ million visits per month 8.000+ employees ONE OF EUROPE’S LARGEST ONLINE FASHION RETAILERS
  4. 4. A BRIEF HISTORY OF ZALANDO TECHNOLOGY
  5. 5. 2010
  6. 6. 2010 Gütersloh
  7. 7. 2012
  8. 8. 2012 Gütersloh Berlin
  9. 9. Platform Platform team request servers deploy
  10. 10. Platform THE CHALLENGE 80+ delivery teams Platform team deploy request servers request storage
  11. 11. RADICAL AGILITY
  12. 12. AUTONOMY
  13. 13. Compliance Innovation
  14. 14. STUPS STUPS To Unleash Penguin Swarms
  15. 15. AWS STUPS DOCKER DEPLOY SSH ACCESS AUDIT REPORTS FULL AWS ACCESS STUPS: A PLATFORM ON TOP OF AMAZON WEB SERVICES
  16. 16. ONE DATA CENTER PER TEAM
  17. 17. Internet *.abc.example.org *.xyz.example.org Team ABC Team XYZ ISOLATED AWS ACCOUNTS EC2EC2 ELBELB EC2
  18. 18. DEPLOYMENT
  19. 19. IMMUTABLE STACKS ELB myapp-1 myapp.example.org EC2 + Docker EC2 + Docker EC2 + Docker
  20. 20. IMMUTABLE STACKS ELB myapp-1 EC2 + Docker EC2 + Docker EC2 + Docker ELB myapp-2 EC2 + Docker EC2 + Docker myapp.example.org
  21. 21. ELB myapp-2 EC2 + Docker EC2 + Docker myapp.example.org IMMUTABLE STACKS
  22. 22. AWS DEPLOYMENT WITH SENZA Senza CLI Pier One docker pull docker push Taupage
  23. 23. $ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 . DOCKER BUILD & PUSH
  24. 24. $ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 . $ pierone login Getting OAuth2 token "pierone".. OK Storing Docker client configuration in ~/.dockercfg.. OK $ docker push ↲ pierone.example.org/myteam/hello-world:0.2 DOCKER BUILD & PUSH
  25. 25. SENZA: STACK DEPLOYMENT $ senza create hello-world.yaml 1 0.2 Generating Cloud Formation template.. OK Creating Cloud Formation stack hello-world-1.. OK
  26. 26. SENZA: STACK DEPLOYMENT $ senza create hello-world.yaml 1 0.2 Generating Cloud Formation template.. OK Creating Cloud Formation stack hello-world-1.. OK $ senza events hello-world.yaml 1 Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event Time hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago ... hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago
  27. 27. LOGGING
  28. 28. SSH ACCESS
  29. 29. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER
  30. 30. OAUTH
  31. 31. OAUTH: CREDENTIAL DISTRIBUTION VIA S3 BUCKETS AWS WEB UI get access token Taupage OAuth Provider store passwords get password S3 rotate passwords
  32. 32. NETWORK
  33. 33. ● ELB for inbound ● NAT for outbound ● HTTPS Only ● Internal subnets for app instances ● odd SSH bastion DMZ DMZ DMZ internalinternal eu-west-1a eu-west-1b eu-west-1c ELB EC2 internal EC2 NAT AWS ACCOUNT VPC SETUP odd
  34. 34. ● 800+ in Zalando Tech ● 90+ AWS Accounts ● 160+ Applications SOME NUMBERS..
  35. 35. 5 THINGS WE LEARNED
  36. 36. AUTONOMOUS TEAMS NEED THEIR OWN DATA CENTER #1
  37. 37. SOME THINGS ONLY WORK IN “THE CLOUD” #2
  38. 38. AWS IS FAR FROM PERFECT #3
  39. 39. TRANSPARENCY IS KEY #4
  40. 40. OUR NEEDS REQUIRED OUR SOLUTION #5
  41. 41. Questions? STUPS Frontpage stups.io GitHub Repositories github.com/zalando-stups tech.zalando.com @try_except_
  42. 42. BACKUP
  43. 43. Docker Registry build approved EC2 Instance Docker Container Ticket System Application Registry SCM Docker Image Ticket Commit ✓ TRACEABILITY Application Version
  44. 44. STUPS COMPONENTS

×