Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker Berlin Meetup June 2015: Docker powering Radical Agility @ Zalando Tech

2,987 views

Published on

How Docker is used in the open source STUPS cloud infrastructure @ Zalando Tech.

Published in: Technology

Docker Berlin Meetup June 2015: Docker powering Radical Agility @ Zalando Tech

  1. 1. Docker Powering Radical Agility Docker Berlin Meetup 2015-06-23 henning.jacobs@zalando.de @try_except_
  2. 2. DOCKER-BASED DEPLOYMENT STUPS CLOUD PLATFORM HISTORY RADICAL AGILITY & ARCHITECTURE PRINCIPLES AGENDA
  3. 3. 15 countries 3 fulfillment centers 15+ million active customers 2.2+ billion € revenue 2014 130+ million visits per month 8.000+ employees ONE OF EUROPE’S LARGEST ONLINE FASHION RETAILERS Visit us: tech.zalando.com
  4. 4. A BRIEF HISTORY OF ZALANDO TECHNOLOGY
  5. 5. A BRIEF HISTORY OF ZALANDO TECH
  6. 6. ZALANDO PLATFORM
  7. 7. ~70% of all applications WAR deployment Single deployment tool On-premise data center MAIN PRODUCTION STACK SINCE 2010
  8. 8. Platform THE CHALLENGE Platform team request servers deploy
  9. 9. Platform THE CHALLENGE 80+ delivery teams Platform team deploy request servers request storage
  10. 10. RADICAL AGILITY
  11. 11. GOAL DELIVER AMAZING PRODUCTS EFFICIENTLY AT SCALE, AND FEELING GREAT ABOUT IT.
  12. 12. 3 PRINCIPLES
  13. 13. PURPOSE
  14. 14. AUTONOMY
  15. 15. MASTERY
  16. 16. ARCHITECTURE AN ARCHITECTURE FOR INNOVATION
  17. 17. API FIRST
  18. 18. REST
  19. 19. SAAS
  20. 20. MICRO SERVICES
  21. 21. CLOUD
  22. 22. STUPS STUPS To Unleash Penguin Swarms
  23. 23. AWS STUPS DOCKER DEPLOY SSH ACCESS AUDIT REPORTS FULL AWS ACCESS A PLATFORM ON TOP OF AMAZON WEB SERVICES
  24. 24. AUTONOMY AND COMPLIANCE STUPS offers maximum freedom for developers while enabling near-real-time audit compliance for every single application.
  25. 25. One AWS account per Team Deployment with Docker Managed SSH Access REST/OAuth 2.0 mandatory Supports Traceability of Changes STUPS IN A NUTSHELL
  26. 26. Public Internet *.foo.example.org *.bar.example.org Team “Foo” Team “Bar”ELB ELB EC2 Instance EC2 InstanceEC2 InstanceEC2 Instance EC2 InstanceEC2 Instance Data Center LB EC2 InstanceEC2 InstanceLegacy Instances ISOLATED AWS ACCOUNTS
  27. 27. DEPLOYMENT
  28. 28. IMMUTABLE STACKS
  29. 29. AWS DEPLOYMENT WITH SENZA Senza CLI Pier One docker pull docker push Taupage
  30. 30. FROM zalando/openjdk:8u40-b09-4 EXPOSE 8080 COPY target/hello-world.jar / COPY target/scm-source.json / CMD java $(java-dynamic-memory-opts) ↲ -jar /hello-world.jar DOCKERFILE
  31. 31. $ docker build -t ↲ pierone.example.org/myteam/hello-world:0.2 . $ pierone login Getting OAuth2 token "pierone".. OK Storing Docker client configuration in ~/.dockercfg.. OK $ docker push pierone.example.org/myteam/hello-world:0.2 DOCKER BUILD & PUSH
  32. 32. $ pierone tags myteam hello-world Team │Artifact │Tag │Created│By | myteam hello-world 0.1-andre-test 13d ago ahartmann myteam hello-world 0.1 3d ago ahartmann myteam hello-world 0.2 3m ago hjacobs $ pierone scm myteam hello-world 0.2 Tag│Author │URL │Revision │Status│Created│By | 0.2 hjacobs git:git@github.. 442b7502 10m ago hjacobs VERIFY IMAGE UPLOAD
  33. 33. PIER ONE DOCKER REGISTRY ✓ S3 backend to store images ✓ OAuth2 integration ✓ Team repositories ✓ Immutable tags & scm-source.json ✓ JVM-based (Clojure) ✓ Command line interface (Python)
  34. 34. SENZA: DEFINITION YAML SenzaInfo: StackName: hello-world Parameters: - ImageVersion: Description: "Docker image version of Hello World." SenzaComponents: - Configuration: Type: Senza::StupsAutoConfiguration # auto-detect network setup - AppServer: # will create a launch configuration and ASG with scaling triggers Type: Senza::TaupageAutoScalingGroup InstanceType: t2.micro SecurityGroups: [app-hello-world] ElasticLoadBalancer: AppLoadBalancer TaupageConfig: runtime: Docker source: "stups/hello-world:{{Arguments.ImageVersion}}" ports: 8080: 8080
  35. 35. SENZA: STACK DEPLOYMENT $ senza create hello-world.yaml 1 0.2 Generating Cloud Formation template.. OK Creating Cloud Formation stack hello-world-1.. OK $ senza events hello-world.yaml 1 Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event Time hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago ... hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago
  36. 36. docker run -d --log-driver=syslog ↲ --restart=on-failure:10 ↲ -e DB_SUBNAME=.. ↲ -v /meta:/meta:ro ↲ -e CREDENTIALS_DIR=/meta/credentials ↲ -p 8080:8080 -p 7979:7979 ↲ -u 999 ↲ pierone.example.org/stups/pierone:0.5 TAUPAGE: DOCKER COMMAND LINE
  37. 37. SENZA: MANAGE STACKS
  38. 38. LOGGING
  39. 39. docker run .. --log-driver=syslog .. /etc/rsyslog.d/24-application.conf :syslogtag, startswith, "docker" ↲ /var/log/application.log /etc/logrotate.d/.. Don’t forget log rotation.. TAUPAGE: DOCKER SYSLOG
  40. 40. APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR
  41. 41. SSH ACCESS
  42. 42. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER
  43. 43. MONITORING
  44. 44. TODO: Screenshot ZMON
  45. 45. ZMON APPLIANCE *.foo.example.org *.bar.example.org Team “Foo” Team “Bar” EC2 Instance EC2 InstanceEC2 Instance EC2 Instance ZMON Appliance ZMON Appliance KairosDB EC2 Instance EC2 Instance ZMON Controller ELB ELB
  46. 46. HYSTRIX TURBINE
  47. 47. FULLSTOP: REPORT VIOLATIONS
  48. 48. DOCKER?
  49. 49. ● Ubuntu & OpenJDK base image ● Log to STDOUT ● Config via environ. vars (+ KMS decryption) ● Non-root execution ● Persistence via EBS mounts ● Immutable stacks, no orchestration ● DNS endpoints, etcd e.g. for Hystrix streams RECAP: DOCKER IN STUPS
  50. 50. STUPS Frontpage http://stups.io STUPS Documentation http://docs.stups.io GitHub Repositories https://github.com/zalando-stups Trying out Senza and Taupage http://docs.stups.io/en/latest/user-guide/standalone-deployment.html LINKS
  51. 51. QUESTIONS? http://stups.io @try_except_
  52. 52. BACKUP
  53. 53. STUPS COMPONENTS
  54. 54. ● ELB for inbound traffic ● NAT instances for outbound ● HTTPS Only ● Internal subnets for app instances DMZ DMZ DMZ internalinternal eu-west-1a eu-west-1b eu-west-1c ELB EC2 internal EC2 NAT STUPS: AWS ACCOUNT VPC SETUP
  55. 55. Pier One Docker Reg. build approve EC2 Instances Docker Container Application “myapp” issue_management: Jira Application Version “1.0” artifact: docker/myart:1.0 Taupage AMI Ticket System Kio Application Registry Ticket System SCM Image “docker/myart:1.0” commit: afb123Issue “ABC-123” spec: [...] Commit “afb123” msg: ABC-123.. ✓ specs approved ✓ artifact tested ✓ artifact approved STUPS: TRACEABILITY

×