This talk shed lights on how Zalando, Europe’s leading eCommerce platform for fashion, has developed an architecture and set of open source cloud tools that scales as its technology team (100 autonomous engineering teams and counting) grows and evolves. It also describes the challenges regarding compliance and innovation that we face, and how we built our cloud infrastructure on top of AWS and Docker to manage these challenges. Talk given by Henning Jacobs at the WHD.local event in Frankfurt on September 10th 2015.

1. STUPS.io
an Open Source Cloud Framework for Scale
henning.jacobs@zalando.de / @try_except_
WHD.local Frankfurt, 2015-09-10
background image based on
https://commons.wikimedia.org/wiki/File:CERN_Server_03.jpg
by Florian Hirzinger - www.fh-ap.com

2. Henning Jacobs
● STUPS Hacker
● Twitter: @try_except_
● henning.jacobs@zalando.de

3. 15 countries
3 fulfillment centers
15+ million active customers
2.2+ billion € revenue 2014
130+ million visits per month
8.000+ employees
ONE OF EUROPE’S LARGEST ONLINE FASHION RETAILERS

4. A BRIEF
HISTORY OF
ZALANDO
TECHNOLOGY

7. 2010

8. 2010
Gütersloh

9. 2012

10. 2012
Gütersloh Berlin

11. Platform Platform team
request servers
deploy

12. Platform
THE CHALLENGE
80+ delivery teams
Platform team
deploy
request servers
request storage

14. RADICAL
AGILITY

15. AUTONOMY

16. Compliance Innovation

17. STUPS
STUPS To Unleash Penguin Swarms

18. AWS
STUPS
DOCKER
DEPLOY
SSH
ACCESS
AUDIT
REPORTS
FULL AWS
ACCESS
STUPS: A PLATFORM ON TOP OF AMAZON WEB SERVICES

19. ONE DATA CENTER PER TEAM

20. Internet
*.abc.example.org *.xyz.example.org
Team ABC Team XYZ
ISOLATED AWS ACCOUNTS
EC2EC2
ELBELB
EC2

21. DEPLOYMENT

22. IMMUTABLE STACKS
ELB myapp-1
myapp.example.org
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker

23. IMMUTABLE STACKS
ELB myapp-1
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
ELB myapp-2
EC2
+ Docker
EC2
+ Docker
myapp.example.org

24. ELB myapp-2
EC2
+ Docker
EC2
+ Docker
myapp.example.org
IMMUTABLE STACKS

25. AWS
DEPLOYMENT WITH SENZA
Senza CLI
Pier One
docker pull
docker push
Taupage

26. $ docker build -t ↲
pierone.example.org/myteam/hello-world:0.2 .
DOCKER BUILD & PUSH

27. $ docker build -t ↲
pierone.example.org/myteam/hello-world:0.2 .
$ pierone login
Getting OAuth2 token "pierone".. OK
Storing Docker client configuration in ~/.dockercfg.. OK
$ docker push ↲
pierone.example.org/myteam/hello-world:0.2
DOCKER BUILD & PUSH

28. SENZA: STACK DEPLOYMENT
$ senza create hello-world.yaml 1 0.2
Generating Cloud Formation template.. OK
Creating Cloud Formation stack hello-world-1.. OK

29. SENZA: STACK DEPLOYMENT
$ senza create hello-world.yaml 1 0.2
Generating Cloud Formation template.. OK
Creating Cloud Formation stack hello-world-1.. OK
$ senza events hello-world.yaml 1
Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event
Time
hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago
...
hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago

30. LOGGING

32. SSH ACCESS

33. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER

34. OAUTH

35. OAUTH: CREDENTIAL DISTRIBUTION VIA S3 BUCKETS
AWS
WEB UI
get access
token
Taupage
OAuth
Provider
store
passwords
get password
S3
rotate
passwords

36. NETWORK

37. ● ELB for inbound
● NAT for outbound
● HTTPS Only
● Internal subnets
for app instances
● odd SSH bastion
DMZ DMZ DMZ
internalinternal
eu-west-1a eu-west-1b eu-west-1c
ELB
EC2
internal
EC2
NAT
AWS ACCOUNT VPC SETUP
odd

38. ● 800+ in Zalando Tech
● 90+ AWS Accounts
● 160+ Applications
SOME NUMBERS..

39. 5 THINGS
WE LEARNED

40. AUTONOMOUS
TEAMS NEED THEIR
OWN DATA CENTER
#1

41. SOME THINGS
ONLY WORK
IN “THE CLOUD”
#2

42. AWS IS FAR
FROM PERFECT
#3

43. TRANSPARENCY
IS KEY
#4

44. OUR NEEDS
REQUIRED
OUR SOLUTION
#5

45. Questions?
STUPS Frontpage
stups.io
GitHub Repositories
github.com/zalando-stups
tech.zalando.com
@try_except_

46. BACKUP

47. Docker Registry
build
approved
EC2 Instance
Docker
Container
Ticket System
Application Registry
SCM
Docker Image
Ticket
Commit
✓
TRACEABILITY
Application Version

49. STUPS COMPONENTS