STUPS.io is an open source cloud framework for AWS that provides maximum developer freedom while enabling near-real-time audit compliance for applications. It utilizes isolated AWS accounts with Docker deployment, managed SSH access, and immutable infrastructure to give development teams autonomy while ensuring security and compliance.

1. STUPS.io
an Open Source Cloud Framework for AWS
jan.loeffler@zalando.de / @jlsoft2
DevOpsCon Munich 2015-11-25

2. We shape our buildings;
thereafter they shape us

3. Conway’s Law
“organizations which design
systems ... are constrained
to produce designs which
are copies of the
communication structures of
these organizations”
Melvin Conway

4. AN
ARCHITECTURE
FOR
INNOVATION

9. A BRIEF
HISTORY OF
ZALANDO
TECHNOLOGY

13. November
1.000+
Apps
900+
Tech employees

14. PLATFORM
Platform team
request servers
deploy
Datacenter

15. 70+ delivery teams
Platform team
deploy
request servers
request storage
PLATFORM
Datacenter

18. DELIVER AMAZING
PRODUCTS
EFFICIENTLY AT
SCALE, AND
FEELING GREAT
ABOUT IT.

19. PURPOSE - AUTONOMY - MASTERY

23. DRIVE
The Surprising Truth
About What
Motivates Us
Daniel Pink

24. FROM
CONTROL &
COMMAND
TO PURPOSE
AND TRUST

25. DELIVERY
LEAD
PEOPLE
LEAD

26. GLOBAL
REGRESSION
INNOVATION LAB
TECH
SERVICE
PRODUCT
OWNER
PRODUCT
SPECIALIST
DELIVERY
LEAD
PEOPLE
LEAD
BUSINESS
ASSURANCE
PRODUCT
OVERARCHING
ADMIN & SUPPORT
CONTROLLING
EXECUTIVE
SUPPORT
COMPLIANCE
RISK ,
SECURITY &
STRATEGY
ONBOARDING &
TECHADEMY
AGILE
COACHING PROJECT
MANGEMENT
ENGINEERING
PRODUCTIVITY
DELIVERY
LEAD
PEOPLE LEAD
DELIVERY

29. API FIRST

30. REST

31. SAAS

32. MICRO
SERVICES

33. CLOUD

34. OPEN
SOURCE

36. Compliance Innovation

37. file:///Users/kwalckermaye/Downloads/Mobil
e-Developers-look-ov-008.jpg
file:///Users/kwalckermaye/Downloads/deskt
op_death-600x369.jpg
file:///Users/kwalckermaye/Downloads/07235
8-wired.gif.jpeg
file:///Users/kwalckermaye/Downloads/the-
death-of-the-desktop.jpg

39. WHERE
TO GO
DataCenter I DataCenter II AWS
APP
1
APP
2
APP
3
APP
4
APP
5
APP
6
APP
1
APP
2
APP
3
APP
4
APP
5
APP
6
APP
1
APP
2
APP
3
APP
4

41. DEPLOYMENT

42. MANAGED
SSH ACCESS

43. TRACABILITY

44. ACCOUNT
ISOLATION

45. DOCKER?

46. STUPS
STUPS To Unleash Penguin Swarms
https://stups.io

47. ➊ One AWS account per Team
➋ Deployment with Docker
➌ Managed SSH Access
➍ REST/OAuth 2.0 mandatory
➎ Traceability of changes
IN A NUTSHELL
STUPS

48. AWS
STUPS
DOCKER
DEPLOY
SSH ACCESS AUDIT
REPORTS
FULL AWS
ACCESS
A PLATFORM ON TOP OF AMAZON WEB SERVICES

49. “We provide maximum freedom for
developers while enabling near-real-
time audit compliance for every
single application.”
-- Zalando STUPS Delivery Team

50. Public Internet
*.jacob.example.org
*.edward.example.or
g
Team “Jacob”
Team
“Edward”
ELB
ELB
ISOLATED AWS ACCOUNTS & OAUTH 2.0 & SECURITY
Data CenterLB
AWS

51. IMMUTABLE STACKS
ELB myapp-v1
myapp.example.org
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
100%

52. IMMUTABLE STACKS
ELB myapp-v1
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
ELB myapp-v2
EC2
+ Docker
EC2
+ Docker
myapp.example.org
90% 10%
$ senza traffic myapp v2
10

53. IMMUTABLE STACKS
ELB myapp-v1
EC2
+ Docker
EC2
+ Docker
EC2
+ Docker
ELB myapp-v2
EC2
+ Docker
EC2
+ Docker
myapp.example.org
0% 100%
$ senza traffic myapp v2 100

54. EXAMPLE STACK
AWS EC2
Taupage AMI
Docker Container
Application
✓ In isolated
team
account
✓ Created by
senza
through
Cloud
Formation
✓ Docker Runtime
✓ Managed SSH access
✓ Audit Logging
✓ Log Collection
✓ Monitoring
✓ Reviewed security
additions
✓ Ubuntu
✓ OpenJDK
✓ Zalando CA
certificate
✓ scm-source
…

55. AWS
Senza CLI
Docker
Registry
docker pull
docker push
AMI

56. DEMO TIME

57. Securing REST based Microservices
Lock all doors
● Open only ports that you need
permanently
● Grant SSH access only temporary
● Use a firewall for each app container
Encrypt all channels & critical data
● Keep all transferred data confidential
● Use TLS
● Encrypt data in databases & use HSM
Authenticate & Authorize
● Make sure you talk to the right guy
● Check if access is permitted
● Use OAuth 2.0
Monitor & Limit your API calls
● Control who accesses your API how
often, when and from where
● Use Rate Limiting against fraud
● Use external Security Services against
DDoSIsolate your apps
● Only one app per container / server
● Use Docker or CoreOS Rkt

58. AWS
Developer
Console
get access
token
AMI
Password
Rotator
OAuth
Provider
store
passwords
get password
S3
rotate
passwords
Application
Registry

59. SCM
Ticket system
Issue “ABC-123”
Commit “afb123”
msg: ABC-123..

60. SCM
Pier One Docker Registry
build
Ticket system
Image “docker/myart:1.0”
commit: afb123Issue “ABC-123”
Commit “afb123”
msg: ABC-123..

61. Pier One Docker Registry
build
approved
Application Version “1.0”
artifact: docker/myart:1.0
Ticket system
Application Registry
SCM
Image “docker/myart:1.0”
commit: afb123Issue “ABC-123”
Commit “afb123”
msg: ABC-123..
✓ Specification
✓ Artefact tested

62. Pier One Docker Registry
build
approved
EC2 Instance
Docker
Container
Application Version “1.0”
artifact: docker/myart:1.0
AMI
Ticket system
Application Registry
SCM
Image “docker/myart:1.0”
commit: afb123Issue “ABC-123”
Commit “afb123”
msg: ABC-123..
✓ Specification
✓ Artefact tested

64. Spilo
Highly available open-source
PostgreSQL appliance
https://github.com/zalando/spilo

66. Apache License
Version 2.0STUPS
stups.io & github.com/zalando-stups

67. Radical Agility
Purpose - Autonomy - Mastery
Architecture principles
Immutable Infrastructure
Security is key
SUMMARY

68. Build a trustful “devops” culture
&
Utilize cloud services
to give teams autonomy
MESSAGE

69. Try STUPS.io
https://stups.io
https://docs.stups.io
All components are Open Source
https://github.com/zalando
https://github.com/zalando-stups

70. Jan Löffler
● Head of Platform Engineering
● Twitter: @jlsoft2
● jan.loeffler@zalando.de
● http://www.slideshare.net/jlsoft/

71. We shape our buildings;
thereafter they shape us

72. STUPS website & docs
https://stups.io
https://docs.stups.io
All components are Open Source
https://github.com/zalando
https://github.com/zalando-stups

73. BACKUP

76. FROM zalando/openjdk:8u40-b09-4
EXPOSE 8080
COPY target/hello-world.jar /
COPY target/scm-source.json /
CMD java $(java-dynamic-memory-opts) ↲
-jar /hello-world.jar
DOCKERFILE

77. $ docker build -t ↲
pierone.example.org/myteam/hello-world:0.2 .
$ pierone login
Getting OAuth2 token "pierone".. OK
Storing Docker client configuration in ~/.dockercfg.. OK
$ docker push pierone.example.org/myteam/hello-world:0.2
DOCKER BUILD & PUSH

78. $ pierone tags myteam hello-world
Team │Artifact │Tag │Created│By |
myteam hello-world 0.1-andre-test 13d ago ahartmann
myteam hello-world 0.1 3d ago ahartmann
myteam hello-world 0.2 3m ago hjacobs
$ pierone scm myteam hello-world 0.2
Tag│Author │URL │Revision │Status│Created│By |
0.2 hjacobs git:git@github.. 442b7502 10m ago hjacobs
VERIFY IMAGE UPLOAD

79. SENZA: DEFINITION YAML
SenzaInfo:
StackName: hello-world
Parameters:
- ImageVersion:
Description: "Docker image version of Hello World."
SenzaComponents:
- Configuration:
Type: Senza::StupsAutoConfiguration # auto-detect network setup
- AppServer: # will create a launch configuration and ASG with scaling triggers
Type: Senza::TaupageAutoScalingGroup
InstanceType: t2.micro
SecurityGroups: [app-hello-world]
ElasticLoadBalancer: AppLoadBalancer
TaupageConfig:
runtime: Docker
source: "stups/hello-world:{{Arguments.ImageVersion}}"
ports:
8080: 8080

80. SENZA: STACK DEPLOYMENT
$ senza create hello-world.yaml 1 0.2
Generating Cloud Formation template.. OK
Creating Cloud Formation stack hello-world-1.. OK
$ senza events hello-world.yaml 1
Stack Name│Ver.│Resource Type │Resource ID │Status │Status Reason │Event Time
hello-world 1 CloudFormation::Stack hello-world-1 CREATE_IN_PROGRESS User Initiated 10m ago
...
hello-world 1 CloudFormation::Stack hello-world-1 CREATE_COMPLETE 6m ago

81. docker run -d --log-driver=syslog ↲
--restart=on-failure:10 ↲
-e DB_SUBNAME=.. ↲
-v /meta:/meta:ro ↲
-e CREDENTIALS_DIR=/meta/credentials ↲
-p 8080:8080 -p 7979:7979 ↲
-u 999 ↲
pierone.example.org/stups/pierone:0.5
TAUPAGE: DOCKER COMMAND LINE

82. SENZA: MANAGE STACKS

84. docker run .. --log-driver=syslog ..
/etc/rsyslog.d/24-application.conf
:syslogtag, startswith, "docker" ↲
/var/log/application.log
/etc/logrotate.d/..
Don’t forget log rotation..
TAUPAGE: DOCKER SYSLOG

85. APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR

87. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER

89. TODO: Screenshot
ZMON

90. ZMON APPLIANCE
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”
EC2
Instance
EC2
InstanceEC2
Instance
EC2
Instance
ZMON
Appliance
ZMON
Appliance
KairosDB
EC2
Instance
EC2
Instance
ZMON
Controller
ELB ELB

91. HYSTRIX TURBINE

93. Ubuntu & OpenJDK base image
Log to STDOUT
Config via environ. vars (+ KMS decryption)
Non-root execution
Persistence via EBS mounts
Immutable stacks, no orchestration
DNS endpoints, etcd e.g. for Hystrix streams
RECAP: DOCKER IN STUPS