The document discusses a transition from dedicated virtual machines to container-based clusters for a company called Styla, addressing challenges faced during this process, such as infrastructure management and team communication. Key insights included the importance of understanding AWS services, the issues with Docker and ECS, and the ultimate benefits of creating a unified deployment pipeline and clearer responsibilities among teams. The conclusion emphasizes the investment required in ECS for its successful implementation and the potential benefits as technology matures.

1. MONDAY MEETING
IN CHARGE:
From dedicated virtual
machines to container-
based clusters
28.09.2016 - Dr. Sven Ehlert

2. MONDAY MEETING
IN CHARGE:
What is Styla
Fundamental MARKET NEEDS
Provided in a SHOPPABLE MAGAZINE SAAS

3. MONDAY MEETING
IN CHARGE:
What is Styla
Feeds Automatically Laid Out by STYLA. All Products Are
Instantly SHOPPABLE.

4. MONDAY MEETING
IN CHARGE:
Status Quo
1/5

5. MONDAY MEETING
IN CHARGE:
RDS
Our (Previous) Technology Stack
Story API and
static assets
EC2
Servers Database
Browser
Story Screenshots
ElasticCache
Image resizer service
SEO tags service
Product
integration
service
No proper
Prod / Stage / Dev
separation

6. MONDAY MEETING
IN CHARGE:Our Challenges
Growing-Up Pains
Team size increased.
Communication challenges and ownership changes.
Cross-functional teams.
Microservices on the organizational level.

7. MONDAY MEETING
IN CHARGE:Our Challenges
Infrastructure Management
Phoenix server deployment via ansible.
Interruptions due to network errors and version changes during
provision.
Very hard to replicate to developer workspace.
Business requirement to support 2nd region.
All network infrastructure still handled manually (VPC, Subnets,
Security Groups).

8. MONDAY MEETING
IN CHARGE:
Getting Ready
2/5

9. MONDAY MEETING
IN CHARGE:Options?
Immutable Services
Bake server images (AMIs) or use containers?
Containers are more lightweight!

10. MONDAY MEETING
IN CHARGE:Options?
Container Deployment Battle
Containers = Docker.
But the fight for deployments is not yet over.
Docker Swarm vs Kubernetes vs AWS Elastic Container Service

11. MONDAY MEETING
IN CHARGE:False Expectations
AWS ECS Is Not a Service - It's a
Building Block
You need to know your way around all AWS. We didn't at the
beginning.
If you're looking into "Container-as-a-Service" on AWS,
Check out convox.

12. MONDAY MEETING
IN CHARGE:Implementation
If You Are Lost, Get a Guide
AWS partner agency support for one week.
Full transformation took us about 3 MM.

13. MONDAY MEETING
IN CHARGE:Early Mistakes
IT DOES NOT WORK!!
Simple Demo service runs slow in ECS, but fast on localhost.
For no obvious reasons.
Deeply diving into docker – filesystems, distributions, but to no
avail.
Use the force – use strace!
Also: permissions ...

14. MONDAY MEETING
IN CHARGE:
Success!
3/5

15. MONDAY MEETING
IN CHARGE:ECS - the not so nice parts
Get Married to AWS.
And Accept Your Partner's Whims.
Configuration via JSON. Say goodbye to meaningful comments for
your infrastructure.
Only basic docker-compose support. Use some
taskdefinition.json instead.

16. MONDAY MEETING
IN CHARGE:ECS - the not so nice parts
Getting a Cluster up and Running
Should Be Easy
There's a nice first run wizard in the console. But only for your first
cluster!
Then, it's you and CloudFormation, basically.

17. MONDAY MEETING
IN CHARGE:ECS - the not so nice parts
Managing Services Should Be Easy
Ecs-cli is the "cluster-as-a-service" tool for that. But it is not.
Minimal cluster configuration support. No load balancer service
support. Updates seem to have stalled.
Look into convox instead.

18. MONDAY MEETING
IN CHARGE:ECS - the not so nice parts
Tooling Support Is Lacking
Trust no one except aws CLI latest version.
Not even CloudFormation.
Don't even think of trusting ansible.

19. MONDAY MEETING
IN CHARGE:ECS - the not so nice parts
It Is Moving Fast...
… because it has to.
Be prepared to refactor!

20. MONDAY MEETING
IN CHARGE:ECS the awesome parts!
Get Married to AWS.
And Benefit from Your Partner's
Superpowers!
High availability - Scalability -
no-downtime deploy - central logging -
monitoring

21. MONDAY MEETING
IN CHARGE:ECS the awesome parts!
Unified Deploy Pipeline
No matter how many services you have and what frameworks you
use!
→ Github: Source Code + Dockerfile + docker-compose.yml
→ Jenkins: Build images and pushes to Registry (ECR)
→ Jenkins + ansible: converts docker-compose.yml to
taskdefinition.json, adds servicedefinition.json,
triggers ECS
→ AWS ECS: redeploy service

22. MONDAY MEETING
IN CHARGE:ECS the awesome parts!
Handle Deploy Errors with Ease
Because you can easily rollback to a previous release.
(Once your tooling agrees on that …)

23. MONDAY MEETING
IN CHARGE:ECS the awesome parts!
It Is Moving Fast...
Application Load Balancers!
Image registry in Frankfurt!
Memory limits improvements!

24. MONDAY MEETING
IN CHARGE:
… to Remember
4/5

25. MONDAY MEETING
IN CHARGE:Learnings
Docker Is Simple to Get Into.
But...
… it brings its own technology to master:
Caching - file system - networking, especially DNS
Docker still evolves. And it does not like cron jobs.

26. MONDAY MEETING
IN CHARGE:Learnings
Say Hello to Demons from the Past
While transforming servers to services expects some surprises.
Hard-coded paths. Tight coupling. Undocumented code.

27. MONDAY MEETING
IN CHARGE:Learnings
Be Prepared to Sail in Uncharted
Waters
Docker unit testing handling.
Tags and namespaces for prod/stage/dev environments.

28. MONDAY MEETING
IN CHARGE:Learnings
Ramp Up Your Educational Skills
Lots of new concepts and tools for your team to master.
Lots of debugging concepts to master.

29. MONDAY MEETING
IN CHARGE:
Profit!
5/5

30. MONDAY MEETING
IN CHARGE:Profit!
Clear responsibilities
Each team now manages its own services.

31. MONDAY MEETING
IN CHARGE:Profit!
Unified Deploy Pipeline
Less confusion about how to deploy.
Clear separation between develop and operation.

32. MONDAY MEETING
IN CHARGE:Profit!
Stable Deployments - Happy Team
No more network errors during deployment.
No more version dependencies.
No more role dependencies.
Deployments fail for different reasons now ...

33. MONDAY MEETING
IN CHARGE:Profit??
Open Questions
Will it scale?
What about memory usage in production environments?
Will it reduce server costs?

34. MONDAY MEETING
IN CHARGE:Conclusion
ECS is worth it. But you have to invest in it. And
you need someone who thoroughly understands
all of it.
Team will love it - after some time.
Introduce it ideally when your team grows.
Technology has matured. Use it!

35. MONDAY MEETING
IN CHARGE:About Me
Head of Technical Quality Assurance @styla
devOps engineer at night
IT counselor during weekends
Past
PhD in Computer Science Networking
Built my own start-up hoersport.de
sven@styla.com @sven_ehlert

36. MONDAY MEETING
IN CHARGE:
THANK YOU FOR YOUR
TIME AND ATTENTION