- The document discusses using AWS to build a large-scale, Dockerized microservices architecture for moovel Group GmbH's mobility app. - It outlines moovel's goals of simplifying mobility now and solving future challenges through a mobility-as-a-service platform. - The architecture uses AWS services like ECS, SQS, and Kinesis along with Docker containers, microservices, and an API gateway to achieve scalability, independent development, and minimal operational effort across teams.

1. Using AWS to Build a Large Scale
Dockerized Microservices Architecture
Dr. Oliver Wahlen
moovel Group GmbH
Frankfurt, 30. Juni 2016

2. The moovel Group GmbH
Our vision is an ecosystem that
• simplifies mobility now
• solves the mobility challenges of the future

3. Mobility as a Service
Urban areas will become more crowded:
• Today 54% of people live in cities  In 2050 there will be 66%
• More people on less space
• Driving and parking will become more regulated
• High demand for new transportation concepts
Autonomous driving will become reality:
• Taxi and car sharing business will change completely
• Cost efficiency similar to public transport but faster
• Owning a car becomes less relevant
The future will be Mobility as a Service

4. Entwicklung 3: Autonomes Fahren
Wir verändern die Art,
wie wir uns fortbewegen.
We are building apps, widgets and services that integrate
existing transportation providers
Benefits for the user:
• Only one registration: no need to remember passwords
or provide payment data several times
• Transparency on mobility alternatives
• Intermodal routing
Benefits for the partners:
• Get new users from other mobility providers
• Get new users from different geographic regions
moovel Technology

5. Entwicklung 3: Autonomes Fahren
Wir verändern die Art,
wie wir uns fortbewegen.
• Scalability up to millions of users
• Future prove: Easy to modify components and update technologies
• Usable on global scale
• Minimal operational effort
• Teams can develop independently
Design Goals of Our System Architecture

6. Entwicklung 3: Autonomes Fahren
Wir verändern die Art,
wie wir uns fortbewegen.
Avoid Vendor Lock-In vs. Reduced Operational Effort
Services currently
used by moovel

7. Entwicklung 3: Autonomes FahrenWhat and Why Docker?
Docker Containers
• Provide application(s) with a virtual operating system
• Isolate applications from the host OS and other docker containers
(e.g. Files, Network)
• Are self contained  run on any machine without other installs
• Are simple to build  images reference a base image
Docker Registry
• Stores docker images produced by the build system
• Provides access control and maintenance of tags
• Dockerhub (Docker Inc.) vs. Amazon EC2 Container Registry

8. Entwicklung 3: Autonomes FahrenWhat and Why EC2 Container Service (ECS)?
ECS is a highly scalable, fast, container management service that makes
it easy to run, stop, and manage Docker containers on a cluster of
Amazon EC2 instances.
– AWS documentation
Technology similar to Kubernetes on Google Container Engine
but tailored for AWS and fully managed
Terminology
• ECS cluster: Group of EC2 instances that allow docker execution
• ECS container instance: EC2 instance with ECS container agent
• ECS task: The running docker container on an instance
• ECS task definition: docker image, CPU, mem, port requirements, …
• ECS service: number of parallel tasks, scalability, health checks, …

9. Entwicklung 3: Autonomes FahrenWhat and Why Microservices?
http://martinfowler.com/articles/microservices.html
• Services independently deployable
 Continuous Deployment
• Scalable on service level
 Specific to service needs:
CPU, memory
• Services have firm boundary
allows for:
 different languages per service
 different teams per service group
 evolution of logic and technology
ECS Cluster
EC2 Instance
Docker Task
= ECS Task
Web Service

10. Entwicklung 3: Autonomes FahrenConway’s Law And Microservices: A Good Fit
Organizations which design
systems [...] are constrained to
produce designs which are copies
of the communication structures
of these organizations
– Melvin Conway 1967
Product Owner
App-Developers
Backend-Devs
Search
Squad
Book
Squad
User
Squad
Pay
Squad
moovel product dev is organized in squads (feature teams):
• with a long term business vision
• ability to build and modularize mobile apps / componets
• build their own set of microservices
• choose the technology that best fits their needs

11. Entwicklung 3: Autonomes FahrenContinuous Integration with Docker and AWS
Developer
code push to
SaaS Build SystemVersion Control
triggers dockerfile.tgz
S3 Bucket
Build Path
SaaS Build System Dockerhub
Deployment triggers:
develop branch: automatic
master branch: manually
Deployment Path
docker image
push to
start update
Cloudformation
Elastic Container
Services: dev/prod
update docker

12. Entwicklung 3: Autonomes Fahrenmoovel System Architecture
moovel VPC
Client App
External ELB
• TLS termination
Kong API Gateway
• reverse proxying
• LUA plugins
Consul
• service discovery
• client load balancing
• service configuration
synchronous REST call
ECS microservice
SNS
Topic
SQS
Queue
asynchronously
publish changes
logging Kinesis ELK Stack

13. Entwicklung 3: Autonomes FahrenLimitations of Existing Solution
• Difficult to realize dynamic port allocation
 Currently each service has a fixed port
• Two levels of scaling introduce additional complexity:
 Cluster nodes = No. of EC2 instances
 ECS Tasks = No. of running Docker containers for a service
 had to introduced a custom metric taskless instance count
• Service discovery with Consul:
 Automates node and service registry
 Provides hierarchical key value store with ACL for configuration
 Consul is a very critical component
 Must be operated and maintained
• Kong API gateway:
 Extremely flexible and powerful nginx based reverse proxy
 Requires cassandra DB
 Must be operated and maintained
• Elasticsearch Logstash Kibana (ELK):
 Do not use it for long term data storage!

14. Entwicklung 3: Autonomes FahrenCheck-up of Design Goals
Scalability
 moovel runs 80 services on 21 ECS clusters (dev+prod)
Future prove
 Exchanging microservice technology is easy
 We have microservices in Java, Node.js, Python, Kotlin, …
Global Scale
 The world is currently operated from eu-west-1
 Pub/Sub is good starting point for multi region deployment
Minimal Operational Effort
 Technology is operated by 10 squads
 Currently no DevOps team needed: squads build it and run
 Deployments are easy, frequently done and without downtime
Teams can develop independently
 Technology is operated by 10 squads
 Microservices, REST and Pub/Sub are a good basis
 UI decomposition in app is difficult: modularization ongoing

15. Thank You