How Bridestory design and implement the Container-Based Microservice in AWS with aspects of deployment pipeline, logging and monitoring.

1. Container-Based Microservice Architecture
M A K I N G D R E A M W E D D I N G S P O S S I B L E
Docker ID Meetup - Jakarta, 28 March 2018

2. ADITYA HILMAN
M A K I N G D R E A M W E D D I N G S P O S S I B L E
ABOUT ME
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
PROFESSIONAL
● Linux System Consultant, Indolinux (2008 - 2011)
● Senior System Administrator, FirstWAP (2011-2014)
● System Architect Engineer, Bridestory (2014-now)
CONTACT
● Personal Email : aditya.hilman@gmail.com
● Business Email : aditya.hilman@bridestory.com
● Linkedin : http://bit.ly/adityahilman

3. WHAT IS
BRIDESTORY?
M A K I N G D R E A M W E D D I N G S P O S S I B L E
WHAT IS BRIDESTORY
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

4. OUR VISION
M A K I N G D R E A M W E D D I N G S P O S S I B L E
OUR VISION
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
TO HELP CREATE DREAM WEDDINGS POSSIBLE FOR
EVERYONE, EVERYWHERE

5. OUR MISSION
M A K I N G D R E A M W E D D I N G S P O S S I B L E
OUR MISSION
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
TO BECOME THE MOST POWERFUL TOOL FOR
BRIDES-TO-BE AROUND THE WORLD THAT ENABLES
THEM TO:
FIND MILLION OF
WEDDING
INSPIRATIONS
FIND THE BEST
WEDDING VENDORS
ANYWHERE
BOOK THEIR SERVICE
THROUGH OUR WEDDING
CONSULTANT SERVICE

6. BRIDESTORY IN
NUMBERS
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY IN NUMBERS
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
*according to Similar Web, since August 2015

7. BRIDESTORY PRODUCTS
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

8. BRIDESTORY PRODUCTS
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

9. BRIDESTORY PRODUCTS
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

10. BRIDESTORY PRODUCTS
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

11. BRIDESTORY PRODUCTS
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

12. CONTAINER-BASED
MICROSERVICE ARCHITECTURE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

13. DISCLAIMER
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Emphasis on infrastructure and deployment point of view, not on application or
software development point of view

14. MONOLITHIC
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Pros :
● Simple infra architecture
● Easy to troubleshoot
Cons :
● Slow iteration: small change requires full
deployment
● Difficult to isolate application services
● Difficult to deploy
● Difficult to scale
From infrastructure point of view

15. MONOLITHIC BIG PROBLEM
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
v1.0 v2.0 v2.1
Local Staging Production

16. DOCKERIZE ARCHITECTURE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Pros :
● Isolate
● Easy deployment
Cons :
● Difficult to scale

17. MONOLITHIC vs MICROSERVICE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

18. MONOLITHIC vs MICROSERVICE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

19. MICROSERVICE ARCHITECTURE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Pros :
● Easy to scale
● Infra cost effective (start with 1 instance)
Cons :
● More complex deployment pipeline
● Challenging in Logging, need to build
Centralized Logging
From infrastructure point of view

20. AWS ELASTIC CONTAINER SERVICE (ECS)
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

21. ECS COMPONENTS
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

22. IMPLEMENTATION
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
● URL Application Health Check
○ /microservice/healthcheck (Custom Url)
● Application Load Balancer
○ Distribute traffic to container
● Task Definition
○ Group of container
● Cluster
○ Group of services
● Service
○ Group of tasks

23. TASK DEFINITION
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
● Docker image for each container
● CPU and Memory requirement for each container
● Links between containers
● Networking and port settings
● Log settings

24. SERVICES
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
● Manage long-running workloads
● Monitor running tasks
● Restart tasks if they fail

25. Application Load Balancer-Based Service Discovery
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
● Service health checks
● Automatic registration/de-registration services
● Path and Host based routing

26. SECURING ECS CLUSTER
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

27. BRIDESTORY ARCHITECTURE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

28. ECS DEPLOYMENT
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

29. DEPLOYMENT PIPELINE
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
● Checkout Repo
● Build & Tag Docker Image
● Container Testing
● Push Docker Image to ECR
● Create New Task
● Update Service Cluster

30. JENKINS DEPLOYMENT WORKFLOW
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

31. JENKINS DEPLOYMENT WORKFLOW
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Stage : Build & Docker Image
sh 'docker build -f dockerfile-staging -t microservice-app:staging-${BUILD_NUMBER} .'
● Different dockerfile for staging and production
● Tag ${BUILD_NUMBER} => Jenkins Env Variable

32. JENKINS DEPLOYMENT WORKFLOW
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Stage : Container Testing
sh './container-testing.bash'
docker run --rm --name ms-staging -d -P microservice-app:staging-${BUILD_NUMBER}
MS_IP=`docker inspect --format '{{ .NetworkSettings.IPAddress }}' ms-staging`
sleep 30s
RESPONSE_CODE=`curl -I -X GET http://$MS_IP/v2/endpoint | head -n1 | awk '{ print $2 }'`
if [ $RESPONSE_CODE != "200" ]
then
echo "Tests Failed! - Return code $RESPONSE_CODE"
echo "Remove ms-staging container"
docker stop ms-staging
exit 1
else
echo "Tests Passed"
echo "Remove ms-staging container"
docker stop ms-staging
exit 0
fi

33. JENKINS DEPLOYMENT WORKFLOW
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Stage : Create New Task
sh 'sed -e "s:BUILD_NUMBER:${BUILD_NUMBER}:g"
task-template/staging/microservice-app.json > microservice-app-${BUILD_NUMBER}.json'
sh 'aws ecs register-task-definition --family microservice-staging-task --cli-input-json
file://microservice-app-${BUILD_NUMBER}.json'
● Tag ${BUILD_NUMBER} => Jenkins Env Variable

34. JENKINS DEPLOYMENT WORKFLOW
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Task Template
{
"containerDefinitions": [
{
"portMappings": [
{
"hostPort": 0,
"containerPort": 3000,
"protocol": "tcp"
}
],
"essential": true,
"name": "chat-staging-container",
"image": "xxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/chat-web:staging-BUILD_NUMBER",
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "cluster-staging",
"awslogs-region": "ap-southeast-1",
"awslogs-stream-prefix": "chat-staging"
}
},
"cpu": 0,
"memoryReservation": 128
}
],
"family": "chat-staging-task"
}

35. JENKINS DEPLOYMENT WORKFLOW
M A K I N G D R E A M W E D D I N G S P O S S I B L E© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.
Stage : Update Service Cluster
#!/bin/bash
TASK_REVISION=`aws ecs describe-task-definition --task-definition
microservice-staging-task | egrep "revision" | tr "/" " " | awk '{ print $2 }' | sed
's/,$//'`
DESIRED_COUNT=`aws ecs describe-services --services
arn:aws:ecs:ap-southeast-1:xxxxxxx:service/microservice-staging-service --cluster
arn:aws:ecs:ap-southeast-1:xxxxxxx:cluster/microservice-staging-cluster-private | egrep
"desiredCount" | tr "/" " " | awk '{print $2}' | sed 's/,$//' | head -n1`
if [ ${DESIRED_COUNT} = "0" ]; then
DESIRED_COUNT="1"
fi
aws ecs update-service --cluster
arn:aws:ecs:ap-southeast-1:xxxxxxx:cluster/microservice-staging-cluster-private
--service arn:aws:ecs:ap-southeast-1:xxxxxxx:service/microservice-staging-service
--task-definition microservice-staging-task:${TASK_REVISION} --desired-count
${DESIRED_COUNT}

36. MONITORING
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

37. MONITORING & ALERTING
● GRAFANA
● GOOGLE STACK DRIVER
● TELEGRAM
● EMAIL
CENTRALIZED LOGGING
● CLOUDWATCH LOGS
● ELK STACK
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

38. GRAFANA
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

39. TELEGRAM
M A K I N G D R E A M W E D D I N G S P O S S I B L E
BRIDESTORY - DOCKER ID MEETUP
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.

40. Q & A
M A K I N G D R E A M W E D D I N G S P O S S I B L E
Q & A
© 2018 BRIDESTORY. All content & materials in this presentation belongs to Bridestory and its partners.