Speaker: Michał Kobus & Bartłomiej Jakubowski Language: English Microservices architecture is well-known not free lunch. It allows high scalability for price in extra effort of specialized devops team. Not easy to develop, not easier in operations field, microservices demand reliable monitoring and infrastructure automation. In Syncano, we want to use right tools for right tasks, that's why we test and apply recent open-source projects in our product. In the presentation we will share the knowledge we gained while working with Ansible, Prometheus and Docker. Visit our website: http://2015.devopsdays.pl

1. AutomatingmicroservicesinSyncano
Bartek Jakubowski & Michał Kobus

2. Who?
Bartek
Senior Devops
Engineer with solid
sysadmin background
Michał
M.Tech in Mechatronics
Python developer
sysadmin padawan

3. Whatis ?
EngineeringCustomer
Growth
HEADS

4. WhatisSyncano?
➔ API builder
➔ data storage
➔ code execution
➔ sharing

5. WhatisSyncano?
➔ API builder
➔ data storage
➔ code execution
➔ sharing

6. WhatisSyncano?
➔ API builder
➔ data storage
➔ code execution
➔ sharing

7. WhatisSyncano?
➔ API builder
➔ data storage
➔ code execution
➔ sharing

8. Beginning
done:
➔ architecture
➔ EB deploy
➔ autoscaling
➔ load balancing
➔ CI
➔ logging
➔ MQ
no:
➔ monitoring
➔ metrics collected
➔ regular load tests

9. Microservices…que?
➔ small services
➔ separate processes
➔ communication
➔ business logic
➔ deployable separately
➔ decentralized management
http://martinfowler.com/microservices/

10. Whydowebuildmicroservices?
➔ it’s the future ;)
➔ partial deployment
➔ rollbacks
➔ availability
➔ scalability
➔ technology diversity
➔ 3rd-party services
http://blog.circleci.com/its-the-future/

11. Sweet…butnotfree
➔ complexity boost
➔ debugging...
➔ requires:
◆ fast provisioning
◆ fast deploy
◆ monitoring
◆ automation
◆ devops culture
http://martinfowler.com/articles/microservice-trade-offs.html

12. AgileinOPSworld(DevOpsculture)
➔ continuous integration
➔ git flow
◆ feature
◆ review
➔ testing
➔ configuration management

13. AgileinOPSworld(DevOpsculture)
➔ continuous integration
➔ git flow
➔ testing
◆ syntax
◆ result
➔ configuration management

14. AgileinOPSworld(DevOpsculture)
➔ continuous integration
➔ git flow
➔ testing
➔ configuration management
◆ documentation
◆ version control

15. TheWayofTheAutomation
1. Containerization Docker
2. Build CI server
3. Deploy AWS Elastic Beanstalk
4. Monitor Prometheus / Icinga2
5. Orchestrate Ansible
6. Test Runscope

16. TheWayofTheAutomation
Ansible mixed inventory design

17. dockerandtheContainers
➔ open platform to deliver apps faster
➔ run (almost) any app in securely
isolated container
➔ lightweight nature - getting more
from your hardware

18. dockerandtheContainers
Misconceptions about using docker
➔ one process peer docker container
➔ dockerize everything
➔ use docker to get speed and
consistency
➔ docker is a lightweight VM
➔ docker is widely used in production
➔ docker makes your app more secure

19. dockerandtheContainers
Containers!
➔ share resources at kernel level
➔ improve the speed and reliability of CD/CI
➔ orchestrating with zero downtime
➔ easy deployment rollbacks

20. dockerandtheContainers
container resources?
➔ run on linux
(kernel 2.6.32+)
➔ physical or virtual -
cloud or not
➔ four (to six) times more
server application
instances then
virtualization on the
same hardware

21. dockerandtheContainers
How to store?
➔ dockerhub registry and possible alternatives
➔ docker distribution (registry 2.0)
➔ distribution for private use

22. dockerandtheContainers
how to version? (layer FS)
➔ docker final image as a set of layers
➔ layers as building blocks for application stacks
➔ reusable images make builds much faster
➔ versioning and power of rollbacks

23. dockerandtheContainers
How to build?
➔ approaches to image building (interactive vs dockerfile)
➔ dockerfile and automated builds
➔ which approach is right?
➔ prototyping and troubleshooting

24. WhydoesDocker(sometimes)hurt?
1. unresponsive Docker daemon (AWS autoscaling)
2. non-trivial debugging
3. layers (download time)
4. leaves garbage
5. registry issues
6. tricky network isolation
7. Docker issues (concurrent downloads)

25. dockerandtheContainers
Best and worst of docker
The bright side
➔ deploy code faster & roll-
backs
➔ easy to orchestrate
➔ run apps anywhere
➔ build process speedup
➔ devops favorite
➔ software version pinning
➔ nice API
The dark side
➔ level of complexity (big envs)
➔ direction of the company
behind
➔ debug and troubleshooting hell
➔ networking step back

26. ContinuousIntegration(build)
1. push to feature branch
2. build app image
3. build environment
4. run tests

27. ContinuousIntegration(deploy)
1. merge to staging branch
2. build app image
3. push new app image to
registry
4. deploy app container to
staging
5. run checks on app
container

28. Autoscaling
➔ AWS elastic beanstalk apps
➔ Elastic Beanstalk solution stack with docker
➔ scalability, security and content delivery
➔ orchestrating environment configs - ebextensions
➔ custom metrics for autoscaling

29. Monitoring
Design

30. Monitoring
Prometheus - open-source monitoring and alerting toolkit

31. Monitoring
Why we use prometheus?
➔ multidimensional data
model
➔ flexible query language
➔ multiple graphing modes
and dashboard support
➔ docker and devops
friendly

32. Monitoringfor…Monitoring
Icinga2
➔ object based, rule driven
configuration format
➔ apply and assign
attributes
➔ runtime macros and
conditional behaviors
➔ dynamic notification
➔ scalability and devops
friendly
➔ good old nagios look :)

33. Monitoring
Integration point
➔ AWS SNS http/https
endpoint
➔ python code to update
configs with ansible
➔ slack notifications for
autoscaling groups
Dashboards
➔ promdash from prometheus
➔ dashing

34. TheFuture
➔ continuous delivery
➔ automated everything
➔ faster builds and deploys
➔ smarter Icinga checks
➔ more reliable Ansible playbooks testing
➔ load testing in pipeline
➔ provider independence
➔ moving to self-hosted services

35. Thanksforlistening!
Timeforgoodquestions
andniceAnswers