This document discusses ways to use Apache Mesos, including: 1) Running frameworks directly on Mesos by developing a scheduler or containerizing components and using Marathon. 2) Building platform as a service (PaaS) using Mesos, Marathon, Docker, Jenkins, Gitlab/GitHub, and metrics for monitoring. 3) Implementing continuous integration/continuous delivery (CI/CD) pipelines based on Mesos.

1. what can be done
around Mesos
weitao zhou@DataMan Inc.
www.shurenyun.com

2. Overview
• Frameworks on Mesos
• PaaS Based on Mesos
• CI/CD Based on Mesos
• Big Data Based on Mesos

3. Frameworks on Mesos
http://mesos.apache.org/documentation/latest/framework
s/

4. ways of running on Mesos
• 3 ways, while every coin has two sides
✓ develop scheduler running on Mesos
✓ components containerized(options) +
marathon
✓ batch restful api

6. develop scheduler running
on Mesos
• Myriad: yarn-on-mesos

7. Components containerized
+ Marathon

8. PaaS
• Platform as a Service
• You can focus on your application, not the
infrastructure
• We can focus on making the platform resilient
and self-healing

9. PaaS
• marathon
• mesos
• private docker registry
• Jenkins
• gitlab/github
• metrics! metrics! metrics!
• log&debug

10. Marathon
• PaaS on Mesos
• init.d for the cluster
• Docker support
• Scales at the click of a button
• roll-back
• Blue-Green Deployment
• RESTful API
• Manages edge routes －
HAproxy

11. Marathon
• health check, instead of supervisord
• constraints
• resources
• force pull image
• zookeeper is the bottleneck
• docker images warm-up or dfs

12. Private docker registry
• Storage Strategy
• data disk: size planning, inode
• storage driver: device mapper, aufs
• Auth
• authorized push, allowed pull
• Frontend

13. docker
• docker build image
• base image + wget executable program: run-
time
• tag image by code release version
• exited containers GC

14. Service Discovery
• Mesos-DNS(actively developing)
• one mesos-dns one slave
• stateless, replica
• RESTful API
• HAproxy/Bamboo
• 1 container 1 IP
• IPAM
• mesos container, instead docker container
• docker-Weave integration
• network isolation IS NOT Service Discovery
• host or bridge
• performance vs security

15. Load Balance
• Mesos-DNS
• round robin
• dynamic load
• HAproxy/Bamboo
• ip/source hash
• round robin
• least conn
• dynamic load

16. Schedule
• Docker-Swarm
• constraint: storage==ssd io
• affinity: container or image affinity
• volume-from, link, net: container
• strategy: bin-pack, spread, random
• Mesos
• resource: cpus:24;mem:24576;disk:409600;ports:[21000-24000,30000-
34000];bugs(debug_role):{a,b,c}'
• attributes: 'rack:abc;zone:west;os:centos5;level:10;keys:[1000-1500]'

17. Metrics&Auto-Scaling
• Metrics
• host
• docker
• app

18. Metrics&Auto-Scaling
• the trigger depends
• cpu/mem metrics
• app request rate
• app force/load
• how
• IaaS
• PaaS
• where

19. Log&Debug
• client
• stdout/stderr VS. /var/log/app/*.log
• logspout/heka VS. logstash file input
• Server
• ES
• frontend
• Kibana

20. Multiple Roles
• DRF
• Static Reservation VS Dynamic Reservation

21. Q&A