Downloaded 1,016 times
![“Divide et impera [et sarcina]”
“Divide and rule [and package]”
—Philip II of Macedon, paraphrased](https://image.slidesharecdn.com/dockerconeualexmesosphere-141210192612-conversion-gate01/85/Building-Web-Scale-Apps-with-Docker-and-Mesos-by-Alex-Rukletsov-Mesosphere-7-320.jpg)
![Native Docker support in Mesos
message DockerInfo {
required string image = 1;
// Network options.
enum Network {
HOST = 1;
BRIDGE = 2;
NONE = 3;
}
message PortMapping {
required uint32 host_port = 1;
required uint32 container_port = 2;
optional string protocol = 3; // Protocol to expose (ie: tcp, udp).
}
optional Network network = 2 [default = HOST];
repeated PortMapping port_mappings = 3;
optional bool privileged = 4 [default = false];
// Allowing arbitrary parameters to be passed to docker CLI.
repeated Parameter parameters = 5;
}](https://image.slidesharecdn.com/dockerconeualexmesosphere-141210192612-conversion-gate01/85/Building-Web-Scale-Apps-with-Docker-and-Mesos-by-Alex-Rukletsov-Mesosphere-20-320.jpg)
![Native Docker support in Marathon
// nginx-task.json
{
"container": {
"type": "DOCKER",
"docker": {
"image": "nginx",
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 80,
"hostPort": 0,
"servicePort": 80,
"protocol": "tcp" }
]
}
},
"id": "nginx",
"instances": "1",
"cpus": "0.25",
"mem": "256",
"uris": []
}
$ cat nginx-task.json | http http://dev1.mesosphere.com:8080/v2/apps](https://image.slidesharecdn.com/dockerconeualexmesosphere-141210192612-conversion-gate01/85/Building-Web-Scale-Apps-with-Docker-and-Mesos-by-Alex-Rukletsov-Mesosphere-22-320.jpg)
Uploaded byDocker, Inc.
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
The document discusses building web-scale applications using Docker and Apache Mesos, emphasizing the evolution from client-server to cloud applications. It highlights Mesos as a distributed operating system kernel capable of managing containerized applications, including Docker, with features like multi-tenancy and advanced resource scheduling. Additionally, it outlines various cluster configurations, showcasing Mesos’ scalability and flexibility for both large and small deployments.
More Related Content
Similar to Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
- 1. Building Web ScaleApps with Docker and Mesos Alexander Rukletsov Software Engineer @ Mesosphere
- 2. Why should wecare?
- 3. Applications in theCloud Era App App App App Client-Server Era: Small apps, big servers App Serv Serv Serv Serv Cloud Era: Big apps, small servers Virtualization Aggregation Server
- 4. Major Components ●Hardware ● Orchestration, deployment and isolation ● Cluster and resource management ● Scale-aware applications, service discovery, etc.
- 5. Major Components ●Hardware ● Orchestration, deployment and isolation ● Cluster and resource management ● Scale-aware applications, service discovery
- 6. Applications in theCloud Era App App App App Client-Server Era: Small apps, big servers App Serv Serv Serv Serv Cloud Era: Big apps, small servers Virtualization Aggregation Server
- 7. “Divide et impera[et sarcina]” “Divide and rule [and package]” —Philip II of Macedon, paraphrased
- 8. Containers ● LightweightLinux execution environment ● Static application composition ● Reliable deployment ● Unit of resource isolation ● Execution isolation ● Multi-tenancy without heavyweight VMs
- 9. Docker ● Opensource ● Configurable layers ● Reproducible ● Version-controlled ● Plenty of other people’s containers
- 10. Docker ● Opensource ● Configurable layers ● Reproducible ● Version-controlled ● Plenty of other people’s containers ● First-class citizen in Mesos and the Mesosphere stack ● Kubernetes employs and promotes Docker
- 11. © Gerard Julien/AFP Run everything in containers!
- 12. What about containermanagement?
- 13.
- 14.
- 15. Apache Mesos facts ● Created in 2009 at UC Berkeley, hardened in Twitter ● Top-level Apache project ● Mesosphere, Twitter, and Airbnb are major users / contributors ● Scales to 10 000s of nodes, production grade ● Packages and support through Mesosphere ● Google officially endorsed Mesos for Kubernetes ● Built-in containerization, including Docker
- 16. Mesos as aDistributed OS kernel ● Two level resource scheduling ● Launch tasks across the cluster ● Communication between tasks (like IPC) ● APIs for building “native” applications (aka frameworks): program against the datacenter ● APIs in C++, Python, JVM-languages, Go and counting ● Pluggable CPU, memory, IO isolation ● Multi-tenant workloads ● Failure detection ● Easy failover and HA
- 17. How Mesos works(HA mode) Application Scheduler Zookeeper Mesos MasteMr esos Master Mesos Master Mesos Master Mesos Slave Executor Task Task Executor Framework Task Task Task Executor Mesos Slave
- 18. Manage containers withMesos! © ect.nl
- 19.
- 20. Native Docker supportin Mesos message DockerInfo { required string image = 1; // Network options. enum Network { HOST = 1; BRIDGE = 2; NONE = 3; } message PortMapping { required uint32 host_port = 1; required uint32 container_port = 2; optional string protocol = 3; // Protocol to expose (ie: tcp, udp). } optional Network network = 2 [default = HOST]; repeated PortMapping port_mappings = 3; optional bool privileged = 4 [default = false]; // Allowing arbitrary parameters to be passed to docker CLI. repeated Parameter parameters = 5; }
- 21. Native Docker supportin Mesos CommandInfo command; command.set_value("dd if=/dev/zero of=/dev/null"); ContainerInfo::DockerInfo dockerInfo; dockerInfo.set_image("mesosphere/inky"); ContainerInfo containerInfo; containerInfo.set_type(ContainerInfo::DOCKER); containerInfo.mutable_docker()->CopyFrom(dockerInfo); TaskInfo task; task.set_name(""); task.mutable_task_id()->set_value("1"); task.mutable_slave_id()->CopyFrom(offer.slave_id()); task.mutable_resources()->CopyFrom(offer.resource()); task.mutable_command()->CopyFrom(command); task.mutable_container()->CopyFrom(containerInfo); vector<TaskInfo> tasks; tasks.push_back(task); driver.launchTasks(offer.id(), tasks);
- 22. Native Docker supportin Marathon // nginx-task.json { "container": { "type": "DOCKER", "docker": { "image": "nginx", "network": "BRIDGE", "portMappings": [ { "containerPort": 80, "hostPort": 0, "servicePort": 80, "protocol": "tcp" } ] } }, "id": "nginx", "instances": "1", "cpus": "0.25", "mem": "256", "uris": [] } $ cat nginx-task.json | http http://dev1.mesosphere.com:8080/v2/apps
- 23. Future Docker SwarmAPI support App App App Mesos Framework Mesos App Serv Docker Docker Swarm API Docker App Docker Docker Docker App Docker App Docker Serv Serv Serv Serv
- 24. What Mesos contributes ● Multi-framework: weighted fair sharing, roles, etc. ● Run Docker containers alongside other popular frameworks (e.g. Spark, Rails, Hadoop, …) ● Run services and batch apps in the same cluster ● Advanced scheduling: resources, constraints, global view ● High resource availability, cluster self-healing ● Proven at scale, battle-tested in production ● GUI / CLI cluster management console
- 25. Ways of runningDockers ● Marathon easy to setup, reliable orchestration ● Multiple Marathons benefit from two level scheduling, reservations, framework roles ● Custom framework fine-grained management
- 26. “L’homme est libreau moment qu’il veut l’être” “Man is free at the instant he wants to be” —Voltaire
- 27. Cluster configuration example#1 Services Mesos App App AWS, DigitalOcean, GCE Apps API Kernel Hardware Docker Docker Services REST API “Marathon” App Docker App App Docker App
- 28. Cluster configuration example#2 Mesos App App AWS, DigitalOcean, GCE Apps API Kernel Hardware Services Services REST API “Marathon” App App Kubernetes Docker Docker Docker App Docker App
- 29. Cluster configuration example#3 Docker Services REST API “Marathon” (init) Serv Serv Serv Serv Mesos Spark, MPI, Hadoop, Storm Mesos SDK App Java, Python, C++, Go Serv Serv Serv Serv Batch REST API “Chronos” (cron) Serv App Recurring Jobs (ETL, backups) Hardware Native Long running Batch Apps API Kernel App Docker
- 30. Cluster configuration example#4 Docker AWS, DO, GCE Mesos Spark, MPI, Hadoop, Storm Mesos SDK App Java, Python, C++, Go Serv Serv Services REST API “Marathon” (init) Batch REST API “Chronos” (cron) Serv App Recurring Jobs (ETL, backups) Hardware Native Long running Batch Apps API Kernel App Docker Serv
- 31. Mesos is notonly for “big” players! © Reuters
- 32. Tiny clusters foreverybody ● Mesosphere on GCE ● Mesosphere on DigitalOcean ● Mesosphere on AWS ● Portable Mesosphere on a USB stick
- 33.
- 34.
- 35.
- 36.
- 37. Mesos-on-Mesosphere Task TaskTask Zookeeper Mesos Marathon Slave Mesos Master Mesos MasMteersos MasteMresos Master Mesos Slave ... Task
- 38. Mesos-on-Mesosphere Mesos Slave Mesos Slave Docker Docker Docker Docker Zookeeper Mesos Marathon Slave Mesos Master Mesos MasMteersos MasteMresos Master Mesos Slave ... Mesos Master Docker Mesos Slave Docker Mesos Slave Mesos Master
- 39. Summary ● Completestack for large (and small) distributed apps ● Multi-tenancy ● Resource optimizations ● Easy to deploy ● No vendor lock-in
- 40. Thank You. alex@mesosphere.io #mesos on irc.freenode.net