Downloaded 34 times
Uploaded byDatio Big Data
DC/OS: The definitive platform for modern apps
DC/OS is an open source platform that provides container orchestration and management using Mesos. It allows running applications and services across data center infrastructure including bare metal, VMs, and cloud. DC/OS provides services like Marathon for container orchestration, security, monitoring, load balancing and service discovery. It has features like high resource utilization, mixed workload support, elastic scalability, high availability and zero downtime upgrades.
More Related Content
Similar to DC/OS: The definitive platform for modern apps
DC/OS: The definitive platform for modern apps
- 1.
- 2. Contents Introduction to Mesos1 2 3 MesosArchitecture Mesos Ecosystem 4 DC/OS
- 3. 1. Introduction to Mesos Let’sstart with the first set of slides
- 4. ✓ Modern applicationsare highly depend on data ✓ Rise of the data center: Clusters of commodity servers have become a major computing framework Datacenter hardware on-premise physical/virtual machines public/private cloud instances
- 5. Different frameworks for cluster computing, data integrationor storage are as important as the hardware they run on Stream Processing Batch Processing Data Storage Data Integration
- 6. Platform for sharingcommodity clusters between multiple and diverse computing frameworks Frameworks Datacenter Hardware What is mesos?
- 7.
- 8. Mesos Master Main Components MesosAgent Framework Executor Task 1 Task 2 Framework Scheduler Job 1 Job 2 Framework Scheduler Job 1 Job 2 Framework Scheduler Job 1 Job 2 Mesos Agent Framework Executor Task 1 Task 2 Mesos Agent Framework Executor Task 1 Task 2 Features Scalability → 50k nodes Efficiency → Resource Allocation Resource Isolation → Containers High Availabilty → Fault Tolerance Flexibility → Extensible r
- 9. Mesos Master Resource Allocation Task2 Job 2 Job 2 Framework Scheduler Job 1 Job 2 Task 1 Task 2 Mesos Agent 1 Framework Executor Task 1 Task 2 <slave1, 4cpu, 4 gb, ..> <slave1, 4cpu, 4 gb, ..> <task1, agent1, 2 cpu, 1 gb, ..> <task2, agent1, 1 cpu, 2 gb, ..> <framework A, task1, agent1, 2 cpu, 1 gb, ..> <framework Atask2, agent1, 1 cpu, 2 gb, ..> Allocation Module Fine-Grained Resource Sharing Mesos introduces a distributed two- level scheduling mechanism called resource offers Mesos decides HOW many resources to offer each framework, while frameworks decide WHICH resources to accept and WHICH computations to run on them
- 10. Mesos Agent Mesos Containerizer Mesos Master Framework Executor Task1 Task 2 Job 2 Job 2 Framework Scheduler Job 1 Job 2 ContainerContainer Containers limit the amount of resources that the process can access Isolation + scheduling → service level agreements (SLAs) Mesos Agent Docker Containerizer ContainerContainer Mesos Containerizer Linux Cgroups to provide isolation for CPU, memory, I/O and file system Docker Containerizer High-level API to run a lightweight Docker container in a portable manner Resource Isolation
- 11. Mesos Agent Framework Executor Task 1 FaultTolerance Task 2 Mesos Agent Framework Executor Task 1 Task 2 Mesos Agent Framework Executor Task 1 Task 2 Framework Scheduler Job 1 Job 2 Framework Scheduler Job 1 Job 2 Framework Scheduler Job 1 Job 2 Mesos Master Standby Mesos Master Standby Zookeeper quorum Agent Fails Master notices and informs the frameworks. They can choose to reschedule task in other healthy agents Framework Fails Framework can implement its own failover mechanism Master Fails Leader election Mesos Master Soft state Active agents Active fws Running tasks
- 12.
- 13. Frameworks Mesos Ecosystem Data Center Mesos Components Auxiliary Services Auxiliary Services ❖ Deployment ❖Packaging ❖ Configuration ❖ Service Discovery ❖ Load Balancing ❖ Monitoring ❖ Orchestration
- 14. Which instance shoulda given request to go? Load Balancing Packaging The deployment has to make sure that its dependencies (libs, other apps) are met while deploying Where are the instances of a particular service running? Service Discovery Orchestration Long-running services and cron jobs Deployment Monitoring Configuration ❖ Framework to manage Frameworks ❖ Orchestrates apps and frameworks ❖ It simplifies launching your distributed (or not) applications in Mesos without needing to develop a custom framework for your application ❖ Can run in HA using ZK ❖ It will monitor your app checking if it is healthy
- 15.
- 16. Data Center Infrastructure Mesos Services Apps Introduction Container & Bigdata Open source platform based on Mesos Ecosystem Insfrastructure Agnostic
- 17. Services Apps Data Center Infrastructure BareMetal Virtual Machines Cloud Mesos Security User Interface (CLI + GUI) Container Orchestration Monitoring Components SQL/NoSQLStream/Batch Processing Microservices Mesos Ecosystem Stack DC/OS Stack Cosmos Package Manager Admin Router Nginx Metronome Scheduler Universe Repository Minuteman Load balancer Exhibitor Zookeeper Mgmt
- 18. Zookeeper Mesos Master Mesos DNS Marathon AdminRouter Mesos Agent Process Mesos Agent Process DCOS Master (1..3) DCOS Private Agent (0..n) DCOS Public Agent (0..n) Architecture Public Internet ❖ High Resource Utilization ❖ Mixed Workload Colocation ❖ Container Orchestration ❖ Extensible Resource Isolation ❖ Stateful Storage Support ❖ Public and Private Package Repositories ❖ Cloud-Agnostic Installer ❖ Web and Command Line Interfaces ❖ Elastic Scalability ❖ High Availability ❖ Zero Downtime Upgrades ❖ Integration-Tested Components ❖ Service Discovery and Load Balancing Features
- 19.
- 20.