The document presents an overview of cloud development with Mantl, detailing various components including Apache Mesos, Marathon, Cassandra, Spark, Docker, and Consul. It explains how Mesos abstracts hardware resources to build scalable and fault-tolerant applications, while Marathon manages long-running applications within this environment. Additionally, the document highlights the integration of these technologies for big data processing, application deployment, and system management.

1. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
We’re ready. Are you?

2. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Introducing Cloud
Development with Mantl
Brian Hicks (Mantl Core Committer)
Ryan Eschinger (Mantl Core Committer)

3. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
• Introduction
• Mesos
• Frameworks
• Docker
• Consul
• Mantl
Agenda
3

4. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Introduction
4
Brian Hicks Ryan Eschinger

5. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Mesos
What is it?
Program against your
datacenter like it’s a single
pool of resources
Apache Mesos abstracts CPU, memory, storage,
and other compute resources away from machines
(physical or virtual), enabling fault-tolerant and
elastic distributed systems to easily be built and
run effectively.
5

6. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Mesos
2-Tier Scheduler
Frameworks:
• Receive resources offered by the
master nodes
• Schedule work on offered
resources
• Are notified if workloads fail via
internal and external mechanisms
6
Mesos:
• Determines resources available
on nodes in the cluster
• Offers those resources to
frameworks
• Keep track of node status (online,
offline, remaining resources)

7. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Mesos
Architecture
Slaves (Agents in 1.0):
• Tell the masters which resources
are available
• Run tasks on claimed resources
7
Masters:
• Communicate with Schedulers
• Keep track of Agents
• Provide a UI for Ops / DevOps

8. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Mesos
Architecture
Executor:
• A binary or script that runs on the
slave to perform work
• Runs in an isolated environment
(can use Docker)
8
Framework (Scheduler):
• Schedules work based on offers
received
• Domain specific

9. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Framework Demos
9

10. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Marathon
What is it?
1
0
A cluster-wide init and
control system for services
in cgroups and Docker
containers
Marathon is an Apache Mesos framework for long-
running applications. Given that you have Mesos
running as the kernel for your datacenter, Marathon
is the init or upstart daemon.

11. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Marathon
Overview
1
1
• Developed by Mesosphere
• Controls app resources, ports,
environment variables, and
scaling
• Runs “long-running” tasks

12. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Cassandra
What is it?
Distributed database
designed for scalability and
high availability
Apache Cassandra is an open source distributed
database management system designed to handle
large amounts of data across many commodity
servers, providing high availability with no single
point of failure.
1
2

13. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Cassandra
Overview
1
3
• Designed to handle big data workloads across multiple nodes (ring) with no
single point of failure
• Data is distributed among all nodes in the cluster
• Uses the Gossip protocol for peer-to-peer communication
• Eventually consistent (C): prioritizes Availability and Partitioning tolerance
(AP)

14. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Cassandra
Mesos Framework
1
4
• Automates bootstrapping and operations of Cassandra
clusters
• Bootstraps initial seed nodes and expands cluster to
desired number of nodes
• Runs periodic operational tasks
• Restart, remove, and replace nodes on failure
• Self-contained: does not require specialized software
installed on agent nodes

15. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Spark
What is it?
A fast and general engine for
large-scale data processing
Apache Spark is an open source, high performance
big data processing framework. It provides high-level
APIs in Java, Scala, Python, and R. Spark includes
higher-level libraries, including support for SQL
queries, streaming data, machine learning and graph
processing.
1
5

16. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Spark
Overview
1
6
• Originally developed at UC Berkeley in 2009
• Up to 100x faster than Hadoop for large scale data processing
• Execution engine works both in-memory and on-disk
• Architecture
• Data storage: supports Hadoop-compatible data sources like HDFS,
HBase, Cassandra, etc.
• API: Scala, Java, Python, and R.
• Resource Management: standalone, Mesos, Yarn

17. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Spark
on Mesos
1
7
• Runs as a framework
• Supports the Docker containerizer
• 2-level scheduling: Spark can make decisions about the
offers it receives from Mesos
• Enables sophisticated scheduling scenarios

18. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Spark
Run Modes
1
8
• Coarse-grained
• Runs long-running Spark executors on every node
• Fast startup, better for interactive sessions, but can
be inefficient
• Fine Grained
• Launches a Spark executor per task
• Slower startup but can utilize resources better

19. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Docker
What is it?
A shipping container
system for applications
A container virtualization platform that abstracts
underlying Linux resource isolation technologies like
cgroups and kernel namespaces. Allows developers
to build, package, and ship applications that can run
anywhere*.
*anywhere that runs a 3.10+ Linux kernel
1
9

20. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Docker
Overview
2
0
• Compared to virtual machines, containers are smaller, lighter weight, more
portable, and easier to deploy
• Package an application, along with all of its dependencies, in a single
artifact
• Use the same artifact throughout development, testing, and production
• Reduces concerns about the compatibility in different environments

21. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Consul
2
1

22. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Consul
Discovery (HTTP)
2
2
curl localhost:8500/v1/catalog/service/marathon
[{
"Address": "10.0.113.214",
"Node": "mantl-control-01",
"ServiceAddress": "",
"ServiceID": "marathon",
"ServiceName": "marathon",
"ServicePort": 18080,
"ServiceTags": ["marathon"]
}]

23. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Consul
Discovery (DNS)
2
3
dig +short marathon.service.consul
10.0.113.214
10.0.220.54
10.0.131.25
dig +short marathon.service.consul SRV
1 1 18080 mantl-control-02.node.mantl.consul
1 1 18080 mantl-control-01.node.mantl.consul
1 1 18080 mantl-control-03.node.mantl-consul

24. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Consul
Key/Value Store
2
4
curl -X PUT -d world http://localhost:8500/v1/kv/hello
true

25. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Consul
Key/Value Store
2
5
curl -X GET http://localhost:8500/v1/kv/hello
[{
"CreateIndex": 223,
"Flags": 0,
"Key": "hello",
"LockIndex": 0,
"ModifyIndex": 223,
"Value": "d29ybGQ="
}]

26. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Consul
consul-template
2
6
global
maxconn {{or (key "service/haproxy/maxconn") 256}}
debug
{{range services}}
frontend {{.Name}}
bind *:{{key (printf "ports/%s" .Name)}}
{{range service .Name}}
server {{.Node}} {{.Address}}:{{.Port}}{{end}}
{{end}}

27. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Mantl
2
7

28. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID
Thank you
© 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID 28

29. © 2015 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation ID