Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Orchestration Tool Roundup -
Docker Swarm vs. Kubernetes, TerraForm vs.
TOSCA/Cloudify vs. Heat
Speakers..
Agenda
• Orchestration 101..
• Different approaches for orchestration
• Method of comparison
• Comparison
• Synergies
• Su...
Orchestration 101
Orchestration is a mean to Automate Manual Process
Orchestration 101
• Common Characteristics
– Use DSL to define blueprint
– Execute a process based on
input from the bluep...
Goals of this Exercise
Explore the
different
approaches to
orchestration
Infrastructure
Centric
Pure Play
Container
Centric
Method of Comparison
• Same Application Requirements
• Full Production Deployment
• Broken into three main groups
– Contai...
The Test Application
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Balancer
VM
N...
Orchestration Process - Setup
VM
VM VM
VM
VM
VM
Load Balancer
VM
VM
VM
VM
VM
VM
VM
VM
VM
Create network and
compute resour...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestration Process - Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-cfg
VM
Mongo-cfg
VM
Mongo-cfg
Load Bala...
Orchestrating in Production
• Monitoring and log collection
• Manual/Auto healing
• Manual/Auto scaling
• Maintenance:
– B...
Common Requirements
• Dependency management
•Reproducible
•Cloneable
• Recoverable
Series 1: Container Centric
Quick Overview of Docker Swarm
A Docker-native clustering system
• Use a pool of hosts through a single swarm
master endpo...
Swarm Architecture
Solution Overview - Deploy - Create
Replica Sets
for i in 1..{number_of_replica_sets}
for j in 1..{number_of_nodes_for_rep...
Solution Overview - Deploy - Start
Node.js application containers
Make sure you inject all mongos endpoints for
the applic...
Solution Overview - Deploy -
Reconfigure HAProxy
Extract Node.js container IPs using docker
inspect and then:
for i in 1.....
Solution Overview - Mongodb scale
out
Identical to the process of deploying the initial
mongodb shards, mongodb will take ...
Docker Swarm - Pros and Cons
Pros
● Easy modeling
● Placement/Affinity
Cons
● Basic infrastructure
handling
● Manual handl...
Kubernetes
Quick Overview to Kubernetes
Container cluster manager
• Pods: tightly coupled group of containers
• Replication controlle...
Kubernetes Architecture
Sample Replication Controller
apiVersion: v1beta3
kind: ReplicationController
spec:
replicas: 5
selector:
name: mongod-rs1...
Sample Service Configuration
apiVersion: v1beta3
kind: Service
metadata:
labels:
type: nodejs
name: nodejs
spec:
ports:
- ...
Solution Overview - Deploy
• Create mongod config servers
for i in 1..3
kubectl create -f mongod-configsvr{i}-controller.y...
Solution Overview - Deploy - Create
Data nodes
for i in 1..{number_of_replica_sets}
kubectl create -f 
mongod-rs{i}-contro...
Solution Overview - Node.js Heal
Failing pods are identified by kubernetes and
are automatically rescheduled
Solution Overview - Node.js
continuous deployment
# initially configured with 0 replicas
kubectl create -f nodejs-v{new_ve...
Kubernetes - Pros and Cons
Pros
● (almost) zero configuration autoheal
● Out of the box load balancer
● Simple scaling
Con...
Series 2: Pure Play Orchestration
Introduction to Terraform
• By Hashicorp
• Simple (in a good way) command
line tool
– Resources
– Providers and provisione...
Sample Configuration
resource "openstack_compute_secgroup_v2" "nodejs_security_group" {
name = "nodejs_security_group"
des...
Sample Configuration
#
# Create a Network
#
resource "openstack_networking_network_v2" "tf_network" {
region = ""
name = "...
Sample Configuration
resource "openstack_compute_instance_v2" "mongod_host" {
count = "3"
region = ""
name = "mongod_host"...
Solution Overview
• Single top level configuration file
• Creates: Network, subnet, router, floating IP,
security groups, ...
Master Assignment & Registration of Shards
• Issue - no "cluster wide" way of invoking
provisioners
– Needed for configuri...
Terraform - Pros and Cons
Pros
● Infrastructure &
Framework neutrality
● Solid support for
OpenStack
● Simple and elegant
...
TOSCA / Cloudify
What is TOSCA?
TOSCA defines the
interoperable
description of
applications; including
their components,
relationships,
dep...
Cloudify – Open Source
Implementation of TOSCA
Provision
ConfigureMonitor
Manage
Infrastructure
Can be used as a
command l...
Cloudify – Open Source
Implementation of TOSCA
Provision
ConfigureMonitor
Manage
Monitoring &
Alarming
Infrastructure
Can ...
Hosted
On
Software
ComponentContainer
(Docker Runtime
Capability)
Containee
(Docker Runtime
Requirement)
Requirements
Capa...
Solution Overview
Mongod-shard
Mongo replica-
set
Output:
Mogoconfig hosts
Shards endpoint
Subsitutable
*Scalable *Scalabl...
Infrastructure setup
node_templates:
nodecellar_security_group:
type: cloudify.openstack.nodes.SecurityGroup
properties:
s...
Create Mongo Shards
mongodb:
type: tosca.nodes.mongodb.Shard
directives: [substitutable]
properties:
count: { get_input: s...
Create Compute Instances
mongo_server:
type: tosca.nodes.Compute
capabilities:
host:
properties: *host_capabilities
os:
pr...
Create MongoDB Replica Set
mongo_db_replica_set:
type: tosca.nodes.DBMS
requirements:
- host:
node: mongo_server
interface...
Creat NodeJS Containers
nodecellar_container:
type: tosca.nodes.NodeCellarAppContainer
properties:
port: { get_input: node...
Create Load Balancer
haproxy:
type: tosca.nodes.Proxy
properties:
frontend_port: 80
statistics_port: 9000
backend_app_port...
Handling Post Deployment through
Workflow & Policies
● Cloudify Workflows
● Built in workflows
o Install
o Uninstall
o Hea...
Summary TOSCA/Cloudify
Pros
● Infrastructure &
Framework neutrality
● Complete Life Cycle
Management
● Handles Infrastruct...
Series 3: Infrastructure Centric
• Overview of Heat
• Orchestrating NodeJS/MongoDB with Heat
• Summary – Benefits/ Limitat...
What is Heat?
Heat provides a
mechanism for
orchestrating
OpenStack resources
through the use of
modular templates.
Heat Architecture
Solution Overview
Mogo replica-
set
Output:
Replica set node hosts
ssh-key, private ip to the
init node
Input:
#Replica se...
Infrastructure setup
resources:
secgroup:
type: OS::Neutron::SecurityGroup
properties:
name:
str_replace:
template: mongod...
Create Compute Instances
mongo_host:
type: OS::Nova::Server
properties:
name:
str_replace:
template: $stackprefix-$stackst...
Create MongoDB Replica Servers
mongodb_peer_servers:
type: "OS::Heat::ResourceGroup"
properties:
count: { get_param: peer_...
Configure the Replica Servers
server_setup:
type: "OS::Heat::ChefSolo"
depends_on:
- mongodb_peer_servers
properties:
user...
Create NodeJS Container
nodestack_chef_run:
type: 'OS::Heat::ChefSolo'
depends_on: nodestack_node
properties:
...
node:
no...
Summary
Pros
● Native To OpenStack
● Built-in mapping of all
the OpenStack
infrastructure resource
types
Cons
● Limited to...
Potential Synergies
• Magnum -
Kubernetes + Docker,
Heat
• Cloudify/TOSCA +
Docker
• Cloudify/TOSCA +
Heat
Which orchestration tool should I
choose?
Final Words..
The Only Constant Is Change!
More Change Ahead..
Further Reading..
OpenStack Vancouver Session
Upcoming SlideShare
Loading in …5
×

Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs. tosca (1)

80,914 views

Published on

Video recording: https://www.youtube.com/watch?v=tGlIgUeoGz8

It’s no news that containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running container workloads. However, where it usually becomes more complex is that many times an application is often built out of multiple containers. What’s more, setting up a cluster of container images can be fairly cumbersome because you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host.




These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestation projects out there - from Heat to Kubernetes to TOSCA - and help you choose the right tool for the job.


Session link from teh summit: https://openstacksummitmay2015vancouver.sched.org/event/abd484e0dedcb9774edda1548ad47518#.VV5eh5NViko

Published in: Technology
  • Does Penis Size REALLY Matter? The truth comes out... ■■■ https://bit.ly/30G1ZO1
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Discover A Simple Holistic System For Curing Acne Once And For All using 100% Guaranteed All-Natural Method. Click Here ◆◆◆ http://t.cn/AiWGkfAm
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Women usually act out of emotion, not logic. Take advantage of this and get your Ex back today! See how to ▲▲▲ http://t.cn/R50e5nn
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • A testimony on how i won lottery with the help of Dr. OCUSODO Good day everybody, Please i want you all to stop whatever you are doing right now and read this testimony. I am Raymond Ellerbe from United Kingdom...I won Lottery of 120,000 Pounds last week in Nsw 6 pool from 38 lottery play.I was introduced to a spell caster called Dr. OCUSODO his email is (drocusodospellcaster@gmail.com) a month ago by a colleague who he helped with marriage and financial issues.All what he said and what he promised to do really happened.I never believed in magic or spells because i thought they were mostly the same acts or tricksters until i met him,everything changed,it exists but only in the right hands and with it everything is possible.I don't know how to pay him for this but i really hope i can do something important and special for him I bought him a good car but not enough i guess by telling other people how i became rich now, he is good in love spell, get your ex back, HIV spell, Pregnancy Spell And other spell if you are in need of any help contact him here drocusodospellcaster@yahoo.com. Emai: drocusodospellcaster@yahoo.com or drocusodospellcaster@gmail.com WhatsApp or call him now: +2349067457724
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs. tosca (1)

  1. 1. Orchestration Tool Roundup - Docker Swarm vs. Kubernetes, TerraForm vs. TOSCA/Cloudify vs. Heat
  2. 2. Speakers..
  3. 3. Agenda • Orchestration 101.. • Different approaches for orchestration • Method of comparison • Comparison • Synergies • Summary - which tool to choose?
  4. 4. Orchestration 101 Orchestration is a mean to Automate Manual Process
  5. 5. Orchestration 101 • Common Characteristics – Use DSL to define blueprint – Execute a process based on input from the blueprint – Pass context information between the deployed entities • Different assumptions lead to different approaches – Application Architecture – Infrastructure – Scope of automation
  6. 6. Goals of this Exercise Explore the different approaches to orchestration Infrastructure Centric Pure Play Container Centric
  7. 7. Method of Comparison • Same Application Requirements • Full Production Deployment • Broken into three main groups – Container Centric – Kubernetes, Docker – Pure Play –Cloudify/TOSCA, Terraform, –Infrastructure Centric - Heat • Out of scope* – PaaS, Configuration Management (e.g Chef, Puppet, Ansible,..) – Covering all orchestrations solutions – Deep Dive into each orchestration technology
  8. 8. The Test Application VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod
  9. 9. Orchestration Process - Setup VM VM VM VM VM VM Load Balancer VM VM VM VM VM VM VM VM VM Create network and compute resources: VMs, security group, network, subnet, routers, LB pool 1
  10. 10. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Install Mongo and Node Binaries 2
  11. 11. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongod processes Start mongod processes Start mongod processes 3
  12. 12. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongo-cfg proecesses Start mongo-cfg proecesses Start mongo-cfg processes 4
  13. 13. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongos processes, pointing to config servers Start mongos processes, pointing to config servers Start mongos processes, pointing to mongo-cfg servers 5
  14. 14. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and initialize replica set Pick one mongos and initialize replica set Pick one VM per shard and initialize replica set 6
  15. 15. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and add shards, one at a time 7
  16. 16. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and initialize data in mongodb 8
  17. 17. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start nodejs processes Start nodejs processes Start nodejs processes 9
  18. 18. Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Add nodejs VMs to LB pool 10
  19. 19. Orchestrating in Production • Monitoring and log collection • Manual/Auto healing • Manual/Auto scaling • Maintenance: – Backup and restore – Continuous deployment – Infrastructure upgrades and patches
  20. 20. Common Requirements • Dependency management •Reproducible •Cloneable • Recoverable
  21. 21. Series 1: Container Centric
  22. 22. Quick Overview of Docker Swarm A Docker-native clustering system • Use a pool of hosts through a single swarm master endpoint • Placement constraints, affinity/anti-affinity docker run -name rs1 -e affinity:container!=rs* ...
  23. 23. Swarm Architecture
  24. 24. Solution Overview - Deploy - Create Replica Sets for i in 1..{number_of_replica_sets} for j in 1..{number_of_nodes_for_replica_set} docker run -name rs{i}_srv{j} -e affinity:container!=rs* -e affinity:container!=cfg* -e constraint:daemon==mongodb -d example/mongodb --replSet rs{i} Then, SSH into one host per replica set to configure it.
  25. 25. Solution Overview - Deploy - Start Node.js application containers Make sure you inject all mongos endpoints for the application. for i in 1..{number_of_nodejs_servers} docker run -P -name nodejs{i}_v1 -e constraint:daemon==nodejs -e affinity:container!=nodejs* -e MONGO_HOSTS=<LIST_OF_MONGOS_IPs> -d example/nodejs_v1 nodejs server.js
  26. 26. Solution Overview - Deploy - Reconfigure HAProxy Extract Node.js container IPs using docker inspect and then: for i in 1..{number_of_nodejs_servers} docker exec haproxy1 reconfigure.sh --add=<IP_of_nodejs{i}:port>
  27. 27. Solution Overview - Mongodb scale out Identical to the process of deploying the initial mongodb shards, mongodb will take care of migrating data to the new shard
  28. 28. Docker Swarm - Pros and Cons Pros ● Easy modeling ● Placement/Affinity Cons ● Basic infrastructure handling ● Manual handling multiple instances ● “Manual” workflow ● Requires other tools for production aspects - monitoring, healing, scaling
  29. 29. Kubernetes
  30. 30. Quick Overview to Kubernetes Container cluster manager • Pods: tightly coupled group of containers • Replication controller: ensures that a specified number of pod "replicas" are running at any one time. • Networking: Each pod gets its own IP address • Service: Load balanced endpoint for a set of pods
  31. 31. Kubernetes Architecture
  32. 32. Sample Replication Controller apiVersion: v1beta3 kind: ReplicationController spec: replicas: 5 selector: name: mongod-rs1 template: metadata: labels: name: mongod-rs1 spec: containers: - command: [mongod, --port, 27017, --replSet, rs1] image: example/mongod name: mongod-rs1 - command: [mongod-rs-manager, --replSet, rs1] image: example/mongod-rs-manager name: mongod-rs1-manager
  33. 33. Sample Service Configuration apiVersion: v1beta3 kind: Service metadata: labels: type: nodejs name: nodejs spec: ports: - port: 80 targetPort: 8080 selector: type: nodejs createExternalLoadBalancer: true
  34. 34. Solution Overview - Deploy • Create mongod config servers for i in 1..3 kubectl create -f mongod-configsvr{i}-controller.yaml kubectl create -f mongod-configsvr{i}-service.yaml • Create mongos router kubectl create -f mongos-controller.yaml kubectl create -f mongos-service.yaml
  35. 35. Solution Overview - Deploy - Create Data nodes for i in 1..{number_of_replica_sets} kubectl create -f mongod-rs{i}-controller.yaml # Now configure each replicate set # by picking pod to be the initial “master” # of each replica set and extract all # containers IPs using “kubectl get -l ...” # dynamically update replica set # members (this will kick of this process) kubectl create -f mongod-rs{i}-service.yaml
  36. 36. Solution Overview - Node.js Heal Failing pods are identified by kubernetes and are automatically rescheduled
  37. 37. Solution Overview - Node.js continuous deployment # initially configured with 0 replicas kubectl create -f nodejs-v{new_version}-controller.yaml for i in 1..{number_of_nodejs_replicas} kubectl resize rc nodejs_v{new_version} --current-replicas={i - 1} --replicas={i} # smoke test and rollback everything if testing failed kubectl resize rc nodejs_v{previous_version} --current-replicas={number_of_nodejs_replicas - i + 1} --replicas={number_of_nodejs_replicas - i}
  38. 38. Kubernetes - Pros and Cons Pros ● (almost) zero configuration autoheal ● Out of the box load balancer ● Simple scaling Cons ● No placement (yet) ● Not simple to manage stateful services
  39. 39. Series 2: Pure Play Orchestration
  40. 40. Introduction to Terraform • By Hashicorp • Simple (in a good way) command line tool – Resources – Providers and provisioners – Modules – Variables and outputs
  41. 41. Sample Configuration resource "openstack_compute_secgroup_v2" "nodejs_security_group" { name = "nodejs_security_group" description = "security group for mongodb" rule { from_port = 22 to_port = 22 ip_protocol = "tcp" cidr = "0.0.0.0/0" } rule { from_port = "${var.nodejs_port}" to_port = "${var.nodejs_port}" ip_protocol = "tcp" cidr = "0.0.0.0/0" } }
  42. 42. Sample Configuration # # Create a Network # resource "openstack_networking_network_v2" "tf_network" { region = "" name = "tf_network" admin_state_up = "true" } # # Create a subnet in our new network # Notice here we use a TF variable for the name of our network above. # resource "openstack_networking_subnet_v2" "tf_net_sub1" { region = "" network_id = "${openstack_networking_network_v2.tf_network.id}" cidr = "192.168.1.0/24" ip_version = 4 }
  43. 43. Sample Configuration resource "openstack_compute_instance_v2" "mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-keypair-1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-exec" { scripts = [ "scripts/install_mongo.sh" "start_mongod.sh" ] } }
  44. 44. Solution Overview • Single top level configuration file • Creates: Network, subnet, router, floating IP, security groups, VMs, LBaaS pool • TF module to model a mongodb shard – No easy way to specify "I want X occurrences of this module" – Just copy and paste...
  45. 45. Master Assignment & Registration of Shards • Issue - no "cluster wide" way of invoking provisioners – Needed for configuring shard masters and adding shards to the cluster • Option 1: use Consul – e.g. first instance acquires a lock and waits for other to join • Option 2: Static allocation in the configuration • Option 3: local-exec with locks
  46. 46. Terraform - Pros and Cons Pros ● Infrastructure & Framework neutrality ● Solid support for OpenStack ● Simple and elegant ● Present plan before applying ● Support for incremental updates Cons ● Configurations are not portable across cloud providers ● Hard to model non- infrastructure components ● Everything is done in the context of a single resource instance
  47. 47. TOSCA / Cloudify
  48. 48. What is TOSCA? TOSCA defines the interoperable description of applications; including their components, relationships, dependencies, requirements, and capabilities….
  49. 49. Cloudify – Open Source Implementation of TOSCA Provision ConfigureMonitor Manage Infrastructure Can be used as a command line tool or as a managed service Plugins CM Monitoring & Alarming
  50. 50. Cloudify – Open Source Implementation of TOSCA Provision ConfigureMonitor Manage Monitoring & Alarming Infrastructure Can be used as a command line tool or as a managed service Plugins CM
  51. 51. Hosted On Software ComponentContainer (Docker Runtime Capability) Containee (Docker Runtime Requirement) Requirements Capabilities # NOT YET IN TOSCA SPEC. TO BE INVENTED… repositories: docker_hub: url: xxx credentials: yyy node_templates: docker_webserver: type: tosca.nodes.Container requirements: - host: # omitted for brevity artifacts: - my_image: < URI of Docker Image in Repo. > type: tosca.artifacts.impl.Docker.Image: repository: docker_repo Container Container Docker Rocket … artifact_types: tosca.artifacts.impl.Docker.Image: derived_from: tosca.artifacts.Root description: Docker Image TAR mime_type: TBD file_ext: [ tar ] Docker Hub (Repo.) • URI of DockerImage • Relative to Repo. Artifacts • Docker Image • .TAR) Containers Portability in TOSCA Source: Vmware Proposal
  52. 52. Solution Overview Mongod-shard Mongo replica- set Output: Mogoconfig hosts Shards endpoint Subsitutable *Scalable *Scalable Input: #config instances #Shards #Replica set per shard Input: #nodeJS instances mongodb deployment id or MongoConfig Mogo Shards Output: App EndPoint = Load-Balancer IP/path Mongo cfg *Scalable Initialization Initialization Load Balancer NodeJS MongoS *Scalable *Scalable
  53. 53. Infrastructure setup node_templates: nodecellar_security_group: type: cloudify.openstack.nodes.SecurityGroup properties: security_group: name: nodecellar_security_group rules: - remote_ip_prefix: 0.0.0.0/0 port: { get_property: [ nodecellar, port ] }
  54. 54. Create Mongo Shards mongodb: type: tosca.nodes.mongodb.Shard directives: [substitutable] properties: count: { get_input: servers_count_in_replica_set } requirements: - host: node: mongo_server capabilities: scalable: properties: min_instances: 1 max_instances: 10 default_instances: { get_input: mongodb_rs_count } MongoDB ReplicaSet MongoDB Server *scalable
  55. 55. Create Compute Instances mongo_server: type: tosca.nodes.Compute capabilities: host: properties: *host_capabilities os: properties: *os_capabilities scalable: properties: min_instances: 1 max_instances: 10 default_instances: 5
  56. 56. Create MongoDB Replica Set mongo_db_replica_set: type: tosca.nodes.DBMS requirements: - host: node: mongo_server interfaces: Standard: create: Scripts/mongodb/create.sh configure: implementation: Scripts/mongodb/config.sh inputs: mongodb_ip: { get_attribute: [mongo_server, addr] } start: Scripts/mongodb/start.sh
  57. 57. Creat NodeJS Containers nodecellar_container: type: tosca.nodes.NodeCellarAppContainer properties: port: { get_input: nodejs_app_port } interfaces: cloudify.interfaces.lifecycle: create: inputs: .... command: nodejs server.js environment: NODECELLAR_PORT: { get_property: [SELF, port] } MONGO_PORT: { get_property: [SELF, database_connection, port] } MONGO_HOST: { get_attribute: [SELF, database_connection, private_address] } …..
  58. 58. Create Load Balancer haproxy: type: tosca.nodes.Proxy properties: frontend_port: 80 statistics_port: 9000 backend_app_port: { get_property: [ nodecellar, port ] } requirements: - host: node: haproxy_frontend_host - member: node: nodecellar_container Get the web containers through relationship and update the load balancer accordingly
  59. 59. Handling Post Deployment through Workflow & Policies ● Cloudify Workflows ● Built in workflows o Install o Uninstall o Heal o Scale ● Discovery through graph navigation ● Remote/Local execution Script execution in python with context to the deployment graph cfy executions start -w install ...
  60. 60. Summary TOSCA/Cloudify Pros ● Infrastructure & Framework neutrality ● Complete Life Cycle Management ● Handles Infrastructure & Software ● Production Orchestration* o Monitoring o Workflow o Policies o Logging *Implementation specific Cons ● The spec is still evolving ● Cloudify isn’t 100% complaint yet ● Limited set of tooling
  61. 61. Series 3: Infrastructure Centric • Overview of Heat • Orchestrating NodeJS/MongoDB with Heat • Summary – Benefits/ Limitations
  62. 62. What is Heat? Heat provides a mechanism for orchestrating OpenStack resources through the use of modular templates.
  63. 63. Heat Architecture
  64. 64. Solution Overview Mogo replica- set Output: Replica set node hosts ssh-key, private ip to the init node Input: #Replica set per shardInput: #nodeJS instances MongoConfig hosts Mogo Shards hosts Output: mongos node hosts App EndPoint = Load-Balancer IP/path mogocfg Output: mogocfg node hosts Input: #config instances Initialize replica- set-script Initialize MongoS- Script Load Balancer NodeJS MongoS
  65. 65. Infrastructure setup resources: secgroup: type: OS::Neutron::SecurityGroup properties: name: str_replace: template: mongodb-$stackstr-secgroup params: $stackstr: get_attr: - stack-string - value rules: - protocol: icmp - protocol: tcp port_range_min: 22 port_range_max: 22 - protocol: tcp port_range_min: 27017 port_range_max: 27019
  66. 66. Create Compute Instances mongo_host: type: OS::Nova::Server properties: name: str_replace: template: $stackprefix-$stackstr params: $stackprefix: get_param: stack-prefix $stackstr: get_attr: - stack-string - value image: get_param: image flavor: get_param: flavor security_groups: - get_param: security_group
  67. 67. Create MongoDB Replica Servers mongodb_peer_servers: type: "OS::Heat::ResourceGroup" properties: count: { get_param: peer_server_count } resource_def: type: { get_param: child_template } properties: server_hostname: str_replace: template: '%name%-0%index%' params: '%name%': { get_param: server_hostname } image: { get_param: image } flavor: { get_param: flavor } ssh_key: { get_resource: ssh_key } ssh_private_key: { get_attr: [ssh_key, private_key] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version }
  68. 68. Configure the Replica Servers server_setup: type: "OS::Heat::ChefSolo" depends_on: - mongodb_peer_servers properties: username: root private_key: { get_attr: [ssh_key, private_key] } host: { get_attr: [mongodb_peer_servers, accessIPv4, 0] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version } node: mongodb: ruby_gems: mongo: '1.12.0' bson_ext: '1.12.0' bind_ip: { get_attr: [mongodb_peer_servers, privateIPv4, 0] } use_fqdn: false replicaset_members: { get_attr: [mongodb_peer_servers, privateIPv4] } config: replset: myreplset run_list: [ "recipe[config_replset]" ]
  69. 69. Create NodeJS Container nodestack_chef_run: type: 'OS::Heat::ChefSolo' depends_on: nodestack_node properties: ... node: nodejs_app: ... deployment: id: { get_param: stack_id } app_id: nodejs run_list: ["recipe[apt]", "recipe[nodejs]", "recipe[ssh_known_hosts]", "recipe[nodejs_app]"] data_bags: nodejs: id: { get_param: stack_id } nodejs_app: password: { get_attr: [nodejs_user_password, value] } deploy_key: { get_param: deploy_key } database_url: str_replace: template: 'mongodb://%dbuser%:%dbpasswd%@%dbhostname%' params: '%dbuser%': { get_param: database_username } '%dbpasswd%': { get_param: database_user_password } '%dbhostname%': { get_param: db_server_ip }
  70. 70. Summary Pros ● Native To OpenStack ● Built-in mapping of all the OpenStack infrastructure resource types Cons ● Limited to OpenStack ● Software configuration is limited ● Lack of built-in workflow ● Production orchestration is limited o Requires integration with other tools/ projects
  71. 71. Potential Synergies • Magnum - Kubernetes + Docker, Heat • Cloudify/TOSCA + Docker • Cloudify/TOSCA + Heat
  72. 72. Which orchestration tool should I choose?
  73. 73. Final Words..
  74. 74. The Only Constant Is Change!
  75. 75. More Change Ahead..
  76. 76. Further Reading..
  77. 77. OpenStack Vancouver Session

×