Uri Cohen, profile picture
Uploaded byUri Cohen
PPTX, PDF1,618 views

Orchestration tool roundup - OpenStack Israel summit - kubernetes vs. docker vs. heat vs. terra form vs. TOSCA

The document reviews various orchestration tools, comparing Docker Swarm, Kubernetes, Terraform, TOSCA/Cloudify, and Heat. It outlines their characteristics, advantages, and limitations while providing insights into orchestration processes and applications in production. It emphasizes the choice of orchestration tool based on specific application needs and infrastructure setups.

Embed presentation

Downloaded 31 times
Orchestration Tool Roundup - Docker Swarm vs. Kubernetes, TerraForm vs. TOSCA/Cloudify vs. Heat
Speakers..
Agenda • Orchestration 101.. • Different approaches for orchestration • Method of comparison • Comparison • Synergies • Summary - which tool to choose?
Orchestration 101 Orchestration is a mean to Automate Manual Process
Orchestration 101 • Common Characteristics – Use DSL to define blueprint – Execute a process based on input from the blueprint – Pass context information between the deployed entities • Different assumptions lead to different approaches – Application Architecture – Infrastructure – Scope of automation
Goals of this Exercise Explore the different approaches to orchestration Infrastructure Centric Pure Play Container Centric
Method of Comparison • Same Application Requirements • Full Production Deployment • Broken into three main groups – Container Centric – Kubernetes, Docker – Pure Play –Cloudify/TOSCA, Terraform, –Infrastructure Centric - Heat • Out of scope* – PaaS, Configuration Management (e.g Chef, Puppet, Ansible,..) – Covering all orchestrations solutions – Deep Dive into each orchestration technology
The Test Application VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod
Orchestration Process - Setup VM VM VM VM VM VM Load Balancer VM VM VM VM VM VM VM VM VM Create network and compute resources: VMs, security group, network, subnet, routers, LB pool 1
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Install Mongo and Node Binaries 2
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongod processes Start mongod processes Start mongod processes 3
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongo-cfg proecesses Start mongo-cfg proecesses Start mongo-cfg processes 4
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongos processes, pointing to config servers Start mongos processes, pointing to config servers Start mongos processes, pointing to mongo-cfg servers 5
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and initialize replica set Pick one mongos and initialize replica set Pick one VM per shard and initialize replica set 6
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and add shards, one at a time 7
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and initialize data in mongodb 8
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start nodejs processes Start nodejs processes Start nodejs processes 9
Orchestration Process - Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Add nodejs VMs to LB pool 10
Orchestrating in Production • Monitoring and log collection • Manual/Auto healing • Manual/Auto scaling • Maintenance: – Backup and restore – Continuous deployment – Infrastructure upgrades and patches
Common Requirements • Dependency management • Reproducible • Cloneable • Recoverable
Series 1: Container Centric
Quick Overview of Docker Swarm A Docker-native clustering system • Use a pool of hosts through a single swarm master endpoint • Placement constraints, affinity/anti-affinity docker run -name rs1 -e affinity:container!=rs* ...
Swarm Architecture
Solution Overview - Deploy - Create Replica Sets for i in 1..{number_of_replica_sets} for j in 1..{number_of_nodes_for_replica_set} docker run -name rs{i}_srv{j} -e affinity:container!=rs* -e affinity:container!=cfg* -e constraint:daemon==mongodb -d example/mongodb --replSet rs{i} Then, SSH into one host per replica set to configure it.
Solution Overview - Deploy - Start Node.js application containers Make sure you inject all mongos endpoints for the application. for i in 1..{number_of_nodejs_servers} docker run -P -name nodejs{i}_v1 -e constraint:daemon==nodejs -e affinity:container!=nodejs* -e MONGO_HOSTS=<LIST_OF_MONGOS_IPs> -d example/nodejs_v1 nodejs server.js
Solution Overview - Deploy - Reconfigure HAProxy Extract Node.js container IPs using docker inspect and then: for i in 1..{number_of_nodejs_servers} docker exec haproxy1 reconfigure.sh --add=<IP_of_nodejs{i}:port>
Solution Overview - Mongodb scale out Identical to the process of deploying the initial mongodb shards, mongodb will take care of migrating data to the new shard
Docker Swarm - Pros and Cons Pros ● Easy modeling ● Placement/Affinity Cons ● Basic infrastructure handling ● Manual handling multiple instances ● “Manual” workflow ● Requires other tools for production aspects - monitoring, healing, scaling
Kubernetes
Quick Overview to Kubernetes Container cluster manager • Pods: tightly coupled group of containers • Replication controller: ensures that a specified number of pod "replicas" are running at any one time. • Networking: Each pod gets its own IP address • Service: Load balanced endpoint for a set of pods
Kubernetes Architecture
Sample Replication Controller apiVersion: v1beta3 kind: ReplicationController spec: replicas: 5 selector: name: mongod-rs1 template: metadata: labels: name: mongod-rs1 spec: containers: - command: [mongod, --port, 27017, --replSet, rs1] image: example/mongod name: mongod-rs1 - command: [mongod-rs-manager, --replSet, rs1] image: example/mongod-rs-manager name: mongod-rs1-manager
Sample Service Configuration apiVersion: v1beta3 kind: Service metadata: labels: type: nodejs name: nodejs spec: ports: - port: 80 targetPort: 8080 selector: type: nodejs createExternalLoadBalancer: true
Solution Overview - Deploy • Create mongod config servers for i in 1..3 kubectl create -f mongod-configsvr{i}-controller.yaml kubectl create -f mongod-configsvr{i}-service.yaml • Create mongos router kubectl create -f mongos-controller.yaml kubectl create -f mongos-service.yaml
Solution Overview - Deploy - Create Data nodes for i in 1..{number_of_replica_sets} kubectl create -f mongod-rs{i}-controller.yaml # Now configure each replicate set # by picking pod to be the initial “master” # of each replica set and extract all # containers IPs using “kubectl get -l ...” # dynamically update replica set # members (this will kick of this process) kubectl create -f mongod-rs{i}-service.yaml
Solution Overview - Node.js Heal Failing pods are identified by kubernetes and are automatically rescheduled
Solution Overview - Node.js continuous deployment # initially configured with 0 replicas kubectl create -f nodejs-v{new_version}-controller.yaml for i in 1..{number_of_nodejs_replicas} kubectl resize rc nodejs_v{new_version} --current-replicas={i - 1} --replicas={i} # smoke test and rollback everything if testing failed kubectl resize rc nodejs_v{previous_version} --current-replicas={number_of_nodejs_replicas - i + 1} --replicas={number_of_nodejs_replicas - i}
Kubernetes - Pros and Cons Pros ● (almost) zero configuration autoheal ● Out of the box load balancer ● Simple scaling Cons ● No placement (yet) ● Not simple to manage stateful services
Series 2: Pure Play Orchestration
Introduction to Terraform • By Hashicorp • Simple (in a good way) command line tool – Resources – Providers and provisioners – Modules – Variables and outputs
Sample Configuration resource "openstack_compute_secgroup_v2" "nodejs_security_group" { name = "nodejs_security_group" description = "security group for mongodb" rule { from_port = 22 to_port = 22 ip_protocol = "tcp" cidr = "0.0.0.0/0" } rule { from_port = "${var.nodejs_port}" to_port = "${var.nodejs_port}" ip_protocol = "tcp" cidr = "0.0.0.0/0" } }
Sample Configuration # # Create a Network # resource "openstack_networking_network_v2" "tf_network" { region = "" name = "tf_network" admin_state_up = "true" } # # Create a subnet in our new network # Notice here we use a TF variable for the name of our network above. # resource "openstack_networking_subnet_v2" "tf_net_sub1" { region = "" network_id = "${openstack_networking_network_v2.tf_network.id}" cidr = "192.168.1.0/24" ip_version = 4 }
Sample Configuration resource "openstack_compute_instance_v2" "mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-keypair-1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-exec" { scripts = [ "scripts/install_mongo.sh" "start_mongod.sh" ] } }
Solution Overview • Single top level configuration file • Creates: Network, subnet, router, floating IP, security groups, VMs, LBaaS pool • TF module to model a mongodb shard – No easy way to specify "I want X occurrences of this module" – Just copy and paste...
Master Assignment & Registration of Shards • Issue - no "cluster wide" way of invoking provisioners – Needed for configuring shard masters and adding shards to the cluster • Option 1: use Consul – e.g. first instance acquires a lock and waits for other to join • Option 2: Static allocation in the configuration • Option 3: local-exec with locks
Terraform - Pros and Cons Pros ● Infrastructure & Framework neutrality ● Solid support for OpenStack ● Simple and elegant ● Present plan before applying ● Support for incremental updates Cons ● Configurations are not portable across cloud providers ● Hard to model non- infrastructure components ● Everything is done in the context of a single resource instance
TOSCA / Cloudify
What is TOSCA? TOSCA defines the interoperable description of applications; including their components, relationships, dependencies, requirements, and capabilities….
Cloudify – Open Source Implementation of TOSCA Provision ConfigureMonitor Manage Infrastructure Can be used as a command line tool or as a managed service Plugins CM Monitoring & Alarming
Cloudify – Open Source Implementation of TOSCA Provision ConfigureMonitor Manage Monitoring & Alarming Infrastructure Can be used as a command line tool or as a managed service Plugins CM
Hosted On Software ComponentContainer (Docker Runtime Capability) Containee (Docker Runtime Requirement) Requirements Capabilities # NOT YET IN TOSCA SPEC. TO BE INVENTED… repositories: docker_hub: url: xxx credentials: yyy node_templates: docker_webserver: type: tosca.nodes.Container requirements: - host: # omitted for brevity artifacts: - my_image: < URI of Docker Image in Repo. > type: tosca.artifacts.impl.Docker.Image: repository: docker_repo Container Container Docker Rocket … artifact_types: tosca.artifacts.impl.Docker.Image: derived_from: tosca.artifacts.Root description: Docker Image TAR mime_type: TBD file_ext: [ tar ] Docker Hub (Repo.) • URI of DockerImage • Relative to Repo. Artifacts • Docker Image • .TAR) Containers Portability in TOSCA Source: Vmware Proposal
Solution Overview Mongod-shard Mongo replica- set Output: Mogoconfig hosts Shards endpoint Subsitutable *Scalable *Scalable Input: #config instances #Shards #Replica set per shard Input: #nodeJS instances mongodb deployment id or MongoConfig Mogo Shards Output: App EndPoint = Load-Balancer IP/path Mongo cfg *Scalable Initialization Initialization Load Balancer NodeJS MongoS *Scalable *Scalable
Infrastructure setup node_templates: nodecellar_security_group: type: cloudify.openstack.nodes.SecurityGroup properties: security_group: name: nodecellar_security_group rules: - remote_ip_prefix: 0.0.0.0/0 port: { get_property: [ nodecellar, port ] }
Create Mongo Shards mongodb: type: tosca.nodes.mongodb.Shard directives: [substitutable] properties: count: { get_input: servers_count_in_replica_set } requirements: - host: node: mongo_server capabilities: scalable: properties: min_instances: 1 max_instances: 10 default_instances: { get_input: mongodb_rs_count } MongoDB ReplicaSet MongoDB Server *scalable
Create Compute Instances mongo_server: type: tosca.nodes.Compute capabilities: host: properties: *host_capabilities os: properties: *os_capabilities scalable: properties: min_instances: 1 max_instances: 10 default_instances: 5
Create MongoDB Replica Set mongo_db_replica_set: type: tosca.nodes.DBMS requirements: - host: node: mongo_server interfaces: Standard: create: Scripts/mongodb/create.sh configure: implementation: Scripts/mongodb/config.sh inputs: mongodb_ip: { get_attribute: [mongo_server, addr] } start: Scripts/mongodb/start.sh
Creat NodeJS Containers nodecellar_container: type: tosca.nodes.NodeCellarAppContainer properties: port: { get_input: nodejs_app_port } interfaces: cloudify.interfaces.lifecycle: create: inputs: .... command: nodejs server.js environment: NODECELLAR_PORT: { get_property: [SELF, port] } MONGO_PORT: { get_property: [SELF, database_connection, port] } MONGO_HOST: { get_attribute: [SELF, database_connection, private_address] } …..
Create Load Balancer haproxy: type: tosca.nodes.Proxy properties: frontend_port: 80 statistics_port: 9000 backend_app_port: { get_property: [ nodecellar, port ] } requirements: - host: node: haproxy_frontend_host - member: node: nodecellar_container Get the web containers through relationship and update the load balancer accordingly
Handling Post Deployment through Workflow & Policies ● Cloudify Workflows ● Built in workflows o Install o Uninstall o Heal o Scale ● Discovery through graph navigation ● Remote/Local execution Script execution in python with context to the deployment graph cfy executions start -w install ...
Summary TOSCA/Cloudify Pros ● Infrastructure & Framework neutrality ● Complete Life Cycle Management – day 0,1,2 operations (monitoring, log aggregation, healing, scaling, policies)* ● Model for both infrastructure & software Cons ● The spec is still evolving ● Cloudify isn’t 100% complaint yet ● Limited set of tooling
Series 3: Infrastructure Centric • Overview of Heat • Orchestrating NodeJS/MongoDB with Heat • Summary – Benefits/ Limitations
What is Heat? Heat provides a mechanism for orchestrating OpenStack resources through the use of modular templates.
Heat Architecture
Solution Overview Mongo replica- set Output: Replica set node hosts ssh-key, private ip to the init node Input: # of replicas per shardInput: # of nodeJS instances MongoConfig hosts Mogo Shards hosts Output: mongos node hosts App EndPoint = Load-Balancer IP/path mogocfg Output: mogocfg node hosts Input: # of config instances Initialize replica- set-script Initialize MongoS- Script Load Balancer NodeJS MongoS
Infrastructure setup resources: secgroup: type: OS::Neutron::SecurityGroup properties: name: str_replace: template: mongodb-$stackstr-secgroup params: $stackstr: get_attr: - stack-string - value rules: - protocol: icmp - protocol: tcp port_range_min: 22 port_range_max: 22 - protocol: tcp port_range_min: 27017 port_range_max: 27019
Create Compute Instances mongo_host: type: OS::Nova::Server properties: name: str_replace: template: $stackprefix-$stackstr params: $stackprefix: get_param: stack-prefix $stackstr: get_attr: - stack-string - value image: get_param: image flavor: get_param: flavor security_groups: - get_param: security_group
Create MongoDB Replica Servers mongodb_peer_servers: type: "OS::Heat::ResourceGroup" properties: count: { get_param: peer_server_count } resource_def: type: { get_param: child_template } properties: server_hostname: str_replace: template: '%name%-0%index%' params: '%name%': { get_param: server_hostname } image: { get_param: image } flavor: { get_param: flavor } ssh_key: { get_resource: ssh_key } ssh_private_key: { get_attr: [ssh_key, private_key] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version }
Configure the Replica Servers server_setup: type: "OS::Heat::ChefSolo" depends_on: - mongodb_peer_servers properties: username: root private_key: { get_attr: [ssh_key, private_key] } host: { get_attr: [mongodb_peer_servers, accessIPv4, 0] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version } node: mongodb: ruby_gems: mongo: '1.12.0' bson_ext: '1.12.0' bind_ip: { get_attr: [mongodb_peer_servers, privateIPv4, 0] } use_fqdn: false replicaset_members: { get_attr: [mongodb_peer_servers, privateIPv4] } config: replset: myreplset run_list: [ "recipe[config_replset]" ]
Create NodeJS Container nodestack_chef_run: type: 'OS::Heat::ChefSolo' depends_on: nodestack_node properties: ... node: nodejs_app: ... deployment: id: { get_param: stack_id } app_id: nodejs run_list: ["recipe[apt]", "recipe[nodejs]", "recipe[ssh_known_hosts]", "recipe[nodejs_app]"] data_bags: nodejs: id: { get_param: stack_id } nodejs_app: password: { get_attr: [nodejs_user_password, value] } deploy_key: { get_param: deploy_key } database_url: str_replace: template: 'mongodb://%dbuser%:%dbpasswd%@%dbhostname%' params: '%dbuser%': { get_param: database_username } '%dbpasswd%': { get_param: database_user_password } '%dbhostname%': { get_param: db_server_ip }
Summary Pros ● Native To OpenStack ● Built-in mapping of all the OpenStack infrastructure resource types Cons ● Only applicable to OpenStack ● Software configuration is limited ● Setup / Teardown focused ● Limited coverage for production aspects (monitoring, log aggregation, healing, scaling)
Potential Synergies • Magnum - Kubernetes + Docker, Heat • Cloudify/TOSCA + Docker • Cloudify/TOSCA + Heat
Which orchestration tool should I choose?
Final Words..
The Only Constant Is Change!
More Change Ahead..
Further Reading..
OpenStack Vancouver Session

More Related Content

PDF
Orchestrating Redis & K8s Operators
byDoiT International
 
PPTX
Cloudify workshop at CCCEU 2014
byUri Cohen
 
PDF
Performance Benchmarking of Clouds Evaluating OpenStack
byPradeep Kumar
 
PDF
Rally--OpenStack Benchmarking at Scale
byMirantis
 
PPTX
Your journey into the serverless world
byRed Hat Developers
 
PDF
Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...
byDocker, Inc.
 
PPTX
Openstack study-nova-02
byJinho Shin
 
PDF
Clocker - The Docker Cloud Maker
byAndrew Kennedy
 
Orchestrating Redis & K8s Operators
byDoiT International
 
Cloudify workshop at CCCEU 2014
byUri Cohen
 
Performance Benchmarking of Clouds Evaluating OpenStack
byPradeep Kumar
 
Rally--OpenStack Benchmarking at Scale
byMirantis
 
Your journey into the serverless world
byRed Hat Developers
 
Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...
byDocker, Inc.
 
Openstack study-nova-02
byJinho Shin
 
Clocker - The Docker Cloud Maker
byAndrew Kennedy
 

What's hot

PDF
[NYC Meetup] Docker at Nuxeo
byNuxeo
 
PDF
[OpenInfra Days Korea 2018] Day 2 - E6 - OpenInfra monitoring with Prometheus
byOpenStack Korea Community
 
PDF
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
byDocker, Inc.
 
PDF
Docker storage designing a platform for persistent data
byDocker, Inc.
 
PPTX
Openstack Rally - Benchmark as a Service. Openstack Meetup India. Ananth/Rahul.
byRahul Krishna Upadhyaya
 
PDF
Spark on Kubernetes - Advanced Spark and Tensorflow Meetup - Jan 19 2017 - An...
byChris Fregly
 
PPTX
Openstack Magnum: Container-as-a-Service
byChhavi Agarwal
 
PDF
Micro services infrastructure with AWS and Ansible
byBamdad Dashtban
 
PPTX
Introduction to Apache Mesos
byJoe Stein
 
PDF
Spring Boot Revisited with KoFu and JaFu
byVMware Tanzu
 
PPTX
Benchmarking Openstack Installations using Rally
byRama Krishna B
 
PPTX
DCUS17 : Docker networking deep dive
byMadhu Venugopal
 
PPTX
To Build My Own Cloud with Blackjack…
bySergey Dzyuban
 
PDF
Docker 進階實務班
byPhilip Zheng
 
PDF
Musings on Mesos: Docker, Kubernetes, and Beyond.
byTimothy St. Clair
 
PPTX
A fun cup of joe with open liberty
byAndy Mauer
 
PDF
Commit to excellence - Java in containers
byRed Hat Developers
 
PPTX
Scaling Development Environments with Docker
byDocker, Inc.
 
PPTX
Nashorn: JavaScript that doesn't suck - Tomer Gabel, Wix
byCodemotion Tel Aviv
 
PDF
Docker 對傳統 DevOps 工具鏈的衝擊 (Docker's Impact on traditional DevOps toolchain)
byWilliam Yeh
 
[NYC Meetup] Docker at Nuxeo
byNuxeo
 
[OpenInfra Days Korea 2018] Day 2 - E6 - OpenInfra monitoring with Prometheus
byOpenStack Korea Community
 
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
byDocker, Inc.
 
Docker storage designing a platform for persistent data
byDocker, Inc.
 
Openstack Rally - Benchmark as a Service. Openstack Meetup India. Ananth/Rahul.
byRahul Krishna Upadhyaya
 
Spark on Kubernetes - Advanced Spark and Tensorflow Meetup - Jan 19 2017 - An...
byChris Fregly
 
Openstack Magnum: Container-as-a-Service
byChhavi Agarwal
 
Micro services infrastructure with AWS and Ansible
byBamdad Dashtban
 
Introduction to Apache Mesos
byJoe Stein
 
Spring Boot Revisited with KoFu and JaFu
byVMware Tanzu
 
Benchmarking Openstack Installations using Rally
byRama Krishna B
 
DCUS17 : Docker networking deep dive
byMadhu Venugopal
 
To Build My Own Cloud with Blackjack…
bySergey Dzyuban
 
Docker 進階實務班
byPhilip Zheng
 
Musings on Mesos: Docker, Kubernetes, and Beyond.
byTimothy St. Clair
 
A fun cup of joe with open liberty
byAndy Mauer
 
Commit to excellence - Java in containers
byRed Hat Developers
 
Scaling Development Environments with Docker
byDocker, Inc.
 
Nashorn: JavaScript that doesn't suck - Tomer Gabel, Wix
byCodemotion Tel Aviv
 
Docker 對傳統 DevOps 工具鏈的衝擊 (Docker's Impact on traditional DevOps toolchain)
byWilliam Yeh
 

Viewers also liked

PPTX
Lessons Learned with Unity and WebGL
byLior Tal
 
PPTX
Joy of scala
byMaxim Novak
 
PDF
Storm at Forter
byRe'em Bensimhon
 
PDF
How does the Internet Work?
byDina Goldshtein
 
PDF
Guice - dependency injection framework
byEvgeny Barabanov
 
PDF
Scrum. software engineering seminar
byAlexandr Gavrishev
 
PPTX
Continuous Deployment into the Unknown with Artifactory, Bintray, Docker and ...
byGilad Garon
 
PPTX
1953 and all that. A tale of two sciences (Kitcher, 1984)
byYoav Francis
 
ODP
Optimizing DevOps strategy in a large enterprise
byEyal Edri
 
PPTX
JavaScript TDD
byUri Lavi
 
PDF
Responsive Web Design
byNir Elbaz
 
DOCX
HagayOnn_EnglishCV_ 2016
byHagay Onn (the Spot)
 
PPTX
Not your dad's h base new
byYaniv Rodenski
 
PDF
How fast ist it really? Benchmarking in practice
byTobias Pfeiffer
 
PPTX
Scala does the Catwalk
byAriel Kogan
 
PDF
Work-for-hire Game Studios: Elevate Your Game
byGur Dotan
 
PDF
What's the Magic in LinkedIn?
byEfrat Fenigson
 
PDF
Elasticsearch na prática
byBreno Oliveira
 
PDF
מכתב המלצה - לירן פרידמן
byLiran Fridman
 
PDF
טלפונים חכמים ואתם
byIdan ofek
 
Lessons Learned with Unity and WebGL
byLior Tal
 
Joy of scala
byMaxim Novak
 
Storm at Forter
byRe'em Bensimhon
 
How does the Internet Work?
byDina Goldshtein
 
Guice - dependency injection framework
byEvgeny Barabanov
 
Scrum. software engineering seminar
byAlexandr Gavrishev
 
Continuous Deployment into the Unknown with Artifactory, Bintray, Docker and ...
byGilad Garon
 
1953 and all that. A tale of two sciences (Kitcher, 1984)
byYoav Francis
 
Optimizing DevOps strategy in a large enterprise
byEyal Edri
 
JavaScript TDD
byUri Lavi
 
Responsive Web Design
byNir Elbaz
 
HagayOnn_EnglishCV_ 2016
byHagay Onn (the Spot)
 
Not your dad's h base new
byYaniv Rodenski
 
How fast ist it really? Benchmarking in practice
byTobias Pfeiffer
 
Scala does the Catwalk
byAriel Kogan
 
Work-for-hire Game Studios: Elevate Your Game
byGur Dotan
 
What's the Magic in LinkedIn?
byEfrat Fenigson
 
Elasticsearch na prática
byBreno Oliveira
 
מכתב המלצה - לירן פרידמן
byLiran Fridman
 
טלפונים חכמים ואתם
byIdan ofek
 

Similar to Orchestration tool roundup - OpenStack Israel summit - kubernetes vs. docker vs. heat vs. terra form vs. TOSCA

PPTX
Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...
byNati Shalom
 
PPTX
MongoDB Ops Manager and Kubernetes - James Broadhead
byMongoDB
 
PDF
FIWARE Tech Summit - Docker Swarm Secrets for Creating Great FIWARE Platforms
byFIWARE
 
PPTX
Orchestration Tool Roundup - Arthur Berezin & Trammell Scruggs
byCloud Native Day Tel Aviv
 
PDF
Docker Swarm and Traefik 2.0
byJakub Hajek
 
PDF
Kubernetes and CoreOS @ Athens Docker meetup
byMist.io
 
PDF
Containerizing MongoDB with kubernetes
byBrian McNamara
 
PDF
What is the Difference Between Kubernetes and Docker?
byRavendra Singh
 
PPTX
Uri Cohen & Dan Kilman, GigaSpaces - Orchestration Tool Roundup - OpenStack l...
byCloud Native Day Tel Aviv
 
PPTX
Containers orchestrators: Docker vs. Kubernetes
byDmitry Lazarenko
 
PPTX
Docker Swarm secrets for creating great FIWARE platforms
byFederico Michele Facca
 
PPTX
Webinar: Enabling Microservices with Containers, Orchestration, and MongoDB
byMongoDB
 
PDF
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
byMario Ishara Fernando
 
PDF
Kubernetes Walk Through from Technical View
byLei (Harry) Zhang
 
PPTX
Hybrid cloud openstack meetup
bydfilppi
 
PPTX
A docker love story
byBrian de Heus
 
PDF
Kubernetes: https://youtu.be/KnjnQj-FvfQ
byRahul Malhotra
 
PDF
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
byLee Calcote
 
PDF
Managing containers at scale
bySmruti Ranjan Tripathy
 
PDF
Orchestrating Linux Containers
byBergamo Linux Users Group
 
Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...
byNati Shalom
 
MongoDB Ops Manager and Kubernetes - James Broadhead
byMongoDB
 
FIWARE Tech Summit - Docker Swarm Secrets for Creating Great FIWARE Platforms
byFIWARE
 
Orchestration Tool Roundup - Arthur Berezin & Trammell Scruggs
byCloud Native Day Tel Aviv
 
Docker Swarm and Traefik 2.0
byJakub Hajek
 
Kubernetes and CoreOS @ Athens Docker meetup
byMist.io
 
Containerizing MongoDB with kubernetes
byBrian McNamara
 
What is the Difference Between Kubernetes and Docker?
byRavendra Singh
 
Uri Cohen & Dan Kilman, GigaSpaces - Orchestration Tool Roundup - OpenStack l...
byCloud Native Day Tel Aviv
 
Containers orchestrators: Docker vs. Kubernetes
byDmitry Lazarenko
 
Docker Swarm secrets for creating great FIWARE platforms
byFederico Michele Facca
 
Webinar: Enabling Microservices with Containers, Orchestration, and MongoDB
byMongoDB
 
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
byMario Ishara Fernando
 
Kubernetes Walk Through from Technical View
byLei (Harry) Zhang
 
Hybrid cloud openstack meetup
bydfilppi
 
A docker love story
byBrian de Heus
 
Kubernetes: https://youtu.be/KnjnQj-FvfQ
byRahul Malhotra
 
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
byLee Calcote
 
Managing containers at scale
bySmruti Ranjan Tripathy
 
Orchestrating Linux Containers
byBergamo Linux Users Group
 

More from Uri Cohen

PPTX
Deployment Automation on OpenStack with TOSCA and Cloudify
byUri Cohen
 
PPTX
GigaSpaces XAP - Don't Call Me Cache!
byUri Cohen
 
PPTX
DevOps Meets PaaS - NY Meetup with Chef (OpsCode)
byUri Cohen
 
PPTX
MongoDB in the Clouds
byUri Cohen
 
PPTX
Carrier Paas - CloudStack Collaboration Event 2012
byUri Cohen
 
PPTX
OpenStack Israel Summit 2013 - It’s the App, Stupid!
byUri Cohen
 
PPTX
Cassandra summit - Big Data Apps on the cloud
byUri Cohen
 
PPTX
SSDs, IMDGs and All the Rest - Jax London
byUri Cohen
 
PPTX
Your Apps on the Cloud - What it really takes
byUri Cohen
 
PPTX
App Centric Devops - CloudStack 2014 Collaboration Conference #CCNA14
byUri Cohen
 
PPTX
In Memory Data Grids, Demystified!
byUri Cohen
 
PPTX
One Does Not Simply Walk Into Devops
byUri Cohen
 
PPTX
GigaSpaces XAP for Financial Services
byUri Cohen
 
PPTX
Oscon 2013 -Your OSS Project Is now served
byUri Cohen
 
PPTX
Its the app stupid - CloudStack 2014 Collaboration Conference #CCNA14
byUri Cohen
 
PPTX
Cloud stack collabiration conference - It's the app, stupid!
byUri Cohen
 
PPTX
Trade and Event Processing at a Massive Scale - QCon NY 2012
byUri Cohen
 
PPTX
Oscon 2013 - Lessons from building an open source community
byUri Cohen
 
PPTX
Alef event - going open source
byUri Cohen
 
PPTX
Changing organizational culture - a sweaty usecase
byUri Cohen
 
Deployment Automation on OpenStack with TOSCA and Cloudify
byUri Cohen
 
GigaSpaces XAP - Don't Call Me Cache!
byUri Cohen
 
DevOps Meets PaaS - NY Meetup with Chef (OpsCode)
byUri Cohen
 
MongoDB in the Clouds
byUri Cohen
 
Carrier Paas - CloudStack Collaboration Event 2012
byUri Cohen
 
OpenStack Israel Summit 2013 - It’s the App, Stupid!
byUri Cohen
 
Cassandra summit - Big Data Apps on the cloud
byUri Cohen
 
SSDs, IMDGs and All the Rest - Jax London
byUri Cohen
 
Your Apps on the Cloud - What it really takes
byUri Cohen
 
App Centric Devops - CloudStack 2014 Collaboration Conference #CCNA14
byUri Cohen
 
In Memory Data Grids, Demystified!
byUri Cohen
 
One Does Not Simply Walk Into Devops
byUri Cohen
 
GigaSpaces XAP for Financial Services
byUri Cohen
 
Oscon 2013 -Your OSS Project Is now served
byUri Cohen
 
Its the app stupid - CloudStack 2014 Collaboration Conference #CCNA14
byUri Cohen
 
Cloud stack collabiration conference - It's the app, stupid!
byUri Cohen
 
Trade and Event Processing at a Massive Scale - QCon NY 2012
byUri Cohen
 
Oscon 2013 - Lessons from building an open source community
byUri Cohen
 
Alef event - going open source
byUri Cohen
 
Changing organizational culture - a sweaty usecase
byUri Cohen
 

Recently uploaded

PDF
Everything You Ever Wanted to Know About Quarkus and LangChain4j
byMarkus Eisele
 
PDF
IAAM Meetup #7 chez Onepoint - Construire un Rag-as-a-service en production. ...
byiaaixmarseille
 
PDF
DSD-INT 2025 Delft3D FM Suite 2026.01 2D3D - New features + Improvements - Sp...
byDeltares
 
PDF
How Device, OS, and Network Variability Affects App Experience - And How to T...
bykalichargn70th171
 
PPTX
Understanding-ROM-RAM-Cache-and-Buffer-Memory.pptx.pptx
bylata2850
 
PPTX
The Sync Strikes Back: Tales from the MOPs Trenches
bybbedford2
 
PPTX
Future of Software Testing: AI-Powered Open Source Testing Tools
bySophie Lane
 
PDF
DSD-INT 2025 Hydrodynamic and Morphodynamic Modeling with Delft3D FM for an I...
byDeltares
 
PDF
Fundamental of Information Systems introduction.pdf
byYeabsraLakew
 
PDF
DSD-INT 2025 Flood Early Warning System for the Trans-African Hydrometeorolog...
byDeltares
 
PDF
Internship Project Training Report-3.pdf
byPawank36
 
PPTX
Boost Productivity the Smart Way with AI Automation Solutions
byEllocent Labs
 
PDF
DSD-INT 2025 Quantifying Flood Mitigation Strategies Under Sea Level Rise - H...
byDeltares
 
PDF
DSD-INT 2025 Century-Scale Impacts of Sediment-Based Coastal Restoration unde...
byDeltares
 
PDF
DSD-INT 2025 Next-Generation Flood Inundation Mapping for Taiwan - Challenges...
byDeltares
 
PDF
DSD-INT 2025 3D-modelling of salt intrusion into Germany’s busiest waterway -...
byDeltares
 
PDF
DSD-INT 2025 Climate and impact attribution of compound flooding induced by t...
byDeltares
 
PDF
DSD-INT 2025 Reviving a Lost Meander - Deen
byDeltares
 
PDF
DSD-INT 2025 Modelling Non-Newtonian Slurry Beaching with Delft3D-Slurry - Bi
byDeltares
 
PDF
DSD-INT 2025 Timor-Leste Flood exposure and Climate Change assessment - Ogunwumi
byDeltares
 
Everything You Ever Wanted to Know About Quarkus and LangChain4j
byMarkus Eisele
 
IAAM Meetup #7 chez Onepoint - Construire un Rag-as-a-service en production. ...
byiaaixmarseille
 
DSD-INT 2025 Delft3D FM Suite 2026.01 2D3D - New features + Improvements - Sp...
byDeltares
 
How Device, OS, and Network Variability Affects App Experience - And How to T...
bykalichargn70th171
 
Understanding-ROM-RAM-Cache-and-Buffer-Memory.pptx.pptx
bylata2850
 
The Sync Strikes Back: Tales from the MOPs Trenches
bybbedford2
 
Future of Software Testing: AI-Powered Open Source Testing Tools
bySophie Lane
 
DSD-INT 2025 Hydrodynamic and Morphodynamic Modeling with Delft3D FM for an I...
byDeltares
 
Fundamental of Information Systems introduction.pdf
byYeabsraLakew
 
DSD-INT 2025 Flood Early Warning System for the Trans-African Hydrometeorolog...
byDeltares
 
Internship Project Training Report-3.pdf
byPawank36
 
Boost Productivity the Smart Way with AI Automation Solutions
byEllocent Labs
 
DSD-INT 2025 Quantifying Flood Mitigation Strategies Under Sea Level Rise - H...
byDeltares
 
DSD-INT 2025 Century-Scale Impacts of Sediment-Based Coastal Restoration unde...
byDeltares
 
DSD-INT 2025 Next-Generation Flood Inundation Mapping for Taiwan - Challenges...
byDeltares
 
DSD-INT 2025 3D-modelling of salt intrusion into Germany’s busiest waterway -...
byDeltares
 
DSD-INT 2025 Climate and impact attribution of compound flooding induced by t...
byDeltares
 
DSD-INT 2025 Reviving a Lost Meander - Deen
byDeltares
 
DSD-INT 2025 Modelling Non-Newtonian Slurry Beaching with Delft3D-Slurry - Bi
byDeltares
 
DSD-INT 2025 Timor-Leste Flood exposure and Climate Change assessment - Ogunwumi
byDeltares
 

Orchestration tool roundup - OpenStack Israel summit - kubernetes vs. docker vs. heat vs. terra form vs. TOSCA

  • 1.
    Orchestration Tool Roundup- Docker Swarm vs. Kubernetes, TerraForm vs. TOSCA/Cloudify vs. Heat
  • 3.
  • 4.
    Agenda • Orchestration 101.. •Different approaches for orchestration • Method of comparison • Comparison • Synergies • Summary - which tool to choose?
  • 5.
    Orchestration 101 Orchestration isa mean to Automate Manual Process
  • 6.
    Orchestration 101 • CommonCharacteristics – Use DSL to define blueprint – Execute a process based on input from the blueprint – Pass context information between the deployed entities • Different assumptions lead to different approaches – Application Architecture – Infrastructure – Scope of automation
  • 7.
    Goals of thisExercise Explore the different approaches to orchestration Infrastructure Centric Pure Play Container Centric
  • 8.
    Method of Comparison •Same Application Requirements • Full Production Deployment • Broken into three main groups – Container Centric – Kubernetes, Docker – Pure Play –Cloudify/TOSCA, Terraform, –Infrastructure Centric - Heat • Out of scope* – PaaS, Configuration Management (e.g Chef, Puppet, Ansible,..) – Covering all orchestrations solutions – Deep Dive into each orchestration technology
  • 9.
    The Test Application VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg LoadBalancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod
  • 10.
    Orchestration Process -Setup VM VM VM VM VM VM Load Balancer VM VM VM VM VM VM VM VM VM Create network and compute resources: VMs, security group, network, subnet, routers, LB pool 1
  • 11.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Install Mongo and Node Binaries 2
  • 12.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongod processes Start mongod processes Start mongod processes 3
  • 13.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongo-cfg proecesses Start mongo-cfg proecesses Start mongo-cfg processes 4
  • 14.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start mongos processes, pointing to config servers Start mongos processes, pointing to config servers Start mongos processes, pointing to mongo-cfg servers 5
  • 15.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and initialize replica set Pick one mongos and initialize replica set Pick one VM per shard and initialize replica set 6
  • 16.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and add shards, one at a time 7
  • 17.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Pick one mongos and initialize data in mongodb 8
  • 18.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Start nodejs processes Start nodejs processes Start nodejs processes 9
  • 19.
    Orchestration Process -Setup VM mongod VM NodeJS mongos VM NodeJS mongos VM Mongo-cfg VM Mongo-cfg VM Mongo-cfg Load Balancer VM NodeJS mongos VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod VM mongod Add nodejs VMs to LB pool 10
  • 20.
    Orchestrating in Production •Monitoring and log collection • Manual/Auto healing • Manual/Auto scaling • Maintenance: – Backup and restore – Continuous deployment – Infrastructure upgrades and patches
  • 21.
    Common Requirements • Dependencymanagement • Reproducible • Cloneable • Recoverable
  • 22.
  • 23.
    Quick Overview ofDocker Swarm A Docker-native clustering system • Use a pool of hosts through a single swarm master endpoint • Placement constraints, affinity/anti-affinity docker run -name rs1 -e affinity:container!=rs* ...
  • 24.
  • 25.
    Solution Overview -Deploy - Create Replica Sets for i in 1..{number_of_replica_sets} for j in 1..{number_of_nodes_for_replica_set} docker run -name rs{i}_srv{j} -e affinity:container!=rs* -e affinity:container!=cfg* -e constraint:daemon==mongodb -d example/mongodb --replSet rs{i} Then, SSH into one host per replica set to configure it.
  • 26.
    Solution Overview -Deploy - Start Node.js application containers Make sure you inject all mongos endpoints for the application. for i in 1..{number_of_nodejs_servers} docker run -P -name nodejs{i}_v1 -e constraint:daemon==nodejs -e affinity:container!=nodejs* -e MONGO_HOSTS=<LIST_OF_MONGOS_IPs> -d example/nodejs_v1 nodejs server.js
  • 27.
    Solution Overview -Deploy - Reconfigure HAProxy Extract Node.js container IPs using docker inspect and then: for i in 1..{number_of_nodejs_servers} docker exec haproxy1 reconfigure.sh --add=<IP_of_nodejs{i}:port>
  • 28.
    Solution Overview -Mongodb scale out Identical to the process of deploying the initial mongodb shards, mongodb will take care of migrating data to the new shard
  • 29.
    Docker Swarm -Pros and Cons Pros ● Easy modeling ● Placement/Affinity Cons ● Basic infrastructure handling ● Manual handling multiple instances ● “Manual” workflow ● Requires other tools for production aspects - monitoring, healing, scaling
  • 30.
  • 31.
    Quick Overview toKubernetes Container cluster manager • Pods: tightly coupled group of containers • Replication controller: ensures that a specified number of pod "replicas" are running at any one time. • Networking: Each pod gets its own IP address • Service: Load balanced endpoint for a set of pods
  • 32.
  • 33.
    Sample Replication Controller apiVersion:v1beta3 kind: ReplicationController spec: replicas: 5 selector: name: mongod-rs1 template: metadata: labels: name: mongod-rs1 spec: containers: - command: [mongod, --port, 27017, --replSet, rs1] image: example/mongod name: mongod-rs1 - command: [mongod-rs-manager, --replSet, rs1] image: example/mongod-rs-manager name: mongod-rs1-manager
  • 34.
    Sample Service Configuration apiVersion:v1beta3 kind: Service metadata: labels: type: nodejs name: nodejs spec: ports: - port: 80 targetPort: 8080 selector: type: nodejs createExternalLoadBalancer: true
  • 35.
    Solution Overview -Deploy • Create mongod config servers for i in 1..3 kubectl create -f mongod-configsvr{i}-controller.yaml kubectl create -f mongod-configsvr{i}-service.yaml • Create mongos router kubectl create -f mongos-controller.yaml kubectl create -f mongos-service.yaml
  • 36.
    Solution Overview -Deploy - Create Data nodes for i in 1..{number_of_replica_sets} kubectl create -f mongod-rs{i}-controller.yaml # Now configure each replicate set # by picking pod to be the initial “master” # of each replica set and extract all # containers IPs using “kubectl get -l ...” # dynamically update replica set # members (this will kick of this process) kubectl create -f mongod-rs{i}-service.yaml
  • 37.
    Solution Overview -Node.js Heal Failing pods are identified by kubernetes and are automatically rescheduled
  • 38.
    Solution Overview -Node.js continuous deployment # initially configured with 0 replicas kubectl create -f nodejs-v{new_version}-controller.yaml for i in 1..{number_of_nodejs_replicas} kubectl resize rc nodejs_v{new_version} --current-replicas={i - 1} --replicas={i} # smoke test and rollback everything if testing failed kubectl resize rc nodejs_v{previous_version} --current-replicas={number_of_nodejs_replicas - i + 1} --replicas={number_of_nodejs_replicas - i}
  • 39.
    Kubernetes - Prosand Cons Pros ● (almost) zero configuration autoheal ● Out of the box load balancer ● Simple scaling Cons ● No placement (yet) ● Not simple to manage stateful services
  • 40.
    Series 2: PurePlay Orchestration
  • 41.
    Introduction to Terraform •By Hashicorp • Simple (in a good way) command line tool – Resources – Providers and provisioners – Modules – Variables and outputs
  • 42.
    Sample Configuration resource "openstack_compute_secgroup_v2""nodejs_security_group" { name = "nodejs_security_group" description = "security group for mongodb" rule { from_port = 22 to_port = 22 ip_protocol = "tcp" cidr = "0.0.0.0/0" } rule { from_port = "${var.nodejs_port}" to_port = "${var.nodejs_port}" ip_protocol = "tcp" cidr = "0.0.0.0/0" } }
  • 43.
    Sample Configuration # # Createa Network # resource "openstack_networking_network_v2" "tf_network" { region = "" name = "tf_network" admin_state_up = "true" } # # Create a subnet in our new network # Notice here we use a TF variable for the name of our network above. # resource "openstack_networking_subnet_v2" "tf_net_sub1" { region = "" network_id = "${openstack_networking_network_v2.tf_network.id}" cidr = "192.168.1.0/24" ip_version = 4 }
  • 44.
    Sample Configuration resource "openstack_compute_instance_v2""mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-keypair-1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-exec" { scripts = [ "scripts/install_mongo.sh" "start_mongod.sh" ] } }
  • 45.
    Solution Overview • Singletop level configuration file • Creates: Network, subnet, router, floating IP, security groups, VMs, LBaaS pool • TF module to model a mongodb shard – No easy way to specify "I want X occurrences of this module" – Just copy and paste...
  • 46.
    Master Assignment &Registration of Shards • Issue - no "cluster wide" way of invoking provisioners – Needed for configuring shard masters and adding shards to the cluster • Option 1: use Consul – e.g. first instance acquires a lock and waits for other to join • Option 2: Static allocation in the configuration • Option 3: local-exec with locks
  • 47.
    Terraform - Prosand Cons Pros ● Infrastructure & Framework neutrality ● Solid support for OpenStack ● Simple and elegant ● Present plan before applying ● Support for incremental updates Cons ● Configurations are not portable across cloud providers ● Hard to model non- infrastructure components ● Everything is done in the context of a single resource instance
  • 48.
  • 49.
    What is TOSCA? TOSCAdefines the interoperable description of applications; including their components, relationships, dependencies, requirements, and capabilities….
  • 50.
    Cloudify – OpenSource Implementation of TOSCA Provision ConfigureMonitor Manage Infrastructure Can be used as a command line tool or as a managed service Plugins CM Monitoring & Alarming
  • 51.
    Cloudify – OpenSource Implementation of TOSCA Provision ConfigureMonitor Manage Monitoring & Alarming Infrastructure Can be used as a command line tool or as a managed service Plugins CM
  • 52.
    Hosted On Software ComponentContainer (Docker Runtime Capability) Containee (Docker Runtime Requirement) Requirements Capabilities #NOT YET IN TOSCA SPEC. TO BE INVENTED… repositories: docker_hub: url: xxx credentials: yyy node_templates: docker_webserver: type: tosca.nodes.Container requirements: - host: # omitted for brevity artifacts: - my_image: < URI of Docker Image in Repo. > type: tosca.artifacts.impl.Docker.Image: repository: docker_repo Container Container Docker Rocket … artifact_types: tosca.artifacts.impl.Docker.Image: derived_from: tosca.artifacts.Root description: Docker Image TAR mime_type: TBD file_ext: [ tar ] Docker Hub (Repo.) • URI of DockerImage • Relative to Repo. Artifacts • Docker Image • .TAR) Containers Portability in TOSCA Source: Vmware Proposal
  • 53.
    Solution Overview Mongod-shard Mongo replica- set Output: Mogoconfighosts Shards endpoint Subsitutable *Scalable *Scalable Input: #config instances #Shards #Replica set per shard Input: #nodeJS instances mongodb deployment id or MongoConfig Mogo Shards Output: App EndPoint = Load-Balancer IP/path Mongo cfg *Scalable Initialization Initialization Load Balancer NodeJS MongoS *Scalable *Scalable
  • 54.
    Infrastructure setup node_templates: nodecellar_security_group: type: cloudify.openstack.nodes.SecurityGroup properties: security_group: name:nodecellar_security_group rules: - remote_ip_prefix: 0.0.0.0/0 port: { get_property: [ nodecellar, port ] }
  • 55.
    Create Mongo Shards mongodb: type:tosca.nodes.mongodb.Shard directives: [substitutable] properties: count: { get_input: servers_count_in_replica_set } requirements: - host: node: mongo_server capabilities: scalable: properties: min_instances: 1 max_instances: 10 default_instances: { get_input: mongodb_rs_count } MongoDB ReplicaSet MongoDB Server *scalable
  • 56.
    Create Compute Instances mongo_server: type:tosca.nodes.Compute capabilities: host: properties: *host_capabilities os: properties: *os_capabilities scalable: properties: min_instances: 1 max_instances: 10 default_instances: 5
  • 57.
    Create MongoDB ReplicaSet mongo_db_replica_set: type: tosca.nodes.DBMS requirements: - host: node: mongo_server interfaces: Standard: create: Scripts/mongodb/create.sh configure: implementation: Scripts/mongodb/config.sh inputs: mongodb_ip: { get_attribute: [mongo_server, addr] } start: Scripts/mongodb/start.sh
  • 58.
    Creat NodeJS Containers nodecellar_container: type:tosca.nodes.NodeCellarAppContainer properties: port: { get_input: nodejs_app_port } interfaces: cloudify.interfaces.lifecycle: create: inputs: .... command: nodejs server.js environment: NODECELLAR_PORT: { get_property: [SELF, port] } MONGO_PORT: { get_property: [SELF, database_connection, port] } MONGO_HOST: { get_attribute: [SELF, database_connection, private_address] } …..
  • 59.
    Create Load Balancer haproxy: type:tosca.nodes.Proxy properties: frontend_port: 80 statistics_port: 9000 backend_app_port: { get_property: [ nodecellar, port ] } requirements: - host: node: haproxy_frontend_host - member: node: nodecellar_container Get the web containers through relationship and update the load balancer accordingly
  • 60.
    Handling Post Deploymentthrough Workflow & Policies ● Cloudify Workflows ● Built in workflows o Install o Uninstall o Heal o Scale ● Discovery through graph navigation ● Remote/Local execution Script execution in python with context to the deployment graph cfy executions start -w install ...
  • 61.
    Summary TOSCA/Cloudify Pros ● Infrastructure& Framework neutrality ● Complete Life Cycle Management – day 0,1,2 operations (monitoring, log aggregation, healing, scaling, policies)* ● Model for both infrastructure & software Cons ● The spec is still evolving ● Cloudify isn’t 100% complaint yet ● Limited set of tooling
  • 62.
    Series 3: InfrastructureCentric • Overview of Heat • Orchestrating NodeJS/MongoDB with Heat • Summary – Benefits/ Limitations
  • 63.
    What is Heat? Heatprovides a mechanism for orchestrating OpenStack resources through the use of modular templates.
  • 64.
  • 65.
    Solution Overview Mongo replica- set Output: Replicaset node hosts ssh-key, private ip to the init node Input: # of replicas per shardInput: # of nodeJS instances MongoConfig hosts Mogo Shards hosts Output: mongos node hosts App EndPoint = Load-Balancer IP/path mogocfg Output: mogocfg node hosts Input: # of config instances Initialize replica- set-script Initialize MongoS- Script Load Balancer NodeJS MongoS
  • 66.
    Infrastructure setup resources: secgroup: type: OS::Neutron::SecurityGroup properties: name: str_replace: template:mongodb-$stackstr-secgroup params: $stackstr: get_attr: - stack-string - value rules: - protocol: icmp - protocol: tcp port_range_min: 22 port_range_max: 22 - protocol: tcp port_range_min: 27017 port_range_max: 27019
  • 67.
    Create Compute Instances mongo_host: type:OS::Nova::Server properties: name: str_replace: template: $stackprefix-$stackstr params: $stackprefix: get_param: stack-prefix $stackstr: get_attr: - stack-string - value image: get_param: image flavor: get_param: flavor security_groups: - get_param: security_group
  • 68.
    Create MongoDB ReplicaServers mongodb_peer_servers: type: "OS::Heat::ResourceGroup" properties: count: { get_param: peer_server_count } resource_def: type: { get_param: child_template } properties: server_hostname: str_replace: template: '%name%-0%index%' params: '%name%': { get_param: server_hostname } image: { get_param: image } flavor: { get_param: flavor } ssh_key: { get_resource: ssh_key } ssh_private_key: { get_attr: [ssh_key, private_key] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version }
  • 69.
    Configure the ReplicaServers server_setup: type: "OS::Heat::ChefSolo" depends_on: - mongodb_peer_servers properties: username: root private_key: { get_attr: [ssh_key, private_key] } host: { get_attr: [mongodb_peer_servers, accessIPv4, 0] } kitchen: { get_param: kitchen } chef_version: { get_param: chef_version } node: mongodb: ruby_gems: mongo: '1.12.0' bson_ext: '1.12.0' bind_ip: { get_attr: [mongodb_peer_servers, privateIPv4, 0] } use_fqdn: false replicaset_members: { get_attr: [mongodb_peer_servers, privateIPv4] } config: replset: myreplset run_list: [ "recipe[config_replset]" ]
  • 70.
    Create NodeJS Container nodestack_chef_run: type:'OS::Heat::ChefSolo' depends_on: nodestack_node properties: ... node: nodejs_app: ... deployment: id: { get_param: stack_id } app_id: nodejs run_list: ["recipe[apt]", "recipe[nodejs]", "recipe[ssh_known_hosts]", "recipe[nodejs_app]"] data_bags: nodejs: id: { get_param: stack_id } nodejs_app: password: { get_attr: [nodejs_user_password, value] } deploy_key: { get_param: deploy_key } database_url: str_replace: template: 'mongodb://%dbuser%:%dbpasswd%@%dbhostname%' params: '%dbuser%': { get_param: database_username } '%dbpasswd%': { get_param: database_user_password } '%dbhostname%': { get_param: db_server_ip }
  • 71.
    Summary Pros ● Native ToOpenStack ● Built-in mapping of all the OpenStack infrastructure resource types Cons ● Only applicable to OpenStack ● Software configuration is limited ● Setup / Teardown focused ● Limited coverage for production aspects (monitoring, log aggregation, healing, scaling)
  • 72.
    Potential Synergies • Magnum- Kubernetes + Docker, Heat • Cloudify/TOSCA + Docker • Cloudify/TOSCA + Heat
  • 73.
    Which orchestration toolshould I choose?
  • 74.
  • 75.
  • 76.
  • 77.
  • 78.