Cloud foundry Docker Openstack - Leading Open Source Triumvirate

12,636 views

Published on

OpenStack, Docker, and Cloud Foundry are the three most popular open source projects according to a recent cloud software survey. Docker has taken the cloud world by storm as a revolutionary way to not only run isolated application containers, but also to package them. But how does Docker fit into the paradigm of IaaS and PaaS? More specifically, how does it integrate with OpenStack and Cloud Foundry, the world's most popular infrastructure and platform service implementations? OpenStack, Docker, and Cloud Foundry are the three most popular open source projects according to a recent cloud software survey. Docker has taken the cloud world by storm as a revolutionary way to not only run isolated application containers, but also to package them. But how does Docker fit into the paradigm of IaaS and PaaS? More specifically, how does it integrate with OpenStack and Cloud Foundry, the world's most popular infrastructure and platform service implementations?

These charts from our OpenStack Summit talk Vancouver talk how the three leading open source cloud technologies are evolving to work together to support next generation workloads!

Published in: Technology
1 Comment
41 Likes
Statistics
Notes
No Downloads
Views
Total views
12,636
On SlideShare
0
From Embeds
0
Number of Embeds
2,505
Actions
Shares
0
Downloads
960
Comments
1
Likes
41
Embeds 0
No embeds

No notes for slide

Cloud foundry Docker Openstack - Leading Open Source Triumvirate

  1. 1. © IBM Corporation 1 OpenStack, Docker, Cloud Foundry How does the leading open source triumvirate come together? Animesh Singh Manuel Silveyra Daniel Krook
  2. 2. © IBM Corporation 2 The leading open source triumvirate @AnimeshSingh @DanielKrook@Manuel_Silveyra
  3. 3. © IBM Corporation 3 In the top 5 of open source cloud projects http://analystpov.com/cloud-computing/top-15-open-source-cloud-computing-technologies-2014-24727
  4. 4. © IBM Corporation 4 IBM Cloud and open technologies At all tiers, IBM is committed to building its cloud on an open cloud architecture     OAuth   OSLC Infrastructure as a Service Platform as a Service Software as a Service API Economy Cloud operating environment Software- defined environment TOSCA http://www.ibm.com/developerworks/cloud/library/cl-open-architecture/ Products and services built on open source and open standards benefit IBM and customers
  5. 5. © IBM Corporation 5 IBM Cloud and open technologies At all tiers, IBM is committed to building its cloud on an open cloud architecture     OAuth   OSLC Infrastructure as a Service Platform as a Service Software as a Service API Economy TOSCA http://www.ibm.com/developerworks/cloud/library/cl-open-architecture/ Products and services built on open source and open standards benefit IBM and customers
  6. 6. © IBM Corporation 6 Agenda •  OpenStack overview •  Cloud Foundry introduction and architectural overview •  Docker introduction and architectural overview •  How Cloud Foundry and OpenStack intersect •  How OpenStack and Docker intersect •  How Cloud Foundry and Docker intersect •  How they all come together
  7. 7. © IBM Corporation 7
  8. 8. © IBM Corporation 8 OpenStack overview Collection of well integrated IaaS modules for compute, network, and storage •  Compute (Nova) •  Networking (Neutron) •  Object Storage (Swift) •  Block Storage (Cinder) •  Identity(Keystone) •  Image Service (Glance) •  Dashboard (Horizon) OpenStack is the ubiquitous open source cloud computing platform for public and private clouds, regardless of size.
  9. 9. © IBM Corporation 9
  10. 10. © IBM Corporation 10 Cloud Foundry overview Open Cloud Platform •  There is an increasing appetite for cloud-based mobile, social and analytics applications from line-of-business executives - drives the need for a more open cloud development platform. Meets Developers’ Needs •  Focus on app development, not provisioning VMs, databases, messaging servers, etc. •  Agile development model •  Deploy and scale in seconds Compelling Community •  Cloud Foundry has an emerging ecosystem as well as a mature set of capabilities and robustness Cloud Foundry is the industry’s Open PaaS and provides a choice of clouds, frameworks, and application services.
  11. 11. © IBM Corporation 11 Cloud Foundry architecture •  The platform is abstracted as a set of large-scale distributed services. •  It uses Cloud Foundry BOSH to operate the underlying infrastructure from the IaaS providers. •  Components are dynamically discoverable and loosely coupled. •  Health is exposed through HTTP endpoints so agents can collect state information and act on it.
  12. 12. © IBM Corporation 12
  13. 13. © IBM Corporation 13 Docker overview Build Ship Run Application portability •  Run Docker containers unchanged in any environment, on any infrastructure Organizational speed & agility via separation of concerns •  Development owns the “inside” of the container – the app and its dependencies •  Operations owns the “outside” of the container – host, cluster, and policies for deployment •  Each team can add their part of the image (e.g. OS, configuration, security, monitoring tools, etc.) Service reliability •  Faithful representation of app with encapsulated dependencies Docker is an open platform for building distributed applications for developers and operators.
  14. 14. © IBM Corporation 14 Docker architecture •  Docker uses a client-server architecture. •  The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your Docker containers. •  Both the Docker client and the daemon can run on the same system, or you can connect a Docker client to a remote Docker daemon. •  The Docker client and daemon communicate via sockets or through a RESTful API.
  15. 15. © IBM Corporation 15 Docker containers •  A Docker container consists of an operating system, user-added files, and metadata – Basically a way to run mini operating systems in your host operating system with strong guarantees of isolation •  The Docker image is read-only. When Docker runs a container from an image, it adds a read-write layer on top of the image (using a union file system) in which your application can then run. •  Underlying Technology : Written in Go and makes use of several Linux kernel features •  Namespaces - pid, net, mnt, ipc, etc. •  Control Groups - cgroups (memory, cpu, blkio, devices) •  Union File Systems - UnionFS (AUFS, btrfs, vfs) •  Container Format - libcontainer or LXC
  16. 16. © IBM Corporation 16 Docker containers vs. virtual machines Virtual Machines •  Each virtualized application includes not only the application - which may be only 10s of MB - and the necessary binaries and libraries, but also an entire guest operating system - which may weigh 10s of GB. Docker •  The Docker Engine container comprises just the application and its dependencies. It runs as an isolated process in userspace on the host operating system, sharing the kernel with other containers Compared with Hypervisors, Docker which is OS-Level Virtualization: •  CPU Performance => native performance •  Memory Performance => few % for (optional) accounting •  Network Performance => small overhead; can be optimized to zero overhead •  creating a new base image takes a few seconds (copy-on-write) •  apps in different containers can share the same binaries / libs
  17. 17. © IBM Corporation 17
  18. 18. © IBM Corporation 18 Cloud Foundry and OpenStack integration Cloud  Provider  Interface   Cloud DEA  Pool      Apps   Build  Packs   Service  Nodes   Cloud  Controller   Service  Connector   Messaging   Health  Manager   UAA   Router   PaaS IaaS Integration BOSH  
  19. 19. © IBM Corporation 19 BOSH deploys Cloud Foundry resources to OpenStack Deployment Manifest •  Release name/version •  # VMs, job params •  Stemcells to use Stemcell •  Base OS •  BOSH agent Release •  Name •  Software packages •  Config templates •  Scripts BOSH Cloud Foundry Virtual Machine •  Configuration •  Software Packages Virtual Machine •  Configuration •  Software Packages Virtual Machine •  Configuration •  Software Packages Virtual Machine •  Configuration •  Software packages
  20. 20. © IBM Corporation 20 Sample manifest snippet for a BOSH deployed component
  21. 21. © IBM Corporation 21
  22. 22. © IBM Corporation 22 Docker in OpenStack via the Nova Driver and Glance •  Docker is hypervisor driver for OpenStack Nova Compute. It was introduced with the Havana release, but lives out-of- tree for Icehouse, Juno, and Kilo. •  The Nova driver embeds a tiny HTTP client which talks with the Docker internal REST API through a Unix socket. It uses the HTTP API to control containers and fetch information about them. •  The driver will fetch images from the OpenStack Image Service (Glance) and load them into the Docker file system. Images may be placed in Glance by exporting them from Docker using the 'docker save' command. •  Well integrated with Horizon UI.
  23. 23. © IBM Corporation 23 Heat template plugin for Docker in OpenStack •  Not a replacement for Docker Nova Driver. •  OpenStack Orchestration (Heat) is a solution for providing orchestration of resources inside OpenStack clouds. •  Using the Heat plugin, users may deploy and manage Docker Containers on top of traditional OpenStack deployments, making it compatible with existing OpenStack clouds. •  It allows to use the whole Docker API from a Heat template, and makes linking of containers easier. •  In the example here, multiple containers may be created and linked together by simply adding more sections like “my_docker_container”. They’re not constrained by the OpenStack APIs and may leverage the full power of the Docker Remote API.
  24. 24. © IBM Corporation 24 User interface integration with Horizon Horizon UI > Orchestration > Stacks Configure the Stack (parameters of the Heat template)
  25. 25. © IBM Corporation 25 OpenStack Kolla runs OpenStack itself in Containers •  Containers used as building blocks for an OpenStack control plane (compute, storage, network, and others) to simplify the operational experience •  Containerized services become independently manageable atomic microservices •  Container sets group related OpenStack services •  Database control •  Messaging control •  High availability control •  OpenStack interface •  OpenStack control •  OpenStack compute operation •  OpenStack network operation •  OpenStack storage operation
  26. 26. © IBM Corporation 26 OpenStack Magnum runs Containers on OpenStack in VMs •  Magnum is a Platform-as-a-Service (or Containers- as-a-Service) solution for OpenStack. Joined March 2015. Not just Docker images/containers. •  Magnum combines OpenStack with a pluggable container orchestration engine of your choice (Kubernetes or Docker Swarm so far) •  Users leverage the same cloud account and credentials they use to provision other cloud resources, such as Compute, Storage, Network. •  Different users and project groups are properly isolated from each other so that they have no visibility into, or access to, other containers in the system
  27. 27. © IBM Corporation 27
  28. 28. © IBM Corporation 28 Cloud Foundry developer experience Eclipse IDE CLI Browser    cf  push   hBp      
  29. 29. © IBM Corporation 29 Cloud Foundry runtime architecture Eclipse IDE CLI Browser    cf  push   hBp       Cloud  Controller   (API)     Router   DEA  Pool       DEA  Pool       VM  Pool       Apps   Apps   Apps   Apps  
  30. 30. © IBM Corporation 30 Cloud Foundry runtime architecture Eclipse IDE CLI Browser    cf  push   hBp       Cloud  Controller   (API)     Router   DEA  Pool       DEA  Pool      Droplet  ExecuGon     Agent  (DEA)    VM  Pool       NATS   (Message  bus)   Health   Manager  
  31. 31. © IBM Corporation 31 Warden Containers in Cloud Foundry architecture Eclipse IDE CLI Browser    cf  push   hBp       Cloud  Controller   (API)     Router   DEA  Pool       DEA  Pool      Droplet  ExecuGon     Agent  (DEA)    VM  Pool       NATS   (Message  bus)   Health   Manager   Warden Server Container Manager
  32. 32. © IBM Corporation 32 Cloud Foundry services architecture – Docker service Eclipse IDE CLI Browser    cf  create-­‐service   Cloud  Controller   (API)   Docker   Service   Broker   Provision Instance (PUT /v2/service instances/:id) Create Binding (PUT /v2/ service_instances/:id Fetch Catalog (GET /v2/catalog DEA  Pool       Docker Backend Docker instance Docker instance DEA  Pool       DEA Pool App App Docker instance App    cf  bind-­‐service  
  33. 33. © IBM Corporation 33 Current Cloud Foundry architecture is being rewritten Cloud  Controller   (API)     Router   DEA  Pool       DEA  Pool      Droplet  ExecuGon     Agent  (DEA)    VM  Pool       NATS   (Message  bus)   Health   Manager   Warden Server Container Manager
  34. 34. © IBM Corporation 34 Cloud  Controller   (API)     Router   DEA  Pool       DEA  Pool      Droplet  ExecuGon     Agent  (DEA)    VM  Pool       NATS   (Message  bus)   Health   Manager   Warden Server Container Manager What’s being rewritten
  35. 35. © IBM Corporation 35 Why rewrite Cloud  Controller   (API)     Router   •  Tight Coupling Between CC and rest of the components Between Warden Server and backend •  Triangular Dependencies Between CC, Health Monitor and DEAs •  Domain Specific Apps specific Hard to extend to new domains (e.g. cron-like jobs) •  Platform Specific Custom Linux containers only – Warden containers
  36. 36. © IBM Corporation 36 Rewrite Goal : Run loosely coupled one off tasks and LRPs…. Executor Pool Run Tasks! Launch Long Running Processes! Run Task! Launch LRP!
  37. 37. © IBM Corporation 37 ..using generic recipes across various container backends Run Task! Launch LRP! Executor Pool Run Tasks! Launch LRPs! Rep Exec Recipes! Exec Garden Manage Containers! Garden Backend Run Containers!
  38. 38. © IBM Corporation 38 Introducing Diego Brain BBS CELLS Cloud Foundry Diego Diego is a distributed system that orchestrates containerized workloads GARDEN Garden Garden defines an interface to be implemented by container- runners, and runs Linux, Widnows and Docker containers BBS An up-to-date cache of the state of the Diego cluster (including a picture-in-time of all desired LRPs, running LRP instances, and inflight Tasks) is maintained in the BBS (Bulletin Board System/Store). Brain Schedule Deployments using Auction, and Monitor the health
  39. 39. © IBM Corporation 39 Diego Cloud Foundry Diego Diego is a distributed system that orchestrates containerized workloads https://github.com/cloudfoundry-incubator/diego-design-notes
  40. 40. © IBM Corporation 40 Lattice Doppler/   Traffic  Controller   Router   Router   EmiBer   AWS GAE
  41. 41. © IBM Corporation 41 CC     Bridge   Cloud Foundry V3 Cloud     Controller   (API)     Router   Router   EmiBer   Doppler/   Traffic  Controller   Run App! Stager Stage App! App Mgr Run Task! Launch LRP!
  42. 42. © IBM Corporation 42 CC     Bridge   Cloud Foundry V3 Cloud     Controller   (API)     Doppler/   Traffic  Controller   Run App! Stager Stage App! App Mgr Run Task! Launch LRP! Eclipse IDE CLI Browser hBp          cf  push      cf  push  -­‐docker      cf  push  –stack                  windows   Router   Router   EmiBer  
  43. 43. © IBM Corporation 43 Same Cloud Foundry developer experience – with added features Eclipse IDE CLI Browser hBp          cf  push      cf  push  -­‐docker      cf  push  –stack                  windows  
  44. 44. © IBM Corporation 44
  45. 45. © IBM Corporation 45 What’s possible now Cloud DEA  Pool      Apps   Build  Packs   Service  Nodes   Cloud  Controller   Service  Connector   Messaging   Health  Manager   UAA   Router   PaaS IaaS Integration BOSH   Docker Service Broker
  46. 46. © IBM Corporation 46 What’s possible now Cloud DEA  Pool      Apps   Build  Packs   Cloud  Controller   Service  Connector   Messaging   Health  Manager   UAA   Router   Docker  Service  Nodes   PaaS IaaS Integration BOSH   Docker Service Broker
  47. 47. © IBM Corporation 47 What’s coming next Cloud DIEGO  Pool       Apps   Docker  Files/Build  Packs   Cloud  Controller   Service  Connector   Messaging   UAA   Router   Docker  Service  Nodes   PaaS IaaS Integration BOSH   Docker Service Broker
  48. 48. © IBM Corporation 48 What’s coming next Cloud DIEGO  Pool       Apps   Docker  Files/Build  Packs   Cloud  Controller   Service  Connector   Messaging   UAA   Router   Docker  Service  Nodes   PaaS IaaS Integration BOSH   Docker Service Broker
  49. 49. © IBM Corporation 49 What’s possible future Cloud DIEGO  Pool       Apps   Docker  Files/Build  Packs   Cloud  Controller   Service  Connector   Messaging   UAA   Router   Docker  Service  Nodes   PaaS IaaS Integration BOSH Docker Service Broker
  50. 50. © IBM Corporation 50 IBM Bluemix •  IBM Platform as a Services offering •  IBM and partner cloud services •  Integrated DevOps with both Browser and Eclipse-based tools Services Lifecycle Management IDS Application Runtime Runtimes & Frameworks Middleware Application Operational Mobile ExternalData Node Java Ruby Worklight WebSphere Liberty Eclipse IDE Application Composition Environment Create & Manage Services Test/Run Test/Run Explore Services Explore Services IBM Bluemix Check In Code Check In Code Web IDE (Eclipse Orion)
  51. 51. © IBM Corporation 51 IBM Bluemix
  52. 52. © IBM Corporation 52 Object Storage Service in IBM Bluemix
  53. 53. © IBM Corporation 53 Container Service in IBM Bluemix
  54. 54. © IBM Corporation 54 Virtual Machine Service in IBM Bluemix
  55. 55. © IBM Corporation 55 IBM “Open by design” projects and products Infrastructure-as-a-Service Features OpenStack Self-serve virtual machines, storage, networking, and other infrastructure services Docker Linux container management with host, cluster, and policy management Platform-as-a-Service Features Cloud Foundry Lifecycle managed application and service management Docker Application and service container encapsulation and management IBM products Features Bluemix (instant runtimes) Hosted Cloud Foundry based PaaS to manage scalable application and services lifecycle Bluemix (containers) Hosted OpenStack-based environment for build, deployment of Docker containers Bluemix (virtual machines) Hosted OpenStack-based virtual machine environment Bluemix Local On premises version of Cloud Foundry and OpenStack based Bluemix Cloud Manager with OpenStack IBM branded distribution of OpenStack Cloud Orchestrator IBM branded distribution of OpenStack with patterns and automation Cloud OpenStack Services Hosted dedicated OpenStack environment Object Storage and Elastic Storage Easily consumable storage services based on OpenStack components Docker Hub Enterprise IBM resold and integrated version of a Docker image registry
  56. 56. © IBM Corporation 56 Agenda •  OpenStack overview •  Cloud Foundry introduction and architectural overview •  Docker introduction and architectural overview •  How Cloud Foundry and OpenStack intersect •  How OpenStack and Docker intersect •  How Cloud Foundry and Docker intersect •  How they all come together
  57. 57. © IBM Corporation 57 Questions? Reach out at the following Twitter handles @AnimeshSingh @DanielKrook@Manuel_Silveyra
  58. 58. Meet the IBM Team at these sessions Monday, May 18th 12:05 pm - A Conversation with Cinder Developers - Jay Bryant 4:40 pm - Tales From the Gate: How Debugging the Gate Helps Your Enterprise - Matt Riedemann 4:40 pm - From Archive to Insight: Debunking Myths of Analytics on Object Stores - Dean Hildebrand, Simon Lorenz 5:30 pm - OpenStack, Docker, and Cloud Foundry - How does the Leading Open Source Triumvirate Come Together - Animesh Singh, Daniel Krook, Manuel Silveyra, Kalonji Bankole Tuesday, May 19th 11:15 am How to Configure your Cloud and Tempest for Interoperability Testing Catherine Diep 12:05 pm - Past, Present and Future of Fibre Channel in OpenStack - Jay Bryant 2:00 pm - Building a Production Grade PaaS platform like Bluemix on OpenStack, leveraging Container based scalable services - Animesh Singh, James Busche 2:00 pm - Standing Tall in the Room - Sponsored by the Women of OpenStack - Radha Ratnaparkhi 5:30 pm New Advances in Federated Identity and Federated Service Provider Support for OpenStack Clouds - Brad Topol, Steve Martinelli Wednesday, May 20th 9:50 am - Network Connectivity in a Hybrid OpenStack Cloud - John Kasperski, Vinit Jain 1:50 pm - Leveraging open source tools to gain insight into OpenStack Swift - Dmitry Sotnikov, Michael Factor 1:50 pm - Keystone advanced authentication methods - Steve Martinelli, Henry Nash 2:40 pm - Helping Telcos go Green and save OpEx via Policy - Dilip Krishnaswamy Thursday, May 21st 9:00 am - Big Data Analytics and Docker: The Thrilla in Manila - Bill Owen, Dean Hildebrand, Michael Hines, Nilesh Bhosale 9:50 am - Role of NFV Research in Open Source and Open Standards - Dilip Krishnaswamy 1:30 pm - On-demand Disaster Recovery (DR) service enablement through Software Defined Environments under hybrid clouds - Venkata Jagana, Ramesh Palakodeti, CV Venugopal, Mike Williams, Ann Corrao 1:30pm - OpenStack Networking: It's time to talk Performance - Bengi Karacali, John Tracey,Mohammad Banikazemi, George Almasi 4:10 pm - Beyond the Horizon: Innovating and Customizing Horizon using AngularJS - Cindy Lu, Thai Tran
  59. 59. © IBM Corporation 59 http://www.meetup.com/OpenStack http://www.meetup.com/CloudFoundry Tomorrow – 2PM : Building a Production Grade PaaS Platform like Bluemix on OpenStack Apps @AnimeshSingh @jim1348b @Bodine_Andrew

×