The document outlines the integration and functionalities of key open source cloud technologies: OpenStack, Cloud Foundry, and Docker. It provides detailed architectural overviews and highlights how these technologies intersect to enhance cloud computing capabilities, focusing on their respective modules, application portability, and orchestration. The document also discusses IBM's commitment to building cloud services on open standards and offers insights into workflows and deployments across these platforms.

1. © IBM Corporation 1
OpenStack, Docker, Cloud Foundry
How does the leading open source triumvirate
come together?
Animesh Singh
Manuel Silveyra
Daniel Krook

2. © IBM Corporation 2
The leading open source triumvirate
@AnimeshSingh @DanielKrook@Manuel_Silveyra

3. © IBM Corporation 3
In the top 5 of open source cloud projects
http://analystpov.com/cloud-computing/top-15-open-source-cloud-computing-technologies-2014-24727

4. © IBM Corporation 4
IBM Cloud and open technologies
At all tiers, IBM is committed to building its cloud on an open cloud architecture
OAuth
OSLC Infrastructure
as a Service
Platform
as a Service
Software
as a Service
API
Economy
Cloud
operating
environment
Software-
defined
environment
TOSCA
http://www.ibm.com/developerworks/cloud/library/cl-open-architecture/
Products and services built on open source and open standards benefit IBM and customers

5. © IBM Corporation 5
IBM Cloud and open technologies
At all tiers, IBM is committed to building its cloud on an open cloud architecture
OAuth
OSLC Infrastructure
as a Service
Platform
as a Service
Software
as a Service
API
Economy
TOSCA
http://www.ibm.com/developerworks/cloud/library/cl-open-architecture/
Products and services built on open source and open standards benefit IBM and customers

6. © IBM Corporation 6
Agenda
• OpenStack overview
• Cloud Foundry introduction and architectural overview
• Docker introduction and architectural overview
• How Cloud Foundry and OpenStack intersect
• How OpenStack and Docker intersect
• How Cloud Foundry and Docker intersect
• How they all come together

7. © IBM Corporation 7

8. © IBM Corporation 8
OpenStack overview
Collection of well integrated IaaS modules for compute, network, and storage
• Compute (Nova)
• Networking (Neutron)
• Object Storage (Swift)
• Block Storage (Cinder)
• Identity(Keystone)
• Image Service (Glance)
• Dashboard (Horizon)
OpenStack is the ubiquitous open source cloud computing
platform for public and private clouds, regardless of size.

9. © IBM Corporation 9

10. © IBM Corporation 10
Cloud Foundry overview
Open Cloud Platform
• There is an increasing appetite for cloud-based mobile, social
and analytics applications from line-of-business executives -
drives the need for a more open cloud development platform.
Meets Developers’ Needs
• Focus on app development, not provisioning VMs, databases,
messaging servers, etc.
• Agile development model
• Deploy and scale in seconds
Compelling Community
• Cloud Foundry has an emerging ecosystem as well as a
mature set of capabilities and robustness
Cloud Foundry is the
industry’s Open PaaS and
provides a choice of
clouds, frameworks, and
application services.

11. © IBM Corporation 11
Cloud Foundry architecture
• The platform is abstracted as a set of large-scale
distributed services.
• It uses Cloud Foundry BOSH to operate the
underlying infrastructure from the IaaS providers.
• Components are dynamically discoverable and
loosely coupled.
• Health is exposed through HTTP endpoints so
agents can collect state information and act on it.

12. © IBM Corporation 12

13. © IBM Corporation 13
Docker overview
Build Ship Run
Application portability
• Run Docker containers unchanged in any environment, on any infrastructure
Organizational speed & agility via separation of concerns
• Development owns the “inside” of the container – the app and its dependencies
• Operations owns the “outside” of the container – host, cluster, and policies for deployment
• Each team can add their part of the image (e.g. OS, configuration, security, monitoring tools, etc.)
Service reliability
• Faithful representation of app with encapsulated dependencies
Docker is an open platform for building distributed
applications for developers and operators.

14. © IBM Corporation 14
Docker architecture
• Docker uses a client-server architecture.
• The Docker client talks to the Docker daemon,
which does the heavy lifting of building,
running, and distributing your Docker
containers.
• Both the Docker client and the daemon can run
on the same system, or you can connect a
Docker client to a remote Docker daemon.
• The Docker client and daemon communicate
via sockets or through a RESTful API.

15. © IBM Corporation 15
Docker containers
• A Docker container consists of an operating system,
user-added files, and metadata – Basically a way to run
mini operating systems in your host operating system
with strong guarantees of isolation
• The Docker image is read-only. When Docker runs a
container from an image, it adds a read-write layer on top
of the image (using a union file system) in which your
application can then run.
• Underlying Technology : Written in Go and makes use of
several Linux kernel features
• Namespaces - pid, net, mnt, ipc, etc.
• Control Groups - cgroups (memory, cpu, blkio, devices)
• Union File Systems - UnionFS (AUFS, btrfs, vfs)
• Container Format - libcontainer or LXC

16. © IBM Corporation 16
Docker containers vs. virtual machines
Virtual Machines
• Each virtualized application includes not only the application - which may be
only 10s of MB - and the necessary binaries and libraries, but also an entire
guest operating system - which may weigh 10s of GB.
Docker
• The Docker Engine container comprises just the application and its
dependencies. It runs as an isolated process in userspace on the host
operating system, sharing the kernel with other containers
Compared with Hypervisors, Docker which is OS-Level Virtualization:
• CPU Performance => native performance
• Memory Performance => few % for (optional) accounting
• Network Performance => small overhead; can be optimized to zero overhead
• creating a new base image takes a few seconds (copy-on-write)
• apps in different containers can share the same binaries / libs

17. © IBM Corporation 17

18. © IBM Corporation 18
Cloud Foundry and OpenStack integration
Cloud
Provider
Interface
Cloud
DEA
Pool
Apps
Build
Packs
Service
Nodes
Cloud
Controller
Service
Connector
Messaging
Health
Manager
UAA
Router
PaaS
IaaS
Integration BOSH

19. © IBM Corporation 19
BOSH deploys Cloud Foundry resources to OpenStack
Deployment Manifest
• Release name/version
• # VMs, job params
• Stemcells to use
Stemcell
• Base OS
• BOSH agent
Release
• Name
• Software packages
• Config templates
• Scripts
BOSH
Cloud Foundry
Virtual Machine
• Configuration
• Software Packages
Virtual Machine
• Configuration
• Software Packages
Virtual Machine
• Configuration
• Software Packages
Virtual Machine
• Configuration
• Software packages

20. © IBM Corporation 20
Sample manifest snippet for a BOSH deployed component

21. © IBM Corporation 21

22. © IBM Corporation 22
Docker in OpenStack via the Nova Driver and Glance
• Docker is hypervisor driver for OpenStack Nova Compute.
It was introduced with the Havana release, but lives out-of-
tree for Icehouse, Juno, and Kilo.
• The Nova driver embeds a tiny HTTP client which talks
with the Docker internal REST API through a Unix socket.
It uses the HTTP API to control containers and fetch
information about them.
• The driver will fetch images from the OpenStack Image
Service (Glance) and load them into the Docker file
system. Images may be placed in Glance by exporting
them from Docker using the 'docker save' command.
• Well integrated with Horizon UI.

23. © IBM Corporation 23
Heat template plugin for Docker in OpenStack
• Not a replacement for Docker Nova Driver.
• OpenStack Orchestration (Heat) is a solution for providing
orchestration of resources inside OpenStack clouds.
• Using the Heat plugin, users may deploy and manage Docker
Containers on top of traditional OpenStack deployments, making
it compatible with existing OpenStack clouds.
• It allows to use the whole Docker API from a Heat template, and
makes linking of containers easier.
• In the example here, multiple containers may be created and
linked together by simply adding more sections like
“my_docker_container”. They’re not constrained by the
OpenStack APIs and may leverage the full power of the Docker
Remote API.

24. © IBM Corporation 24
User interface integration with Horizon
Horizon UI > Orchestration > Stacks Configure the Stack
(parameters of the Heat template)

25. © IBM Corporation 25
OpenStack Kolla runs OpenStack itself in Containers
• Containers used as building blocks for an OpenStack
control plane (compute, storage, network, and others)
to simplify the operational experience
• Containerized services become independently
manageable atomic microservices
• Container sets group related OpenStack services
• Database control
• Messaging control
• High availability control
• OpenStack interface
• OpenStack control
• OpenStack compute operation
• OpenStack network operation
• OpenStack storage operation

26. © IBM Corporation 26
OpenStack Magnum runs Containers on OpenStack in VMs
• Magnum is a Platform-as-a-Service (or Containers-
as-a-Service) solution for OpenStack. Joined March
2015. Not just Docker images/containers.
• Magnum combines OpenStack with a pluggable
container orchestration engine of your choice
(Kubernetes or Docker Swarm so far)
• Users leverage the same cloud account and
credentials they use to provision other cloud
resources, such as Compute, Storage, Network.
• Different users and project groups are properly
isolated from each other so that they have no visibility
into, or access to, other containers in the system

27. © IBM Corporation 27

28. © IBM Corporation 28
Cloud Foundry developer experience
Eclipse
IDE
CLI
Browser
cf
push
hBp

29. © IBM Corporation 29
Cloud Foundry runtime architecture
Eclipse
IDE
CLI
Browser
cf
push
hBp
Cloud
Controller
(API)
Router
DEA
Pool
DEA
Pool
VM
Pool
Apps
Apps
Apps
Apps

30. © IBM Corporation 30
Cloud Foundry runtime architecture
Eclipse
IDE
CLI
Browser
cf
push
hBp
Cloud
Controller
(API)
Router
DEA
Pool
DEA
Pool
Droplet
ExecuGon
Agent
(DEA)
VM
Pool
NATS
(Message
bus)
Health
Manager

31. © IBM Corporation 31
Warden Containers in Cloud Foundry architecture
Eclipse
IDE
CLI
Browser
cf
push
hBp
Cloud
Controller
(API)
Router
DEA
Pool
DEA
Pool
Droplet
ExecuGon
Agent
(DEA)
VM
Pool
NATS
(Message
bus)
Health
Manager
Warden Server
Container
Manager

32. © IBM Corporation 32
Cloud Foundry services architecture – Docker service
Eclipse
IDE
CLI
Browser
cf
create-­‐service
Cloud
Controller
(API)
Docker
Service
Broker
Provision Instance
(PUT /v2/service
instances/:id)
Create Binding
(PUT /v2/
service_instances/:id
Fetch Catalog
(GET /v2/catalog
DEA
Pool
Docker Backend
Docker instance
Docker instance
DEA
Pool
DEA Pool
App
App
Docker instance
App
cf
bind-­‐service

33. © IBM Corporation 33
Current Cloud Foundry architecture is being rewritten
Cloud
Controller
(API)
Router
DEA
Pool
DEA
Pool
Droplet
ExecuGon
Agent
(DEA)
VM
Pool
NATS
(Message
bus)
Health
Manager
Warden Server
Container
Manager

34. © IBM Corporation 34
Cloud
Controller
(API)
Router
DEA
Pool
DEA
Pool
Droplet
ExecuGon
Agent
(DEA)
VM
Pool
NATS
(Message
bus)
Health
Manager
Warden Server
Container
Manager
What’s being rewritten

35. © IBM Corporation 35
Why rewrite
Cloud
Controller
(API)
Router
• Tight Coupling
Between CC and rest of the components
Between Warden Server and backend
• Triangular Dependencies
Between CC, Health Monitor and DEAs
• Domain Specific
Apps specific
Hard to extend to new domains (e.g. cron-like jobs)
• Platform Specific
Custom Linux containers only – Warden containers

36. © IBM Corporation 36
Rewrite Goal : Run loosely coupled one off tasks and LRPs….
Executor Pool
Run
Tasks!
Launch
Long Running
Processes!
Run Task!
Launch LRP!

37. © IBM Corporation 37
..using generic recipes across various container backends
Run Task!
Launch LRP!
Executor Pool
Run
Tasks!
Launch
LRPs!
Rep
Exec
Recipes!
Exec Garden
Manage
Containers!
Garden
Backend
Run
Containers!

38. © IBM Corporation 38
Introducing Diego
Brain
BBS
CELLS
Cloud Foundry Diego
Diego is a distributed
system that orchestrates
containerized workloads
GARDEN
Garden
Garden defines an interface to
be implemented by container-
runners, and runs Linux,
Widnows and Docker
containers
BBS
An up-to-date cache of the
state of the Diego cluster
(including a picture-in-time
of all desired LRPs, running
LRP instances, and inflight
Tasks) is maintained in the
BBS (Bulletin Board
System/Store).
Brain
Schedule Deployments
using Auction, and Monitor
the health

39. © IBM Corporation 39
Diego
Cloud Foundry Diego
Diego is a distributed
system that orchestrates
containerized workloads
https://github.com/cloudfoundry-incubator/diego-design-notes

40. © IBM Corporation 40
Lattice
Doppler/
Traffic
Controller
Router
Router
EmiBer
AWS
GAE

41. © IBM Corporation 41
CC
Bridge
Cloud Foundry V3
Cloud
Controller
(API)
Router
Router
EmiBer
Doppler/
Traffic
Controller
Run App!
Stager
Stage App!
App Mgr
Run Task!
Launch LRP!

42. © IBM Corporation 42
CC
Bridge
Cloud Foundry V3
Cloud
Controller
(API)
Doppler/
Traffic
Controller
Run App!
Stager
Stage App!
App Mgr
Run Task!
Launch LRP!
Eclipse
IDE
CLI
Browser hBp
cf
push
cf
push
-­‐docker
cf
push
–stack
windows
Router
Router
EmiBer

43. © IBM Corporation 43
Same Cloud Foundry developer experience – with added
features
Eclipse
IDE
CLI
Browser hBp
cf
push
cf
push
-­‐docker
cf
push
–stack
windows

44. © IBM Corporation 44

45. © IBM Corporation 45
What’s possible now
Cloud
DEA
Pool
Apps
Build
Packs
Service
Nodes
Cloud
Controller
Service
Connector
Messaging
Health
Manager
UAA
Router
PaaS
IaaS
Integration
BOSH
Docker
Service
Broker

46. © IBM Corporation 46
What’s possible now
Cloud
DEA
Pool
Apps
Build
Packs
Cloud
Controller
Service
Connector
Messaging
Health
Manager
UAA
Router
Docker
Service
Nodes
PaaS
IaaS
Integration
BOSH
Docker
Service
Broker

47. © IBM Corporation 47
What’s coming next
Cloud
DIEGO
Pool
Apps
Docker
Files/Build
Packs
Cloud
Controller
Service
Connector
Messaging
UAA
Router
Docker
Service
Nodes
PaaS
IaaS
Integration
BOSH
Docker
Service
Broker

48. © IBM Corporation 48
What’s coming next
Cloud
DIEGO
Pool
Apps
Docker
Files/Build
Packs
Cloud
Controller
Service
Connector
Messaging
UAA
Router
Docker
Service
Nodes
PaaS
IaaS
Integration
BOSH
Docker
Service
Broker

49. © IBM Corporation 49
What’s possible future
Cloud
DIEGO
Pool
Apps
Docker
Files/Build
Packs
Cloud
Controller
Service
Connector
Messaging
UAA
Router
Docker
Service
Nodes
PaaS
IaaS
Integration
BOSH
Docker
Service
Broker

50. © IBM Corporation 50
IBM Bluemix
• IBM Platform as a
Services offering
• IBM and partner cloud
services
• Integrated DevOps with
both Browser and
Eclipse-based tools
Services
Lifecycle
Management
IDS
Application
Runtime
Runtimes &
Frameworks
Middleware Application Operational Mobile ExternalData
Node Java Ruby Worklight
WebSphere
Liberty
Eclipse IDE
Application
Composition
Environment
Create & Manage Services
Test/Run Test/Run
Explore
Services
Explore
Services
IBM Bluemix
Check In Code Check In Code
Web IDE
(Eclipse Orion)

51. © IBM Corporation 51
IBM Bluemix

52. © IBM Corporation 52
Object Storage Service in IBM Bluemix

53. © IBM Corporation 53
Container Service in IBM Bluemix

54. © IBM Corporation 54
Virtual Machine Service in IBM Bluemix

55. © IBM Corporation 55
IBM “Open by design” projects and products
Infrastructure-as-a-Service Features
OpenStack Self-serve virtual machines, storage, networking, and other infrastructure services
Docker Linux container management with host, cluster, and policy management
Platform-as-a-Service Features
Cloud Foundry Lifecycle managed application and service management
Docker Application and service container encapsulation and management
IBM products Features
Bluemix (instant runtimes) Hosted Cloud Foundry based PaaS to manage scalable application and services lifecycle
Bluemix (containers) Hosted OpenStack-based environment for build, deployment of Docker containers
Bluemix (virtual machines) Hosted OpenStack-based virtual machine environment
Bluemix Local On premises version of Cloud Foundry and OpenStack based Bluemix
Cloud Manager with OpenStack IBM branded distribution of OpenStack
Cloud Orchestrator IBM branded distribution of OpenStack with patterns and automation
Cloud OpenStack Services Hosted dedicated OpenStack environment
Object Storage and Elastic Storage Easily consumable storage services based on OpenStack components
Docker Hub Enterprise IBM resold and integrated version of a Docker image registry

56. © IBM Corporation 56
Agenda
• OpenStack overview
• Cloud Foundry introduction and architectural overview
• Docker introduction and architectural overview
• How Cloud Foundry and OpenStack intersect
• How OpenStack and Docker intersect
• How Cloud Foundry and Docker intersect
• How they all come together

57. © IBM Corporation 57
Questions? Reach out at the following Twitter handles
@AnimeshSingh @DanielKrook@Manuel_Silveyra

58. Meet the IBM Team at these sessions
Monday, May 18th
12:05 pm - A Conversation with Cinder Developers - Jay Bryant
4:40 pm - Tales From the Gate: How Debugging the Gate Helps Your Enterprise
- Matt Riedemann
4:40 pm - From Archive to Insight: Debunking Myths of Analytics on Object
Stores - Dean Hildebrand, Simon Lorenz
5:30 pm - OpenStack, Docker, and Cloud Foundry - How does the Leading Open
Source Triumvirate Come Together - Animesh Singh, Daniel Krook, Manuel
Silveyra, Kalonji Bankole
Tuesday, May 19th
11:15 am How to Configure your Cloud and Tempest for Interoperability Testing
Catherine Diep
12:05 pm - Past, Present and Future of Fibre Channel in OpenStack - Jay Bryant
2:00 pm - Building a Production Grade PaaS platform like Bluemix on
OpenStack, leveraging Container based scalable services - Animesh Singh,
James Busche
2:00 pm - Standing Tall in the Room - Sponsored by the Women of OpenStack -
Radha Ratnaparkhi
5:30 pm New Advances in Federated Identity and Federated Service Provider
Support for OpenStack Clouds - Brad Topol, Steve Martinelli
Wednesday, May 20th
9:50 am - Network Connectivity in a Hybrid OpenStack Cloud - John Kasperski,
Vinit Jain
1:50 pm - Leveraging open source tools to gain insight into OpenStack Swift -
Dmitry Sotnikov, Michael Factor
1:50 pm - Keystone advanced authentication methods - Steve Martinelli, Henry
Nash
2:40 pm - Helping Telcos go Green and save OpEx via Policy - Dilip
Krishnaswamy
Thursday, May 21st
9:00 am - Big Data Analytics and Docker: The Thrilla in Manila - Bill Owen,
Dean Hildebrand, Michael Hines, Nilesh Bhosale
9:50 am - Role of NFV Research in Open Source and Open Standards - Dilip
Krishnaswamy
1:30 pm - On-demand Disaster Recovery (DR) service enablement through
Software Defined Environments under hybrid clouds - Venkata Jagana,
Ramesh Palakodeti, CV Venugopal, Mike Williams, Ann Corrao
1:30pm - OpenStack Networking: It's time to talk Performance - Bengi Karacali,
John Tracey,Mohammad Banikazemi, George Almasi
4:10 pm - Beyond the Horizon: Innovating and Customizing Horizon using
AngularJS - Cindy Lu, Thai Tran

59. © IBM Corporation 59
http://www.meetup.com/OpenStack http://www.meetup.com/CloudFoundry
Tomorrow – 2PM : Building a Production Grade PaaS Platform like Bluemix on OpenStack
Apps
@AnimeshSingh @jim1348b @Bodine_Andrew