A Complete Open Cloud
Storage, Virt, IaaS, PaaS
Dave Neary
Open Source and Standards,
Red Hat
1
Agenda
1. Traditional virtualization
2. The move to IaaS
3. Storage
4. PaaS, application encapsulation and orchestration
5. Cloud monitoring, service brokerage

2
Agenda
1. Traditional virtualization
2. The move to IaaS
3. Storage
4. PaaS, application encapsulation and orchestration
5. Cloud monitoring, service brokerage

3
Virtualization

After

Before

4
Virtualization
●

●

●

●

●

5

Consolidation of hardware – fewer, bigger servers
Workload management – over-subscribed services get more
hardware
Protect your apps – fault tolerance, high availability, live
migration, Network, disk, compute, memory Quality of Service
Ability to scale up – add more hardware and modify resources
allocated to VM on the fly
No change to legacy apps
Open Source virtualization

KVM management application
Open Source alternative to vSphere

Widely deployed Hypervisor
Hosted by Linux Foundation since April 2013

Ganeti
6

Developed by Google to manage clusters
Manages KVM and Xen nodes
Agenda
1. Traditional virtualization
2. The move to IaaS
3. Storage
4. PaaS, application encapsulation and orchestration
5. Cloud monitoring, service brokerage

7
Infrastructure as a Service

●

Self-service provision

●

Many identical servers

●

Stateless services

●

●

8

Scale-out easily – load balancing, public IPs, database as
service
Move to object storage
Pets vs Cattle (yes, again)

Scale Up

- Servers are like pets.
Pets are given names, are unique,
lovingly hand raised and cared for.
When they get ill, you nurse them
back to health

“

9

Scale Out

- Servers are like cattle.
Cattle are given numbers and
are almost identical to each
other. When they get ill, you get
another one.

“Future application architectures should use Cattle but Pets with
strong configuration management are viable and still needed”
- Tim Bell, CERN
OpenStack architecture

●

●

Designed to easily scale out

●

10

Modular architecture

Based on (growing) set of core services
Instance types

XL
16 cores
30 GB memory

XL

M
L

L

L
M

Server 19234
32 cores
60GB memory
11

Server 19235
32 cores
60GB memory
Sample cloud application

6. Shared storage

1. Load balancer
with public IP

4. Load balanced
application server

2. Load balanced
3. Load balancer
web servers

Web service
12

5. Replicated
database
Sample cloud application
●

●

●

13

Stateless – no side-effects in instance, memory only
Persistent storage requires block storage or object storage
service, or shared filesystem
Requires rearchitecture of legacy applications
Agenda
1. Traditional virtualization
2. The move to IaaS
3. Storage
4. PaaS, application encapsulation and orchestration
5. Cloud monitoring, service brokerage

14
Storage
●

Clouds needs scale-out storage

●

Disk drives fail. All the time

●

●

●

15

Image, block, and object storage, and/or shared
filesystem
Scale-out storage on commodity hardware GlusterFS, Ceph, RiakCS, Swift
Virtualization aware, tight integration with virtualization
and Infrastructure as a Service desirable
Scale-out storage

16
Scale-out storage

17
Agenda
1. Traditional virtualization
2. The move to IaaS
3. Storage
4. PaaS, application encapsulation and orchestration
5. Cloud monitoring, service brokerage

18
PaaS = Platform as a Service
A Cloud Application Platform

Code

Code your app

19

Deploy
Pushbutton
Deploy, and
your App is
running in
the Cloud!

Enjoy
Save Time and Money
Consolidated B-24 Liberator
Incredibly sophisticated. ~500k parts, assembled
by unskilled labor.
No manufacturing process. Parts were cast in
rubber molds, so every part was slightly different.
Assembled in the heat of San Diego, which warped
the metal and required whole assemblies to be
adjusted.
Ford Motor Co. brought a manufacturing process
…
●
unit production went from 250 planes a year to
650 planes a month.

THIS IS OUR CHALLENGE TODAY

20
Physical
How to Build an App:
1.
2.
3.
4.
5.
6.
7.
8.

9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.

21

Have Idea
Get Budget
Submit hardware acquisition request
Wait
Get Hardware
Rack and Stack Hardware
Install Operating System
Install Operating System
Patches/Fix-Packs
Create user Accounts
Deploy framework/appserver
Deploy testing tools
Test testing tools
Code
Configure Prod servers (and buy
them if needed)
Push to Prod
Launch
Order more servers to meet demand
Wait…
Deploy new servers
Etc.

With PaaS

Virtualized
How to Build an App:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.

Have Idea
Get Budget
Submit VM Request request
Wait
Deploy framework/appserver
Deploy testing tools
Test testing tools
Code
Configure Prod VMs
Push to Prod
Launch
Request More Prod VMs to
meet demand
Wait
Deploy app to new VMs
Etc.

How to Build an App:
1.
2.
3.
4.
5.
6.

Have Idea
Get Budget
Code
Test
Launch
Automatically Scale

“The use of Platform-as-a-Service technologies will
enable IT organizations to become more agile and more
responsive to the business needs.” –Gartner*
Control vs automation

IaaS

PaaS

SaaS

APPLICATION
APPLICATION PLATFORM
(JBOSS, PHP, RUBY, ETC)
OPERATING SYSTEM
(RHEL)
VIRTUALIZATION
(KVM)
HARDWARE
(x86)
STORAGE
(GlusterFS)
Managed and Controlled
by Customer (IT, Dev, or
User)
Automated and Managed
by the Public or Private
Cloud Offering
22

Increased Control
Increased Automation
OpenShift Origin

Multi-language,
Auto-Scaling,
Self-service,
Elastic,
Cloud Application Platform

23
CloudFoundry

24
Docker

25
Orchestration and deployment
●

Containerization – Docker, OpenShift cartridges

●

Orchestration templates – Heat, CloudFormations

●

26

Configuration management – Foreman, Puppet, Chef,
Ansible, CFEngine...
Agenda
1. Traditional virtualization
2. The move to IaaS
3. Storage
4. PaaS, application encapsulation and orchestration
5. Cloud monitoring, service brokerage

27
Industry Trend – Hybrid Cloud

Existing IT + Private Cloud + Public Cloud = “Hybrid Cloud”
Private Cloud

Physical IT

Virtual IT

Private Cloud

Public Cloud

It's all coming together!!!
28

Hybrid Cloud
One Ring to Rule them All
●

●

Cloud management software
“Single pane of glass” to allow management of
traditional Virt and private and public IaaS
●
●

29

Facilitates service migration

●

●

Enables policy enforcement
Frees you from vendor lock-in

CloudForms Management Engine
Open Hybrid Cloud

Legacy
virtualization
(VMware,
oVirt/RHEV)

30

Private
cloud
(eg. OpenStack)

Public
cloud
(AWS, GCE)
Open Hybrid Cloud

Cloud services broker, Cloud management
Legacy
virtualization
(VMware,
oVirt/RHEV)

31

Private
cloud
(eg. OpenStack)

Public
cloud
(AWS, GCE)
Open Hybrid Cloud

Developer environment
Platform as a Service

Guest
instances

Cloud services broker, Cloud management
Legacy
virtualization
(VMware,
oVirt/RHEV)

32

Private
cloud
(eg. OpenStack)

Public
cloud
(AWS, GCE)
Open Hybrid Cloud

Software Defined Networking

Developer environment
Platform as a Service

Cloud services broker, Cloud management
Legacy
virtualization
(VMware,
oVirt/RHEV)

Private
cloud
(eg. OpenStack)

Shared storage (object, block)

33

Guest
instances

Public
cloud
(AWS, GCE)
Software Defined Networking

Developer environment
Platform as a Service

Cloud services broker, Cloud management
Legacy
virtualization
(VMware,
oVirt/RHEV)

Private
cloud
(eg. OpenStack)

Shared storage (object, block)

34

Guest
instances

Public
cloud
(AWS, GCE)

Development, deployment, orchestration

Open Hybrid Cloud
Open Hybrid Cloud needs Open Source

participate
35

integrate

stabilize
THANK YOU!
QUESTIONS?

Dave Neary
dneary@redhat.com
@nearyd
36

A complete Open Source cloud: Storage, Virt, IaaS, PaaS

  • 1.
    A Complete OpenCloud Storage, Virt, IaaS, PaaS Dave Neary Open Source and Standards, Red Hat 1
  • 2.
    Agenda 1. Traditional virtualization 2.The move to IaaS 3. Storage 4. PaaS, application encapsulation and orchestration 5. Cloud monitoring, service brokerage 2
  • 3.
    Agenda 1. Traditional virtualization 2.The move to IaaS 3. Storage 4. PaaS, application encapsulation and orchestration 5. Cloud monitoring, service brokerage 3
  • 4.
  • 5.
    Virtualization ● ● ● ● ● 5 Consolidation of hardware– fewer, bigger servers Workload management – over-subscribed services get more hardware Protect your apps – fault tolerance, high availability, live migration, Network, disk, compute, memory Quality of Service Ability to scale up – add more hardware and modify resources allocated to VM on the fly No change to legacy apps
  • 6.
    Open Source virtualization KVMmanagement application Open Source alternative to vSphere Widely deployed Hypervisor Hosted by Linux Foundation since April 2013 Ganeti 6 Developed by Google to manage clusters Manages KVM and Xen nodes
  • 7.
    Agenda 1. Traditional virtualization 2.The move to IaaS 3. Storage 4. PaaS, application encapsulation and orchestration 5. Cloud monitoring, service brokerage 7
  • 8.
    Infrastructure as aService ● Self-service provision ● Many identical servers ● Stateless services ● ● 8 Scale-out easily – load balancing, public IPs, database as service Move to object storage
  • 9.
    Pets vs Cattle(yes, again) Scale Up - Servers are like pets. Pets are given names, are unique, lovingly hand raised and cared for. When they get ill, you nurse them back to health “ 9 Scale Out - Servers are like cattle. Cattle are given numbers and are almost identical to each other. When they get ill, you get another one. “Future application architectures should use Cattle but Pets with strong configuration management are viable and still needed” - Tim Bell, CERN
  • 10.
    OpenStack architecture ● ● Designed toeasily scale out ● 10 Modular architecture Based on (growing) set of core services
  • 11.
    Instance types XL 16 cores 30GB memory XL M L L L M Server 19234 32 cores 60GB memory 11 Server 19235 32 cores 60GB memory
  • 12.
    Sample cloud application 6.Shared storage 1. Load balancer with public IP 4. Load balanced application server 2. Load balanced 3. Load balancer web servers Web service 12 5. Replicated database
  • 13.
    Sample cloud application ● ● ● 13 Stateless– no side-effects in instance, memory only Persistent storage requires block storage or object storage service, or shared filesystem Requires rearchitecture of legacy applications
  • 14.
    Agenda 1. Traditional virtualization 2.The move to IaaS 3. Storage 4. PaaS, application encapsulation and orchestration 5. Cloud monitoring, service brokerage 14
  • 15.
    Storage ● Clouds needs scale-outstorage ● Disk drives fail. All the time ● ● ● 15 Image, block, and object storage, and/or shared filesystem Scale-out storage on commodity hardware GlusterFS, Ceph, RiakCS, Swift Virtualization aware, tight integration with virtualization and Infrastructure as a Service desirable
  • 16.
  • 17.
  • 18.
    Agenda 1. Traditional virtualization 2.The move to IaaS 3. Storage 4. PaaS, application encapsulation and orchestration 5. Cloud monitoring, service brokerage 18
  • 19.
    PaaS = Platformas a Service A Cloud Application Platform Code Code your app 19 Deploy Pushbutton Deploy, and your App is running in the Cloud! Enjoy Save Time and Money
  • 20.
    Consolidated B-24 Liberator Incrediblysophisticated. ~500k parts, assembled by unskilled labor. No manufacturing process. Parts were cast in rubber molds, so every part was slightly different. Assembled in the heat of San Diego, which warped the metal and required whole assemblies to be adjusted. Ford Motor Co. brought a manufacturing process … ● unit production went from 250 planes a year to 650 planes a month. THIS IS OUR CHALLENGE TODAY 20
  • 21.
    Physical How to Buildan App: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21 Have Idea Get Budget Submit hardware acquisition request Wait Get Hardware Rack and Stack Hardware Install Operating System Install Operating System Patches/Fix-Packs Create user Accounts Deploy framework/appserver Deploy testing tools Test testing tools Code Configure Prod servers (and buy them if needed) Push to Prod Launch Order more servers to meet demand Wait… Deploy new servers Etc. With PaaS Virtualized How to Build an App: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. Have Idea Get Budget Submit VM Request request Wait Deploy framework/appserver Deploy testing tools Test testing tools Code Configure Prod VMs Push to Prod Launch Request More Prod VMs to meet demand Wait Deploy app to new VMs Etc. How to Build an App: 1. 2. 3. 4. 5. 6. Have Idea Get Budget Code Test Launch Automatically Scale “The use of Platform-as-a-Service technologies will enable IT organizations to become more agile and more responsive to the business needs.” –Gartner*
  • 22.
    Control vs automation IaaS PaaS SaaS APPLICATION APPLICATIONPLATFORM (JBOSS, PHP, RUBY, ETC) OPERATING SYSTEM (RHEL) VIRTUALIZATION (KVM) HARDWARE (x86) STORAGE (GlusterFS) Managed and Controlled by Customer (IT, Dev, or User) Automated and Managed by the Public or Private Cloud Offering 22 Increased Control Increased Automation
  • 23.
  • 24.
  • 25.
  • 26.
    Orchestration and deployment ● Containerization– Docker, OpenShift cartridges ● Orchestration templates – Heat, CloudFormations ● 26 Configuration management – Foreman, Puppet, Chef, Ansible, CFEngine...
  • 27.
    Agenda 1. Traditional virtualization 2.The move to IaaS 3. Storage 4. PaaS, application encapsulation and orchestration 5. Cloud monitoring, service brokerage 27
  • 28.
    Industry Trend –Hybrid Cloud Existing IT + Private Cloud + Public Cloud = “Hybrid Cloud” Private Cloud Physical IT Virtual IT Private Cloud Public Cloud It's all coming together!!! 28 Hybrid Cloud
  • 29.
    One Ring toRule them All ● ● Cloud management software “Single pane of glass” to allow management of traditional Virt and private and public IaaS ● ● 29 Facilitates service migration ● ● Enables policy enforcement Frees you from vendor lock-in CloudForms Management Engine
  • 30.
  • 31.
    Open Hybrid Cloud Cloudservices broker, Cloud management Legacy virtualization (VMware, oVirt/RHEV) 31 Private cloud (eg. OpenStack) Public cloud (AWS, GCE)
  • 32.
    Open Hybrid Cloud Developerenvironment Platform as a Service Guest instances Cloud services broker, Cloud management Legacy virtualization (VMware, oVirt/RHEV) 32 Private cloud (eg. OpenStack) Public cloud (AWS, GCE)
  • 33.
    Open Hybrid Cloud SoftwareDefined Networking Developer environment Platform as a Service Cloud services broker, Cloud management Legacy virtualization (VMware, oVirt/RHEV) Private cloud (eg. OpenStack) Shared storage (object, block) 33 Guest instances Public cloud (AWS, GCE)
  • 34.
    Software Defined Networking Developerenvironment Platform as a Service Cloud services broker, Cloud management Legacy virtualization (VMware, oVirt/RHEV) Private cloud (eg. OpenStack) Shared storage (object, block) 34 Guest instances Public cloud (AWS, GCE) Development, deployment, orchestration Open Hybrid Cloud
  • 35.
    Open Hybrid Cloudneeds Open Source participate 35 integrate stabilize
  • 36.