OpenStack
Architecture
Past and Future
Ken Pepple
Chief Technology Officer
http://www.solinea.com
HAPPY 3RD BIRTHDAY
AUTOMATEALLTHETHINGS
About Me
  CTO, Solinea
  Former Director of Cloud
Development, Internap
Public Cloud
  Author of O’Reilly
“Deploying OpenStack”
  OpenStack Code
contributor since Bexar
release
  Twitter @ken_pepple
  IRC kpepple
2
Early Development (Austin – Diablo)
  OpenStack started with Nova and Swift
  No interaction between two services until Glance in
Bexar release
  Developed completely on Launchpad
  My humble first commit
3
commit 16eeac71055ffa9fe0fc7a13032da4e6397121b1
Author: Ken Pepple <ken.pepple@gmail.com>
Date: Sat Jan 8 17:40:06 2011 -0800
fixed doc make process for new nova version (rev530) mechanism
Bexar/Cactus/Diablo Architecture (2011)
4
Image
Compute
Object
Storage
Stores
images in
Stores disk
files in
Conceptual Architecture Logical Architecture
Essex Development
  Web interface released
  Centralization of authentication released on
second try when “keystone light” was made the full
keystone
–  https://lists.launchpad.net/openstack/msg07063.html
  Moved codebase to GitHub
5
6
Essex Architecture (2012)
ObjectImage
Compute
Dashboard
Identity
authenticates
with
authenticates
with authenticates
with
retrieves / stores
images in
provides UI for
provides UI forprovides UI for
retrieves/stores
disk files in
http://ken.pepple.info
7
OpenStack Object Store
OpenStack
Image Service
OpenStack Compute
OpenStack
Dashboard
OpenStack
Identity
Service
OpenStack Compute
API /
Admin API
keystone
(service & admin APIs)
nova-api
(OS, EC2, Admin)
nova-consoleauth
nova-cert/
objectstore
nova-console
VNC/VMRC
OpenStack
Object API
Queue
nova-compute
nova-network
nova-volume
nova-scheduler
nova
database
volume
provider
(iSCSI, etc)
OpenStack
Compute API
OpenStack
Image API
Horizon
OpenStack
Image API
identity
backend
(kvs, pam, sql,
etc.)
swift-proxy
objectcontaineraccount
object
store
account
DB
container
DB
OpenStack
Object API
HTTP(S)
OpenStack
Object API
Horizon
Database
OpenStack
Identity API
OpenStack
Identity
API
OpenStack
Identity
API
OpenStack End Users
OpenStack
Image API
Amazon
Web Services
EC2 API
OpenStack
Identity
API
OpenStack
Image
API
catalog
backend
(kvs, catalog,
etc.)
token backend
(kvs, memcache,
etc.)
OpenStack
Identity
API
hypervisor
libvirt, XenAPI, etc.
HTTP(S)
glance-api
glance-registry
glance
database
http://ken.pepple.info/openstack/2011/04/22/openstack-nova-architecture/
Folsom Development
  Development and project pace quickens
  Cinder and Quantum (now Neutron) introduced
  Major influx of industry vendors support new services
8
Folsom Architecture (2012)
Identity
Dashboard
Image
Compute
Object
Storage
Block
Storage
Network
Provides
UI for
Provides
UI for
Provides
UI for Provides
UI for
Provides
UI for
Provides
Auth for
Provides
Auth for Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
volumes
for
Provide
network
connectivity
for
Stores
images in
Stores disk
files in
http://ken.pepple.info
OpenStack Object Store
OpenStack
Image Service
OpenStack Compute
OpenStack
Dashboard
OpenStack
Identity
Service
OpenStack Compute
API /
Admin API
keystone
(service & admin APIs)
nova-api
(OS, EC2, Admin)
nova-consoleauth
nova-cert/
objectstore
nova-console
VNC/VMRC
OpenStack
Object API
http://ken.pepple.info
Queue
nova-compute
nova-network
nova-volume
nova-scheduler
nova
database
OpenStack
Compute API
OpenStack
Image API
Horizon
OpenStack
Image API
identity
backend
(kvs, pam, sql,
ldap, etc.)
swift-proxy
objectcontaineraccount
object
store
account
DB
container
DB
OpenStack
Object API
HTTP(S)
OpenStack
Object API
Horizon
Database
OpenStack
Identity API
OpenStack
Identity
API
OpenStack
Identity
API
OpenStack End Users
OpenStack
Image API
OpenStack
Identity
API
OpenStack
Image
API
catalog
backend
(kvs, sql,
etc.)
token backend
(kvs, memcache,
etc.)
OpenStack
Identity
API
hypervisor
libvirt, XenAPI, etc.
HTTP(S)
Amazon
Web Services
EC2 API
OpenStack
Network Service
glance-api
glance-registry
glance
database
OpenStack
Block Storage
OpenStack
Block Storage API
cinder-api
cinder-volume
OpenStack
Identity
API
quantum-server
quantum
plugin(s)
OpenStack
Identity
API
cinder-scheduler
cinder
database
OpenStack
Network API
AMQP
AMQP
network
provider
OpenStack
Block Storage API
OpenStack
Network API
policy
backend
(rules, custom)
Queue
OpenStack
Network API
quantum
database
quantum
agent(s)
volume
provider
9http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
Grizzly Development
  No new services
  Simplification of some services
  Maturing of code on others
–  nova-conductor introduced to isolate nova-compute from
database access
10
Grizzly Architecture (2013)
11
Identity
Dashboard
Image
Compute
Object
Storage
Block
Storage
Network
Provides
UI for Provides
UI for
Provides
UI for Provides
UI for
Provides
UI for
Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
volumes
for
Provide
network
connectivity
for
Stores
images in
Stores disk
files in
http://www.solinea.com
http://www.solinea.com/2013/06/15/openstack-grizzly-architecture-revisited/
OpenStack Object Store
OpenStack
Image Service OpenStack Compute
OpenStack
Dashboard
OpenStack
Identity
Service
OpenStack Compute
API /
Admin API
keystone
(service & admin APIs)
nova-api
(OS, EC2, Metadata, Admin)
nova-consoleauth
nova-cert/
objectstore
nova-console
nova-*proxy
VNC/VMRC
/Spice
OpenStack
Object API
http://www.solinea.com
Queue
nova-compute
nova-scheduler
nova
database
OpenStack
Compute API
OpenStack
Image API
Horizon
OpenStack
Image API
identity
backend
swift-proxy
objectcontaineraccount
object
store
account
DB
container
DB
OpenStack
Object API
HTTP(S)
OpenStack
Object API
OpenStack
Identity API
OpenStack
Identity
API
OpenStack
Identity
API
Internet
OpenStack
Image API
OpenStack
Identity
API
OpenStack
Image
API
catalog
backend
token backend
OpenStack
Identity
API
hypervisor
libvirt, XenAPI, etc.
HTTP(S)
Amazon
Web Services
EC2 API
OpenStack
Network Service
glance-api
glance-registry
glance
database
OpenStack
Block Storage
OpenStack
Block Storage API
cinder-api
cinder-volume
quantum-server
quantum
plugin(s)
OpenStack
Identity
API
cinder-scheduler
cinder
database
OpenStack
Network API
network
provider
OpenStack
Block Storage API
OpenStack
Network API
policy
backend
Queue
OpenStack
Network API
quantum
database
quantum
agent(s)
nova-conductor
memcached
OpenStack
Identity
API
⁃ OpenStack Command Line Tools (nova-client, swift-client, etc.)
⁃ Cloud Management Tools (Rightscale, Enstratius, etc.)
⁃ GUI tools (Cyberduck, iPhone client, etc.)
volume provider
cinder-backup
OpenStack Object API
OpenStack
Identity
API
Queue
OpenStack
Block Storage API
Havana Development
  Ceilometer is a metering
project. The project offers
metering. Metering lets you
know what actions have
taken place, rating enables
pricing and line items, and
billing gathers the line items
to create a bill to send to
the consumer and collect
payment.
  Heat provides a REST API to
orchestrate multiple cloud
applications implementing
standards such as AWS
CloudFormation.
12
Havana Architecture (~Oct 2013)
13
Dashboard
Compute
Block
Storage
Network
Provides
UI for Provides
UI for
Provides
UI for
Provides
UI for
Provides
UI for
Provides
Auth for
Provides
Auth for
Provides
Auth for
Provides
Auth for Provides
Auth for
Provides
Auth for
Provides
volumes
for
Provide
connectivity
for
Stores
images in
Stores disk
files in
Identity
Object
Storage
Image
http://www.solinea.com
Orchestraton
Metering
Meters
usage of
Meters
usage of
Meters
usage of
Meters
usage of
Automates Automates
Automates
Automates
14
  November 5-8, 2013 – Hong Kong!
  Registration and sponsorships now open
–  New: Two tiers of registration, please read carefully
  Call for speakers deadline July 31
  Book your travel early, room blocks are filling up!
  Travel Assistance Program – applications in July
  More details at openstack.org/summit
Ken Pepple
ken@solinea.com
http://www.solinea.com
Accelerating the adoption of Cloud Computing

OpenStack Architecture: Past and Future

  • 1.
    OpenStack Architecture Past and Future KenPepple Chief Technology Officer http://www.solinea.com HAPPY 3RD BIRTHDAY AUTOMATEALLTHETHINGS
  • 2.
    About Me   CTO,Solinea   Former Director of Cloud Development, Internap Public Cloud   Author of O’Reilly “Deploying OpenStack”   OpenStack Code contributor since Bexar release   Twitter @ken_pepple   IRC kpepple 2
  • 3.
    Early Development (Austin– Diablo)   OpenStack started with Nova and Swift   No interaction between two services until Glance in Bexar release   Developed completely on Launchpad   My humble first commit 3 commit 16eeac71055ffa9fe0fc7a13032da4e6397121b1 Author: Ken Pepple <ken.pepple@gmail.com> Date: Sat Jan 8 17:40:06 2011 -0800 fixed doc make process for new nova version (rev530) mechanism
  • 4.
    Bexar/Cactus/Diablo Architecture (2011) 4 Image Compute Object Storage Stores imagesin Stores disk files in Conceptual Architecture Logical Architecture
  • 5.
    Essex Development   Webinterface released   Centralization of authentication released on second try when “keystone light” was made the full keystone –  https://lists.launchpad.net/openstack/msg07063.html   Moved codebase to GitHub 5
  • 6.
  • 7.
    Essex Architecture (2012) ObjectImage Compute Dashboard Identity authenticates with authenticates withauthenticates with retrieves / stores images in provides UI for provides UI forprovides UI for retrieves/stores disk files in http://ken.pepple.info 7 OpenStack Object Store OpenStack Image Service OpenStack Compute OpenStack Dashboard OpenStack Identity Service OpenStack Compute API / Admin API keystone (service & admin APIs) nova-api (OS, EC2, Admin) nova-consoleauth nova-cert/ objectstore nova-console VNC/VMRC OpenStack Object API Queue nova-compute nova-network nova-volume nova-scheduler nova database volume provider (iSCSI, etc) OpenStack Compute API OpenStack Image API Horizon OpenStack Image API identity backend (kvs, pam, sql, etc.) swift-proxy objectcontaineraccount object store account DB container DB OpenStack Object API HTTP(S) OpenStack Object API Horizon Database OpenStack Identity API OpenStack Identity API OpenStack Identity API OpenStack End Users OpenStack Image API Amazon Web Services EC2 API OpenStack Identity API OpenStack Image API catalog backend (kvs, catalog, etc.) token backend (kvs, memcache, etc.) OpenStack Identity API hypervisor libvirt, XenAPI, etc. HTTP(S) glance-api glance-registry glance database http://ken.pepple.info/openstack/2011/04/22/openstack-nova-architecture/
  • 8.
    Folsom Development   Developmentand project pace quickens   Cinder and Quantum (now Neutron) introduced   Major influx of industry vendors support new services 8
  • 9.
    Folsom Architecture (2012) Identity Dashboard Image Compute Object Storage Block Storage Network Provides UIfor Provides UI for Provides UI for Provides UI for Provides UI for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides volumes for Provide network connectivity for Stores images in Stores disk files in http://ken.pepple.info OpenStack Object Store OpenStack Image Service OpenStack Compute OpenStack Dashboard OpenStack Identity Service OpenStack Compute API / Admin API keystone (service & admin APIs) nova-api (OS, EC2, Admin) nova-consoleauth nova-cert/ objectstore nova-console VNC/VMRC OpenStack Object API http://ken.pepple.info Queue nova-compute nova-network nova-volume nova-scheduler nova database OpenStack Compute API OpenStack Image API Horizon OpenStack Image API identity backend (kvs, pam, sql, ldap, etc.) swift-proxy objectcontaineraccount object store account DB container DB OpenStack Object API HTTP(S) OpenStack Object API Horizon Database OpenStack Identity API OpenStack Identity API OpenStack Identity API OpenStack End Users OpenStack Image API OpenStack Identity API OpenStack Image API catalog backend (kvs, sql, etc.) token backend (kvs, memcache, etc.) OpenStack Identity API hypervisor libvirt, XenAPI, etc. HTTP(S) Amazon Web Services EC2 API OpenStack Network Service glance-api glance-registry glance database OpenStack Block Storage OpenStack Block Storage API cinder-api cinder-volume OpenStack Identity API quantum-server quantum plugin(s) OpenStack Identity API cinder-scheduler cinder database OpenStack Network API AMQP AMQP network provider OpenStack Block Storage API OpenStack Network API policy backend (rules, custom) Queue OpenStack Network API quantum database quantum agent(s) volume provider 9http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
  • 10.
    Grizzly Development   Nonew services   Simplification of some services   Maturing of code on others –  nova-conductor introduced to isolate nova-compute from database access 10
  • 11.
    Grizzly Architecture (2013) 11 Identity Dashboard Image Compute Object Storage Block Storage Network Provides UIfor Provides UI for Provides UI for Provides UI for Provides UI for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides volumes for Provide network connectivity for Stores images in Stores disk files in http://www.solinea.com http://www.solinea.com/2013/06/15/openstack-grizzly-architecture-revisited/ OpenStack Object Store OpenStack Image Service OpenStack Compute OpenStack Dashboard OpenStack Identity Service OpenStack Compute API / Admin API keystone (service & admin APIs) nova-api (OS, EC2, Metadata, Admin) nova-consoleauth nova-cert/ objectstore nova-console nova-*proxy VNC/VMRC /Spice OpenStack Object API http://www.solinea.com Queue nova-compute nova-scheduler nova database OpenStack Compute API OpenStack Image API Horizon OpenStack Image API identity backend swift-proxy objectcontaineraccount object store account DB container DB OpenStack Object API HTTP(S) OpenStack Object API OpenStack Identity API OpenStack Identity API OpenStack Identity API Internet OpenStack Image API OpenStack Identity API OpenStack Image API catalog backend token backend OpenStack Identity API hypervisor libvirt, XenAPI, etc. HTTP(S) Amazon Web Services EC2 API OpenStack Network Service glance-api glance-registry glance database OpenStack Block Storage OpenStack Block Storage API cinder-api cinder-volume quantum-server quantum plugin(s) OpenStack Identity API cinder-scheduler cinder database OpenStack Network API network provider OpenStack Block Storage API OpenStack Network API policy backend Queue OpenStack Network API quantum database quantum agent(s) nova-conductor memcached OpenStack Identity API ⁃ OpenStack Command Line Tools (nova-client, swift-client, etc.) ⁃ Cloud Management Tools (Rightscale, Enstratius, etc.) ⁃ GUI tools (Cyberduck, iPhone client, etc.) volume provider cinder-backup OpenStack Object API OpenStack Identity API Queue OpenStack Block Storage API
  • 12.
    Havana Development   Ceilometeris a metering project. The project offers metering. Metering lets you know what actions have taken place, rating enables pricing and line items, and billing gathers the line items to create a bill to send to the consumer and collect payment.   Heat provides a REST API to orchestrate multiple cloud applications implementing standards such as AWS CloudFormation. 12
  • 13.
    Havana Architecture (~Oct2013) 13 Dashboard Compute Block Storage Network Provides UI for Provides UI for Provides UI for Provides UI for Provides UI for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides Auth for Provides volumes for Provide connectivity for Stores images in Stores disk files in Identity Object Storage Image http://www.solinea.com Orchestraton Metering Meters usage of Meters usage of Meters usage of Meters usage of Automates Automates Automates Automates
  • 14.
    14   November 5-8,2013 – Hong Kong!   Registration and sponsorships now open –  New: Two tiers of registration, please read carefully   Call for speakers deadline July 31   Book your travel early, room blocks are filling up!   Travel Assistance Program – applications in July   More details at openstack.org/summit
  • 15.