© 2014 VMware Inc. All rights reserved.
OpenStack 101
Or: “Take the red pill”
Mark T. Voelker, OpenStack Architect
Oct. 20, 2015
All Things Open 2015
CONFIDENTIAL 2
Remember the first time you
were asked to stand up a
distributed system?
Or maybe a series of
cooperating ones?
CONFIDENTIAL 3
Turns out, distributed systems are both very powerful
…and sometimes hard to learn.
CONFIDENTIAL 4
But at least the fine manuals are all pretty clear for these things.
…right?
?
CONFIDENTIAL 5
A little help making that first leap goes a long way.
So where do you start?
6
“A computer nerd….is somebody who uses a computer in order to use a computer.”
–Douglas Adams
Mark T. Voelker (@marktvoelker)
• OpenStack Architect @ VMware, OpenStack ATC, Former Puppet-OpenStack core dev, Triangle
OpenStack Meetup founder, OS Foundation Member #54, DefCore Committee Member
• Fact: can be bribed with doughnuts
• OpenStack community member since 2011, VMware since 11 months ago.
• In copious (hah!) spare time: data center OS’s, Big Data, Massively Scalable Data Centers, DevOps,
outdoorsey stuff, making sawdust
Let’s begin.
“OpenStack software controls large pools of compute, storage, and networking resources throughout a datacenter,
managed through a dashboard or via the OpenStack API. OpenStack works with popular enterprise and open source
technologies making it ideal for heterogeneous infrastructure…The software is built by a thriving community of
developers, in collaboration with users, and is designed in the open…”
What if I told you….
…ok, let me rephrase that.
Basically: it’s software to run cloud services (compute, storage, network, etc) and
the community behind that software.
History
• Founded July 2010 by Rackspace, NASA, & friends
– NASA contributed a compute controller (Nova)
– Rackspace contributed an object storage controller (Swift)
• 12th release (Liberty) just went live a few days ago
– The project now follows a 6-month release cycle
• Hundreds of companies and thousands of people contribute
– 1,933 contributors from 164 organizations in the last release
– It’s probably easier to list IT/cloud companies that aren’t involved somehow than it is to list the ones that are:
Structure
Structure
• The OpenStack Foundation
– Membership free for individuals
– Platinum, Gold, Corporate memberships paid for by member companies
– Board of Directors comprised of Platinum, some Gold, and generally elected members
• Provides strategic & financial oversight of Foundation resources & staff
– Controls the OpenStack brand & logo, coordinates events, etc
• The Technical Committee
– Provides technical leadership for OpenStack as a whole
– Enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality)
– Elected by active technical contributors (ATC’s) to the project
• Project Team Leaders
– Elected to lead individual projects by contributors to those projects
• User Committee
– Represents users with the Technical Committee and Board of Directors
All to give you this: power.
The power to build a cloud out of the stuff you want, or use one
Someone else has built for you to consume.
How to Participate
• IRC Channels and Mailing Lists
• User/Meetup Groups
– Local to RTP? http://meetup.com/Triangle-OpenStack-Meetup/
• Code is in git, reviewed in Gerrit, mirrored on GitHub, Bugs in Launchpad
• Two annual Design Summit/Conferences
– I leave for Tokyo this Saturday
– Spring 2016: Austin
– Fall 2016: Barcelona
• There’s a welcome guide here.
• You can try out OpenStack as a user here or get a development instance running on your
laptop to start hacking on here.
What are people using it for?
Much more info from
annual User Survey data
With hundreds of projects
available, it’s a very flexible
platform.
You get to choose which
projects you deploy.
But a few
projects are
where the
majority of the
focus (and
users) are.
Horizon (GUI)
Nova (Compute) Neutron (Networking)
Swift (Object Storage)
Cinder (Block storage)
Glance
(VM Image Service)
Keystone
(Identity Service)
OpenStack: The Software
AWS Management Console
EC2 VPC/ELB
S3
EBS
Ceilometer
(Telemetry Service)
Trove (Database as a Service)
Heat (Orchestration)
Sahara (Data Processing)
EMR
RDS
AWS CloudFormation
and AWS comparables
DefCore: the new interoperability standard for OpenStack Powered™ products
• The DefCore Committee creates Guidelines to
which products must adhere if they use the
OpenStack name or OpenStack Powered logo.
• Guidelines contain a list of Capabilities that
products must expose and tests they must pass in
order to prove it.
• Guidelines also contain Designated Sections of
code products must use to provide those
Capabilities
Keystone: the Identity Service
• Provides a central service for authentication and authorization as well as a service catalog
(e.g. a list of where the API endpoints of other services are)
• Abstracts various backend auth services
– SQL databases
– LDAP/AD
• Uses a bearer token model
– Clients are assigned a token which they present to other services in headers
• Multiple token types
– UUID
– PKI/PKIz
– Fernet
• Can federate
– E.g. use another keystone as a service provider via SAML assertion
Primitives:
• Tokens
• Services
• Endpoints
• Domains
• Projects
• Groups
• Credentials
• Roles
• Policies
Glance:
The Image Service
• Houses virtual machine images that can later be launched as instances
• Abstracts various image containers and disk formats
– Note: this does not mean that we magically make all image formats work on all hypervisors although there
are some ways to do conversion
– Bare, OVF, AKI, ARI, AMI
– Qcow2, raw, VHD, AKI/ARI/AMI, ISO, VDI, VMDK
• Multiple storage backends
– File, Swift, Ceph, etc
Primitives:
• Images
• Metadata
• Tags
• Tasks
Nova: The Compute Controller
• Basically, it’s what takes care of launching VM instances (think Amazon EC2) and plugging
things into them.
• Nova abstracts hypervisors and pools of computer hardware.
• Most operations can be involved with a REST API call, a CLI client, or a few clicks in the
Horizon web GUI.
• A few high-level features:
– Supports most hypervisors
– Distributed, mostly asynchronous architecture
– Public REST API, SQL backend DB, AMQP for RPC
– Supports security groups
– Several means of providing resource segregation
• Host aggregates
• Availability zones
• Regions
• Cells
Some primitives:
• Flavors
• Servers
• Keypairs
• Quotas
• Aggregates
Neutron: the network controller
• Provides tenants with the ability to create isolated or shared L2 and L3 virtual networks,
route between them, and connect compute instances to them
• Abstracts various networking backends
– SDN controllers
– Physical switches
– Dozens of backend plugins, both open source and proprietary
• Supports IPv4 and IPv6 (depending on plugin)
• Organized as a “stadium” project to house many networking subprojects with their own
lieutenants
• Can also provide L4-L7 services
– Load Balancing as a Service
– VPN as a Service
– Firewall as a Service
Primitives:
• Networks
• Subnets
• Ports
• Quotas
• Security groups
• Routers
• VIPs
• Health Monitors
• Pools
• Members
Swift: The object storage service
• Provides highly available, distributed, eventually consistent object storage.
• Can (and often is) run completely independently of OpenStack Compute.
• Optimized around durability and availability
• One of the most long-lived API’s in OpenStack (still v1)
• Similar in some respects to Hadoop HDFS and Amazon S3
– Replicates objects over multiple machines (usually at least 3)
– Replicas lost due to hardware failures can be re-replicated
– Clusters can be rebalanced
Primitives:
• Accounts
• Containers
• Objects
Cinder: the block storage service
• Provides persistent block storage volumes to compute instances
• Abstracts underlying storage systems
• Originally part of Nova itself, but split out into it’s own project since the Folsom release
• Dozens of drivers
– EMC, NetApp, LVM, VMware, Gluster, Nexenta, NFS, Ceph, SolidFire, etc etc etc
• Volumes appear to instances as block devices
– E.g. a virtual hard drive
• There’s a separate service for shared file systems called Manila
Primitives:
• Volumes
• Backups
• Snapshots
• Quota sets
• QoS Specs
And much more!
• Documentation
• CI & Infrastructure
• Client libraries
• Oslo (common libs)
• DevStack
• Tempest (integration tests)
• Rally (benchmarking/scale
test)
• Modules for deploying with
Ansible, Puppet, Chef, Salt
• Metering service
• DNS as a Service
• Data Processing Service
• Bare metal service
• Container service
• Orchestration service
• Key management service
• Queue service
• Database as a service
Questions?
Thank You
@marktvoelker

OpenStack 101 - All Things Open 2015

  • 1.
    © 2014 VMwareInc. All rights reserved. OpenStack 101 Or: “Take the red pill” Mark T. Voelker, OpenStack Architect Oct. 20, 2015 All Things Open 2015
  • 2.
    CONFIDENTIAL 2 Remember thefirst time you were asked to stand up a distributed system? Or maybe a series of cooperating ones?
  • 3.
    CONFIDENTIAL 3 Turns out,distributed systems are both very powerful …and sometimes hard to learn.
  • 4.
    CONFIDENTIAL 4 But atleast the fine manuals are all pretty clear for these things. …right? ?
  • 5.
    CONFIDENTIAL 5 A littlehelp making that first leap goes a long way. So where do you start?
  • 6.
  • 7.
    “A computer nerd….issomebody who uses a computer in order to use a computer.” –Douglas Adams Mark T. Voelker (@marktvoelker) • OpenStack Architect @ VMware, OpenStack ATC, Former Puppet-OpenStack core dev, Triangle OpenStack Meetup founder, OS Foundation Member #54, DefCore Committee Member • Fact: can be bribed with doughnuts • OpenStack community member since 2011, VMware since 11 months ago. • In copious (hah!) spare time: data center OS’s, Big Data, Massively Scalable Data Centers, DevOps, outdoorsey stuff, making sawdust
  • 8.
  • 9.
    “OpenStack software controlslarge pools of compute, storage, and networking resources throughout a datacenter, managed through a dashboard or via the OpenStack API. OpenStack works with popular enterprise and open source technologies making it ideal for heterogeneous infrastructure…The software is built by a thriving community of developers, in collaboration with users, and is designed in the open…” What if I told you….
  • 10.
    …ok, let merephrase that.
  • 11.
    Basically: it’s softwareto run cloud services (compute, storage, network, etc) and the community behind that software.
  • 12.
    History • Founded July2010 by Rackspace, NASA, & friends – NASA contributed a compute controller (Nova) – Rackspace contributed an object storage controller (Swift) • 12th release (Liberty) just went live a few days ago – The project now follows a 6-month release cycle • Hundreds of companies and thousands of people contribute – 1,933 contributors from 164 organizations in the last release – It’s probably easier to list IT/cloud companies that aren’t involved somehow than it is to list the ones that are:
  • 13.
  • 14.
    Structure • The OpenStackFoundation – Membership free for individuals – Platinum, Gold, Corporate memberships paid for by member companies – Board of Directors comprised of Platinum, some Gold, and generally elected members • Provides strategic & financial oversight of Foundation resources & staff – Controls the OpenStack brand & logo, coordinates events, etc • The Technical Committee – Provides technical leadership for OpenStack as a whole – Enforces OpenStack ideals (Openness, Transparency, Commonality, Integration, Quality) – Elected by active technical contributors (ATC’s) to the project • Project Team Leaders – Elected to lead individual projects by contributors to those projects • User Committee – Represents users with the Technical Committee and Board of Directors
  • 15.
    All to giveyou this: power. The power to build a cloud out of the stuff you want, or use one Someone else has built for you to consume.
  • 16.
    How to Participate •IRC Channels and Mailing Lists • User/Meetup Groups – Local to RTP? http://meetup.com/Triangle-OpenStack-Meetup/ • Code is in git, reviewed in Gerrit, mirrored on GitHub, Bugs in Launchpad • Two annual Design Summit/Conferences – I leave for Tokyo this Saturday – Spring 2016: Austin – Fall 2016: Barcelona • There’s a welcome guide here. • You can try out OpenStack as a user here or get a development instance running on your laptop to start hacking on here.
  • 17.
    What are peopleusing it for? Much more info from annual User Survey data
  • 18.
    With hundreds ofprojects available, it’s a very flexible platform.
  • 19.
    You get tochoose which projects you deploy.
  • 20.
    But a few projectsare where the majority of the focus (and users) are.
  • 21.
    Horizon (GUI) Nova (Compute)Neutron (Networking) Swift (Object Storage) Cinder (Block storage) Glance (VM Image Service) Keystone (Identity Service) OpenStack: The Software AWS Management Console EC2 VPC/ELB S3 EBS Ceilometer (Telemetry Service) Trove (Database as a Service) Heat (Orchestration) Sahara (Data Processing) EMR RDS AWS CloudFormation and AWS comparables
  • 22.
    DefCore: the newinteroperability standard for OpenStack Powered™ products • The DefCore Committee creates Guidelines to which products must adhere if they use the OpenStack name or OpenStack Powered logo. • Guidelines contain a list of Capabilities that products must expose and tests they must pass in order to prove it. • Guidelines also contain Designated Sections of code products must use to provide those Capabilities
  • 23.
  • 24.
    • Provides acentral service for authentication and authorization as well as a service catalog (e.g. a list of where the API endpoints of other services are) • Abstracts various backend auth services – SQL databases – LDAP/AD • Uses a bearer token model – Clients are assigned a token which they present to other services in headers • Multiple token types – UUID – PKI/PKIz – Fernet • Can federate – E.g. use another keystone as a service provider via SAML assertion
  • 25.
    Primitives: • Tokens • Services •Endpoints • Domains • Projects • Groups • Credentials • Roles • Policies
  • 26.
  • 27.
    • Houses virtualmachine images that can later be launched as instances • Abstracts various image containers and disk formats – Note: this does not mean that we magically make all image formats work on all hypervisors although there are some ways to do conversion – Bare, OVF, AKI, ARI, AMI – Qcow2, raw, VHD, AKI/ARI/AMI, ISO, VDI, VMDK • Multiple storage backends – File, Swift, Ceph, etc
  • 28.
  • 29.
  • 30.
    • Basically, it’swhat takes care of launching VM instances (think Amazon EC2) and plugging things into them. • Nova abstracts hypervisors and pools of computer hardware. • Most operations can be involved with a REST API call, a CLI client, or a few clicks in the Horizon web GUI. • A few high-level features: – Supports most hypervisors – Distributed, mostly asynchronous architecture – Public REST API, SQL backend DB, AMQP for RPC – Supports security groups – Several means of providing resource segregation • Host aggregates • Availability zones • Regions • Cells
  • 31.
    Some primitives: • Flavors •Servers • Keypairs • Quotas • Aggregates
  • 32.
  • 33.
    • Provides tenantswith the ability to create isolated or shared L2 and L3 virtual networks, route between them, and connect compute instances to them • Abstracts various networking backends – SDN controllers – Physical switches – Dozens of backend plugins, both open source and proprietary • Supports IPv4 and IPv6 (depending on plugin) • Organized as a “stadium” project to house many networking subprojects with their own lieutenants • Can also provide L4-L7 services – Load Balancing as a Service – VPN as a Service – Firewall as a Service
  • 34.
    Primitives: • Networks • Subnets •Ports • Quotas • Security groups • Routers • VIPs • Health Monitors • Pools • Members
  • 35.
    Swift: The objectstorage service
  • 36.
    • Provides highlyavailable, distributed, eventually consistent object storage. • Can (and often is) run completely independently of OpenStack Compute. • Optimized around durability and availability • One of the most long-lived API’s in OpenStack (still v1) • Similar in some respects to Hadoop HDFS and Amazon S3 – Replicates objects over multiple machines (usually at least 3) – Replicas lost due to hardware failures can be re-replicated – Clusters can be rebalanced
  • 37.
  • 38.
    Cinder: the blockstorage service
  • 39.
    • Provides persistentblock storage volumes to compute instances • Abstracts underlying storage systems • Originally part of Nova itself, but split out into it’s own project since the Folsom release • Dozens of drivers – EMC, NetApp, LVM, VMware, Gluster, Nexenta, NFS, Ceph, SolidFire, etc etc etc • Volumes appear to instances as block devices – E.g. a virtual hard drive • There’s a separate service for shared file systems called Manila
  • 40.
    Primitives: • Volumes • Backups •Snapshots • Quota sets • QoS Specs
  • 41.
    And much more! •Documentation • CI & Infrastructure • Client libraries • Oslo (common libs) • DevStack • Tempest (integration tests) • Rally (benchmarking/scale test) • Modules for deploying with Ansible, Puppet, Chef, Salt • Metering service • DNS as a Service • Data Processing Service • Bare metal service • Container service • Orchestration service • Key management service • Queue service • Database as a service
  • 42.
  • 43.

Editor's Notes

  • #8 Hi, I’m Mark Voelker, and I am a software architect who in comparison to modern enterprise apps is only slightly more evolved than the donut I ate for breakfast.