Successfully reported this slideshow.

Openstack 101


Published on

Do you think of cheetahs not RabbitMQ when you hear the word Swift? Think a Nova is just a giant exploding star, not a cloud compute engine. This deck (presented at the OpenStack Boston meetup) provides introduction will answer your many questions. It covers the basic components including: Nova, Swift, Cinder, Keystone, Horizon and Glance.

Published in: Technology

Openstack 101

  1. 1. © 2013 Cloud Technology Partners, Inc.1OpenStack 101Technology IntroductionBoston OpenStack MeetupJune 25th, 2013
  2. 2. © 2013 Cloud Technology Partners, Inc.2Theme: An Open Source Cloud Computing“Is OpenStack the new Linux?Or is this open source cloud operating system just a launchingpad for a million new cloud businesses? Either way, theexcitement is contagious”Oliver Rist,InfoWorld
  3. 3. © 2013 Cloud Technology Partners, Inc.3• What is OpenStack?– A brief history– The OpenStack Foundation• The OpenStack projects– Compute: Nova, Glance– Storage – Swift, Cinder– Networks: Quantum– Tools: Horizon, Ceilometer, Heat– Security: Keystone• Pulling it all together• Deploying OpenStackAgendaLike Linux, OpenStack aims to provide akernel around which all kinds of softwarevendors can build businesses. But withOpenStack, were talking multipleprojects. Its hugely ambitious, perhapsthe most far-reaching open sourceproject ever, although still at a very earlystage.
  4. 4. © 2013 Cloud Technology Partners, Inc.4● Open Source Cloud software● Launched by NASA and Rackspace in 2010● Massively scalable● Managed by the OpenStack Foundation● Rapidly taking over the Cloud world!Quick Start for the Terminally Busy…What is OpenStack?
  5. 5. © 2013 Cloud Technology Partners, Inc.5A Brief History• Jointly founded in July 2010 byRackspace and NASA with mergerof two projects:– Swift object storage based onRackspaces Cloud Files platform– Nova based on Nebula computeplatform• Fastest growing Open Sourceproject to date!Strong Community Support• Red Hat• Rackspace• IBM• HP• Intel• Vmware (through Nicira)• Canonical (Ubuntu)• Cloudscaling• And several hundred more…What is OpenStack?
  6. 6. © 2013 Cloud Technology Partners, Inc.6• Releases are timed tocorrespond with thedeveloper Summit meeting• Currently no reliableupgrade paths betweenreleases• Expect large deltas betweenreleases for the next year orso as new features and corefunctionality are addedSix Month Cycle – Currently working on HavanaOpenStack Project Release History
  7. 7. © 2013 Cloud Technology Partners, Inc.7• Independent home for OpenStack with more than $10 million in funding• Uses the Apache licensing model• Serves developers, users, and the entire ecosystem by providing:– Shared resources– Enable technology vendors targeting the platform• Individual membership is free and accessible to anyoneAn Open Source FoundationWhat is OpenStack?
  8. 8. © 2013 Cloud Technology Partners, Inc.8• Open source software for building private and public clouds– Designed for flexibility and many different use cases– Mix and match components– Kit of administrative tools• Enables multi-tenancy– Quota for different users– Users can be associated with multiple tenants• Provides virtual machines (VM) on demand– Self service provisioning– Snapshotting capability• Storage volumes– Block storage for VM images– Object storage for VM images and arbitrary filesOpen Source Cloud PlatformWhat is OpenStack?
  9. 9. © 2013 Cloud Technology Partners, Inc.9OpenStack: The Open Source Cloud Operating System
  10. 10. © 2013 Cloud Technology Partners, Inc.10High Level Architecture - GrizzlyAdapted from : NetworkingCinderKeystoneNovaGlance Swift
  11. 11. © 2013 Cloud Technology Partners, Inc.11An Open Source Set of Cloud TechnologiesWhat is OpenStack?• Compute (codenamed "Nova") provides virtual servers upon demand.• Image (codenamed "Glance") provides a catalog and repository for virtualdisk images. These disk images are mostly commonly used in OpenStackCompute.• Dashboard (codenamed "Horizon") provides a modular web-based userinterface for all the OpenStack services. Used to perform most operationslike launching an instance, assigning IP addresses and setting accesscontrols.• Identity (codenamed "Keystone") provides authentication andauthorization for all the OpenStack services. It also provides a servicecatalog of services within a particular OpenStack cloud.• Object Store (codenamed "Swift") provides object storage. It allows you tostore or retrieve files (but not mount directories like a fileserver).• Block Storage (codenamed "Cinder") provides persistent block storage toguest VMs.• Network (codenamed “Quantum") provides "network connectivity as aservice" between interface devices managed by other OpenStack services.
  12. 12. © 2013 Cloud Technology Partners, Inc.12OpenStack Overall Architecture - ComputeHorizonQuantumCinderKeystoneNovaGlance Swift
  13. 13. © 2013 Cloud Technology Partners, Inc.13• OpenStack Compute is a tool to orchestrate a cloud, includingrunning instances, managing networks, and controlling access to thecloud through users and projects.• The underlying open source projects name is Nova, and it providesthe software that can control an Infrastructure as a Service (IaaS)cloud computing platform.• It is similar in scope to Amazon EC2 and Rackspace Cloud Servers.• OpenStack Compute does not include any virtualization software;rather it defines drivers that interact with underlying virtualizationmechanisms that run on a host operating system, and exposesfunctionality over a web-based API.OpenStack Nova Compute - Introduction
  14. 14. © 2013 Cloud Technology Partners, Inc.14OpenStack Nova Compute – Message QueueNova-compute is a worker daemon, which primarily creates andterminates VMs via hypervisor API.
  15. 15. © 2013 Cloud Technology Partners, Inc.15OpenStack Nova Compute – Message QueueNova-compute is a worker daemon, which primarily creates andterminates VMs via hypervisor API.Nova-computelibvirtXENVMVMVMwareVMVMKVMVMVMXENVMVMQEMUVMVMLXCVMVM
  16. 16. © 2013 Cloud Technology Partners, Inc.16• The process for selecting a hypervisor usually means prioritizing andmaking decisions based on budget and resource constraints as wellas the list of supported features and required technicalspecifications.– Most development is done on KVM and Xen-based hypervisors• With OpenStack Compute, you can orchestrate clouds using multiplehypervisors in different zones.– KVM - Kernel-based Virtual Machine– LXC - Linux Containers (through libvirt)– QEMU - Quick EMUlator– UML - User Mode Linux– VMWare ESX/ESXi 4.1 update 1– Xen - Xen, Citrix XenServer and Xen Cloud Platform (XCP)– Bare Metal - Provisions physical hardware via pluggable sub-drivers.OpenStack Nova Compute – Supported Hypervisors
  17. 17. © 2013 Cloud Technology Partners, Inc.17GlanceOpenStack Overall Architecture – Image StoreHorizonQuantumCinderKeystoneNovaGlance Swift
  18. 18. © 2013 Cloud Technology Partners, Inc.18• The Glance project provides services for discovering, registering, andretrieving virtual machine images. Glance has a RESTful API that allowsquerying of VM image metadata as well as retrieval of the actual image• Basically a database and some tools• VM images made available through Glance can be stored in a variety oflocations:– Simple filesystems like ZFS, LVM, etc.– Direct attached storage– Object-storage systems like OpenStack Swift project or S3– Block storage like OpenStack Cinder projectGlanceOpenStack Glance – Introduction
  19. 19. © 2013 Cloud Technology Partners, Inc.19OpenStack Glance – Reference Architecture
  20. 20. © 2013 Cloud Technology Partners, Inc.20HorizonOpenStack Overall Architecture - DashboardHorizonQuantumCinderKeystoneNovaGlance Swift
  21. 21. © 2013 Cloud Technology Partners, Inc.21Horizon• Provides a baseline user interfacefor managing OpenStack Services– Stateless– Error handling is delegated to back-end– Doesn’t support all API functions– Can use memcached or database tostore sessions– Gets updated via nova-api pollingBasic Operations• From UI– Login in to Horizon– Specify parameters of VM in “createVM” form– Hit “create” button• Under the Hood– Form parameters are converted toPost data– “Create” request initiates HTTPPOST request to back-end• To Keystone if authorization token isnot cached• To nova-api if authorization tokenhas not yet expiredOpenStack Horizon Dashboard – Introduction
  22. 22. © 2013 Cloud Technology Partners, Inc.22OpenStack Horizon Dashboard – Reference ArchitectureHorizonEnd UsersAdministrators
  23. 23. © 2013 Cloud Technology Partners, Inc.23OpenStack Horizon Dashboard – Example
  24. 24. © 2013 Cloud Technology Partners, Inc.24KeystoneOpenStack Overall Architecture - SecurityAdapted from : Swift
  25. 25. © 2013 Cloud Technology Partners, Inc.25• Keystone provides a single point of integration for OpenStack policy,catalog, token and authentication.– Keystone handles API requests as well as providing configurable catalog,policy, token and identity services.• Communicates via OpenStack Identity API (version 2)• Each Keystone function has a pluggable backend which allowsdifferent ways to use the particular service.– Standard backends include LDAP or SQL, as well as Key Value Stores(KVS).• Most commonly used in delegated authorization deployments• Most people will use this as a point of customization for theircurrent authentication services.OpenStack Security - Keystone Introduction
  26. 26. © 2013 Cloud Technology Partners, Inc.26Keystone relationship to other OpenStack Elements
  27. 27. © 2013 Cloud Technology Partners, Inc.27• The Identity service has two primary functions:– User management: keep track of users and what they are permitted todo– Service catalog: Provide a catalog of what services are available andwhere their API endpoints are located• User management– Users - Represents a human user, and has associated information suchas username, password and email.– Tenants - A project, group, or organization. Must specify a tenant tomake requests to OpenStack services– Roles - Captures what operations a user is permitted to perform in agiven tenant.OpenStack Security - Keystone Basic Concepts$ keystone user-create --name=alice --pass=mypassword123$ keystone tenant-create --name=acme$ keystone role-create --name=compute-user
  28. 28. © 2013 Cloud Technology Partners, Inc.28OpenStack Security - Keystone Architecture Flow
  29. 29. © 2013 Cloud Technology Partners, Inc.29Swift and CinderOpenStack Overall Architecture - StorageHorizonQuantumCinderKeystoneNovaGlance Swift
  30. 30. © 2013 Cloud Technology Partners, Inc.30• Storage is found in many parts of the OpenStack stack, and thediffering types can cause confusion to even experienced cloudengineers.OpenStack Storage - IntroductionOn-instance / ephemeral Volumes block storage (Cinder) Object Storage (Swift)Used for running Operating Systemand scratch spaceUsed for adding additionalpersistent storage to a virtualmachine (VM)Used for storing virtual machineimages and dataPersists until VM is terminated Persists until deleted Persists until deletedAccess associated with a VM Access associated with a VM Available from anywhereImplemented as a filesystemunderlying OpenStack ComputeMounted via OpenStack Block-Storage controlled protocol (forexample, iSCSI)REST APIAdministrator configures sizesetting, based on flavorsSizings based on need Easily scalable for future growthExample: 10GB first disk, 30GB/coresecond diskExample: 1TB "extra hard drive"Example: 10s of TBs of datasetstorage
  31. 31. © 2013 Cloud Technology Partners, Inc.31OpenStack Storage - Concepts
  32. 32. © 2013 Cloud Technology Partners, Inc.32• The two common use cases for providing object storage in acompute cloud are:– To provide users with a persistentstorage mechanism– As a scalable, reliable data storefor virtual machine imagesOpenStack Storage - Uses
  33. 33. © 2013 Cloud Technology Partners, Inc.33OpenStack Storage - Swift Key Features
  34. 34. © 2013 Cloud Technology Partners, Inc.34Unique as possible data storageOpenStack Storage - Swift ArchitectureDiskServer/NodeZoneMultiple zonesmake up a Region
  35. 35. © 2013 Cloud Technology Partners, Inc.35• The Ring– Maps names to entities (accounts, containers, objects) on disk– Uses MD5 hashing for object tags– Stores data on zones, devices, nodes and replicas– Weights can be used to balance the distribution of partitions and nodes– Used by proxy server and storage nodes• Proxy Server– Exposes the public API– Makes routing requests: read, write, modify, etc.• Object Server– Blob storage server– Uses xattrs, binary format– Object location based on path from name hash and timestampOpenStack Storage - Swift System Components
  36. 36. © 2013 Cloud Technology Partners, Inc.36• Services run completely autonomously• Designed for generic hardware• Proxy Services - More CPU and network I/O intensive. If you are using10g networking to the proxy, or are terminating SSL traffic at the proxy,greater CPU power will be required.• Object, Container, and Account Services (Storage Services) are moredisk and network I/O intensive.• The easiest deployment is to install all services on each server– Scales each service out horizontally.– If you need more throughput to either Account or Container Services, theymay each be deployed to their own servers. For example you might use faster(but more expensive) SAS or even SSD drives to get faster disk I/O to thedatabases.Main servicesOpenStack Storage - Swift System Components
  37. 37. © 2013 Cloud Technology Partners, Inc.37OpenStack Storage – Cinder Architecture
  38. 38. © 2013 Cloud Technology Partners, Inc.38• Cinder separates out the persistent block storage functionality that was previouslypart of OpenStack Compute into its own service.• The OpenStack Block Storage API allows for manipulation of volumes, volume types(similar to compute flavors) and volume snapshots.– cinder-api accepts API requests and routes them to cinder-volume for action.– cinder-volume acts upon the requests by reading or writing to the Cinder database tomaintain state, interacting with other processes (like cinder-scheduler) through a messagequeue and directly upon block storage providing hardware or software.– It can interact with a variety of storage providers through a driver architecture.– Available drivers: IBM, SolidFire, NetApp, Nexenta, Zadara, linux iSCSI and other storageproviders.– Much like nova-scheduler, the cinder-scheduler daemon picks the optimal block storageprovider node to create the volume on.• Cinder deployments will also make use of a messaging queue to route informationbetween the cinder processes as well as a database to store volume state.• Like Quantum, Cinder will mainly interact with Nova, providing volumes for itsinstances.Cinder Block StorageOpenStack Storage – Cinder Introduction
  39. 39. © 2013 Cloud Technology Partners, Inc.39OpenStack Overall Architecture - NetworkHorizonQuantumCinderKeystoneNovaGlance SwiftQuantum – Software Defined Networking
  40. 40. © 2013 Cloud Technology Partners, Inc.40• Quantum is an OpenStack project to provide "networking as a service"between interface devices (e.g., vNICs) managed by other OpenStackservices (e.g., nova).• Starting in the Folsom release, Quantum is a core and supported part ofthe OpenStack platformQuantum NetworkOpenStack Network – Quantum Introduction
  41. 41. © 2013 Cloud Technology Partners, Inc.41Sample Deployment ModelOpenStack Network - Quantum Deployment
  42. 42. © 2013 Cloud Technology Partners, Inc.42Plugin and Agent SummaryOpenStack Network - Quantum Architecture
  43. 43. © 2013 Cloud Technology Partners, Inc.43OpenStack Architecture – Putting it all together
  44. 44. © 2013 Cloud Technology Partners, Inc.44Putting it all together:Large Cloud Implementation Case
  45. 45. © 2013 Cloud Technology Partners, Inc.45• A $3B global electronics company• Building a cloud in support of its consumerdivision activities• Major issues• Little in-house cloud expertise• Relying on immature cloud technology• Weak middle management support forprojectOrganization• Build an organizational cloud to support millionsof external customers• Create an IT organization to support the cloud infrastructure• Provide a platform for building future applicationsGoalsAs-Is SituationOpenStack Deployment Case Study
  46. 46. © 2013 Cloud Technology Partners, Inc.46• Independent network requirements for physical server nodes andvirtual machines (VM)• Need to isolate VM networking information from the core network forscaling• Many components interact at different levels of the system stack addscomplexity• Need to isolate networks and separate functions for security• Separate networks by function for traffic shaping• Complex data paths – Data between VM’s, East/West and in and out ofthe system, North/South• OpenStack has a weak high availability architectureNetwork Architecture RequirementsOpenStack Cloud Deployment Case Study
  47. 47. © 2013 Cloud Technology Partners, Inc.47Layer 3 with Virtual NetworkingOpenStack Cloud Deployment Case StudyCloud Backbone NetworkNova Compute Node 1 Nova Compute Node 2iSCSI SANVM VM’s192.168.1.5 SwitchEBGP/30182.196.0.2550.0.0.0/0182.196.0.0/22Virtual SwitchCloud Network EdgeVirtual SwitchSuwon Network EdgeNode 1VM viewEBGP/30Eth0 AS eg. AS64512Eth1/ SAN192.168.1.7182.196.0.1
  48. 48. © 2013 Cloud Technology Partners, Inc.48Deployment Automation Network DiagramOpenStack Cloud Deployment Case StudyCombinedServerStage 2: Build Crowbarserver on Managementnetwork and automatedeployment process forSwift and Nova nodes.IPMI configuredIPMI configuredNeed VPN access todeployment serversand nodes
  49. 49. © 2013 Cloud Technology Partners, Inc.49• Think holistically• Top management needs to actively support cross organizational change• Focus on building in-house expertise in cloud:– Architecture– Networking– Applications– Data center operations• Use the rack as the base unit for scaling• Scale the cloud horizontally, not vertically• Automate, automate, automate!Advice from the TrenchesOpenStack Cloud Deployment Case Study
  50. 50. © 2013 Cloud Technology Partners, Inc.50Havana Architecture
  51. 51. © 2013 Cloud Technology Partners, Inc.51• Metering (Ceilometer): Central collection fro metering/monitoringdata– Example: Collect usage information for billing systems• Orchestration (Heat): Template-based orchestration engine forOpenStack– Example: Developers define application deployment patterns• LBaaS: Load Balancer server connected to Quantum– Expect to see more plugins and ecosystem activity• Bare Metal Provisioning (Ironic) incubator project• Havana Release Features andStatus: Havana Roadmap Highlights
  52. 52. © 2013 Cloud Technology Partners, Inc.52• Integrated Projects (Havana release)– OpenStack Compute (nova):– OpenStack Object Storage (swift):– OpenStack Image Service (glance):– OpenStack Identity (keystone):– OpenStack Dashboard (horizon):– OpenStack Networking (quantum):– OpenStack Block Storage service (cinder):– Ceilometer:– Heat:• Incubated Projects (Havana release)– Trove – Formerly Reddwarf, Database as a Service for Open Stack.– Ironic - Bare metal hypervisor API and a set of plugins which interact with thebare metal hypervisors. By default, it will use PXE and IPMI in concert toprovision and turn on/off machinesHavana Official Projects
  53. 53. © 2013 Cloud Technology Partners, Inc.53• Related - unofficial projects with no rights to use OpenStack brand andassets or project resources• TripleO - vision that Openstack can be used to deploy Openstack at amassive scale• Designate - provides DNS-as-a-service for OpenStack• Marconi - message queueing service• Savanna - easily provision and manage Hadoop clusters on OpenStack• Murano - allow a non-experienced user to deploy reliable Windowsbased environments in a “push-the-button” manner• Convection - TaskSystem-as-a-Service project for cloud workloadsUnofficial/related ProjectsOpenStack Havana Incubator Projects
  54. 54. © 2013 Cloud Technology Partners, Inc.54• - Main site•• - Current OpenStack computer AdministrationManual – Dec 18, 2012• - How To Get Started WithOpenStack• -OpenStack Folsom Install Guide• -• OpenStack Cloud Computing Cookbook• - Tag: TrainingAdditional Resources
  55. 55. © 2013 Cloud Technology Partners, Inc.55OpenStack 101 TechnologyIntroductionQuestions?