Open stack architecture overview-meetup-6-6_2013

24,279 views
24,047 views

Published on

Published in: Technology
11 Comments
200 Likes
Statistics
Notes
No Downloads
Views
Total views
24,279
On SlideShare
0
From Embeds
0
Number of Embeds
723
Actions
Shares
0
Downloads
0
Comments
11
Likes
200
Embeds 0
No embeds

No notes for slide

Open stack architecture overview-meetup-6-6_2013

  1. 1. © MIRANTIS 2013 PAGE 1© MIRANTIS 2013Module 1:OpenStackArchitectureOverviewMirantis, 2013
  2. 2. © MIRANTIS 2013 PAGE 2Goals
  3. 3. © MIRANTIS 2013 PAGE 3Goals• Understand OpenStack purpose and use casesUnderstand the OpenStack ecosystem• Definition• History• Projects
  4. 4. © MIRANTIS 2013 PAGE 4Goals• Understand OpenStack purpose and use casesUnderstand the OpenStack ecosystem• Definition• History• ProjectsBACKGROUND
  5. 5. © MIRANTIS 2013 PAGE 5Goals• Understand OpenStack purpose and use casesUnderstand the OpenStack ecosystem• Definition• History• Projects• Understand OpenStack architecture• Logical architecture• Provision VM request flow• Components detailsBACKGROUND
  6. 6. © MIRANTIS 2013 PAGE 6Goals• Understand OpenStack purpose and use casesUnderstand the OpenStack ecosystem• Definition• History• Projects• Understand OpenStack architecture• Logical architecture• Provision VM request flow• Components detailsBACKGROUNDTHEORY
  7. 7. © MIRANTIS 2013 PAGE 7What is OpenStack?As described by the OpenStack Foundation:"Open source software for buildingprivate and public clouds“
  8. 8. © MIRANTIS 2013 PAGE 8OpenStack as IaaS
  9. 9. © MIRANTIS 2013 PAGE 9OpenStack Capabilities
  10. 10. © MIRANTIS 2013 PAGE 10OpenStack Capabilities• VMs on demand• provisioning• snapshotting
  11. 11. © MIRANTIS 2013 PAGE 11OpenStack Capabilities• VMs on demand• provisioning• snapshotting• Volumes
  12. 12. © MIRANTIS 2013 PAGE 12OpenStack Capabilities• VMs on demand• provisioning• snapshotting• Volumes• Networks
  13. 13. © MIRANTIS 2013 PAGE 13OpenStack Capabilities• VMs on demand• provisioning• snapshotting• Volumes• Networks• Object storage for VM images and arbitrary files
  14. 14. © MIRANTIS 2013 PAGE 14OpenStack Capabilities• VMs on demand• provisioning• snapshotting• Volumes• Networks• Object storage for VM images and arbitrary files• Multi-tenancy• quotas for different tenants• user can be associated with multiple tenants
  15. 15. © MIRANTIS 2013 PAGE 15OpenStack HistoryDate Rel Projects Type NoteJul 2010 N/A PoC* Rackspace Hosting & NASAjoint launchOct 2010 Austin Nova, Swift PoCFeb 2011 Bexar Nova, Glance, Swift PoCApr 2011 Cactus Nova, Glance, Swift PoC** 6 month developmentcycle startsSep 2011 Diablo Nova, Glance, Swift Prod1stproduction release(Cactus) at Internap (10/27)Apr 2012 EssexNova, Glance, Swift, Horizon,KeystoneProdCommon web UI and sharedauthentication mechanismaddedSep 2012 FolsomNova, Glance, Swift, Horizon,Keystone, Quantum, Cinder, OsloProdOpenStack FoundationEstablishedApr 2013 GrizzlyNova, Glance, Swift, Horizon,Keystone, Quantum, Cinder, OsloProdCeilometer and Heatintegration projects addedOct 2013 HavanaNova, Glance, Swift, Horizon,Keystone, Quantum, Cinder, Oslo,Heat, CeilometerProd Coming Soon!** Follows similar Ubuntu 6 month release cycle* Pre-July 2010 is predicated by Rackspace Cloud Files project (Swift), NASA Nebula project (Nova)
  16. 16. © MIRANTIS 2013 PAGE 16OpenStack Grizzly Projects• Core Projects:• Nova (Compute Service)• Glance (Image Service)• Quantum (Network Service)• Cinder (Block Storage Service)• Swift (Object Store Service)• Common Projects:• Keystone (Identity Service)• Horizon (Dashboard)• Library Projects:• Oslo (Shared Infrastructure Code)• Incubated Projects (Coming in Havana)• Ceilometer (Metering/Monitoring)• Heat (Orchestration)
  17. 17. © MIRANTIS 2013 PAGE 17Each OpenStack Project
  18. 18. © MIRANTIS 2013 PAGE 18Each OpenStack Project• Is also a “top-level” OpenStack component
  19. 19. © MIRANTIS 2013 PAGE 19Each OpenStack Project• Is also a “top-level” OpenStack component• Has an elected “Project Technical Lead” (PTL)
  20. 20. © MIRANTIS 2013 PAGE 20Each OpenStack Project• Is also a “top-level” OpenStack component• Has an elected “Project Technical Lead” (PTL)• Has separate developers and design teams
  21. 21. © MIRANTIS 2013 PAGE 21Each OpenStack Project• Is also a “top-level” OpenStack component• Has an elected “Project Technical Lead” (PTL)• Has separate developers and design teams• Has a well defined public API• With the exception of Horizon, which is the Web GUI, allother projects have a RESTfull (JSON/HTTP) API
  22. 22. © MIRANTIS 2013 PAGE 22Each OpenStack Project• Is also a “top-level” OpenStack component• Has an elected “Project Technical Lead” (PTL)• Has separate developers and design teams• Has a well defined public API• With the exception of Horizon, which is the Web GUI, allother projects have a RESTfull (JSON/HTTP) API• Has a separate database and isolatedpersistent layer
  23. 23. © MIRANTIS 2013 PAGE 23OpenStack Projects: Begining (Cactus)
  24. 24. © MIRANTIS 2013 PAGE 24OpenStack Projects: Now (Grizzly)
  25. 25. © MIRANTIS 2013 PAGE 25Dev Trends• Decoupling of features (e.g. Nova-volumebecame Cinder)• API-s to communicate• Common generic API/Infrastructure (Oslo)• Backends & drivers (everything’s pluggable)
  26. 26. © MIRANTIS 2013 PAGE 26OpenStack Projects Relationships
  27. 27. © MIRANTIS 2013 PAGE 27OpenStack Projects Relationships
  28. 28. © MIRANTIS 2013 PAGE 28OpenStack Projects Relationships
  29. 29. © MIRANTIS 2013 PAGE 29OpenStack Projects RelationshipsMySQLDatabaseNot a project, butimportant tounderstand therelationship
  30. 30. © MIRANTIS 2013 PAGE 30OpenStack Projects RelationshipsMySQLDatabaseNot a project, butimportant tounderstand therelationshipRabbitMQ Not a project, butimportant tounderstand therelationship
  31. 31. © MIRANTIS 2013 PAGE 31Deployment – Pick up WhatYou Want• The components can be mixed & matched• Base:• Nova• Keystone• Dashboard• Glance• Mutually exclusive scenarios• Some components can conflict
  32. 32. © MIRANTIS 2013 PAGE 32OpenStack: “Typical” Deployment TopologyControl ClusterHorizon KeystoneGlanceQuantumCinderCloud Ctrl.(nova)Compute ClustercomputenodecomputenodecomputenodecomputenodecomputenodecomputenodeStorage Cluster (Swift)storagenodestoragenodestoragenodestoragenodestoragenodestoragenode
  33. 33. © MIRANTIS 2013 PAGE 33OpenStack: “Typical” Deployment TopologyControl ClusterHorizon KeystoneGlanceQuantumCinderCloud Ctrl.(nova)Compute ClustercomputenodecomputenodecomputenodecomputenodecomputenodecomputenodeStorage Cluster (Swift)storagenodestoragenodestoragenodestoragenodestoragenodestoragenodeHeavy CPU andRAM
  34. 34. © MIRANTIS 2013 PAGE 34OpenStack: “Typical” Deployment TopologyControl ClusterHorizon KeystoneGlanceQuantumCinderCloud Ctrl.(nova)Compute ClustercomputenodecomputenodecomputenodecomputenodecomputenodecomputenodeStorage Cluster (Swift)storagenodestoragenodestoragenodestoragenodestoragenodestoragenodeHeavy CPU andRAM Heavy Disk and I/O
  35. 35. © MIRANTIS 2013 PAGE 35NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellOpenStack Projects:Communication TypesHTTPAMQPSQLNative APIiSCSISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLI
  36. 36. © MIRANTIS 2013 PAGE 36NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellOpenStack Projects:Communication TypesHTTPAMQPSQLNative APIiSCSISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWEvery OpenStack service exposes accessto restful API via HTTPUI: Horizon or CLI
  37. 37. © MIRANTIS 2013 PAGE 37NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellOpenStack Projects:Communication TypesHTTPAMQPSQLNative APIiSCSISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWEvery OpenStack service exposes accessto restful API via HTTPUI: Horizon or CLIEach action treated as distributedtransaction, state built as MQ messages
  38. 38. © MIRANTIS 2013 PAGE 38NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellOpenStack Projects:Communication TypesHTTPAMQPSQLNative APIiSCSISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWEvery OpenStack service exposes accessto restful API via HTTPEach service updates it’s own DB withstate information as actions are performedUI: Horizon or CLIEach action treated as distributedtransaction, state built as MQ messages
  39. 39. © MIRANTIS 2013 PAGE 39NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellOpenStack Projects:Communication TypesHTTPAMQPSQLNative APIiSCSISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWEvery OpenStack service exposes accessto restful API via HTTPEach service updates it’s own DB withstate information as actions are performedUI: Horizon or CLIEach action treated as distributedtransaction, state built as MQ messagesDirectaccess calls,ex. Plugins,NetApp,Nicira, etc.
  40. 40. © MIRANTIS 2013 PAGE 40NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellOpenStack Projects:Communication TypesHTTPAMQPSQLNative APIiSCSISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWCinder Blockstorageprovided asiSCSI storageto VMsEvery OpenStack service exposes accessto restful API via HTTPEach service updates it’s own DB withstate information as actions are performedUI: Horizon or CLIEach action treated as distributedtransaction, state built as MQ messagesDirectaccess calls,ex. Plugins,NetApp,Nicira, etc.
  41. 41. © MIRANTIS 2013 PAGE 41OpenStack REST API• OpenStack public API is a RESTful API
  42. 42. © MIRANTIS 2013 PAGE 42OpenStack REST API• OpenStack public API is a RESTful API• REST stands for Representational State Transfer
  43. 43. © MIRANTIS 2013 PAGE 43OpenStack REST API• OpenStack public API is a RESTful API• REST stands for Representational State Transfer• REST is a stateless client/server protocol with a uniforminterface for accessing the object model
  44. 44. © MIRANTIS 2013 PAGE 44OpenStack REST API• OpenStack public API is a RESTful API• REST stands for Representational State Transfer• REST is a stateless client/server protocol with a uniforminterface for accessing the object model• OpenStack RESTful API is implemented using HTTPGET/PUT/POST/DELETE in combination with JSON fordata
  45. 45. © MIRANTIS 2013 PAGE 45Part 1 Recap
  46. 46. © MIRANTIS 2013 PAGE 46Part 1 Recap• OpenStack – open source software for buildingIaaS
  47. 47. © MIRANTIS 2013 PAGE 47Part 1 Recap• OpenStack – open source software for buildingIaaS• OpenStack release cycle is every 6 month
  48. 48. © MIRANTIS 2013 PAGE 48Part 1 Recap• OpenStack – open source software for buildingIaaS• OpenStack release cycle is every 6 month• OpenStack is an umbrella over multipleindependent projects (components)
  49. 49. © MIRANTIS 2013 PAGE 49Part 1 Recap• OpenStack – open source software for buildingIaaS• OpenStack release cycle is every 6 month• OpenStack is an umbrella over multipleindependent projects (components)• All OpenStack Components talk RESTful API
  50. 50. © MIRANTIS 2013 PAGE 50Part 1 Recap• OpenStack – open source software for buildingIaaS• OpenStack release cycle is every 6 month• OpenStack is an umbrella over multipleindependent projects (components)• All OpenStack Components talk RESTful API• Most OpenStack Components have dedicated DB(SQL) and MQ (QP), some talk to 3rd partycomponents using their native APIs
  51. 51. © MIRANTIS 2013 PAGE 51Use case: Provision VM• Most common and complex process• Interacts with all OpenStack components
  52. 52. © MIRANTIS 2013 PAGE 52Initial StateAssumes Tenant is created,provisioning quota is available, userhas an access to Horizon/CLICloud Operator, DevOp, etc.SwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLINovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  53. 53. © MIRANTIS 2013 PAGE 53Step 1: Request Provisioning– From UI
  54. 54. © MIRANTIS 2013 PAGE 54Step 1: Request Provisioning– From UI• Login to Horizon
  55. 55. © MIRANTIS 2013 PAGE 55Step 1: Request Provisioning– From UI• Login to Horizon• Specify params of VM• VM Name• Image (OS type)• Flavor (specifies CPU, Memory, Disk)• Network (required for Folsom or later)• Optional (SSH Keys, Persistent volumes, comments, etc.)
  56. 56. © MIRANTIS 2013 PAGE 56Step 1: Request Provisioning– From UI• Login to Horizon• Specify params of VM• VM Name• Image (OS type)• Flavor (specifies CPU, Memory, Disk)• Network (required for Folsom or later)• Optional (SSH Keys, Persistent volumes, comments, etc.)• Hit "Create" button
  57. 57. © MIRANTIS 2013 PAGE 57What is Horizon"The OpenStack Dashboard(Horizon) provides abaseline user interfacefor managing OpenStack services.“
  58. 58. © MIRANTIS 2013 PAGE 58Horizon Notes
  59. 59. © MIRANTIS 2013 PAGE 59Horizon Notes• "Stateless“, no DB
  60. 60. © MIRANTIS 2013 PAGE 60Horizon Notes• "Stateless“, no DB• Error handling is delegated to back-end
  61. 61. © MIRANTIS 2013 PAGE 61Horizon Notes• "Stateless“, no DB• Error handling is delegated to back-end• Doesnt support all API functions
  62. 62. © MIRANTIS 2013 PAGE 62Horizon Notes• "Stateless“, no DB• Error handling is delegated to back-end• Doesnt support all API functions• Can use memcached or database to storesessions
  63. 63. © MIRANTIS 2013 PAGE 63Horizon Notes• "Stateless“, no DB• Error handling is delegated to back-end• Doesnt support all API functions• Can use memcached or database to storesessions• Gets updated via Nova API polling
  64. 64. © MIRANTIS 2013 PAGE 64Horizon Internals• Subprojects• Horizon – generic Python Django libraries andcomponents to work with REST-based back-end / restfulweb service• Openstack Dashboard - web app itself, exposes UI forOpenStack with styles, locale, etc.• Dashboard for each component = Individualnested Django app• Easily modifiable• Modularly developed
  65. 65. © MIRANTIS 2013 PAGE 65Step 1: Request VMProvisioning via UI/CLICloud Operator, DevOp, etc.SwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLIUser logs in to UISpecifies VM params: name,flavor, keys, etc. and hits"Create" buttonNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  66. 66. © MIRANTIS 2013 PAGE 66Step 1: Request Provisioning– Under the Hood
  67. 67. © MIRANTIS 2013 PAGE 67Step 1: Request Provisioning– Under the Hood• Form params are converted to POST data
  68. 68. © MIRANTIS 2013 PAGE 68Step 1: Request Provisioning– Under the Hood• Form params are converted to POST data• "Create" request initiate HTTP POST request toback-end• To Keystone if auth token is not cached – step 2
  69. 69. © MIRANTIS 2013 PAGE 69What is Keystone?"Keystone providesIdentity, Token, Catalog andPolicy servicesfor use specifically by projects in theOpenStack family.“Keystone was developed by the OpenStack community but is written as a “generic” authentication /authorization mechanism for any 2 or more restful API services to communicate
  70. 70. © MIRANTIS 2013 PAGE 70Keystone ArchitectureDeploys with it’s own DB but canalso be integrated with LDAP orother EASContains user,role, andtenant dataContainstemporarytokensRule managementinterface and rule-basedauthorizationContainsendpointregistry
  71. 71. © MIRANTIS 2013 PAGE 71Keystone Data Model• User: has account credentials, is associated with one or moretenants• Tenant: unit of ownership in OpenStack, contains one or moreusers• Role: a first-class piece of metadata associated with manyuser-tenant pairs• Token: identifying credential associated with a user or userand tenant• Extras: bucket of key-value metadata associated with a user-tenant pair• Rule: describes a set of requirements for performing an action
  72. 72. © MIRANTIS 2013 PAGE 72Keystone Key Concept• What service exposes• http://myservice/instances/* - GET/POST/PUT• http://myservice/images/* - GET• How RBAC mapping works• Role X in Tenant Y can do actions A,B,C• User: GET/POST/PUT on instances, images• Admin: GET/POST/PUT on tenants, users, quotas• Each API Service has it’s own RBACenforcement through policy files
  73. 73. © MIRANTIS 2013 PAGE 73Step 2: Validate Auth DataSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLIHorizon sends HTTP request toKeystone. Auth info is specifiedin HTTP headers.NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  74. 74. © MIRANTIS 2013 PAGE 74Step 2: Validate Auth Data• Horizon sends HTTP request to Keystone• Keystone parses HTTP header info and verifiesthat• The credentials are valid (Authentication)• User-Tenant-Role mapping is valid (Access Control)• The requested action is available for this user(Authorization)
  75. 75. © MIRANTIS 2013 PAGE 75Step 2: Validate Auth Data - SuccessSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLIKeystone sends temporary tokenback to Horizon via HTTP.NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  76. 76. © MIRANTIS 2013 PAGE 76Step 1: Request Provisioning– Under the Hood• Form params are converted to POST data• "Create" request initiate HTTP POST request toback-end• To Keystone if auth token is not cached – step 2
  77. 77. © MIRANTIS 2013 PAGE 77Step 1: Request Provisioning– Under the Hood• Form params are converted to POST data• "Create" request initiate HTTP POST request toback-end• To Keystone if auth token is not cached – step 2• To Nova API if auth token hasnt expired yet – step 3
  78. 78. © MIRANTIS 2013 PAGE 78Nova API“Nova API is aRESTful API web servicewhich is used to interact with Nova"
  79. 79. © MIRANTIS 2013 PAGE 79Nova API Characteristics
  80. 80. © MIRANTIS 2013 PAGE 80Nova API Characteristics• Exposes REST API via HTTP
  81. 81. © MIRANTIS 2013 PAGE 81Nova API Characteristics• Exposes REST API via HTTP• Provides system for managing multiple APIs ondifferent sub-domains• EC2-compatible – Starting to be deprecated• Compute API – all innovation happens here
  82. 82. © MIRANTIS 2013 PAGE 82Nova API Characteristics• Exposes REST API via HTTP• Provides system for managing multiple APIs ondifferent sub-domains• EC2-compatible – Starting to be deprecated• Compute API – all innovation happens here• The only "allowed" way to interact with Nova
  83. 83. © MIRANTIS 2013 PAGE 83Nova API Characteristics• Exposes REST API via HTTP• Provides system for managing multiple APIs ondifferent sub-domains• EC2-compatible – Starting to be deprecated• Compute API – all innovation happens here• The only "allowed" way to interact with Nova• Stateless - HA-ready
  84. 84. © MIRANTIS 2013 PAGE 84Nova API ClientsActive effort in the community to makeone CLI to “rule them all”, currentlymultiple CLIs availableOpenStack dashboard iscurrently the only “unified”OpenStack API client
  85. 85. © MIRANTIS 2013 PAGE 85NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 3: Send API Request to Nova APISwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLIHorizon sends POST request toNova API (signed with given token).
  86. 86. © MIRANTIS 2013 PAGE 86NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 4: Validate API TokenSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLINova API sends HTTP request tovalidate API token to Keystone.
  87. 87. © MIRANTIS 2013 PAGE 87Step 4:Validate Token – Keystone API
  88. 88. © MIRANTIS 2013 PAGE 88Keystone /w PKI - TokenValidation• User gets one-time-password on creation• User uses it to establish a key-pair• Public key is signed and stored on Keystone• From this point user uses client certificate to login• Nova API performs offline check of the validity of token using CA&Cert it has from KeystoneKeystoneKeygenerate_cms_token(meta, keystone_key)user/pass/tenantsigned_cms_tokensigned_cms_tokenNovaCA&Cert fromKeystoneverify(signed_cms_token, Cafile,certfile)
  89. 89. © MIRANTIS 2013 PAGE 89NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 4: Validate API Token - SucessSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLIKeystone validates API token andsends HTTP response with tokenacceptance/rejection info.
  90. 90. © MIRANTIS 2013 PAGE 90Step 5:Process API Request Process
  91. 91. © MIRANTIS 2013 PAGE 91Step 5:Process API Request Process• Validate request params• Typographical errors are verified on code level• Cloud-related params are validated via DB requests
  92. 92. © MIRANTIS 2013 PAGE 92Step 5:Process API Request Process• Validate request params• Typographical errors are verified on code level• Cloud-related params are validated via DB requests• If request cannot be processed then throw anexception
  93. 93. © MIRANTIS 2013 PAGE 93Step 5:Process API Request Process• Validate request params• Typographical errors are verified on code level• Cloud-related params are validated via DB requests• If request cannot be processed then throw anexception• If request can be processed• Save initial state to the Database
  94. 94. © MIRANTIS 2013 PAGE 94Nova Database“Nova Database stores currentstate of all objects in computecluster."
  95. 95. © MIRANTIS 2013 PAGE 95Nova Database• In theory can be any relational database• Most of the deployments are done with MySQL orPostgreSQL• Nova API talks to DB via SQLAlchemy (pythonORM (Object Related Mapper))• DB HA should be done via external tools (likeGalera or Multi-Master replication Model forMySQL)
  96. 96. © MIRANTIS 2013 PAGE 96NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 5: Process API RequestSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLINova API parses request topython object model andvalidates it by fetching datafrom Nova DB. If request isvalid, it saves initia db entryabout VM to the database.
  97. 97. © MIRANTIS 2013 PAGE 97Step 5:Process API Request Process• Validate request params• Typographical errors are verified on code level• Cloud-related params are validated via DB requests• If request cannot be processed then throw anexception• If request can be processed• Save initial state to the database
  98. 98. © MIRANTIS 2013 PAGE 98Step 5:Process API Request Process• Validate request params• Typographical errors are verified on code level• Cloud-related params are validated via DB requests• If request cannot be processed then throw anexception• If request can be processed• Save initial state to the database• Send message with next actions to MQ – step 6
  99. 99. © MIRANTIS 2013 PAGE 99Message Queue"Message Queue is a unified way forcollaboration between novacomponents."
  100. 100. © MIRANTIS 2013 PAGE 100Messaging Process Example• 2 modes:• rpc.cast - dont wait for result (fire and forget)• rpc.call - wait for result (when there is something toreturn)
  101. 101. © MIRANTIS 2013 PAGE 101Messaging Process Example• 2 modes:• rpc.cast - dont wait for result (fire and forget)• rpc.call - wait for result (when there is something toreturn)Ex. Nova API Ex. Nova Scheduler
  102. 102. © MIRANTIS 2013 PAGE 102Messaging Process Example• 2 modes:• rpc.cast - dont wait for result (fire and forget)• rpc.call - wait for result (when there is something toreturn)Ex. Nova API Ex. Nova Scheduler
  103. 103. © MIRANTIS 2013 PAGE 103Messaging Process Example• 2 modes:• rpc.cast - dont wait for result (fire and forget)• rpc.call - wait for result (when there is something toreturn)Ex. Nova API Ex. Nova Scheduler
  104. 104. © MIRANTIS 2013 PAGE 104Messaging Process Example• 2 modes:• rpc.cast - dont wait for result (fire and forget)• rpc.call - wait for result (when there is something toreturn)Ex. Nova API Ex. Nova Scheduler
  105. 105. © MIRANTIS 2013 PAGE 105Messaging Process Example• 2 modes:• rpc.cast - dont wait for result (fire and forget)• rpc.call - wait for result (when there is something toreturn)Ex. Nova API Ex. Nova Scheduler
  106. 106. © MIRANTIS 2013 PAGE 106OpenStack Messagings Notes
  107. 107. © MIRANTIS 2013 PAGE 107OpenStack Messagings Notes• Uses multiple queues within single RabbitMQinstance• Used by services to build machine state• Each compute node has a queue for scheduling
  108. 108. © MIRANTIS 2013 PAGE 108OpenStack Messagings Notes• Uses multiple queues within single RabbitMQinstance• Used by services to build machine state• Each compute node has a queue for scheduling• Messages traffic is not intensive
  109. 109. © MIRANTIS 2013 PAGE 109OpenStack Messagings Notes• Uses multiple queues within single RabbitMQinstance• Used by services to build machine state• Each compute node has a queue for scheduling• Messages traffic is not intensive• Doesnt send broadcast messages, e.g. formonitoring, uses API polling instead
  110. 110. © MIRANTIS 2013 PAGE 110OpenStack Messagings Notes• Uses multiple queues within single RabbitMQinstance• Used by services to build machine state• Each compute node has a queue for scheduling• Messages traffic is not intensive• Doesnt send broadcast messages, e.g. formonitoring, uses API polling instead• HA should be configured separately, e.g.mirrored queues, not handled by OpenStack
  111. 111. © MIRANTIS 2013 PAGE 111NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 6a: Publish Provisioning RequestSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova API makes rpc.call toScheduler. It publishes ashort message to schedulerqueue with VM info. UI: Horizon or CLIRequest has been validated, but no action hasbeen taken yet, i.e. which host, IP address, etc.
  112. 112. © MIRANTIS 2013 PAGE 112NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 7: Pick up Provisioning RequestSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWScheduler picks up themessage from MQ.UI: Horizon or CLI
  113. 113. © MIRANTIS 2013 PAGE 113Nova Scheduler“Nova Scheduler is a daemon, whichdetermines, on which compute hostthe request should run.“• Only provisioning time component, i.e. not like VMware’s DRS• Typically co-located with the Cloud Controller
  114. 114. © MIRANTIS 2013 PAGE 114VM Scheduling:Typical Requirements• provision VM to particular host• provision VMs of the particular tenant toisolated hosts• provision all VMs on different hosts• provision VMs to "higher density" hosts
  115. 115. © MIRANTIS 2013 PAGE 115Nova Scheduler: AvailableSchedulersScheduler Description BehaviorChance Picks a host that is up RandomSimplePicks a host that is up andhas the fewest runninginstancesLeast LoadedFilterPicks the best-suited hostwhich satisfies selectedfilterCustom JSON FiltersMulti (Deprecated, to bereplaced by cells)A scheduler that holdsmultiple sub-schedulersCollection of filterscommonly used formulti-site or customizeddeployments
  116. 116. © MIRANTIS 2013 PAGE 116Nova Scheduler: FilteringAffinity, Anti-affinity,etc.Eliminateinapplicable hosts
  117. 117. © MIRANTIS 2013 PAGE 117Nova Scheduler: FiltersFilter Descriptionaffinity Same host or different hostavailability zone Least cost inside selected availability zonecore Least CPU core utilizationram Only return hosts with sufficient RAMjson Allows simple JSON based grammar. Can be used tobuild custom schedulers.i/o filter out hosts with too many concurrent I/Ooperationscompute capabilities match attributes with compute nodes capabilities(e.g. CPU arch.)
  118. 118. © MIRANTIS 2013 PAGE 118Nova Scheduler: FiltersFilter Descriptionaggregate specs match the attributes for the instance with thoseprovided by aggregateimage properties find compute nodes with capabilities matchingimage specification from glanceisolated host match given image with a group of compute nodestrusted host (by Intel) finds only "attested" hoststype find only compute nodes which do not run anyinstances… A lot more
  119. 119. © MIRANTIS 2013 PAGE 119Nova Scheduler: Filters• Filters are statically configured in nova.conf• Multiple filters can be specified• It is possible to create custom filter• Inherit from BaseHostFilter class• Override host_passes(self, host_state, filter_properties)
  120. 120. © MIRANTIS 2013 PAGE 120Nova Scheduler: FilteringRAM, CPU, etc.Integer values
  121. 121. © MIRANTIS 2013 PAGE 121Nova Scheduler: Weights andCosts• Cost - integer value• Every compute host can have several costfunctions associated with it• If no cost functions associated - use defaultfrom nova.conf• weight = sum(costi + weigth_fni)
  122. 122. © MIRANTIS 2013 PAGE 122NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 8a: Schedule ProvisioningSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWScheduler fetchesinformation about thewhole cluster fromdatabase, filters, andselects compute node andupdates DB with its IDUI: Horizon or CLI
  123. 123. © MIRANTIS 2013 PAGE 123NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 8b: Provision ScheduledSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWScheduler publishesmessage to the computequeue (based on host ID) totrigger VM provisioning UI: Horizon or CLI
  124. 124. © MIRANTIS 2013 PAGE 124VM Provisioning Algorithm• Step 9 – Nova Compute gets message from MQ and asksNova Conductor for VM info from database• Step 10 – Nova Compute queries Quantum (previously Nova-Network in Essex) to allocate networking information• Step 11 – Nova Compute queries Cinder to allocate volumeinformation (optional step for persistent data)• Steps 12-14 – Nova Compute fetches VM image (base OS)from Glance• Step 15 – Nova Compute passes all information about VM (ina single message) to Hypervisor and Hypervisor (KVM / Xen)creates an instance
  125. 125. © MIRANTIS 2013 PAGE 125Nova Compute“Nova Compute is a workerdaemon, which primarily createsand terminates VMs viaHypervisor API."
  126. 126. © MIRANTIS 2013 PAGE 126Nova Compute DriversNova ComputeXCPVMVMVMWareVMVMHyperVVMVMLPARVMVMlibvirtKVMVMVMXenVMVMQemuVMVMLXCVMVMToday only 1 hypervisor type percloud instance. Libvirt / KVM is mostcommon deploymentMaintainedby CitrixMaintained byVMWareMaintainedby MicrosoftMaintainedby IBMNative orthrough libvirtBareMetalVMVMExperimentalat this point
  127. 127. © MIRANTIS 2013 PAGE 127Nova Compute Drivers(Continued)• Functionality is not 100% similar• Exact "run_instance" flow depends on driverimplementation• Most of the features are developed and testedon KVM
  128. 128. © MIRANTIS 2013 PAGE 128Nova Compute Config(nova.conf on each host)• --libvirt_type• Hypervisor being used. In this deployment ‘kvm’ is specified.• --libvirt_uri• URI to use for connection to hypervisor. In this deployment ‘qemu+tcp:///system’ is specified.• --sql_connection• Database connection string in SQLAlchemy format. This is used for connecting to state database (ifNova Conductor is not used)• --rabbit_host• IP address for RabbitMQ host. Non-standard port also can be specified• --glance_host• IP address and port of Glance Image Service host. This is needed for streaming virtual boot images.• --glance_api_server• IP address and port of Glance API server. This is needed for getting virtual boot images meta-data.
  129. 129. © MIRANTIS 2013 PAGE 129NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 9a: Start VM ProvisioningSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute getsmessage from MQUI: Horizon or CLI
  130. 130. © MIRANTIS 2013 PAGE 130NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 9b: Start VM ProvisioningSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolKeystoneKeystone ServerKeystone DBGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute makesrpc.call to Nova Conductorfor Information on VM fromDBUI: Horizon or CLI
  131. 131. © MIRANTIS 2013 PAGE 131Nova Conductor“The Nova Conductor service is keyto completing no-db-compute"
  132. 132. © MIRANTIS 2013 PAGE 132Nova Conductor Notescontroller nodeDBnova-conductorcompute nodenova-computerpc.call()
  133. 133. © MIRANTIS 2013 PAGE 133Nova Conductor Notes• Eliminites remote DB access (security)controller nodeDBnova-conductorcompute nodenova-computerpc.call()
  134. 134. © MIRANTIS 2013 PAGE 134Nova Conductor Notes• Eliminites remote DB access (security)• Horizontal scalability (performance)controller nodeDBnova-conductorcompute nodenova-computerpc.call()
  135. 135. © MIRANTIS 2013 PAGE 135Nova Conductor Notes• Eliminites remote DB access (security)• Horizontal scalability (performance)• Hides DB implementation/schema from the Nova Compute (upgrades)controller nodeDBnova-conductorcompute nodenova-computerpc.call()
  136. 136. © MIRANTIS 2013 PAGE 136Nova Conductor Notes• Eliminites remote DB access (security)• Horizontal scalability (performance)• Hides DB implementation/schema from the Nova Compute (upgrades)• Possible offloading of long-running operations from other services, not just Nova Computecontroller nodeDBnova-conductorcompute nodenova-computerpc.call()
  137. 137. © MIRANTIS 2013 PAGE 137Nova Conductor Notes• Eliminites remote DB access (security)• Horizontal scalability (performance)• Hides DB implementation/schema from the Nova Compute (upgrades)• Possible offloading of long-running operations from other services, not just Nova Compute• Beneficial for operations that cross multiple compute nodes (migration, resizes)controller nodeDBnova-conductorcompute nodenova-computerpc.call()
  138. 138. © MIRANTIS 2013 PAGE 138Nova Conductor Notes• Eliminites remote DB access (security)• Horizontal scalability (performance)• Hides DB implementation/schema from the Nova Compute (upgrades)• Possible offloading of long-running operations from other services, not just Nova Compute• Beneficial for operations that cross multiple compute nodes (migration, resizes)• “This is just one (major) step along the path”controller nodeDBnova-conductorcompute nodenova-computerpc.call()
  139. 139. © MIRANTIS 2013 PAGE 139VM Provisioning Algorithm• Step 9 – Nova Compute gets message from MQ and asksNova Conductor for VM info from Nova Conductor• Step 10 – Nova Compute queries Quantum (previously Nova-Network in Essex) to allocate networking information• Step 11 – Nova Compute queries Cinder to allocate volumeinformation (optional step for persistent data)• Steps 12-14 – Nova Compute fetches VM image (base OS)from Glance• Step 15 – Nova Compute passes all information about VM (ina single message) to Hypervisor and Hypervisor (KVM / Xen)creates an instance
  140. 140. © MIRANTIS 2013 PAGE 140Quantum"network-as-a-service"
  141. 141. © MIRANTIS 2013 PAGE 141Quantum Notes
  142. 142. © MIRANTIS 2013 PAGE 142Quantum Notes• Provides a flexible API (POST / GET) for service providers ortheir tenants to manage OpenStack network topologies• Create networks, associate VMs, set routers, etc.
  143. 143. © MIRANTIS 2013 PAGE 143Quantum Notes• Provides a flexible API (POST / GET) for service providers ortheir tenants to manage OpenStack network topologies• Create networks, associate VMs, set routers, etc.• Presents a logical API and a corresponding plug-inarchitecture that separates the description of networkconnectivity from its implementationion
  144. 144. © MIRANTIS 2013 PAGE 144Quantum Notes• Provides a flexible API (POST / GET) for service providers ortheir tenants to manage OpenStack network topologies• Create networks, associate VMs, set routers, etc.• Presents a logical API and a corresponding plug-inarchitecture that separates the description of networkconnectivity from its implementationion• API evolves independently of the compute API, allowing tointroduce more advanced network capabilities (e.g. QoS,ACLs, etc.)
  145. 145. © MIRANTIS 2013 PAGE 145Quantum Notes• Provides a flexible API (POST / GET) for service providers ortheir tenants to manage OpenStack network topologies• Create networks, associate VMs, set routers, etc.• Presents a logical API and a corresponding plug-inarchitecture that separates the description of networkconnectivity from its implementationion• API evolves independently of the compute API, allowing tointroduce more advanced network capabilities (e.g. QoS,ACLs, etc.)• In Folsom/Grizzly one can choose to stay with nova-network(Essex approach) or to go with Quantum
  146. 146. © MIRANTIS 2013 PAGE 146Quantum Architecture -“Birds Eye" View3rd Party plug-in,networking data is storedoutside of OpenStack /QuantumQuantum nativefunctionality
  147. 147. © MIRANTIS 2013 PAGE 147Network Configuration Flow• Allocate MAC addresses• Allocate IPs (for each network)• Associate IP and MAC with VM (DB)• Setup network - L2:• configure L2 via a quantum plugin• actual action can be variable, depending on the plugin used(with OVS plugin the action is: plugging an instance into theintegration bridge on the hypervisor)• Setup network - L3• Update DHCP config• Initialize gatewayAllocation duringcloud setupAssociation andSetup during VMprovisioning
  148. 148. © MIRANTIS 2013 PAGE 148Available Quantum Plugins• Linux Bridge• OpenVSwitch (most common)• Nicira NVP• Cisco (UCS Blade + Nexus)• Ryu OpenFlow controller• NEC ProgrammableFlow Controller
  149. 149. © MIRANTIS 2013 PAGE 149Step 10: Configure NetworkSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute makes a call toQuantum API to provisionnetwork for the instanceUI: Horizon or CLIKeystoneKeystone ServerKeystone DBNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  150. 150. © MIRANTIS 2013 PAGE 150Step 10: Configure Network (Continued)SwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWQuantum configures IP, gateway,DNS name, L2 connectivity, etc.UI: Horizon or CLIKeystoneKeystone ServerKeystone DBStaticDynamicNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  151. 151. © MIRANTIS 2013 PAGE 151VM Provisioning Algorithm• Step 9 – Nova Compute gets message from MQ and asksNova Conductor for VM info from database• Step 10 – Nova Compute queries Quantum (previously Nova-Network in Essex) to allocate networking information• Step 11 – Nova Compute queries Cinder to allocate volumeinformation (optional step for persistent data)• Steps 12-14 – Nova Compute fetches VM image (base OS)from Glance• Step 15 – Nova Compute passes all information about VM (ina single message) to Hypervisor and Hypervisor (KVM / Xen)creates an instance
  152. 152. © MIRANTIS 2013 PAGE 152Cinder"block storage as-a-service"
  153. 153. © MIRANTIS 2013 PAGE 153Cinder Notes
  154. 154. © MIRANTIS 2013 PAGE 154Cinder Notes• Optional
  155. 155. © MIRANTIS 2013 PAGE 155Cinder Notes• Optional• iSCSI solution which can plug into a number ofstorage backends
  156. 156. © MIRANTIS 2013 PAGE 156Cinder Notes• Optional• iSCSI solution which can plug into a number ofstorage backends• Volume can be attached only to 1 instance at atime
  157. 157. © MIRANTIS 2013 PAGE 157Cinder Notes• Optional• iSCSI solution which can plug into a number ofstorage backends• Volume can be attached only to 1 instance at atime• Persistent volumes keep their stateindependent of instances
  158. 158. © MIRANTIS 2013 PAGE 158Cinder Architecture
  159. 159. © MIRANTIS 2013 PAGE 159Cinder Drivers• iSCSI• Fibre Channel• Xen Storage Manager• Nexenta• NetApp• Zadara VPSA• SAN• NFS (volumes as sparse files)• RBD Ceph• IBM Storwize / XIV• HP 3PAR• Coraid• Huawei• Scality SOFS• GlusterFS• LVM thin provisioning support• Mirrored LVM• XenAPINFS• EMC VNX/VMAX arrays• Solidfire
  160. 160. © MIRANTIS 2013 PAGE 160Step 11: Request VolumeSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWUI: Horizon or CLIKeystoneKeystone ServerKeystone DBNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellIt is assumed a volume isalready created. NovaCompute contacts Cinder toget volume data. Can alsoattach volumes after VM isbuilt.
  161. 161. © MIRANTIS 2013 PAGE 161NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 11: Request volume (Continued)SwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute sets upiSCSI initiator & instructsthe Hypervisor to mountiSCSI vol. as a new blockdevice UI: Horizon or CLIKeystoneKeystone ServerKeystone DB
  162. 162. © MIRANTIS 2013 PAGE 162VM Provisioning Algorithm• Step 9 – Nova Compute gets message from MQ and asksNova Conductor for VM info from database• Step 10 – Nova Compute queries Quantum (previously Nova-Network in Essex) to allocate networking information• Step 11 – Nova Compute queries Cinder to allocate volumeinformation (optional step for persistent data)• Steps 12-14 – Nova Compute fetches VM image (base OS)from Glance• Step 15 – Nova Compute passes all information about VM (ina single message) to Hypervisor and Hypervisor (KVM / Xen)creates an instance
  163. 163. © MIRANTIS 2013 PAGE 163Glance"The Glance project providesservices for discovering,registering, and retrieving virtualmachine images."
  164. 164. © MIRANTIS 2013 PAGE 164Glance Summary• Images-as-a-Service• Can use multiple back-ends for image storage• Supports multiple image formats
  165. 165. © MIRANTIS 2013 PAGE 165Glance Architecture
  166. 166. © MIRANTIS 2013 PAGE 166Glance Capabilities
  167. 167. © MIRANTIS 2013 PAGE 167Glance Capabilities• CRUD images (Create, Read, Update, Delete)
  168. 168. © MIRANTIS 2013 PAGE 168Glance Capabilities• CRUD images (Create, Read, Update, Delete)• Search images via filters• name• container format• disk format• size_min, size_max• status
  169. 169. © MIRANTIS 2013 PAGE 169Glance Capabilities• CRUD images (Create, Read, Update, Delete)• Search images via filters• name• container format• disk format• size_min, size_max• status• Caches images• uses SQLite or FS that supports xattrs for caching• queues images for prefetching• prefetches images• prunes images• cleans invalid cache entries
  170. 170. © MIRANTIS 2013 PAGE 170Glance Image FormatsDisk Format Descriptionraw This is an unstructured disk image formatvhd This is the VHD disk format, a common disk format used by virtual machine monitors fromVMWare, Xen, Microsoft, VirtualBox, and othersvmdk Another common disk format supported by many common virtual machine monitorsvdi A disk format supported by VirtualBox virtual machine monitor and the QEMU emulatoriso An archive format for the data contents of an optical disc (e.g. CDROM).qcow2 A disk format supported by the QEMU emulator that can expand dynamically and supports Copyon Writeaki This indicates what is stored in Glance is an Amazon kernel imageari This indicates what is stored in Glance is an Amazon ramdisk imageami This indicates what is stored in Glance is an Amazon machine image
  171. 171. © MIRANTIS 2013 PAGE 171Custom Image Creation• Get installation ISO• Create VM (qemu-img create)• Start VM and connect to it via VNC console• Install image without LVM• Create default iptables rules• Install and configure cloud-init• With cloud-init configure image• Prepare image for OpenStack• Extract root partition, kernel and ramdisk• cleanup• package
  172. 172. © MIRANTIS 2013 PAGE 172Step 12: Request VM Image from GlanceSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute requests VMimage from Glance via Image IDUI: Horizon or CLIKeystoneKeystone ServerKeystone DBNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  173. 173. © MIRANTIS 2013 PAGE 173Step 13: Get Image URI from GlanceSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWIf image with given image ID can befound - return URI – HTTP Get URIUI: Horizon or CLIKeystoneKeystone ServerKeystone DBNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  174. 174. © MIRANTIS 2013 PAGE 174Step 14: Download Image from SwiftSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute downloads image using URI,given by Glance, from Swif(or Glances back-end)UI: Horizon or CLIKeystoneKeystone ServerKeystone DBNovaNova DBQueueNova:ControllerNova APISchedulerConductorNova Cell
  175. 175. © MIRANTIS 2013 PAGE 175VM Provisioning Algorithm• Step 9 – Nova Compute gets message from MQ and asksNova Conductor for VM info from database• Step 10 – Nova Compute queries Quantum (previously Nova-Network in Essex) to allocate networking information• Step 11 – Nova Compute queries Cinder to allocate volumeinformation (optional step for persistent data)• Steps 12-14 – Nova Compute fetches VM image (base OS)from Glance• Step 15 – Nova Compute passes all information about VM (ina single message) to Hypervisor and Hypervisor (KVM / Xen)creates an instance
  176. 176. © MIRANTIS 2013 PAGE 176NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 15: Start VM Rendering via HypervisorSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWNova Compute fetchesinformation about VM fromDB, creates a command toHypervisor and delegatesVM rendering to Hypervisor. UI: Horizon or CLIKeystoneKeystone ServerKeystone DBIn case of KVM / libvirtd this isa single XML VM config file
  177. 177. © MIRANTIS 2013 PAGE 177NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 16: VM is UPSwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWiSCSI communicationbegins for volumeUI: Horizon or CLIKeystoneKeystone ServerKeystone DBNova Compute sends amessage to NovaConductor to updateDB with VM state
  178. 178. © MIRANTIS 2013 PAGE 178NovaNova DBQueueNova:ControllerNova APISchedulerConductorNova CellStep 17: User is HappySwiftObject StoreProxy ServerQuantumQuantum DBPlugin / AgentQuantum ServerCinderCinder APICinder DBSchedulerQueueCinder VolGlanceGlance APIGlance RegistryGlance DBCompute NodeHypervisorNetworkVMNova:Computenova-computeBlock StorageNodeStorageNetwork NodeDHCP / IPAMRouter / GWHorizon polls Nova API forVM status and power state,which is taken fromDatabase.UI: Horizon or CLIKeystoneKeystone ServerKeystone DB
  179. 179. © MIRANTIS 2013 PAGE 179Recap:
  180. 180. © MIRANTIS 2013 PAGE 180Recap:• Users logs into Horizon and initiates a VM create
  181. 181. © MIRANTIS 2013 PAGE 181Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes
  182. 182. © MIRANTIS 2013 PAGE 182Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB
  183. 183. © MIRANTIS 2013 PAGE 183Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB• Nova Scheduler finds appropriate host
  184. 184. © MIRANTIS 2013 PAGE 184Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB• Nova Scheduler finds appropriate host• Quantum configures networking
  185. 185. © MIRANTIS 2013 PAGE 185Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB• Nova Scheduler finds appropriate host• Quantum configures networking• Cinder provides block device
  186. 186. © MIRANTIS 2013 PAGE 186Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB• Nova Scheduler finds appropriate host• Quantum configures networking• Cinder provides block device• Image URI is looked up through Glance
  187. 187. © MIRANTIS 2013 PAGE 187Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB• Nova Scheduler finds appropriate host• Quantum configures networking• Cinder provides block device• Image URI is looked up through Glance• Image is retrieved via Swift
  188. 188. © MIRANTIS 2013 PAGE 188Recap:• Users logs into Horizon and initiates a VM create• Keystone authorizes• Nova initiates provisioning and saves state to DB• Nova Scheduler finds appropriate host• Quantum configures networking• Cinder provides block device• Image URI is looked up through Glance• Image is retrieved via Swift• VM is rendered

×