OpenStack Framework Introduction

22,197 views

Published on

Presentation of OpenStack survey to Internet Research Lab at National Taiwan University, Taiwan. OpenStack framework and architecture overview. (ppt slide for download.) Materials collected from various resources, not originally produced by the author.
Briefly explained Nova, Swift, Glance, Keystone, and Quantum.

Published in: Technology
3 Comments
42 Likes
Statistics
Notes
No Downloads
Views
Total views
22,197
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
2,784
Comments
3
Likes
42
Embeds 0
No embeds

No notes for slide

OpenStack Framework Introduction

  1. 1. OpenStack Introduction Presenter: Jason, Tsung-Cheng, HOU Advisor: Wanjiun Liao June 7th, 2012 1
  2. 2. Motivation• What does a cloud OS look like?• How are they building IaaS Platform?• What are current industry trend?• How will the cloud system press the network?• OpenStack – Founded by NASA and Rackspace in 2010 – Currently 178 companies and 3386 people – Growing fast now, latest release Essex, Apr. 5th 2
  3. 3. OpenStack Status• OpenStack – Founded by NASA and Rackspace in 2010 – Currently 178 companies and 3386 people – Was only 125 and 1500 in fall, 2011. – Growing fast now, latest release Essex, Apr. 5th• Aligned release cycle with Ubuntu, Apr. / Oct.• Aim to be the “Linux” in cloud computing sys.• Open-source v.s. Amazon and vmware• Start-ups are happening around OpenStack• Still lacks big use cases and implementation 3
  4. 4. 4
  5. 5. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 5
  6. 6. Enterprises are building clouds to... 1. VirtualizationServer Virtualization 2. Cloud Data Center 3. Cloud Federation
  7. 7. Datacenters are being virtualized, Servers are firstHypervisors provide abstraction between SW and HW (Servers) HOST 1 HOST 2 HOST 3 HOST 4, ETC. VMs Hypervisor: Turns 1 server into many “virtual machines” (instances or VMs)Next: Storage, Network…the building blocks1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
  8. 8. But questions arise as the environment grows... APPS USERS ADMINS +1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
  9. 9. Solution: OpenStack, The Cloud Operating SystemA new management layer that adds automation and control APPS USERS ADMINS CLOUD OPERATING SYSTEM1. Server Virtualization Server Virtualization 2. Cloud Data Center 3. Cloud Federation
  10. 10. What‟s next? ’1. Server Virtualization Server Virtualization 2. Cloud Data Center 3. Cloud Federation
  11. 11. Common Platform1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
  12. 12. A common platform is here.OpenStack is open source software powering public and private clouds. Private Cloud: Public Cloud:OpenStack enables cloud federationConnecting clouds to create global resource pools Washington Common software platform making Federation possible Texas California Europe1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
  13. 13. In Summary, the Cloud Operating Systemenables enterprises to:Top 3 Benefits of a Common Platform
  14. 14. Core Components in Essex • Release Apr. 5th, 2012• Dashboard: Access and control portal for admin and users, also web-based• Identity: Unified authentication across whole system• Object Storage: Large-scale redundant storage of static objects, not a file system• Image Service: Store, retrieve, discover, register, and deliver VM images• Compute: Large-scale deployment of automatically provisions VMs and related SWs 14
  15. 15. 15
  16. 16. OpenStack Compute Key Features 1. REST-based API 2. Horizontally and massively scalable 3. Hardware agnostic: supports a variety of standard hardware 4. Hypervisor agnostic: support for Xen ,Citrix XenServer, Microsoft Hyper-V, KVM, UML, LXC and ESX
  17. 17. OpenStack Storage Key Features 1. REST-based API 2. Data distributed evenly throughout system 4. Scalable to3. Runs on standard hardware multiple petabytes, billions of objects 5. No central database required 6. Account/Container/Object structure (not file system, no nesting) plus Replication (N copies of accounts, containers, objects)
  18. 18. OpenStack Image Service Key Features 1. Store & retrieve VM 2. REST-based API images 3. Compatible with all common image formats4. Storage agnostic: Store imageslocally, or use OpenStack ObjectStorage, HTTP, or S3
  19. 19. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 24
  20. 20. Chief Reference• CIS 607: Seminar in Cloud Computing, Spring 2012, by Dr. Allen D. Malony• "Applied Computational Instrument for Scientific Synthesis" (ACISS)• University of Oregon (UO), groups may configure their own VM images and carry out research work as if had dedicated clusters.• This course delivers hands-on operations on ACISS and cloud computing knowledge to students. 25
  21. 21. VM and Hypervisor• Virtual Machine: A software package, sometimes using hardware acceleration, that allows an isolated guest operating system to run within a host operating system.• Stateless: Once shut down, all HW states disappear.• Hypervisor: A software platform that is responsible for creating, running, and destroying multiple virtual machines.• OpenStack is hypervisor agnostic.• Type I and Type II hypervisor 26
  22. 22. 27
  23. 23. Bridged Networking• One network card acts as many devices.• Host does not need an IP address.• Hypervisor sets virtual MAC address for guest machine.• ACISS uses bridges, along with Virtual Local Area Networks (VLANs) to segment traffic and assign network addresses. 28
  24. 24. Network Block Storage• Network Block Storage: Make data persistent by mounting a network block storage device.• NFS Mounts: Many machines may access simultaneously. Limited permissions.• iSCSI Mounts. Only one machine may access at any given time. Unlimited permissions. 29
  25. 25. Object Storage• Persistent storage of objects on a network.• Generally “write once, read many.”• Durable storage with redundant copies• Access Control Lists determine visibility for owner and authorized users.• Amazon‟s S3 is an example of this.• ACISS uses OpenStack Swift.• Swift uses same API as S3. 30
  26. 26. Virtual Machine Images• Disk images that can be booted on a virtual machine by a hypervisor.• Can be a single image that contains boot loader, kernel and operating system.• Boot loader and kernel can be separated.• Allows for custom kernels and resizable images. 31
  27. 27. Image Service• Stores and catalogs virtual machine images. – Keep track of VMs, trace and recover.• Provides for discovery, registration, and delivery of images to hypervisors.• Allows for many image formats and for linking of loaders and kernels to images. – There may be different types of virtualization technologies, different kernels, etc.• Usually built on object storage systems.• Glance on Swift. 32
  28. 28. Cloud Computing• The course defines in the following way: – The orchestration of hypervisors, networking, block storage, and image, and identity services to provide on demand virtual machines.• Hence, meeting required characteristics of cloud computing. – On-demand self-service – Resource pooling – Rapid elasticity –… 33
  29. 29. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 34
  30. 30. Keystone Main Functions• Provides 4 primary services: – Identity: User information authentication – Token: After logged in, replace account-password – Service catalog: Service units registered – Policies: Enforces different user levels• Can be backed by different databases. – LDAP – SQL – Key Value Stores (KVS) 35
  31. 31. Keystone: Identity• User information: – username/password – Metadata (e-mail, etc.) – Tenant - organizes users into projects or group. – Role - define a user‟s role and permissions in a project.• A user must belong to at least one tenant, and may belong to many tenants• Roles are assigned to user/tenant pairs – Common roles: Member, Admin 36
  32. 32. Keystone: Token• Once a user‟s identity has been verified with a acc/pswd pair, a short-lived (24 hr) token is issued.• Tokens are a stand-in for the acc/pswd.• OpenStack services hold on to tokens and use them to query keystone during operations.• For example, once Nova can use a token to determine if an authenticated user has authorization to delete an instance. 37
  33. 33. Keystone: Catalog• OpenStack service endpoints are registered with Keystone to create a service catalog.• A client for a service connects to Keystone, and determines an endpoint to call based on the returned catalog.• Behind the scenes, services can be moved to different endpoints. A client can find online services by querying Keystone endpoint.• Also allows for service load distribution with multiple endpoints to a single service. 38
  34. 34. Keystone: Catalog• Every catalog entry has five elements: – region: the name given to a collection of cloud services – service id: the service the endpoint is associated with (Glance, Nova, Swift, Keystone) – public url: the public facing endpoint for the service – internal url: the internal facing endpoint. Usually the same as the public url – admin url: the endpoint for service administration 39
  35. 35. 40
  36. 36. 41
  37. 37. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 42
  38. 38. Swift• Object storage, objects “live” on an endpoint. – An endpoint could be any storage device• Every object belongs to a user/account pair. – keystone tenant : swift account – keystone user : swift user – keystone role : swift group• Proxy, Ring, and Workers• Account, Container, Object 43
  39. 39. Swift: Proxy Server• Handles incoming requests via the OpenStack Object API or raw HTTP.• Accepts files to upload, modifications to metadata or container creation.• Serve files or container listing to web browsers.• Several types of Ring files• May utilize an optional cache to improve performance. 44
  40. 40. Swift: Workers• Keep a distributed database of replicated objects.• Workers are divided into reliability zones.• Copies of data are distributed across multiple zones.• There are many types of workers: – Account server, container server, object server – Housekeeping: Replication, updater, auditor 45
  41. 41. Swift: Ring• Maps names to entities and locations – Stores data based on zones, devices, partitions, and replicas• There are three types of items: – Account, container, object• The locations are determined by a ring file• Worker IP addresses are loaded into a ring builder.• Storage ids and locations are computed using a hashing algorithm to evenly distribute items across the workers. 47
  42. 42. Swift: Ring• Account and container storage id has a database, storing object metadata.• Proxy makes distributed searches across the databases for item requests.• The ring builder can add / remove nodes, and rebalance distribution of files across servers. 48
  43. 43. Duplicated storage, load balancing↑ Logical view↓Physical arrangement ← Stores real objects ←Stores object metadata ↑Stores container / object metadata
  44. 44. Workers can be a account server, acontainer server, or an object server
  45. 45. # of account < # of container < # of object serversDifferent zones ↑
  46. 46. 55
  47. 47. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 56
  48. 48. Glance• Image storage and indexing.• Keeps a database of metadata associated with an image, discover, register, and retrieve.• Built on top of Swift, images store in Swift• Two servers: – Glance-api: public interface for uploading and managing images. – Glance-registry: private interface to metadata database• Support multiple image formats 57
  49. 49. 58
  50. 50. 59
  51. 51. 61
  52. 52. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 62
  53. 53. Nova• Major components: – API: public facing interface – Message Queue: Broker to handle interactions between services, currently based on RabbitMQ – Scheduler: coordinates all services, determines placement of new resources requested – Compute Worker: hosts VMs, controls hypervisor and VMs when receives cmds on Msg Queue – Volume: manages permanent storage 63
  54. 54. Nova• Major components: – Network: manages networking • Was originally a component in Nova • Default gateway, network controller • DHCP server, address mgmt • The network part in Nova will be enhanced by the project named “Quantum”, to be released. • Will introduce Quantum later. 64
  55. 55. Nova Messaging and Data• Messaging is managed through RabbitMQ – Server that allows messages to be posted to channels. – Subscribers to channels receive messages. – Services regularly announce availability. – Scheduler regularly reads for availability. – Scheduler makes requests to services.• Persistent data stored in a database. – VM metadata, network topology, volume metadata, known services 65
  56. 56. Messaging (RabbitMQ)• Get data from point A to point B• Decouple publishers and consumers• Queueing for later delivery• Load balancing and scalability• RabbitMQ is an AMQP messaging broker• Advanced Message Queueing Protocol• Network wire-level protocol• Internet protocol - like HTTP, TCP - but ASYNCHRONOUS 66
  57. 57. Messaging (RabbitMQ) 67
  58. 58. Messaging (RabbitMQ) 68
  59. 59. 69
  60. 60. 70
  61. 61. 73
  62. 62. 74
  63. 63. Agenda• OpenStack Brief Overview• Some Reviews of Cloud Technology• “Keystone” Identity• “Swift” Storage• “Glance” Image• “Nova” Compute• “Quantum” Networking 75
  64. 64. Without Quantum• Originally, Nova handles all networking by: – Linux bridge networking – Virtual interfaces connecting network through the physical interface – Assigns VM IP address – Fixed IP: Returns when VM shuts down – Floating IP: Can be reassigned online• Network Manager provides VN to enable compute servers to interact with each other and the public network• A Blog states currently 90% Nova bugs are network related 76
  65. 65. Original Network Manager• Each VM network owned by one network host – Simply a Linux running Nova-network daemon• Nova Network node is the only gateway• Flat Network Manager: – Linux networking bridge forms a subnet – All instances attached same bridge – Manually Configure server, controller, and IP• Flat DHCP Network Manager: – Add DHCP server along same bridge• Later: VLAN Network Manager 77
  66. 66. Bridged Networking• One network card acts as many devices.• Host does not need an IP address.• Hypervisor sets virtual MAC address for guest machine.• ACISS uses bridges, along with Virtual Local Area Networks (VLANs) to segment traffic and assign network addresses. 78
  67. 67. Linux running Nova-network daemon Network host will act as the gateway for all the NICs bridged into that network. VMs bridged in to a raw Ethernet deviceThe only gatewayWith security measures 79
  68. 68. • DHCP server also tracks IP leases and releases • Re-uses and assigns IP addresses dynamically • Sets up a routing table for outside forwarding• Compute optionally have public IP• Network host is a single point of failure and bottleneck• Backup network host• A new proposed model → →• Multi-NIC→Multiple networks 80
  69. 69. 81
  70. 70. VLAN Network Manager• Current default mode for OpenStack• Nova creates a VLAN and bridge for each project. – Requires switches with VLAN tagging (IEEE 802.1Q). – A range of private IPs, only accessible inside VLAN.• A special VPN instance (code named cloudpipe) needs to be created. Generates (certificate, key) for users to access VPN automatically.• Provides a private network segment for each project, accessed via dedicated VPN connection from the Internet. Each project with own VLAN, Linux networking bridge, and DHCP server. 82
  71. 71. Plugin• The component where the „virtual networking‟ magic happens. Fulfills API contract by implementing the „Plugin Interface‟• Tenants expect same behavior from Quantum API regardless of the particular plugin employed• Available Quantum Plugins: – Open vSwitch: Builds isolated networks with OVS and L2-in-L3 tunnels. – Cisco UCS: Isolation based on VLAN and net-profiles applied to Cisco UCS – converged network adapters – Linux Bridge: Build isolated networks with VLAN interfaces and linux bridge – NTT-Data Ryu: Acts as a proxy for the NTT Ryu platform – Nicira NVP: Acts as a proxy for the Nicira NVP platform 88
  72. 72. The Quantum Manager• Nova‟s network manager for Quantum. Forwards network related requests.• Also, provides other network services such as IP address management, DHCP, NAT, Floating IPs…• Virtual Networking: A label nowadays applied to too many solutions and products. – Securely partitioning the network – Defining virtual network topologies – Automating network provisioning 89
  73. 73. The Near Future• Folsom release, Fall 2012 – Become a core OpenStack project – Merge with IP Address Management service – Improve API quality and documentation – Improve GUI, i.e. Quantum Horizon plugin – Possible more plugins, Build more network services on top of the basic building block • Each service with its own tenant-facing API • IP routing, Distributed Firewall, LB, NAT, VPN, bridging…• Quantum is NOT SDN, but in theory can transform anything into SDN. 93
  74. 74. Reference• OpenStack Documentation http://docs.openstack.org/• Dr. Allen D. Malony, CIS 607: Seminar in Cloud Computing, Spring 2012, U. Oregon http://prodigal.nic.uoregon.edu/~hoge/cis607/• Bret Piatt, OpenStack Overview, OpenStack Tutorial http://salsahpc.indiana.edu/CloudCom2010/slides/PDF/tutorials/OpenStackTutorialIEEECloudCom.pdf http://www.omg.org/news/meetings/tc/ca-10/special-events/pdf/5-3_Piatt.pdf• Vishvananda Ishaya, Networking in Nova http://unchainyourbrain.com/openstack/13-networking-in-nova• Sandy Walsh, OpenStack 101 Technical Overview http://www.slideshare.net/openstackcommgr/openstack-101-technical-overview• Jaesuk Ahn, OpenStack, XenSummit Asia http://www.slideshare.net/ckpeter/openstack-at-xen-summit-asia http://www.slideshare.net/xen_com_mgr/2-xs-asia11kahnopenstack• Salvatore Orlando, Quantum: Virtual Networks for Openstack http://qconlondon.com/dl/qcon-london- 2012/slides/SalvatoreOrlando_QuantumVirtualNetworksForOpenStackClouds.pdf• Dan Wendlandt, Openstack Quantum: Virtual Networks for OpenStack http://www.ovirt.org/wp-content/uploads/2011/11/Quantum_Ovirt_discussion.pdf• Daneyon Hansen, OpenStack @ CISCO http://www.cisco.com/web/strategy/docs/gov/openstack_presentation.pdf• Rick Clark, Cisco and OpenStack http://www.ogf.org/OGF32/materials/2310/ogf32-isod-Cisco-OpenStack-July2011.pdf 94

×