OpenStack History 陳彥宏 EricCloud Computing HQ, Wistron +886-2-6612-2980 Eric_yh_chen@wistron.com
• Projected by Rackspace Cloud and NASA at July, 2010• Open source with Apache License• Written in Python• Stable Release: 2011/09 Diablo• Next Release: 2012/04 Essex 2012/01 2011/08
Datacenters are being virtualized, Servers are firstHYPERVISORS PROVIDE ABSTRACTION BETWEEN APPS AND HARDWARE (SERVERS) HOST 1 HOST 2 HOST 3 HOST 4, ETC. VMs Hypervisor: Turns 1 server into many “virtual machines” (instances or VMs)1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
Next: Storage, Network…the building blocksABSTRACTION BETWEEN APPS AND HARDWARE1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
But questions arise as the environment grows...“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY APPS USERS ADMINS +1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
But questions arise as the environment grows...“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY APPS USERS ADMINS1. Server Virtualization Server Virtualization 2. Cloud Data Center 3. Cloud Federation
Solution: OpenStack, The Cloud Operating SystemA NEW MANAGEMENT LAYER THAT ADDS AUTOMATION AND CONTROL APPS USERS ADMINS CLOUD OPERATING SYSTEM1. Server Virtualization Server Virtualization 2. Cloud Data Center 3. Cloud Federation
In Summary, the Cloud Operating Systemenables enterprises to:
A common platform is here.OPENSTACK IS OPEN SOURCE SOFTWARE POWERING PUBLIC AND PRIVATE CLOUDS. Private Cloud: Public Cloud:1. Server Virtualization Virtualization 2. Cloud Data Center 3. Cloud Federation
Projects in Openstack 陳彥宏 Eric Cloud Computing HQ, Wistron +886-2-6612-2980 Eric_yh_chen@wistron.com
• Responsible for managing virtual machines 1. 2. 3. 4.
Feature Benefit Racks of commodity servers as pools of computingManage virtualized commodity server resources Improved utilization and automation of resources for greater cost CPU, memory, disk, and network interfaces efficiencies Programmatically allocate IPs and VLANs VLANs Manage Local Area Networks (LAN) (for rapid provisioning of network capabilities and security features) Flat, Flat DHCP, VLAN DHCP, IPv6 Flexible networking models to suit needs of each application and/or user group Designed for automation and security (to make it easy for you to manage who has access to compute resources API with rate limiting and authentication and prevents users from impacting each other with excessive API utilization) Massively scalable and highly available system Distributed and asynchronous architecture (for increased assurance of system uptime) Easily store, import, share, and query images Virtual Machine (VM) image management (to make it easy for you to spin up new standardized VMs) Live VM management (Instance) Increase productivity with lifecycle management Run, reboot, suspend, resize, terminate instances (from a single user interface and using the APIs) Build a menu of options for users to select from Create and manage Instance Types (Flavors) (to enable self service and greater efficiency) Define sizes of VMs for CPU, RAM & Disk Flavors make it easy to size VMs for workloads iSCSI storage volume management Enables data to be managed separate from VMs for fault-tolerance Create, delete, attach and transfer volumes and added flexibility Live migration of instances Minimize downtime with planned maintenance (Diablo v3) Floating IP addresses Keep IPs & DNS correct when managing VMs Flexibility to assign and control access to VM instances by creating Security Groups separation between resource pools Role Based Access Control (RBAC) Ensure security by user, role and project Projects & Quotas Ability to allocate, track and limit resource utilization …click here for Compute VNC Proxy through web browser Quick and easy CLI administration Roadmap…
OpenStack Compute – Roadmap Feature Benefit Open vSwitch in Xen Allows for more granular network control and flexibility, including protection for IPv6(Diablo v1 06/02 – Implemented) Multi-Nic support (Diablo v2 06/30 – Code Offers more flexible networking options Review) Event Notification Pro-active alerting (Diablo v2 06/30 – (e.g.: notification of instance builds, deletions and migrations are useful for monitoring and billing Implemented) applications.) Distributed scheduler Robust scheduler for scalability and high availability (Diablo v2 06/30 – Implemented) (for large scale deployments potentially spanning across DCs) System usage Provides metrics for billing, chargeback, or monitoring purposes (Diablo v2 06/30 – Started) Boot with volume (Diablo v3 07/28 – Code shorter boot time, persistent root partition Review) Virtual storage arrays Allows to emulates Enterprise class storage arrays, storage administrators will be able to choose(Diablo v2 06/30 – In Progress) things like type of drives (SSDs, SAS, SATA), type of interface (iSCSI, AoE, FCoE) Global firewall rules DDOS prevention (Diablo v2 06/30 – Implemented) Drops all traffic from blacklisted IPs before it reaches instances Advanced Scheduler Scheduler decision framework for more efficient mgmt./provisioning (Diablo v3 07/28 – Started) Federated Auth with Zones Allows to control permissions b/w public and private zones (Diablo v4 08/25 – TBD) <Release Name> <Version> <Release Date> - < Status>
• Disk images and associated metadata • Discover, register and retrieve 1. 2. 3.4.
OpenStack Image Service: Feature Benefit Image-as-a-service Store and retrieve virtual machine images at scaleMulti-format/container support Compatible with all common image formats Image status Provides visibility and availability structure Scalable API Image Services scales with OpenStack Metadata Store and retrieve information about the image Image Checksum Ensure data integrity Extensive Logging Provide audit and debugging capability Integrated testing Verify functionality of the virtual machine Back-end store options Greater flexibility with Swift, Local, S3 or HTTP Version control Provides structure and control CLI access Administrative options …click here for Image Service Roadmap…
OpenStack Image Service – Roadmap Feature Benefit Auth. System integration Allows for specific ownership vs public/private, integrate with keystone (Diablo v3 07/28 – In Progress) Open metadata fields (Diablo v3 07/28 – In Additional key pairs for custom association Progress) API improvements (Diablo v1 06/02 – Results limiting, filtering, sorting, and version support Implemented) Shared image groups Capability to allow image sharing and access by groups (Diablo v3 – lmplemented) GZIP compression Increase speed and decrease bandwidth for large queries (Diablo TBD – Blocked) ISO format support (Diablo v1 06/02 – Extend format types Implemented)Delayed deletion of images Increase performance (Diablo v3) <Release Name> <Version> <Release Date> - < Status>
• Object storage, Like Amazon S3 1. 2.3. 4. 5. 6.
OpenStack Object Storage: Feature BenefitStore and Manage files programmatically via API Automates resource management/provisioning Create Public or Private containers Better control. Allows to share data publicly or keep it private Leverages Commodity hardware No lock-in, lower price/GB Self healing HDD/node failure agnostic Reliability, Data redundancy protecting from failures Huge & flat namespace, highly scalable read/write access Unlimited Storage Ability to serve content directly from storage system Multi-dimensional scalability (scale out architecture) Backup/Archive large amounts of data with linear performance Allows to scale vertically and horizontally-Distributed storage Account/Container/Object structure Optimized for scale no nesting, not a traditional file system Allows to scale to multiples Peta-bytes, billions of objects Built-in Replication (N copies of accounts, container, objects) High Availability 3x+ data redundancy compared to 2x on RAID Easily add capacity unlike RAID resize Elastic data scaling with ease No central database Higher performance, No bottlenecks RAID not required Allows to handle lots of small, random reads and writes efficiently Acct. Management: Create, add, verify, delete users Built-in Mgmt. utilities Container Management: upload, download, verify Monitoring: Capacity, Host, Network, Log trawling, cluster health Drive auditing Allows to detect drive failures preempting data corruption …click here for Storage VNC Proxy through web browser Quick and easy CLI administration Roadmap…
OpenStack Object Storage – Roadmap Feature Benefit Improved Client IP Logging Allows granular tracking and auditing for Intrusion detection and protection (Diablo v1.4.0 05/31- Implemented) Transaction ID headers (Diablo v1.4.0 05/31 – Better control over data handling Implemented) Auto Account Create (Diablo 1.4.1 06/20 - Option to automate account creation for authorized requests…saving time Implemented)Multi cluster container syncing (Diablo v1.4.2 TBD High availability – In progress) Allows to synchronize container contents across clusters Object Recon Add object server middleware to allow introspection on the storage (Diablo v1.4.3) processes and nodes Container level stats Collect and report stats on a container level (Diablo v1.4.0) True High availability Multi-region support (Future – Not Started) Allows to register and cross replicate b/w physically isolated external Object store clusters Multi-tenant accounting (Pending Approval) Helps service providers support, track, audit, authorize customer resources Client Bindings (Pending Approval) Higher performance and less data footprint Large Single Uploads (Pending Approval) Allows to upload/store files greater than 5GB Self-destructing files (Pending Approval) Policy based file management Search Service (Pending Approval) Allows to search objects and containers by names and metadata SNIA CDMI Support (Pending Approval) Offers compatibility SNIA standard <Release Name> <Version> <Release Date> - < Status>
Keystone (OpenStack Identity)• Unified tenants / accounts for all services – Provides identity management service – Provide and abstract interface to identity system, ex: LDAP, ActiveDirectory, SAML, OAuth
Horizon (Openstack Dashboard)• Django module to build web UI• Integrate with Keystone, Nova, Glance… – Manage virtual infrastructure, quotas, object store, network and security resources, and more
Quantum (Openstack Network)• Virtual network service – Based on Open-vSwitch, Support for layer 2 over layer 3 tunneling to avoid the limitations of VLANs – VPN-aaS, firewall-aaS, data-center-interconnect-aaS – Monitoring protocols like NetFlow Nova VM1 VM2 VM3 Quantu Net1 Net2 m
• dashboardwireframesopenstack20110928- 111011162319-phpapp01.pdf• Live demo ?!
• http://pypi.python.org/pypi/virtualenv – For development – For running unit test
Contd…OpenStack dev. Pipeline…incubating…draft…pending approval - status Feature Description Block storage service An API-fronted iSCSI-based block storage service that aims to offer (Unknown TBD – TBD) moderate performance with a very low cost/GB of capacity Provides common identity components (user store, authentication Identity Service service, endpoint management) and middleware to integrate with (Diablo v2 06/30 – Beta Available) services. Load Balancing API ReSTful API allowing customized solutions to automate load balancer ( Available) management Scalable relational database service that allows users to quickly and Database-as-a-Service easily utilize the features of a relational database without the burden of (TBD– Needs Approval) handling complex administrative tasks Allows service providers to manage multiple OpenStack clouds and Clustering-as-a-Service share physical resource among these cloud infrastructures and (TBD – Started) platforms Address Management and Discovery Provide network information services for use across OpenStack (TBD– Unknown) services. Initial focus for this project will be on IP address management (IPAM) and address discovery (DHCP/dnsmasq) functionality Provide network connectivity between devices managed by other OpenStack services such as nova • Provide flexibility in creating networks + associating devices to support interesting network topologies between VMs from the same Tenant Network Connectivity-as-a-Service • Example: create multi-tier applications (TBD– TBD) • Provide way to connect interconnect multiple Openstack services (*-aaS). • Example: Nova VM + Atlas LB on same private network. • Open the floodgates to let anyone build services (open or closed) that plug into Openstack networks. • Examples: VPN-aaS, firewall-aaS, IDS-aaS. <Release Name> <Version> <Release Date> - < Status>
OpenStack dev.Pipeline…incubating…draft…pending approval -status Feature Description Message queue that can be used in a variety of environments, from Distributed Message Queuing Service simple in-process queues to multi-tenant cloud services. In addition it (Diablo Needs Approval – Available) provides fan-out event notification mechanism so a single message may be read by multiple readers. Inventory service that allows to gather cloud wide node estate Topology Service information in-order to implement intelligent resource placement (TBD– TBD) mechanism for efficient utilization of DC resources (hardware, networking, etc.) Simplified management of complex resources. Logical grouping of Container Service resources (network, compute, storage) created/managed as one unit. (TBD– TBD) Network containers…initial focus Allows for multiple root nodes (top-level Zones) so business units can Multi-Cluster Zones partition the hosts in different ways for different purposes (i.e. (Cactus– Available) geographical zones vs. functional zones). (Zones are logical groupings of Nova Services and VM Hosts) Cloud Gateway Common interface to manage multiple cloud. Users will will be able to (TBD– TBD) design their cloud application environments once and use it on any cloud type <Release Name> <Version> <Release Date> - < Status>