Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Autoscaling OpenStack Natively with Heat, Ceilometer and LBaaS

Autoscaling OpenStack Natively with Heat, Ceilometer and LBaaS workshop I delivered at OpenStack Vancouver Summit (May, 2015) jointly with Jason and Sharmin from Cisco System.

More details can be found at https://github.com/grimmtheory/autoscale

  • Be the first to comment

Autoscaling OpenStack Natively with Heat, Ceilometer and LBaaS

  1. 1. 201 Walkthrough Autoscaling OpenStack natively with Heat, Ceilometer and LBaaS
  2. 2. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Speakers Jason Grimm Consulting Systems Engineer Cisco OpenStack Private Cloud Sharmin Choksey Technical Leader Cisco Cloud Services Shixiong Shang Chief Technology Officer Nephos6 Agenda Introductions < About The Workshop < Environment Setup < Background & Use Cases < Services Enabling Autoscale < Hands-On <
  3. 3. About The Workshop Tips and Logistics Content and Schedule
  4. 4. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. ▪ We all work for or with Cisco, but this is not a marketing workshop ▪ Highly technical and all content is 100% FOSS (Free and Open-Source Software) ▪ All Materials available at https://github.com/grimmtheory/autoscale ▪ Please connect to http://cisco.webex.com/rooms/jasgrimm ▪ Chat, voting, questions, recording, etc. ▪ Allows us to see when everyone is ready to proceed ▪ Session recording will be made available after the summit ▪ Please work with your peers at your table! ▪ Bandwidth constraints and technical issues mean might not get a test environment up in time ▪ Some of you may have more experience with this technology and can assist others ▪ General collaboration and discussion Workshop Tips and Logistics
  5. 5. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Content and Schedule Introductions About The Workshop ** Environment Setup ** Autoscale Background and Use Case Services Enabling Autoscale Hands-On Workshop Lab 1 Lab 2.1 Lab 2.2 Lab 3 Q & A Open Discussion Close 30 Minutes (or less) 45 Minutes (or more) 15 Minutes (or less, as needed)
  6. 6. Environment Setup Virtual Box, Ubuntu, DevStack Install test environment Install content location and examples Test environment logical architecture Test environment configuration Test environment build process
  7. 7. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Install Test Environment… Instructions Before we can begin working with LBaaS, autoscale, etc. We need an environment. For this purpose we’ve chosen a locally installed environment due to past experiences with WIFI coverage and bandwidth at OpenStack summits. 1) Mount media 2) Copy data folder over locally 3) Install VirtualBox 4) Install Vagrant (Optional) 5) Boot Virtual Machine DVD Local Repo USB Key Internet HTTP HTTPMOUNTMOUNT Step 1 Mount Data Step 2 Copy Data Step 3 Install Apps Step 4 Boot VM
  8. 8. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Install content location and examples Also available at: • My VM - http://192.168.1.100/autoscale • My Cloud Server - http://38.84.67.227/autoscale • VirtualBox - https://www.virtualbox.org/wiki/Downloads Vagrantfile Git clone example Wget example
  9. 9. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Test environment logical architecture
  10. 10. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Test environment configuration Disabled Trove Sahara Cinder Nova Network Tempest Swift Enabled MySQL RabbitMQ Keystone Horizon Glance Neutron Ceilometer Heat OS & OpenStack Configuration $HOME = /home/vagrant/devstack Operating System = Ubuntu 14.04.x OpenStack version = “stable/kilo” eth0 = NAT, default GW eth1 = Management / API network eth2 = Neutron / Bridge network Open vSwitch L3 / DHCP / Floating IP “Flat” networking also available No tenant VLANs, FWaaS or VPNaaS Vagrant / VirtualBox Configuration Vagrant v.1.7.2 VirtualBox 4.3.28 eth0 = NAT eth1 = Host Only 192.168.33.2 /24 eth2 = Host Only 0.0.0.0 VT-x off, APIC-IO off, promiscuous on eth2
  11. 11. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Test environment build process Methodology • Tools-based install and config of the OS, DevStack and post-install tasks. • Automate difficult configuration of services and pre-requisites (Neutron, Heat, Ceilometer, OVS, custom images, etc.) Build Options • Base - OS install & DevStack configured • Staged - Base + DevStack install completed and basic configuration (security groups, networks and subnets created, etc.) • Complete - Staged + all objects created (LBs, VIPs, Ceilometer alarms, Heat templates, etc.) CREATE MACHINE 2 vCPU, 4 GB RAM, VT-x, APIC-IO 3 NICs - eth0 NAT, eth1 API, eth3 VMs Promiscuous mode for Neutron Bridge DEVSTACK PRE INSTALL Install pre-requisites Clone git repositories Configure local.conf, bridges, routes users, groups,iptables, ovs, sysctl, permissions, etc. INSTALL OS Base Install Patch / Update Install VBox extensions DEVSTACK INSTALL Install and monitor DevStack Install DEVSTACK POST INSTALL BASIC Add DNS to subnets Edit security groups Create / install SSH keys DEVSTACK POST INSTALL ADV. Create load balancers Add members to pool Create VIPs COMPLETE Snapshot the virtual machine
  12. 12. Background and Use Cases What is Autoscaling and when / how is it used?
  13. 13. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Autoscale Background and Use Cases What is Autoscaling…? “…a method used in cloud computing, whereby the amount of resources in a server farm scales automatically based on the load on the farm. It is closely related to, and builds upon, the idea of load balancing.” Who uses Autoscaling…? Any workload that benefits from programmatic scaling up or down as load dictates. Particularly true from a cost perspective when high workloads are very infrequent. Some examples are: • Retail - Black Friday, holidays, video game / movie launch, etc. • Academic / Research - HPCC • Media - Video / Audio rendering • Analytics - Big data, data mining, data modeling • Security - Federal or private data heuristic, cyber defense, etc. How does Autoscaling work…? Server Load Stress Meter Alarm Action Server Result+ + + +
  14. 14. Services Enabling Autoscaling An overview of: Heat Ceilometer Neutron w/ LBaaS
  15. 15. Heat Overview
  16. 16. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. HEAT
 OpenStack Orchestration Service
 Heat makes “clouds” rise heat The heat tool is a CLI which communicates with the heat-api to execute AWS CloudFormation APIs. End developers could also use the heat REST API directly. heat-api The heat-api component provides an OpenStack- native REST API that processes API requests by sending them to the heat-engine over RPC. heat-api-cfn The heat-api-cfn component provides an AWS Query API that is compatible with AWS CloudFormation and processes API requests by sending them to the heat-engine over RPC. heat-engine The heat engine’s main responsibility is to orchestrate the launching of templates and provide events back to the API consumer. Infrastructure resource abstraction and orchestration: • Physical compute Power • Physical switches • Physical block Storage • … OpenStack resource abstraction and orchestration: • VM Instances • Virtual Networks • Virtual Routers • … “Typically” Virtual (See Ironic Project)
  17. 17. Ceilometer & Neutron LBaaS Overview Ceilometer & Heat Overview
  18. 18. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Ceilometer Overview 1" Polling"agents" Meter" Transformer(s )" New" Meter" Publishing"Pipeline" API" Database" AMQP" Receiver" Collector" Other" OpenStack"" Service" alarm" evaluator" Publisher(s)" alarm" evaluator" Alarm"
  19. 19. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Ceilometer and Neutron LBaaS 2" Polling"agents" Meter" Neutron" (LBaaS)" Neutron" API" LBaaS"Default"Meters" Type" Unit" network.services.lb.ac0ve.connec0ons1 gauge1 connec0on1 network.services.lb.incoming.bytes1 cumula0ve1 B1 network.services.lb.total.connec0ons1 cumula0ve1 connec0on1 network.services.lb.vip1 gauge1 vip1
  20. 20. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Ceilometer and Neutron LBaaS 3" Transformer(s )" New" Meter" Publishing"Pipeline" Publisher(s)" sinks:& &&&&'&name:&lb_total_conn& &&&&&&transformers:& &&&&&&&&&&'&name:&"rate_of_change"& &&&&&&&&&&&&parameters:& &&&&&&&&&&&&&&&&source:& &&&&&&&&&&&&&&&&&&&map_from:& &&&&&&&&&&&&&&&&&&&&&&&name:&"network.services.lb.total.connec;ons"& &&&&&&&&&&&&&&&&&&&&&&&unit:&"connec;on"& &&&&&&&&&&&&&&&&target:& &&&&&&&&&&&&&&&&&&&&map_to:& &&&&&&&&&&&&&&&&&&&&&&&&name:&"network.services.lb.total.connec;ons.rate"& &&&&&&&&&&&&&&&&&&&&&&&&unit:&"connec;ons/s"& &&&&&&&&&&&&&&&&&&&&type:&"gauge"& &&&&&&publishers:& &&&&&&&&&&'&no;fier://& sources:& &&&&'&name:&lb_total_conn& &&&&&&interval:&60& &&&&&&meters:& &&&&&&&&&&'&"network.services.lb.total.connec;ons"& &&&&&&sinks:& &&&&&&&&&&'&lb_total_conn& Configuration file: /etc/ceilometer/pipeline.yaml
  21. 21. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Ceilometer and Neutron LBaaS 4" API" Database" Receiver" Collector" ceilometer sample-list -m network.services.lb.total.connections.rate! +--------------------------------------+--------------------------------------------+-------+--------+---------------+---------------------+! | Resource ID | Name | Type | Volume | Unit | Timestamp |! +--------------------------------------+--------------------------------------------+-------+--------+---------------+---------------------+! | 9d944aec-7d66-46b5-a71b-434e4e9fa98f | network.services.lb.total.connections.rate | gauge | 0.0 | connections/s | 2015-05-13T02:45:02 |! | 9d944aec-7d66-46b5-a71b-434e4e9fa98f | network.services.lb.total.connections.rate | gauge | 0.0 | connections/s | 2015-05-13T02:44:02 |! | 9d944aec-7d66-46b5-a71b-434e4e9fa98f | network.services.lb.total.connections.rate | gauge | 0.0 | connections/s | 2015-05-13T02:43:02 |! +--------------------------------------+--------------------------------------------+-------+--------+---------------+---------------------+! ! ! ! ceilometer statistics -m network.services.lb.total.connections.rate! +--------+---------------------+---------------------+-----+-----+-----+-----+-------+----------+---------------------+---------------------+! | Period | Period Start | Period End | Max | Min | Avg | Sum | Count | Duration | Duration Start | Duration End |! +--------+---------------------+---------------------+-----+-----+-----+-----+-------+----------+---------------------+---------------------+! | 0 | 2015-05-13T02:16:02 | 2015-05-13T02:47:02 | 0.0 | 0.0 | 0.0 | 0.0 | 32 | 1860.0 | 2015-05-13T02:16:02 | 2015-05-13T02:47:02 |! +--------+---------------------+---------------------+-----+-----+-----+-----+-------+----------+---------------------+---------------------+!
  22. 22. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Ceilometer and Neutron LBaaS 5" API" alarm" evaluator" alarm" evaluator" Alarm" ceilometer alarm-threshold-create --name lb_conn_rate_hi ! --description 'lb connection rate is too high' --severity critical --enabled True ! --meter-name network.services.lb.total.connections.rate ! --period 60 --evaluation-periods 3 --statistic avg --comparison-operator gt --threshold 2.0 ! --repeat-actions True! ceilometer alarm-list! +----------+-----------------+--------+----------+---------+------------+-----------------------------------------------------------------+! | Alarm ID | Name | State | Severity | Enabled | Continuous | Alarm condition |! +----------+-----------------+--------+----------+---------+------------+-----------------------------------------------------------------+! | uuid | lb_conn_rate_hi | ok | critical | True | True | network.services.lb.total.connections.rate > 2.0 during 3 x 60s |! +----------+-----------------+--------+----------+---------+------------+-----------------------------------------------------------------+!
  23. 23. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Ceilometer and Heat 6" Polling"agents" Meter" Transformer(s )" New" Meter" Publishing"Pipeline" API" Database" AMQP" Receiver" Collector" alarm" evaluator" Publisher(s)" alarm" evaluator" Alarm" Heat"
  24. 24. Hands-On Workshop Heat feature progression Basic Heat Operations Advanced Heat Operations Autoscale
  25. 25. ‹#› Auth Model • Icehouse • Security Isolation • Keystone Domain “heat” • Stack Domain Users managed by heat_admin_domain • Pre-configured in devstack • Requires Keystone v3 Resource Groups • Icehouse • De-coupling hierarchical nested stacks Stack Snapshot/Rollback Provider Resources • Icehouse • Useful in Template composition TripleO Heat Templates • Leverages Resource Groups, Provider Resources and SoftwareConfig Resource Key enhancements in Icehouse and • Kilo SoftwareConfig Resources • Icehouse • Useful in Template composition
  26. 26. ‹#› 26 Heat Autoscaling - Lab Exercise - 1 (Simple-Server) Spawn a VM Injects an ssh key Security groups VM port fixed ip + security group Floating ip associated it with the fixed ip Installs a simulated http server via the -init user-data script heat stack-create simple-stack -f simple-server.yaml --parameters "key_name=<key_name> ;node_name=<node_name> ;node_server_flavor=<node_server_flavor> ;node_image_name=<node_image_name>; ;floating_net_id=<floating_net_id>; ;private_net_id=<private_net_id>; ;private_subnet_id=<private_subnet_id>" User-defined Constrained and/or default values Type/Description Fundamental blocks Consumed by the Heat engine to orchestrate
  27. 27. ‹#› 27 Heat Autoscaling - Lab Exercise - 2 Part 1 (Load balancer) lb vip :: floating ip curl -X GET http://128.107.2.111 503 Service Not Available
  28. 28. ‹#› 28 Heat Autoscaling - Lab Ex - 2 Part 2 (Adding Members) web-server.yaml environment.yaml heat stack-create lb-members-stack -f lb-members.yaml -e environment.yaml --parameters "key_name=<key_name> ;node_name=<node_name> ;node_server_flavor=<node_server_flavor> ;node_image_name=<node_image_name>; ;floating_net_id=<floating_net_id>; ;private_net_id=<private_net_id>; ;private_subnet_id=<private_subnet_id>; ;pool_id=<pool_id> ;capacity_count=<capacity_count>" curl -X GET http://128.107.2.111 Welcome to 192.168.5.116 curl -X GET http://128.107.2.111 Welcome to 192.168.5.117 Nested Resource Identically Configured lb-members.yaml Provider Resource
  29. 29. ‹#› Scales arbitrary # of resources Resource to be scaled 29 Heat Autoscaling - Lab Ex - 3 (Autoscaling) lb-autoscale-members.yaml Creates an Alarm on a Action to be taken upon Alarm Raise Action to be executed against resource Connection rate crosses 3 per second for a consecutive period of 60 seconds each, the lb_connection_rate_high will be raised and a scaleup_policy will get executed
  30. 30. References Web Links (Content we used as reference when building this lab) Extra slides
  31. 31. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. Summit Lab / Workshop ▪ Materials for the presentation are available for download at https://github.com/grimmtheory/autoscale DevStack ▪ DevStack Configuration Guide – http://docs.openstack.org/developer/devstack/configuration.html Ceilometer Heat ▪ Heat main page - https://wiki.openstack.org/wiki/Heat ▪ Heat developer documentation -http://docs.openstack.org/developer/heat/ Heat operations documentation Heat main github repository Heat templates github respository Heat user template guide Heat developer template guide ▪ Heat Orchestration Template (HOT) guide – http://docs.openstack.org/developer/heat/template_guide/hot_guide.html ▪ Heat on DevStack guide – http://docs.openstack.org/developer/heat/getting_started/on_devstack.html References
  32. 32. Cisco Confidential ‹#›© 2013-2014 Cisco and/or its affiliates. All rights reserved. References Title Link Autoscale workshop repository https://github.com/grimmtheory/ Heat: main page https://wiki.openstack.org/wiki/Heat Heat: developer documentation http://docs.openstack.org/developer/heat/ Heat: operations documentation Heat main github repository

×