OSMC 2015: Monitor Open stack environments from the bottom up and front to back by Roger Rüttimann
•
1 like•108 views
Elastic virtualization using the popular OpenStack platform is for real. While Sysadmins and DevOps professionals fully embrace these new developments, managing them is still a challenge. Adding layers of abstraction for compute, network and storage resources further increases complexity. Resource sharing, the fully dynamic creation of networks, virtual machines and recently Linux containers inside the framework also increases the challenge of managing these already complex systems. This presentation will provide insights on how to optimize the monitoring and management of OpenStack "from the bottom up", and from front to back to efficiently manage and troubleshoot OpenStack environments using API monitoring techniques and best of breed OpenSource tools such as Icinga 2.4, OpenStack API, Fuel, BoxSpy, OpenTSDB and others.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (20)
Similar to OSMC 2015: Monitor Open stack environments from the bottom up and front to back by Roger Rüttimann
Similar to OSMC 2015: Monitor Open stack environments from the bottom up and front to back by Roger Rüttimann (20)
Recently uploaded
Recently uploaded (20)
OSMC 2015: Monitor Open stack environments from the bottom up and front to back by Roger Rüttimann
- 1. Monitor Open stack environments from the bottom up and front to back Roger Ruttimann VP Engineering, GroundWork OpenSource November 17, 2015
- 2. What’s ahead of us Overview: The Impact of Virtualization on IT operations How Open Stack fits into the Virtualization landscape Monitoring the changed landscape of IT infrastructure New Monitoring concepts Selection of the right tools to fit the right process Conclusion 2OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 3. Start with the facts: Open Stack is real! It’s all OpenSource Has 5000 active members, growing daily Conferences draw close to 10,000 participants Many corporate sponsors 3 OpenStack deployments are not just happening in a far away land ... Not secret potion or evil conspiracy Not just a geek movement Serious deployments for IT Operations not just in the Silicon Valley It’s disruptive and requires serious re- tooling for IT operations Let’s review the challanges and what tools best fit the new realities Box Spy OSCM 2015 Monitor Open Stack environments from bottom up and front to back by Roger Ruttimann
- 4. Once upon a time SysAdmin task: Add To IT infrastructure { Deploy servers into datacenter Provisioning applications Define monitoring for each element Monitoring was done with SSH checks and port gets } repeat Checking for up down and send email 4OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann Switch Router Switch A p p s A p p s A p p s A p p s A p p s Firewall Server Server Server Monitor Port get ssh snmp snmp snmp
- 5. virtualization Then came along compute virtualization 5 Datacenter compute optimization • Server resources were virtualized to improve efficiency which was < 30% • Products like ESX allowed resource optimization not disturbing the provisioning process Rollout • VM machines were provisioned • Applications installed • Monitoring defined as before New was an API to the virtualization manager (example VSphere API) to poll data OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann Switch Router Switch A p p s A p p s A p p s A p p s A p p s Firewall Server Server Server Monitor Port get ssh snmp snmp snmp • REST API
- 6. But it didn’t stop there 6 Software Defined Everything • Compute (Hypervisors) • Storage (SDS) • Network (SDN) Hybrid Cloud Public/Private • Amazon WebServices • Rackspace • Azure Change created blind spots in coverage Suddenly ssh/snmp methods don’t cover all KVM RHEV-M VCenter virtualization Switch Router Switch A p p s A p p s A p p s A p p s A p p s Firewall Server Server Server Monitor Port get ssh snmp snmp snmp • REST API Virtualized Network Virtualized Storage Switches Switches Switches • REST API • REST API OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 7. Infrastructure landscape completely changed 7 Siloview Hypervisormanagerview Apps VMs Devices Files Shared I/O Fog Network & Storage OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann SiloviewSiloviewSiloviewSiloviewSiloview
- 8. And the moment you think you’ve seen it all … 8 … DevOps is pushing the envelope even more Linux Containers Are the new kids in town Application Isolation Rapid deployment Elastic Scalability API for deployment: Monitoring is somebody else’s job virtualization Switch Switch A p p s A p p s A p p s A p p s A p p s Server Server Server Monitor Port get ssh • REST API Virtualized Network Virtualized Storage Switches Switches Switches • REST API • REST API Server Docker Host Server Docker Host Server Docker Host OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 9. Don’t panic 9 What happened over the last 5 years: • Virtualize everything • Private and Public clouds • API centric world not just for Applications • REST API is a standard • CORBA, SOAP & proprietary API’s are now classified as dinosaurs • Browser based UI / JavaScript is king • Learn to speak REST & JSON Diverse Virtualization stacks vSphere API ESX NSX vSan OpenStack API Nova Neutron Cinder Glance AWS API EC2 VPC S3 oVirt API Linux KVM Network NFS OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 10. How does Open Stack fit in? 10 Service oriented architecture Pluggable hypervisors, network, storage to support wide range of technologies Elastic Compute Units for better virtualization efficiency Standardized API’s to all services Sounds really good doesn’t it ? Object Store--Swift Image-- Glance Compute/ Hypervisor--Nova Network— Quantum-NeutronStorage--Cinder Dashboard--Horizon Identity-- Keystone APIAPIAPIAPIAPI API But what’s the reality ? Façade ServiceFaçade Service OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 11. Open Stack Profile Editor 11OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 12. Open Stack Profile Editor 12OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 13. ManageIQ/ Cloud Forms Ansible API Manager Stack API HP HelionAPI VMWare VRealize Mirantis Fuel API API Provisioning/Configuration Servers Servers API is good but comprehensive coverage is better 13 Object Store Image Computer Network Identity Storage Dashboard Open Stack API OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann Container metrics Performance data Virtual infrastructure Legacy network Servers Storage racks Applications, Infrastructure checks ssh/SNMP/port get Unified View Standardized Data Collection
- 14. Unified View 14 Silo tools • Impossible to correlate • Naming mismatch • Over-monitoring • Encourages: “It’s not my problem check your system” • No Big Picture Dashboards Need for • Stack monitoring bottom to top • Aliasing of names • Combine best of breed collectors • Correlated metrics across all infrastructure • Dashboard for each customer/client OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 15. Stack monitoring for Open Stack 15 Function Source Host name Alias Network, Storage Netflow SNMP Hardware IPMI SNMP Drac-server1 os-eng-h1 Identity server check-mysql check-port server1:port os-eng-h1 Operating System check-proc check-mem check-load server1 os-eng-h1 OpenStack-API Hypervisor CPU/Mem metrics, Network storage 10.10.0.1 os-eng-h1 OpenStack API VM CPU/mem, Network, storage 10.10.10.123 eng-slicer-1 Operating system check-proc check-mem check-load eng-cent6-actg eng-slicer-1 DockerHost Memory, CPU eng-cent6-actg eng-slicer-1 Container Memory, CPU, Procs FAC3443DA77 Load-bal-01 Application Check_https 172.28.102.51 Load-bal-01 HypervisorVMAPP
- 16. Automation / Continuous configuration discovery 16 Adding Virtual machines, containers is automatic OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann V C Hypervisor So is the monitoring Management Monitor System Configuration collector Data collector Plugins Dashboards D a t a C C V V V V V C Monitoring is like a flight recorder -- collecting all active data Continuous operations by .. synchronization of configuration re-configuration of plugins Collection of data (availability & performance)
- 17. Open Stack API Icinga2 setup OpenStack Monitoring: Tools selection 17OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann OpenStack Linux OS Platform Hypervisor Hardware IPMI Port Agent OpenStack ControllerKeystone OpenStack StorageCinder Unified Monitoring View Data Management CollectorCollectorCollector Fuel Murano Application Software Catalog Icinga2 1) Fuel, Open Stack deployment 2) Murano, Application deployment 3) Fuel Plugin, Configuration generate 4) Fuel Plugin, Configure monitoring Docker Node 2 Node 3 Docker VM 1 OPEN TSDB Icinga2 VM 1 VM 2 VM 3 VM 4 Box Spy Box Spy Docker OPEN TSDB
- 18. Performance Data 18 Monitoring systems Grafana Dashboards VCenter OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann Centralized Data Collector • Monitoring send Perf Data to single API Expandable storage cluster • Expand on demand Dashboard to visualize and drill down • Historic, raw performance data • Group alike metrics to find outliers
- 19. Reference Architecture monitoring Aliasing and data normalization in backend VCenter A P I Data Collector Hub Collector Collector Collector Collect Data Integration Normalization Aliasing A P I Integrate A P I Visualize OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann Growing number of API’s require integration Hub
- 20. Benefits 20 Automation • Reduced cost of maintenance • Dynamic configuration • Streamlined operations Personnel • System Administrator & Dev Ops functions merge Capacity & Resource planning • Complete Bottom (network) to Top (application) data collection • Allocate resources at the right level Hardware • No vendor lock-in • Transparency through API OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 21. Lessons learned 21 Virtualization and containerization require new monitoring techniques Legacy hardware/software will be around for a while so don’t throw tools away Everything API -- needs integration Aliasing is hard but maybe easier than cross silo cooperation Use the best tool for the task Don’t over-monitor to cover a gap OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 22. Conclusion 22 Open Source tools are the driving force for innovation DevOps just selects the best tool to do the job A single monitoring tool to “rule them all” doesn’t exist Integration is a complex task – Don’t expect teams will agree on naming, process, and workflow Automation and pragmatism will prevail as DevOps has demonstrated Virtualize everything will continue Automation and continuous discovery is necessary for rapid scale out Open Stack is the Open Source Virtualization platform but Monitoring coverage needs major improvements OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann
- 23. 23 Thank you Roger Ruttimann rruttimann@gwos.com OSCM 2015 Monitor OpenStack environments from bottom up and front to back by Roger Ruttimann