Your SlideShare is downloading. ×
0
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Predictive analytics and Visualization. Towards Data Driven Insights for Openstack Clouds

642

Published on

The slidedeck for the presentation by my team at the Atlanta Openstack Summit 2014.

The slidedeck for the presentation by my team at the Atlanta Openstack Summit 2014.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
642
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
38
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Cisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved. Predictive Analytics & Visualisation Towards Data Driven Insights for Openstack Clouds
  • 2. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved. Cloud/Big-data Winnovation Team Alex Holden ajonasho@cisco.com Damian Van Vuuren dvanvuur@cisco.com Ailing Zhang ailzhang@cisco.com Marc Solanas msolanas@cisco.com Matt Weeks maweeks@cisco.com Pengfei Zhang penfzhan@cisco.com Kai Zhang kazhang2@cisco.com Xinyuan Huang xinyuahu@cisco.com Debo Dutta dedutta@cisco.com Yathiraj Udupi yudupi@cisco.com
  • 3. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Visibility in OpenStack today
  • 4. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Tomorrow? Sources: 2001 A Space Oddesy, MGM. Minority Report, Twentieth Century Fox Film Corporation. Mad Men, AMC OpenStack, deploy me an app
  • 5. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Our Approach: Predictive Analytics & Awesome Visualisation Why Real Time? ● OpenStack has lots of moving parts ● Rule/Model/Policy based system hard without insight ● Ops folks want to know right now! ● Digital exhaust to characterize system behaviour. ○ Logs ○ Metrics Why Visualisation? ● Self Evident, easy to observe insights ● East way to consume lots of information ● Focus on the user
  • 6. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Cisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved. Digital Exhaust = Metrics/Logs What information can we get from our cloud? Where can we get it? ● Ceilometer ● OpenStack Syslog ● Project APIs (Nova, Neutron, Cinder etc) What can we do with it? ● Extract Insights ● Figure out what failed and why ● Move workloads around ● Detect attacks on the cloud ● Physical Servers ○ CPU ○ Hardware Faults ○ OpenStack Logs ○ OpenStack Messaging Queue ● Tenant/Project Data ○ Quotas ● Virtual Machines ○ Config ○ CPU ○ Disk ○ Network Flows ○ Performance ○ Latency ● Application Level ○ Application Logs ○ Application Requirements ○ Application Performance
  • 7. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved. AVOS Dashboard
  • 8. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential AVOS: Summary ● Analytics and Visualization on OpenStack ● Stateless: easy deployment ● Client side architecture: ○ Polls some OpenStack APIs ○ Listens to the messaging bus ○ Minimum overhead ● Provides users and service providers full insight into cloud ○ Real time info on all instances/networks etc by project and as admin ○ Historical data on performance ○ Stack-search: Search for all components in the cloud, quickly. ○ Multiple select (See combined data)
  • 9. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Analytics and Optimization on OpenStack AVOS
  • 10. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Bottlenecks Use Cases: Why use AVOS? Attacks
  • 11. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Use Case: Bottlenecks
  • 12. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Use Case: Attack Detection ● Classify Metrics ● Analyse Patterns ● E.G. is a util pattern: ○ Hadoop Traffic ○ Network Faults ○ DDoS Attack
  • 13. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Future Plans • Advanced Search Primitives (E.G. filter by instances that are shutoff quickly) • Reducing the number of relevant log messages • Real-time error analytics • Add to/edit the cluster • Ability to preview these changes Curvature: https://github.com/CiscoSystems/curvature
  • 14. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential OpenStack changes In order for these changes to be possible, some modifications to OpenStack are necessary: • Ceilometer Better API queries, optimizing return structure Custom meters: VM to VM traffic flows • Other suggestions/Ideas: Standardization of the Python clients (several inconsistencies between method calls) Integration into horizon “All in view of making OpenStack easier to build on top of, without the performance gains of applications above it lost in the overhead of running them.”
  • 15. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Conclusion Openstack Predictive Analytics/Visibility = Reduce Ops pains We must 1) gather data 2) extract insights 3) present to ops folks with kickass UX
  • 16. © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Thank you. Please take our survey: http://goo.gl/CMAZje Coming Talk: Storage Visibility and Optimization for OpenStack: Wednesday, May 14 • 3:30pm - 4:10pm

×