The document discusses the integration of predictive analytics and visualization in OpenStack environments to enhance data-driven insights and operational efficiency. It highlights the importance of real-time data access through various OpenStack APIs, enabling users to detect issues, optimize resources, and improve user experiences. Future enhancements and use cases for the analytics tool, including attack detection and bottleneck analysis, are also addressed.

1. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Predictive Analytics & Visualization
Towards Data Driven Insights for Openstack Clouds

2. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Cloud/Big-data Winnovation Team
Alex Holden Damian Van VuurenAiling Zhang
Marc Solanas Matt Weeks Pengfei Zhang
Kai Zhang
Xinyuan Huang
Debo Dutta
Yathiraj Udupi

3. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Visualization in OpenStack today

4. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Tomorrow?
Sources: 2001 A Space Oddesy, MGM. Minority Report, Twentieth Century Fox Film Corporation. Mad Men, AMC
OpenStack,
deploy me an
app

5. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Our Approach:
Predictive Analytics & Awesome Visualisation
Why Predictive?
● OpenStack has lots of moving parts
● Rule/Model/Policy based system hard without insight
Why Predictive?
● Ops folks want to know right now!
Why Visualisation?
● Easy to observe insights by humans
● East way to consume lots of information from loads oflogs/metrics
● Improve user experience

6. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Digital Exhaust = Metrics/Logs
What information can we get from our cloud? Where can we get it?
● Ceilometer
● OpenStack Syslog
● Project APIs (Nova, Neutron, Cinder etc)
What can we do with it?
● Extract Insights
● Figure out what failed and why
● Move workloads around
● Detect attacks on the cloud
● Physical Servers
○ CPU
○ Hardware Faults
○ OpenStack Logs
○ OpenStack Messaging Queue
● Tenant/Project Data
○ Quotas
● Virtual Machines
○ Config
○ CPU
○ Disk
○ Network Flows
○ Performance
○ Latency
● Application Level
○ Application Logs
○ Application Requirements
○ Application Performance

7. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
AVOS Dashboard

8. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
AVOS: Summary
● Analytics and Visualization on OpenStack
● Stateless: easy deployment
● Client side architecture:
○ Polls some OpenStack APIs
○ Listens to the messaging bus
○ Minimum overhead
● Provides users and service providers full insight into cloud
○ Real time info on all instances/networks etc by project and as admin
○ Historical data on performance
○ Stack-search: Search for all components in the cloud, quickly.
○ Multiple select (See combined data)

9. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
https://www.youtube.com/watch?v=vyb-Cd4VEDg
AVOS Demo

10. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Bottlenecks
Use Cases: Why use AVOS?
Attacks

11. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Use Case: Bottlenecks

12. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Use Case: Attack Detection
● Extract Metrics
● Analyse Patterns
● We can differentiate
between:
○ Hadoop Traffic
○ Network Faults
○ DDoS Attack

13. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Future Plans
• Advanced Search Primitives
• Reducing the volume of logs
• Real-time error analytics
• Add to/edit the cluster
• Ability to preview changes
Curvature:
https://github.com/CiscoSystems/curvature

14. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Notes: OpenStack changes
• Ceilometer
Better API queries, optimizing return structure
Custom meters: VM to VM traffic flows
• Other suggestions/Ideas
Standardization of the Python clients (several inconsistencies between method calls)
Integration into horizon

15. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Conclusion
Openstack Predictive Analytics/
Visibility = Reduce Ops pains
We must 1) gather data 2) extract
insights 3) present to ops folks with
awesome UX

16. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Thank you.
Please take our survey:
http://goo.gl/CMAZje
Coming Talk:
Storage Visibility and Optimization for OpenStack:
Wednesday, May 14 • 3:30pm - 4:10pm