Slightly edited deck presented by myself and my awesome team at the Openstack Summit in May 2014 at Atlanta.

1. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Predictive Analytics & Visualization
Towards Data Driven Insights for Openstack Clouds

2. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Cloud/Big-data Winnovation Team
Alex Holden Damian Van VuurenAiling Zhang
Marc Solanas Matt Weeks Pengfei Zhang
Kai Zhang
Xinyuan Huang
Debo Dutta
Yathiraj Udupi

3. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Visualization in OpenStack today

4. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Tomorrow?
Sources: 2001 A Space Oddesy, MGM. Minority Report, Twentieth Century Fox Film Corporation. Mad Men, AMC
OpenStack,
deploy me an
app

5. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Our Approach:
Predictive Analytics & Awesome Visualisation
Why Predictive?
● OpenStack has lots of moving parts
● Rule/Model/Policy based system hard without insight
Why Predictive?
● Ops folks want to know right now!
Why Visualisation?
● Easy to observe insights by humans
● East way to consume lots of information from loads oflogs/metrics
● Improve user experience

6. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Digital Exhaust = Metrics/Logs
What information can we get from our cloud? Where can we get it?
● Ceilometer
● OpenStack Syslog
● Project APIs (Nova, Neutron, Cinder etc)
What can we do with it?
● Extract Insights
● Figure out what failed and why
● Move workloads around
● Detect attacks on the cloud
● Physical Servers
○ CPU
○ Hardware Faults
○ OpenStack Logs
○ OpenStack Messaging Queue
● Tenant/Project Data
○ Quotas
● Virtual Machines
○ Config
○ CPU
○ Disk
○ Network Flows
○ Performance
○ Latency
● Application Level
○ Application Logs
○ Application Requirements
○ Application Performance

7. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
AVOS Dashboard

8. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
AVOS: Summary
● Analytics and Visualization on OpenStack
● Stateless: easy deployment
● Client side architecture:
○ Polls some OpenStack APIs
○ Listens to the messaging bus
○ Minimum overhead
● Provides users and service providers full insight into cloud
○ Real time info on all instances/networks etc by project and as admin
○ Historical data on performance
○ Stack-search: Search for all components in the cloud, quickly.
○ Multiple select (See combined data)

9. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
https://www.youtube.com/watch?v=vyb-Cd4VEDg
AVOS Demo

10. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Bottlenecks
Use Cases: Why use AVOS?
Attacks

11. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Use Case: Bottlenecks

12. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Use Case: Attack Detection
● Extract Metrics
● Analyse Patterns
● We can differentiate
between:
○ Hadoop Traffic
○ Network Faults
○ DDoS Attack

13. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Future Plans
• Advanced Search Primitives
• Reducing the volume of logs
• Real-time error analytics
• Add to/edit the cluster
• Ability to preview changes
Curvature:
https://github.com/CiscoSystems/curvature

14. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Notes: OpenStack changes
• Ceilometer
Better API queries, optimizing return structure
Custom meters: VM to VM traffic flows
• Other suggestions/Ideas
Standardization of the Python clients (several inconsistencies between method calls)
Integration into horizon

15. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Conclusion
Openstack Predictive Analytics/
Visibility = Reduce Ops pains
We must 1) gather data 2) extract
insights 3) present to ops folks with
awesome UX

16. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Thank you.
Please take our survey:
http://goo.gl/CMAZje
Coming Talk:
Storage Visibility and Optimization for OpenStack:
Wednesday, May 14 • 3:30pm - 4:10pm