This document discusses network visibility and analysis for efficient OpenStack operations. It describes a data gathering architecture that collects metrics from Neutron, OpenStack logs, server syslogs, and application logs. This data is analyzed using AVOS, an analytics and visualization tool for OpenStack. AVOS provides insights like network flows between VMs, troubleshooting application errors, and detecting abnormal cloud states by analyzing patterns from multiple metrics. Future plans include incorporating network distance and more integrated data-driven diagnosis.

1. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Network VisibilityFor Efficient OpenStack
Operations

2. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
The Cloud / Big-Data Winnovation Team
Alex Holden
ajonasho@cisco.com
Damian Van Vuuren
dvanvuur@cisco.com
Ailing Zhang
ailzhang@cisco.com
Marc Solanas
msolanas@cisco.com
Matt Weeks
maweeks@cisco.com
Pengfei Zhang
penfzhan@cisco.com
Kai Zhang
kazhang2@cisco.com
Xinyuan Huang
xinyuahu@cisco.com
Debo Dutta
dedutta@cisco.com
Yathiraj Udupi
yudupi@cisco.com

3. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Today...

4. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.

5. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Outline
• Data Gathering Architecture
• Network Visibility
• Network Analysis
• Conclusion

6. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Virtual status
Performance
Why is my job slow?
Operators
Physical status
Hotspots
What are they doing in my cloud?
Users

7. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential

8. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Network flows
New ceilometer agent
Collects and aggregates data from Neutron
VM-VM flow traffic insight

9. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Application logs
OpenStack Logs
Server Syslogs
VM Syslogs
Application (Hadoop, etc) Logs
...

10. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
AVOS:
Analytics and
Visualization on
OpenStack Demo

11. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Data Insights

12. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Data insights - What is going on in a Hadoop cluster?
Networkflows(bit/s)
Outgoing network flows from VMs
Time (s) Time (s)
Incoming network flows to VMs
Networkflows(bit/s)

13. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Data insights - What is going on in a Hadoop cluster?
Time (s)
Networkflows(bit/s)
Outgoing network flows
master <-> slave

14. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Data insights - What is going on in a Hadoop cluster?
Time (s)
Networkflows(bit/s)
Outgoing network flows
slave <-> slave

15. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Data insights - How to troubleshoot?
Normal Server down

16. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Data insights - How to troubleshoot?
INFO org.apache.hadoop.hdfs.DFSClient: Exception in
createBlockOutputStream …
java.io.IOException: Bad connect ack with firstBadLink
as 192.168.0.7:…
INFO org.apache.hadoop.hdfs.DFSClient: Abandoning
blk_…
INFO org.apache.hadoop.hdfs.DFSClient: Excluding
datanode 192.168.0.7:…
WARNING nova.compute.manager … [instance:
6a475088-...] Instance shutdown by itself. Calling the
stop API." …

17. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
● Collect Multiple Metrics
● Analyze Patterns
● Classify Cloud States
Data insights - How to detect abnormals?

18. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Future Plans
Network Distance
Data-driven Diagnoses
More data sources & integrated analysis

19. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Thank you.
Please take our survey:
http://goo.gl/CMAZje