Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MidoNet Trouble Shooting – OpenStack最新情報セミナー 2015年4月

3,358 views

Published on

MidoNet Trouble Shooting
講師:鈴木 孝彰(株式会社ミドクラ)

「Midokura OpenStack Training」の人気講師が本セミナーのためにOpenStackネットワークについて分かりやすく解説いたします。これからOpenStackを始める人やOpenStackをふたたび始める人に最適のセッションとなっております。

Published in: Technology
  • Be the first to comment

MidoNet Trouble Shooting – OpenStack最新情報セミナー 2015年4月

  1. 1. MidoNet - Troubleshooting 1MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  2. 2. Troubleshooting ● General Approach ● Typical Issues ● MidoNet Tools ● Getting Help
  3. 3. General Approach 3MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  4. 4. Network Stack ● Multiple layers Underlay Network Overlay Network Topology Simulation Virtual Network Topology
  5. 5. Network Stack ● Multiple layers Underlay Network → Physical Environment Overlay Network → Datapath / Tunnel Topology Simulation → MidoNet Agent Virtual Network Topology → Neutron, NSDB
  6. 6. Physical Environment 6MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  7. 7. Physical Network ● Cabling o Cannot connect to physical nodes o ethtool → link detected? o ip link → device status? ● Routing o Cannot access VMs on particular node o netstat -nr o ip route Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  8. 8. Security Configuration ● Firewall o Cannot connect to services o iptables -L ● ACL / SELinux o Cannot access services o getenforce o setenforce Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  9. 9. Linux Kernel ● Open vSwitch module o MidoNet Agent does not start o VMs are not reachable o modinfo openvswitch o lsmod | grep openvswitch ● Load manually o modprobe openvswitch o insmod /lib/modules/… Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  10. 10. Clock Synchronization ● Sync clock on all nodes o Same time zone? JTC != UTC o ntpd o ntpdate ● Clocks out of sync? o NSDB troubles o 50% packet loss Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  11. 11. Datapath / Tunnel 11MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  12. 12. Datapath ● VMs unreachable, physical network ok ● Check status of MidoNet datapath o mm-dpctl --show-dp midonet o mm-dpctl --dump-dp midonet o mm-ctl --list-hosts Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  13. 13. Tunnel ● Tunnel Zone o Check tunnel-zone with midonet-cli o Check GRE/VXLAN interfaces → mm-dpctl --show-dp midonet ● MTU o Overhead of tunnel protocols  GRE (MTU 1454)  VxLAN (MTU 1450) Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  14. 14. MidoNet Agent 14MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  15. 15. MidoNet Agent ● MidoNet configuration issues o e.g. cannot start MidoNet Agent o Errors show up in the log file o Check NSDB connections o ping [NSDB_IP] o telnet [NSDB_IP] 2181 o telnet [NSDB_IP] 9160 ● Otherwise MidoNet bugs... Overlay Network Topology Simulation Virtual Network Topology Underlay Network
  16. 16. Neutron / NSDB 16MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  17. 17. Neutron / NSDB setup ● Security Groups o Proper rules added? ● ICMP, SSH, HTTP ● Gateway Configuration o neutron router-gateway-set o neutron router-interface-add Overlay Network Virtual NW Topology Simulation Virtual NW Topology Underlay Network
  18. 18. Neutron / NSDB consistency ● DO NOT delete components via MidoNet o neutron port-list returns error if you delete a port! o Neutron server error if you delete a bridge! ● Manual rebuild o Neutron → NSDB ● Use Neutron / Horizon Overlay Network Virtual NW Topology Simulation Virtual NW Topology Underlay Network
  19. 19. MidoNet Tools 19MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  20. 20. General CLI ● midonet-cli o Interactive or script-controlled o Client to the REST API o Browse and manipulate topology o Maintain tunnel zone o Upgrade MidoNet o man midonet-cli
  21. 21. Control ● mm-ctl o Show Agent hosts o Show VM tap interface mappings o Bind / unbind ports
  22. 22. Datapath ● mm-dpctl o Show flows o Delete flows o grep by IP / port
  23. 23. Packet Tracing ● mm-trace o Filters ● Protocol ● IP / MAC address ● Port number ● … o mm-trace –help
  24. 24. Statistics ● mm-stat o Statistics ● Packets processed ● Packets dropped ● Packets on-hold ● Flows created ● Latency ● ...
  25. 25. Getting Help 25MidoNet Troubleshooting, Copyright (c) 2015 Midokura
  26. 26. Getting Help ● Chat – https://slack.midonet.org ● Mailing Lists – https://lists.midonet.org ● Troubleshooting Guide – http://docs.midonet.org
  27. 27. Getting Help ● Chat – https://slack.midonet.org ● Mailing Lists – https://lists.midonet.org ● Troubleshooting Guide – http://docs.midonet.org
  28. 28. Thank You! 28MidoNet Troubleshooting, Copyright (c) 2015 Midokura

×