The document details the internal architecture and operations of network virtualization technologies used in a cloud environment, specifically focusing on VLANs, packet captures, security groups, and routing during VM creation. It describes components created during VM and router creation, their internal mappings, and the related configurations for firewall rules implemented by security groups. Additionally, it outlines various network configurations such as the creation of provider networks, DHCP functions, and OpenFlow rules for efficient data transmission within virtualized environments.

1. Network internals(advancedparts)
Giuliano Santandrea–CIRI ICT
Universityof Bologna

2. ●Internal-external VLAN translation
●packet captures
●Security groups
●routing

3. Duringthe VM creationtheseelementsare createdin the compute node:
◦qbrZZZ: LB and itsmgmtinterface
◦qvbZZZ: vethpairconnectedto the LB
◦qvoZZZ: vethpairconnectedto the OVS bridge “br-int”
◦tapZZZ: tapinterface, connectedto the LB
ZZZ: first 11 lettersof the Neutron"port" for the VM interface

4. Subnetcreation(network node):
◦tap-YYY: tapinterfaceconnectedto br-int, inside a network namespace(YYY are the first 11 lettersof the "port" of the DHCP server)
Router creation(network node):
◦tap-AAA tapinterfaceconnectedto br-ex, inside a network namespace(AAA are the first 11 lettersof the "port" of the router gateway)
◦tap-BBB tapinterfaceconnectedto br-int, inside a network namespace(BBB are the first 11 lettersof the "port" of the router internalport)

5. On the physicaldata network manynetwork virtualizationtechnologiesare possibile (VLAN,VXLAN,GRE,..).
InternallyOS mapseachvirtualnetwork to an internalVLAN
The Cesena cluster usesthe VLANs. The bridgesin the VNI are configuredto do the translationbetweenexternal-internalVLANs
Otherexample: GRE encapsulation
◦for packetsdirectedto the data network, the bridgesremoveinternalVLAN tagsand encapsulatethemwith a atunnel_id

6. public net
gateway
External net
Mgmtnet
Data net
CPU node 1
Controller
Network node
br-data
br-int
linux
bridge
VM
br-data
br-int
br-ex
br-data
br-int
Internet
Untagged
internal VLAN tag
External VLAN tag
untagged

7. specific
routing
tables
dhcpserver
dhcpserver
Network namespaces
No traffichere

8. VM
eth0
VLAN accessport-based(internalVLAN)
Trunkall

9. Trunkall
VLAN accessport-based(internalVLAN)

10. VM
eth0
TCAM (OpenFlowrules):
priority=4,in_port=8,dl_vlan=1 actions=mod_vlan_vid:1000,NORMAL
priority=2,in_port=8 actions=drop
priority=1 actions=NORMAL
•For allpacketscomingfrom phy-br-data and tag=1: changetag=1000, thendo classicMAC Learning Switching(MLS)
•Discardpacketscomingfrom phy-br-data
•OtherwiseMLS (leastpriority)
VLAN
1 => 1000

11. VM
eth0
priority=3,in_port=17,dl_vlan=1000 actions=mod_vlan_vid:1,NORMAL
priority=2,in_port=17 actions=drop
priority=1 actions=NORMAL
VLAN
1000 => 1

13. VM
eth0
untagged
InternalVLAN tag
externalVLAN tag
ExternalVLAN tag

14. InternalVLAN tag
externalVLAN tag
ExternalVLAN tag
No traffichere!
untagged

15. On the VM wesendan Ethernet frame (ARP) in broadcast:
◦sudo arping–bI eth0 10.0.0.9
broadcastallowstobypassMAClearningofthebridges:eachbridgewillforwardtheframetoeveryport!
On the cluster node:
◦tcpdump–nnveiinterface
Or ifin a netns:
◦sudo ipnetnsexec<netns> bash#enterin the netns
◦tcpdump–nnvleiinterface#flushwith -l

17. stack@hc01:~/devstack$sudo tcpdump-nnveiqvb71cbe0bd-6f
tcpdump: WARNING: qvb71cbe0bd-6f: no IPv4 addressassigned
tcpdump: listeningon qvb71cbe0bd-6f, link-typeEN10MB (Ethernet), capturesize65535 bytes
18:44:23.752905 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertypeARP (0x0806), length42: Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
18:44:24.752998 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertypeARP (0x0806), length42: Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
^C
2 packetscaptured
2 packetsreceivedbyfilter
0 packetsdroppedbykernel
NO VLAN TAG!!!

19. root@hc01:/opt/stack#sudo tcpdump-nnveiint-br-data
tcpdump: WARNING: int-br-data: no IPv4 addressassigned
tcpdump: listeningon int-br-data, link-typeEN10MB (Ethernet), capturesize65535 bytes
18:46:41.212436 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertype802.1Q (0x8100), length46: vlan1, p 0, ethertypeARP, Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
18:46:42.212633 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertype802.1Q (0x8100), length46: vlan1, p 0, ethertypeARP, Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
^C
2 packetscaptured
2 packetsreceivedbyfilter
0 packetsdroppedbykernel

21. root@hc01:/opt/stack#sudo tcpdump-nnveieth0
tcpdump: WARNING: eth0: no IPv4 addressassigned
tcpdump: listeningon eth0, link-typeEN10MB (Ethernet), capturesize65535 bytes
18:49:57.241431 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertype802.1Q (0x8100), length46: vlan1000, p 0, ethertypeARP, Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
18:49:58.020910 d0:7e:28:90:d9:4b > 01:80:c2:00:00:00, 802.3, length64: LLC, dsapSTP (0x42) Individual, ssapSTP (0x42) Command, ctrl0x03: STP 802.1w, RapidSTP, Flags[Forward], bridge-id8000.d0:7e:28:90:d9:3d.800d, length47
message-age0.00s, max-age20.00s, hello-time2.00s, forwarding-delay15.00s
root-id8000.d0:7e:28:90:d9:3d, root-pathcost0, port-roleDesignated
18:49:58.241620 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertype802.1Q (0x8100), length46: vlan1000, p 0, ethertypeARP, Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
^C
3 packetscaptured
3 packetsreceivedbyfilter
0 packetsdroppedbykernel

23. root@hc01:~#tcpdump-nnveitap8356e24c-67
tcpdump: listeningon tap8356e24c-67, link-typeEN10MB (Ethernet), capturesize65535 bytes
^C11:39:28.424480 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertypeARP (0x0806), length42: Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
11:39:29.424638 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertypeARP (0x0806), length42: Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
11:39:30.424733 fa:16:3e:e6:9e:f8 > ff:ff:ff:ff:ff:ff, ethertypeARP (0x0806), length42: Ethernet (len6), IPv4 (len4), Requestwho-has10.0.0.9 (ff:ff:ff:ff:ff:ff) tell10.0.0.66, length28
3 packetscaptured
3 packetsreceivedbyfilter
0 packetsdroppedbykernel
NO VLAN TAG!!!

25. Secgroup: containsfirewall rulesconfiguredby the user (atthe cloudplatformlevel)
◦Duringthe VM creationweassociate oneor more secgroups
Itis«default deny», wecan addrulesto allowingresstraffic
In the default secgroupthereare alreadyrulesallowingegresstraffic
Implementation: iptablesruleson the CPU node
Note: it’s implemented by Neutron applying the native kernel filtering functions (netfilter) to bridged tap interfaces, and this works only with LBs.For this reason an additional LB is needed as an intermediate element to interconnect the tap interface to the integration bridge.

26. Iptables rules (global
namespace) on the linux
bridge port
VM
eth0

27. Wehaveenabledsshand pingin Ingress

28. For allpacketsenteringthe LB, passingthroughthe tap(outboundVM traffic, EGRESS), use the followingchains(iptablesfiltertablein the global netnsof the compute node):
neutron-openvswi-sg-chain
neutron-openvswi-oXXX
neutron-openvswi-FORWARD
FORWARD
Source: http://goo.gl/lD30Vl
VM
eth0

29. …exitingthe LB (inboundtraffic,INGRESS)…:
neutron-openvswi-sg-chain
neutron-openvswi-iXXX
neutron-openvswi-FORWARD
FORWARD
Source: http://goo.gl/lD30Vl
VM
eth0

30. Weenabledssh(TCP porta 22) and ping(ICMP), wecan seetheserules:
-A neutron-openvswi-sg-chain-m physdev--physdev-outtapb5d4535b-8f --physdev- is-bridged-j neutron-openvswi-ib5d4535b-8
-A neutron-openvswi-ib5d4535b-8 -m state --state INVALID -j DROP
-A neutron-openvswi-ib5d4535b-8 -m state --state RELATED,ESTABLISHED -j RETURN
-A neutron-openvswi-ib5d4535b-8 -p tcp-m tcp--dport22 -j RETURN
-A neutron-openvswi-ib5d4535b-8 -p icmp-j RETURN
-A neutron-openvswi-ib5d4535b-8 -s 192.168.101.2/32 -p udp-m udp--sport 67 --dport68 -j RETURN
-A neutron-openvswi-ib5d4535b-8 -j neutron-openvswi-sg-fallback

34. The admincreatesa provider network with the allocationpool 10.250.0.50-10.250.0.70 (20 addresses)
Itisattachedto a virtualrouter
The virtualrouter isattachedto a user private network
The router
◦hasan addresson the provider network (10.250.0.50)
◦hasan addresson the user network (192.168.101.1)
◦actsasa NAT

38. sudoipnetnsexec qrouter-XXX bash
ipaddressshow
1: lo: <LOOPBACK,UP,LOWER_UP> mtu65536 qdiscnoqueuestate UNKNOWN
link/loopback00:00:00:00:00:00brd00:00:00:00:00:00
inet127.0.0.1/8 scope hostlo
inet6 ::1/128 scope host
valid_lftforeverpreferred_lftforever
27: qr-8110d0f8-64: <BROADCAST,UP,LOWER_UP> mtu1500 qdiscnoqueuestate UNKNOWN
link/etherfa:16:3e:98:f7:ddbrdff:ff:ff:ff:ff:ff
inet192.168.101.1/24 brd192.168.101.255 scope global qr-8110d0f8-64
inet6 fe80::f816:3eff:fe98:f7dd/64 scope link
valid_lftforeverpreferred_lftforever
29: qg-64643e7a-3e: <BROADCAST,UP,LOWER_UP> mtu1500 qdiscnoqueuestate UNKNOWN
link/etherfa:16:3e:e9:46:1a brdff:ff:ff:ff:ff:ff
inet10.250.0.50/24brd10.250.0.255 scope global qg-64643e7a-3e
inet6 fe80::f816:3eff:fee9:461a/64 scope link
valid_lftforeverpreferred_lftforever

39. iprouteshow
default via 10.250.0.3devqg-64643e7a-3e
10.250.0.0/24 devqg-64643e7a-3e proto kernelscope link src10.250.0.50
192.168.101.0/24 devqr-8110d0f8-64 proto kernelscope link src192.168.101.1
sudoiptables-t nat–nvL
…
Chainneutron-l3-agent-snat (1 references)
pktsbytestarget protoptin out source destination
12 882 neutron-l3-agent-float-snat all--* * 0.0.0.0/0 0.0.0.0/0
6 426 SNAT all--* * 192.168.101.0/24 0.0.0.0/0 to:10.250.0.50
…

40. In some oldOS docs«br-data» iscalled«br- ethX»
Using GRE tunnel, bridge br-data iscalledbr- tun

41. Provider network can be currentlycreatedonlyvia CLI
◦The creationof a provider network requireto specifythe physicalnetwork (mappedto a virtualbridge, connetedto a physicalnetwork)
The netns/dhcpserver are notimplementedatthe theirdefinitiontime, butonlywhena VM on thatnetwork iscreated

42. To ensure connectivity to a VM:
●The tenant user that booted the VM must have enabled the access by inserting the appropriate rules in the secgroupsand then attaching the secgroupto the VM
●neutron-plugin must have inserted the correct OpenFlowrules in the OVS bridges (br-int, br-data, br- ex)
●The “dnsmasq” linuxprocess (managed by neutron- dhcp) must be working properly as DHCP server for the VM