Talk at ONUG Fall 2014

1. Understanding Network and Service Virtualization
Srini Seetharaman
srini.seetharaman@gmail.com

2. •Status Quo
•SDN and other buzzwords explained
•Deploying Network Virtualization
‒Show and Tell
•Deploying Service Virtualization
•Vendor solution survey and landscape
•Docker Networking
‒Show and Tell
Agenda
2

3. Technology Trends

4. Application Rollout Today
•Poor automation for VLAN, Service contexts, and VRFs
•Poor legacy application design?
Web
Tier
Application
Tier
Database
Tier

5. Typical Data Center Design
5
Rack
Core
Aggregation
Edge
Application group A
Application group B

6. Problem: Network not ready for
Over 70% of today’s servers are Virtual Machines, but VMs are not treated as first class citizens by the network
‒East-west traffic poorly managed
‒Lack of prioritization and rate-limiting at VM level
‒Traffic between VMs on same server often unsupervised
‒IP/MAC overlap not allowed, and addressing limited by VLANs
6
VM
VM
VM
VM
VM
VM
VM
VM
VMs
Containers
Symptoms of a broader problem with lack of proper network abstractions and policy layering

7. Trend #1: Network Virtualization
Dynamic, Programmable, Automated
7
Computing Infrastructure
SDN-based Virtualized Network Platform
Storage Infrastructure

8. Network Virtualization Requirements
8
Integration with
legacy network
End-to-end visibility of VM traffic
Traffic isolation across
virtual networks
•Support bare metal servers, appliances and gateways
•VLAN, VxLAN, GRE support, allowing IP overlap across tenants
•Edge-based control of VM traffic and scalable host tracking
Troubleshooting
support
Application policy
Orchestrating
virtual L4-L7 services
•End-to-end visibility that maps Virtual to Physical scalably
•Provisioning, and chaining of virtual services
•Application level policy across and within virtual networks

9. Trend #2: Service Virtualization
9
Internet
Internet
NFV
Step 1. Virtualizing network functions
Step 2. Chaining/Stitching them

10. NFV in Data Centers
1.Virtualizing the L4-L7 network service appliance (e.g., Load-balancer)
2.Chaining services to ensure that the traffic is routed through virtual appliances
3.Optimizing service delivery for applications
•Increasing number of virtual appliances
•Increasing CPU or memory of each appliance
•Placement of virtual appliances
•Offloading certain tasks to NIC or switch
10
Compute Orchestration
SDN control
Open-source?

11. Trend #3: New Infrastructure Tools
11

12. Business Potential of SDN and NFV
12
Business
How?
Reduced time to revenue
Speed up of service provisioning
OpEx saving
Automated operations and easier management of resources
New revenue
Through new business models centered around on-demand usage
Feature velocity
Introduce changes quickly according to business logic needs
Improved policy compliance
Ensure that cloud workload is compliant with enterprise policies (e.g., access control)
Reduced OpEx during upgrades
Introduce new functions and service by replacing just software stack

13. Software-defined Networking (SDN): Technology behind network virtualization

14. “Software-defined Network”
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Network Operating System
OpenFlow or other API
North-bound interface API
Unchanged mgmt API
Future Mode of Operation:
Lower complexity and cost,
Granular traffic management,
Dynamic and Automated
LB service
FW service
IP routing service
14
Legacy Router

15. Design choice: Insertion
1.In-network: Existing/green-field network fabrics upgraded to support SDN
2.Overlay: WITHOUT changing fabric, the intelligence is added to edge-devices,
‒as an additional appliance (e.g., bump-in-wire managed by controller)
‒as enhanced server kernel bridge (e.g., OpenVSwitch in x86 hypervisors)
15
Control Path
OpenFlow
Hardware switch
Data path (Hardware)
Figure courtesy of Martin Casada @ VMware

16. Design choice: Purist vs Hybrid
Hybrid approaches
1.Exclusively through embedded control plane: e.g., Yang modeled NetConf, OpFlex
2.Embedded control plane exists, but FIB reprogrammable directly: e.g., Hybrid switches with rule overridden by OpenFlow
3.Programming both embedded control plane and FIB: e.g., Open vSwitch
Data plane
Control plane
Mgmt plane Orchestration
Purist SDN architecture,
where flow-based abstraction programs all hardware
Extnl. Control plane
Mgmt plane Orchestration
Hybrid control plane where the hardware contains a more open platform for adding logic
Intl. Control plane
Data plane

17. Centralized Control
17
OpenFlow
Switch
OpenFlow
Switch
OpenFlow
Switch
Controller
Distributed Control
Controller
Controller
Controller
Design choice: Distributed operation
OpenFlow
Switch
OpenFlow
Switch
OpenFlow
Switch

18. Buzzwords: Open vSwitch, OVSDB, OpenDayLight, OpenStack

19. •Embraced by industry (including OpenStack, and Intel ) as de facto server networking software
24
Open vSwitch
Physical switch
OVSDB + Optionally OpenFlow
Open vSwitch
Controller
Open vSwitch
VM
VM
VM
VM
Open vSwitch: Most popular S/w switch
Tunnels

20. •Vendor-driven consortium (with Cisco, Brocade, and others) for developing open-source SDN controller platform
OpenDayLight Controller
25

21. Orchestration
North-bound API
Application
Controller
South-bound API
Dataplane elements
OpenStack Network Mgmt
26
Typical workflow
1.Create a network
2.Associate a subnet with the network
3.Boot a VM and attach it to the network
4.Delete the VM
5.Delete any ports
6.Delete the network
Network Virtualization App
SDN Controller
pSwitch
pSwitch
vSwitch
vSwitch
OVSDB
OpenFlow
Neutron API
ODL Mech driver
ML2 Plugin

22. OpenStack Networking in OpenDaylight
►Overlay-based OpenStack Networking supported today with
L2 forwarding and flooding
VLAN, GRE, VxLAN based segmentation
NAT and Distributed L3 Virtual Routing
Distributed ARP responder
ACL/Security policies for ingress and egress
Stateless load-balancing service
<#>

23. Deploying Network Virtualization

24. Goal
Computing Infrastructure
SDN-based Virtualized Network Platform

25. Deployment mode #1: Underlay
VPN termination, L3 routing
VM
VM
VM
VM
VM
VM
IP 192.168.1.2, MAC 0x1
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
Controller cluster
CLI, REST, GUI
IP 192.168.1.2, MAC 0x2
IP 192.168.2.2, MAC 0x1
IP 192.168.1.2, MAC 0x3
IP 192.168.1.2, MAC 0x2
IP 192.168.1.2, MAC 0x1
IP 192.168.2.1, MAC 0x2
IP 192.168.1.3, MAC 0x4
Tenant membership decided based on {switch-port, MAC, IP} tuple in each flow
30
VNet identified using VLANs, VxLANs or GRE
Internet
Custom routing by controller

26. •Problem: OpenFlow switches have resource limitations
‒Weak CPU incapable of doing traffic summarization, frequent statistics reporting, and packet marking
‒Flow-table limitation in switches (e.g., 1500 exact match entries)
‒Switch-controller communication limits (e.g., 200 packet_in/sec)
‒Firmware does not always expose the full capabilities of the chipset
•Solution:
‒Next generation of hardware customized for OpenFlow
‒New TCAMs with larger capacity
‒Intelligent traffic aggregation
‒Minimal offloading to vSwitches
Performance Limitations
31

27. Legacy L3 routing
Legacy L2 switching
VM
VM
VM
VM
VM
VM
10.1.1.0/24
10.1.2.0/24
10.2.1.0/24
10.1.1.1
10.1.1.2
10.1.2.1
10.1.2.2
10.2.1.1
10.2.1.2
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
vDP
vDP
vDP
vDP
vDP
vDP
Controller cluster
Internet
Logical link
v/p-Gateway
CLI, REST, GUI
Deployment mode #2: Overlay
vDP: Virtual Data Plane
VM addressing masked from fabric
Tunnels
Tenant membership decided by virtual interface on the vSwitch
vDP

28. VxLAN Tunneling
33
•Between VxLAN Tunnel End Points (VTEP) in each host server
•UDP port numbers allows better ECMP hashing
•In absence of SDN control plane, IP multicast is used for layer-2 flooding (broadcasts, multicasts and unknown unicasts)
VTEP outer MAC header
Outer IP header
Outer UDP header
VxLAN header
Original L2 packet
VxLAN flags
Reserved
24bit VN ID
Reserved
Source port
VxLAN port
UDP Length
Checksum

29. •Solution:
‒Offload it to the top-of- rack leaf switch
‒Use hardware gateway
•Problem:
‒Overlay mode is CPU hungry at high line rates and has anecdotally fared poorly in real world
Performance Limitations
34
Throughput
Recv side cpu
Send side cpu
Linux Bridge:
9.3 Gbps
85%
75%
OVS Bridge:
9.4 Gbps
82%
70%
OVS-STT:
9.5 Gbps
70%
70%
OVS-GRE:
2.3 Gbps
75%
97%
Source: http://networkheresy.com/2012/06/08/the-overhead-of-software-tunneling/

30. •Combined overlay and underlay (fabric) to achieve:
‒end-to-end visibility
‒complete control
‒best mix of both worlds
•Also called P+V or Overlay-Underlay
‒Vendors are converging towards this architecture
•The integration may need 1) link-local VLANs or 2) integration with VM manager to detect VM profile
Deployment mode #3: Hybrid
35

31. Deploying Network Service Virtualization
36

32. VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
vNF
vNF
vNF
vNF
vNF
vNF
CLI, REST, GUI
Typical Deployment Mode is Overlay
vNF: Virtualized Network Function
Services can be single-tenanted and multi-tenanted
vNF
vNF
vNF
vNF
Traffic to vFirewall
Traffic to dst VM
Traffic to VIP
Network Controller
Service Controller
Compute Controller

33. Service Type: Stateful and Stateless
38
OVS
VM 1
VM 2
VM 1
Host
OVS
VM 2
Host
Dst = VIP 1
Dst = VIP 2
Stateless service: No additional appliance needed
Stateful service: Virtual function deployed in VM or container
VM 3
Change header and Fwd to specific VM
Traffic proxied to specific VM
Stateless load-balancing and access policies can be done in this manner

34. Service Scaling: Scale-out and Scale-up
•Scale-out:
‒Deploy more network function instances
‒Scale-out of workload is also necessary
•Scale-up:
‒Give more resources to each network function instance
‒Offloading simple tasks to vSwitch, pSwitch or pAppliance
39

35. Combined Solution
OVS
VM 1
VM 2
VM 1
Host 1
OVS
VM 2
Host 2
OpenStack
Dst = VIP 1
Dst = VIP 2
Controller
Orchestration
Network Plumbing
VM 3
Service rollout and chaining
L2-L7 Service orchestration
DC Network Virtualization
Policy/ QoS
Trouble- shooting
UI/ Analytics
Compute
L3 Spine
VTEP Leaf

36. Vendor Landscape: Taxonomy is Important

37. Rack
Four types of SDN solutions
1.SDN-Dataplane
‒Traffic handling devices
Physical
Virtual
2.SDN-Control
‒Decoupled control plane
OpenFlow++
Overlay
3.SDN-Fabric
‒Combined data and control plane
4.SDN-Mgmt
‒Extensible mgmt software and API
Core
Aggregation
Edge
Controller cluster
Management/ Orchestration
Virtual switches
Server manager
42

38. Vendor Ecosystem
Data plane (Elements used for traffic handling)
Controller solutions (Decoupled control plane)
Fabric (Combined data and control plane)
Management (Extensible mgmt software and API)
L2-L4 routing
SDN-D- PSwitch
SDN-D- VSwitch
SDN-C- OpenFlow
SDN-C- Overlay
SDN-D-Fabric
SDN-N-Mgmt
43
(*Not necessarily complete)

39. Vendor Ecosystem
Data plane (Elements used for traffic handling)
Controller solutions (Decoupled control plane)
Fabric (Combined data and control plane)
Management (Extensible mgmt software and API)
L4-L7 services
SDN-S-Dataplane
SDN-S-Control
SDN-S-Fabric
SDN-S- Orchestrator
44
(*Not necessarily complete)

40. Docker Networking
45

41. •Over the past few years, LXC came up as an alternative to VM for running workload on hosts
•Each container is a clone of the host OS
•Docker brought Linux containers to prominence
‒Tracks application configuration and possibly archives to DockerHub
Linux Containers
46
Container 1
App X
Container 2
Container 3
Host OS
Guest root
App Y
Guest root
App Z
Guest root

42. Docker
•Excellent way to track application dependencies and configuration in a portable format.
•For instance the Dockerfile on the right can be used to spawn a container with nginx LB and accessed at a host port
$docker build XYZ
$docker images
$docker run -i --name=nginx1 -d –i nginx
$docker ps
$docker inspect nginx1
47
# Pull base image.
FROM dockerfile/ubuntu
# Install Nginx.
RUN
add-apt-repository -y ppa:nginx/stable &&
apt-get update &&
apt-get install -y nginx &&
rm -rf /var/lib/apt/lists/* &&
echo "ndaemon off;" >> /etc/nginx/nginx.conf &&
chown -R www-data:www-data /var/lib/nginx
# Define mountable directories.
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs", "/etc/nginx/conf.d", "/var/log/nginx"]
# Define working directory.
WORKDIR /etc/nginx
# Define default command.
CMD ["nginx"]
# Expose ports.
EXPOSE 80
EXPOSE 443

43. Networking Still in Early Stages
Today Docker usage is predominantly within a single laptop or host. The default network on right is allocated to the nginx container we spawned.
But, folks are exploring connecting containers across hosts.
48
"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"IPAddress": "172.17.0.15",
"IPPrefixLen": 16,
"MacAddress": "02:42:ac:11:00:0f",
"PortMapping": null,
"Ports": {
"443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "49157"
}
],
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "49158"
}
]
}

44. Nginx container
Container X
Host 1
Bash shell container
Container Y
docker0
Open vSwitch
Host 2
Internet
Open vSwitch
docker0
vxlan
vxlan
vxlan
vxlan
Other cluster hosts
Networking borrowed from VM world

45. Container and VM networking unified
•Edge-based overlays are even more important in container world.
•Open vSwitch already supports network namespaces
•VxLAN provides:
‒isolation,
‒improves L2/L3 scalability,
‒allows overlapping MAC/IP address
Docker Engine
OVS
OVS
OVS
Container
Container
Container
Container
Container
Container
VM
V
VM
Kubernetes
OpenStack
VxLAN Tunneled network
Neutron OVS agent

46. Networking Redefined
Going forward, all Networking is SDN, with varying architectures and networking policy being compiled down.
All operational goodness from the computing world is brought into networking world to make it unified.

47. Summary
•Looking at service virtualization separately is not wise. Recommend a joint evaluation
•VM and container networking work with similar network abstractions
‒But at different scale and velocity
•Edge-based intelligence using Open vSwitch, and VxLAN overlay is powerful.
•Open Daylight is catching up.

48. Thank you.
Slidehare.net/sdnhub