This document introduces new Docker network drivers called Macvlan and Ipvlan. It provides information on setting up and using these drivers. Some key points: - Macvlan and Ipvlan allow containers to have interfaces directly on the host network instead of going through NAT or VPN. This provides better performance and no NAT issues. - The drivers can be used in bridge mode to connect containers to an existing network, or in L2/L3 modes for more flexibility in assigning IPs and routing. - Examples are given for creating networks with each driver mode and verifying connectivity between containers on the same network. - Additional features covered include IP address management, VLAN trunking, and dual-stack IPv4/

1. New Docker Network Drivers:
Macvlan & Ipvlan
Brent Salisbury - @networkstatic
John Willis - @botchagalupe
Docker Inc. at #ONS2016 - 3/16/2016

2. Macvlan Bridge & Ipvlan L2
• Very practical. No Unicorns required but cats welcome.
• Great for both existing and new networks.
• Native to Linux
• Lightweight
• Extremely Fast
• No NAT/PAT
• Docker Macvlan and Ipvlan Experimental Readme:
github.com/docker/docker/blob/master/experimental/vlan-networks.md
• Kernel docs on Macvlan and Ipvlan:
kernel.org/doc/Documentation/networking/ipvlan.txt

3. Getting Started
• Download the experimental binary
$ wget https://experimental.docker.com/builds/Linux/x86_64/docker-latest
$ chmod +x ./docker-latest
# Start the Docker engine daemon
$ ./docker-latest daemon
# Verify running version
$./docker-latest -v
Docker version 1.11.0-dev, build ..., experimental
• Build from source
$ git clone https://github.com/docker/docker.git
$ cd docker
$ DOCKER_EXPERIMENTAL=1 make binary
• Note on VirtualBox: If using, the bridge mode interfaces can be flaky.
VBox NAT mode interface is the path of least promiscuous pain
• Vmware Fusion: works out of the box with both modes.

4. Bridge/L2 Modes

5. $ ip route
default via 172.16.86.2 dev eth0
192.168.1.0/24 dev eth1 proto kernel scope link src
192.168.1.251
172.16.0.0/16 dev eth0 proto kernel scope link src 172.16.86.151
$ ip a show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UP
link/ether 00:50:56:2b:29:40 brd ff:ff:ff:ff:ff:ff
inet 172.16.86.151/16 brd 172.16.255.255 scope global eth0
valid_lft forever preferred_lft forever
Pre-Requisites Subnet+Gateway
• For Macvlan Bridge Mode and Ipvlan L2 modes, get some details
about the existing network.

6. Macvlan Bridge Mode

7. # Create a Docker Network Using the Macvlan Driver
$ docker network create -d macvlan
--subnet=172.16.86.0/24
--gateway=172.16.86.2 -o
parent=eth0 mcv
# Ping the Internetz.
$ docker run --net=mcv -it --rm alpine ping -c 4 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=128 time=3.455 ms
64 bytes from 8.8.8.8: seq=1 ttl=128 time=15.909 ms
64 bytes from 8.8.8.8: seq=2 ttl=128 time=7.843 ms
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 3.455/9.069/15.909 ms
Macvlan Bridge Mode

8. Ipvlan L2 Mode

9. # Create a Docker Network Using the Macvlan Driver
docker network create -d ipvlan
--subnet=192.168.1.0/24
--gateway=192.168.1.1
-o ipvlan_mode=l2
-o parent=eth0 db_net
# Start a container on the db_net network
docker run --net=db_net -it --rm alpine /bin/sh
Ipvlan L2 Mode

10. $ docker run --net=mcv --ip=172.168.86.10 -it --rm alpine /bin/sh
Do Whatever You Want
As of Docker v1.10 users can set container IP addresses explicitly.

11. IPAM
### Network macvlan with --ip-range
$ docker network create -d macvlan
--subnet=192.168.32.0/24
--ip-range=192.168.32.128/25
--gateway=192.168.32.254
-o parent=eth1 mcv
$ docker run --net=mcv -it --rm alpine /bin/sh
# View the address in the container
$ ip a | grep 192
inet 192.168.32.128/24 scope global eth0
# View the gateway you explicitly set
$ ip route
default via 192.168.32.254 dev eth0
192.168.32.0/24 dev eth0 src 192.168.32.128
• There are a lot of features in the default IPAM plugin, here are a couple.
Note: The addresses are not NATed. All addresses whether RFC 1918 or publicly
routable addresses are sent as the src_ip out the parent interface.

12. Moar IPAM
# Network exclude eth0 192.168.41.2
# address from IPAM with --aux-address
# eth0 in --aux-address=exclude1=192.168.41.2
# key/IP ${key} can be named anything
# Example: —aux-address=“favorite_ip_ever_ever=192.168.31.2”
$ docker network create -d macvlan
--subnet=192.168.41.0/24
--aux-address="favorite_ip_ever=192.168.41.2"
--gateway=192.168.41.1
-o parent=eth0 macnet41
# First address is the specified gateway, second is aux
$ docker run --net=macnet41 -it --rm alpine /bin/sh
# Check the IP
$ ip a show eth0 | grep 192
inet 192.168.41.3/24 scope global eth0

13. int gig 0/1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10,20,30
switchport mode trunk
:-)
802.1Q Trunking

14. VLANs

16. Manually Creating IP Links
# create a new sub interface tied to dot1q vlan 40
ip link add link eth0 name foo type vlan id 40
# enable the new sub-interface
ip link set foo up
# now add networks and hosts as you would normally by
# attaching to the master (sub)interface that is tagged
docker network create -d ipvlan
--subnet=192.168.40.0/24 --gateway=192.168.40.1
-o parent=foo ipvlan40
# in two separate terminals, start a Docker container
# and the containers can now ping one another.
docker run --net=ipvlan40 -it --name ivlan_test5 --rm alpine /bin/sh
docker run --net=ipvlan40 -it --name ivlan_test6 --rm alpine /bin/sh

17. Automated 802.1q Trunk Provisioning
# View Links prior to network create `ip link`
$ ip link
# Create multiple macvlan bridge subnets using a sub-interface eth0.215 and VLAN ID 215
docker network create -d macvlan
--subnet=192.168.215.0/24
--subnet=192.168.217.0/24
--gateway=192.168.215.1
-o parent=eth101
-o macvlan_mode=bridge macnet215
# View Links after to network create `ip link`
$ ip link
# Test 192.168.215.0/24 connectivity
docker run --net=macnet215 --ip=192.168.215.10 -itd alpine /bin/sh
docker run --net=macnet215 --ip=192.168.215.9 -it --rm alpine ping -c 2 192.168.215.10
# Test 192.168.217.0/24 connectivity
docker run --net=macnet215 --ip=192.168.217.10 -itd alpine /bin/sh
docker run --net=macnet215 --ip=192.168.217.9 -it --rm alpine ping -c 2 192.168.217.10
# Delete All Containers
$ docker rm -f `docker ps -qa`
# Delete all Networks
$ docker network rm $(docker network ls -q)
# Run ip links again and verify the links are cleaned up
$ ip link

18. Ipvlan L3 Mode

19. Really, Whatever You Want
# Dual Stack Ipvlan L3 mode with an interface
# specified using a dummy interface
# gateways IPs are ignored: (default dev eth0)
# no ARP/Broadcasts allowed
$ docker network create -d ipvlan
--subnet=192.168.8.0/24
--subnet=192.168.9.0/24
--subnet=fded:7a74:dec4:5a18::/64
--subnet=fded:7a74:dec4:5a19::/64
-o ipvlan_mode=l3
dualstack

20. Start Some Targets
# Start containers on 192.168.8.0/24 & 7a74:dec4:5a18::/64
docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::81 -itd alpine /bin/sh
docker run --net=dualstack --ip=192.168.8.80 -itd alpine /bin/sh
docker run --net=dualstack --ip=192.168.8.81 --ip6=fded:7a74:dec4:5a18::80 -itd alpine /bin/sh
# Start containers on 192.168.9.0/24 & 7a74:dec4:5a19::/64
docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::91 -itd alpine /bin/sh
docker run --net=dualstack --ip=192.168.9.90 -itd alpine /bin/sh
docker run --net=dualstack --ip=192.168.9.91 --ip6=fded:7a74:dec4:5a18::90 -itd alpine /bin/sh
# Start containers on a mix of the v4/v6 networks create
docker run --net=dualstack --ip=192.168.9.100 --ip6=fded:7a74:dec4:5a18::100 -itd alpine /bin/sh
docker run --net=dualstack --ip=192.168.8.100 --ip6=fded:7a74:dec4:5a19::100 -itd alpine /bin/sh

21. Ipvlan L3 things it shouldn't be able to do
# Ping from one v6 subnet to another enabled by L3 mode
docker run --net=dualstack --ip6=fded:7a74:dec4:5a19::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a18::81
docker run --net=dualstack --ip6=fded:7a74:dec4:5a19::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a18::100
# Ping from one v6 subnet to another enabled by L3 mode
docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a18::91
docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a19::100
# Ping from one v4 inside a subnet and to another enabled by L3 mode
docker run --net=dualstack --ip=192.168.8.25 -it --rm alpine ping -c 2 192.168.8.80
docker run --net=dualstack --ip=192.168.8.25 -it --rm alpine ping -c 2 192.168.9.91
# Ping from one v4 inside a subnet and to another enabled by L3 mode
docker run --net=dualstack --ip=192.168.9.25 -it --rm alpine ping -c 2 192.168.9.91
docker run --net=dualstack --ip=192.168.9.25 -it --rm alpine ping -c 2 192.168.8.80

22. Create 50+ networks & 125+ Containers in < 60 seconds
- Requires an interface named eth0 or set the ENV for $ETH
or
- modify script ETH=${ETH:-eth0}
$ curl -o vlan-tests.sh
https://raw.githubusercontent.com/nerdalert/dotfiles/master/ipvlan-macvlan-it.sh &&
chmod +x vlan-tests.sh
$ ./vlan-tests.sh
Networks are created twice to validate add/del functionality
Really Fast!

23. • Skunkworks repo to Dockerize network tools, all welcome to contribute!
https://github.com/gopher-net/dockerized-net-tools
$ docker run -it --rm gophernet/nmap -sT 192.168.1.1
Unable to find image 'gophernet/nmap:latest' locally
latest: Pulling from gophernet/nmap
7268d8f794c4: Pull complete
a3ed95caeb02: Pull complete
b45e16452ecd: Pull complete
Digest:
sha256:de08ac219d9d665beaad55f8796c85aba44dafcfc64ba4cbf3d53e8e62b2d95a
Status: Downloaded newer image for gophernet/nmap:latest
Starting Nmap 6.47 ( http://nmap.org ) at 2016-03-16 23:43 UTC
Network Tooling

24. # nmap in a container
# A couple of example usages:
# $ docker run -it --rm networkstatic/nmap --help
# Scan for open ssh (tcp/22) ports on a range of IPs
# $ docker run -it --rm networkstatic/nmap -sT 192.168.1.1-100 -p 22
#
FROM debian
MAINTAINER Brent Salisbury <brent.salisbury@gmail.com>
# build initial cache | install binary | remove cache
RUN apk update && apk add
nmap
&& rm -rf /var/cache/apk/*
ENTRYPOINT ["nmap"]
Network Tooling w/ Docker on HW Switches
• Do you know what your network is doing?
• Run and manage apps on switches without dependency nightmares

25. • drill is a tool from lens that is a replacement of dig.
• fping - tool for measuring latency, status and all around ping on steroids.
• hping is useful for both scanning networks and crafting packets.
• iperf - extremely versatile tool for measuring network bandwidth and performance.
• mz Mausezahn is a fast traffic generator which allows you to send nearly any kind of
packet.
• nmap - security scanner, port scanner and network discovery tool
• netcat - security scanner, port scanner and network discovery tool
• netflow generator - generate generic NetFlow data and send it to the specified
IP/Port of the NetFlow collector.
• sflowtool - sFlow collector
• traceroute print the route that IP packets traverse going to a remote host.
• traceroute6 print the route IPv6 packets will take to a network node.
Network Tooling

26. Questions?