Docker Networking with New Ipvlan and Macvlan Drivers

11,554 views

Published on

Docker Networking presentation at ONS2016.

Docker Macvlan and Ipvlan Networking Drivers Experimental Readme:
github.com/docker/docker/blob/master/experimental/vlan-networks.md

Kernel requirements for Ipvlan mode is v4.2+, Macvlan mode is v3.19.

If using Virtualbox to test with, use NAT mode interfaces unless you have multiple MAC addresses working in your setup. Use the 172.x.x.x subnet and gateway used by the VBox NAT network. Vmware Fusion works out of the box.

Here is a screenshot of a VirtualBox NAT interface:
https://www.dropbox.com/s/w1rf61n18y7q4f1/Screenshot%202016-03-20%2001.55.13.png?dl=0

Published in: Technology

Docker Networking with New Ipvlan and Macvlan Drivers

  1. 1. New Docker Network Drivers: Macvlan & Ipvlan Brent Salisbury - @networkstatic John Willis - @botchagalupe Docker Inc. at #ONS2016 - 3/16/2016
  2. 2. Macvlan Bridge & Ipvlan L2 • Very practical. No Unicorns required but cats welcome. • Great for both existing and new networks. • Native to Linux • Lightweight • Extremely Fast • No NAT/PAT • Docker Macvlan and Ipvlan Experimental Readme: github.com/docker/docker/blob/master/experimental/vlan-networks.md • Kernel docs on Macvlan and Ipvlan: kernel.org/doc/Documentation/networking/ipvlan.txt
  3. 3. Getting Started • Download the experimental binary $ wget https://experimental.docker.com/builds/Linux/x86_64/docker-latest $ chmod +x ./docker-latest # Start the Docker engine daemon $ ./docker-latest daemon # Verify running version $./docker-latest -v Docker version 1.11.0-dev, build ..., experimental • Build from source $ git clone https://github.com/docker/docker.git $ cd docker $ DOCKER_EXPERIMENTAL=1 make binary • Note on VirtualBox: If using, the bridge mode interfaces can be flaky. VBox NAT mode interface is the path of least promiscuous pain • Vmware Fusion: works out of the box with both modes.
  4. 4. Bridge/L2 Modes
  5. 5. $ ip route default via 172.16.86.2 dev eth0 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.251 172.16.0.0/16 dev eth0 proto kernel scope link src 172.16.86.151 $ ip a show eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP link/ether 00:50:56:2b:29:40 brd ff:ff:ff:ff:ff:ff inet 172.16.86.151/16 brd 172.16.255.255 scope global eth0 valid_lft forever preferred_lft forever Pre-Requisites Subnet+Gateway • For Macvlan Bridge Mode and Ipvlan L2 modes, get some details about the existing network.
  6. 6. Macvlan Bridge Mode
  7. 7. # Create a Docker Network Using the Macvlan Driver $ docker network create -d macvlan --subnet=172.16.86.0/24 --gateway=172.16.86.2 -o parent=eth0 mcv # Ping the Internetz. $ docker run --net=mcv -it --rm alpine ping -c 4 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=128 time=3.455 ms 64 bytes from 8.8.8.8: seq=1 ttl=128 time=15.909 ms 64 bytes from 8.8.8.8: seq=2 ttl=128 time=7.843 ms --- 8.8.8.8 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 3.455/9.069/15.909 ms Macvlan Bridge Mode
  8. 8. Ipvlan L2 Mode
  9. 9. # Create a Docker Network Using the Macvlan Driver docker network create -d ipvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 -o ipvlan_mode=l2 -o parent=eth0 db_net # Start a container on the db_net network docker run --net=db_net -it --rm alpine /bin/sh Ipvlan L2 Mode
  10. 10. $ docker run --net=mcv --ip=172.168.86.10 -it --rm alpine /bin/sh Do Whatever You Want As of Docker v1.10 users can set container IP addresses explicitly.
  11. 11. IPAM ### Network macvlan with --ip-range $ docker network create -d macvlan --subnet=192.168.32.0/24 --ip-range=192.168.32.128/25 --gateway=192.168.32.254 -o parent=eth1 mcv $ docker run --net=mcv -it --rm alpine /bin/sh # View the address in the container $ ip a | grep 192 inet 192.168.32.128/24 scope global eth0 # View the gateway you explicitly set $ ip route default via 192.168.32.254 dev eth0 192.168.32.0/24 dev eth0 src 192.168.32.128 • There are a lot of features in the default IPAM plugin, here are a couple. Note: The addresses are not NATed. All addresses whether RFC 1918 or publicly routable addresses are sent as the src_ip out the parent interface.
  12. 12. Moar IPAM # Network exclude eth0 192.168.41.2 # address from IPAM with --aux-address # eth0 in --aux-address=exclude1=192.168.41.2 # key/IP ${key} can be named anything # Example: —aux-address=“favorite_ip_ever_ever=192.168.31.2” $ docker network create -d macvlan --subnet=192.168.41.0/24 --aux-address="favorite_ip_ever=192.168.41.2" --gateway=192.168.41.1 -o parent=eth0 macnet41 # First address is the specified gateway, second is aux $ docker run --net=macnet41 -it --rm alpine /bin/sh # Check the IP $ ip a show eth0 | grep 192 inet 192.168.41.3/24 scope global eth0
  13. 13. int gig 0/1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 10,20,30 switchport mode trunk :-) 802.1Q Trunking
  14. 14. VLANs
  15. 15. Manually Creating IP Links # create a new sub interface tied to dot1q vlan 40 ip link add link eth0 name foo type vlan id 40 # enable the new sub-interface ip link set foo up # now add networks and hosts as you would normally by # attaching to the master (sub)interface that is tagged docker network create -d ipvlan --subnet=192.168.40.0/24 --gateway=192.168.40.1 -o parent=foo ipvlan40 # in two separate terminals, start a Docker container # and the containers can now ping one another. docker run --net=ipvlan40 -it --name ivlan_test5 --rm alpine /bin/sh docker run --net=ipvlan40 -it --name ivlan_test6 --rm alpine /bin/sh
  16. 16. Automated 802.1q Trunk Provisioning # View Links prior to network create `ip link` $ ip link # Create multiple macvlan bridge subnets using a sub-interface eth0.215 and VLAN ID 215 docker network create -d macvlan --subnet=192.168.215.0/24 --subnet=192.168.217.0/24 --gateway=192.168.215.1 -o parent=eth101 -o macvlan_mode=bridge macnet215 # View Links after to network create `ip link` $ ip link # Test 192.168.215.0/24 connectivity docker run --net=macnet215 --ip=192.168.215.10 -itd alpine /bin/sh docker run --net=macnet215 --ip=192.168.215.9 -it --rm alpine ping -c 2 192.168.215.10 # Test 192.168.217.0/24 connectivity docker run --net=macnet215 --ip=192.168.217.10 -itd alpine /bin/sh docker run --net=macnet215 --ip=192.168.217.9 -it --rm alpine ping -c 2 192.168.217.10 # Delete All Containers $ docker rm -f `docker ps -qa` # Delete all Networks $ docker network rm $(docker network ls -q) # Run ip links again and verify the links are cleaned up $ ip link
  17. 17. Ipvlan L3 Mode
  18. 18. Really, Whatever You Want # Dual Stack Ipvlan L3 mode with an interface # specified using a dummy interface # gateways IPs are ignored: (default dev eth0) # no ARP/Broadcasts allowed $ docker network create -d ipvlan --subnet=192.168.8.0/24 --subnet=192.168.9.0/24 --subnet=fded:7a74:dec4:5a18::/64 --subnet=fded:7a74:dec4:5a19::/64 -o ipvlan_mode=l3 dualstack
  19. 19. Start Some Targets # Start containers on 192.168.8.0/24 & 7a74:dec4:5a18::/64 docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::81 -itd alpine /bin/sh docker run --net=dualstack --ip=192.168.8.80 -itd alpine /bin/sh docker run --net=dualstack --ip=192.168.8.81 --ip6=fded:7a74:dec4:5a18::80 -itd alpine /bin/sh # Start containers on 192.168.9.0/24 & 7a74:dec4:5a19::/64 docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::91 -itd alpine /bin/sh docker run --net=dualstack --ip=192.168.9.90 -itd alpine /bin/sh docker run --net=dualstack --ip=192.168.9.91 --ip6=fded:7a74:dec4:5a18::90 -itd alpine /bin/sh # Start containers on a mix of the v4/v6 networks create docker run --net=dualstack --ip=192.168.9.100 --ip6=fded:7a74:dec4:5a18::100 -itd alpine /bin/sh docker run --net=dualstack --ip=192.168.8.100 --ip6=fded:7a74:dec4:5a19::100 -itd alpine /bin/sh
  20. 20. Ipvlan L3 things it shouldn't be able to do # Ping from one v6 subnet to another enabled by L3 mode docker run --net=dualstack --ip6=fded:7a74:dec4:5a19::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a18::81 docker run --net=dualstack --ip6=fded:7a74:dec4:5a19::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a18::100 # Ping from one v6 subnet to another enabled by L3 mode docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a18::91 docker run --net=dualstack --ip6=fded:7a74:dec4:5a18::25 -it --rm alpine ping6 -c 2 fded:7a74:dec4:5a19::100 # Ping from one v4 inside a subnet and to another enabled by L3 mode docker run --net=dualstack --ip=192.168.8.25 -it --rm alpine ping -c 2 192.168.8.80 docker run --net=dualstack --ip=192.168.8.25 -it --rm alpine ping -c 2 192.168.9.91 # Ping from one v4 inside a subnet and to another enabled by L3 mode docker run --net=dualstack --ip=192.168.9.25 -it --rm alpine ping -c 2 192.168.9.91 docker run --net=dualstack --ip=192.168.9.25 -it --rm alpine ping -c 2 192.168.8.80
  21. 21. Create 50+ networks & 125+ Containers in < 60 seconds - Requires an interface named eth0 or set the ENV for $ETH or - modify script ETH=${ETH:-eth0} $ curl -o vlan-tests.sh https://raw.githubusercontent.com/nerdalert/dotfiles/master/ipvlan-macvlan-it.sh && chmod +x vlan-tests.sh $ ./vlan-tests.sh Networks are created twice to validate add/del functionality Really Fast!
  22. 22. • Skunkworks repo to Dockerize network tools, all welcome to contribute! https://github.com/gopher-net/dockerized-net-tools $ docker run -it --rm gophernet/nmap -sT 192.168.1.1 Unable to find image 'gophernet/nmap:latest' locally latest: Pulling from gophernet/nmap 7268d8f794c4: Pull complete a3ed95caeb02: Pull complete b45e16452ecd: Pull complete Digest: sha256:de08ac219d9d665beaad55f8796c85aba44dafcfc64ba4cbf3d53e8e62b2d95a Status: Downloaded newer image for gophernet/nmap:latest Starting Nmap 6.47 ( http://nmap.org ) at 2016-03-16 23:43 UTC Network Tooling
  23. 23. # nmap in a container # A couple of example usages: # $ docker run -it --rm networkstatic/nmap --help # Scan for open ssh (tcp/22) ports on a range of IPs # $ docker run -it --rm networkstatic/nmap -sT 192.168.1.1-100 -p 22 # FROM debian MAINTAINER Brent Salisbury <brent.salisbury@gmail.com> # build initial cache | install binary | remove cache RUN apk update && apk add nmap && rm -rf /var/cache/apk/* ENTRYPOINT ["nmap"] Network Tooling w/ Docker on HW Switches • Do you know what your network is doing? • Run and manage apps on switches without dependency nightmares
  24. 24. • drill is a tool from lens that is a replacement of dig. • fping - tool for measuring latency, status and all around ping on steroids. • hping is useful for both scanning networks and crafting packets. • iperf - extremely versatile tool for measuring network bandwidth and performance. • mz Mausezahn is a fast traffic generator which allows you to send nearly any kind of packet. • nmap - security scanner, port scanner and network discovery tool • netcat - security scanner, port scanner and network discovery tool • netflow generator - generate generic NetFlow data and send it to the specified IP/Port of the NetFlow collector. • sflowtool - sFlow collector • traceroute print the route that IP packets traverse going to a remote host. • traceroute6 print the route IPv6 packets will take to a network node. Network Tooling
  25. 25. Questions?

×