Slides from Chris Swan's ONUG Academy "Hands-On Container Networks" on May 12, 2015 This hands on session will begin by looking at how Docker modifies a Linux host to enable containers to be connected to a network. It will then go through how applications running in containers can be connected together, and the different options for interconnectivity on a host and between hosts. Finally we will take a look at running network application services inside of containers. Syllabus Learn what Docker does to your Linux host on installation. Connect applications running across multiple containers using configuration metadata and compositing tools. Understand the different Docker networking modes (host, container, none). Using Pipework to customise network configuration. Connecting containers across VMs using Open vSwitch. Using containers for application network services sush as proxies, load balancers and for TLS termination Learning Objective 1: Understand how containers relate to the host network, and the consequences that has for services running within containers Learning Objective 2: Understand the different ways that containers can be networked and internetworked. Learning Objective 3: Use containers to run network application services. About the topic: Containers aren’t a new thing, but the Docker project has made them a hot topic as organisations look at new ways to build, ship and run their applications. This brings new challenges for the network as containers are likely to be ten times as numerous as virtual machines. At the same time there is regulatory pressure to move away from the flat LAN model and deliver greater separation and segregation. This presentation will look at how these two forces are coming together, firstly by examining how containers are networked and some of the new approaches and challenges that come with that. This will be followed by a look at how overlay networks are being deployed to achieve ‘microsegmentation’, and ultimately drive a shift towards application centric networking. Of course these forces will collide, bringing us to contained networks of containers.

1. copyright 2015
Container networks
Chris Swan
@cpswan

2. copyright 2015
Syllabus
2
• Default Docker Network
• Multi container apps
• Networking modes
• Pipework
• Connecting containers across VMs using
Open vSwitch.
• Using containers for application network services such
as proxies, load balancers and for TLS termination

3. copyright 2015
Go at your own pace
3
Detailed instructions (and these slides) are available at:
https://github.com/cpswan/container-networking-tutorial
is.gd/onugcn

4. copyright 2015
The default Docker network

5. copyright 2015 5
Let’s start with a regular host
eth0
10.0.1.1

6. copyright 2015 6
Launch an instance
1) console.aws.amazon.com
2)
3)
4)

7. copyright 2015 7
Launch an instance cont.
5)
6)
7) Go ahead and launch it, then go to the instances view
to see private and public IP addresses

8. copyright 2015 8
Connect on SSH and inspect network
Connect:
ssh -i my_key.pem ubuntu@public_ip
Show IPs
ip addr
Look at NAT rules
sudo iptables -t nat -L -n

9. copyright 2015 9
Install Docker
eth0
10.0.1.1
docker0
172.17.42.1

10. copyright 2015 10
Install Docker and inspect network
Install Docker
wget -qO- https://get.docker.com/ | sh
Show IPs
ip addr
Look at NAT rules
sudo iptables -t nat -L -n

11. copyright 2015 11
Start a container
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab

12. copyright 2015 12
Start a container and inspect network
Start container
CON1=$(sudo docker run -d cpswan/hello_onug)
Get IP
CON1IP=$(sudo docker inspect
--format='{{.NetworkSettings.IPAddress}}' $CON1)
Show IP and use it
echo $CON1IP && curl $CON1IP:8080

13. copyright 2015 13
Start another container
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d

14. copyright 2015 14
Start 2nd container and use it
Start container
CON2=$(sudo docker run -d -p 8080:8080 cpswan/hello_onug)
Connect to the container
curl localhost:8080

15. copyright 2015 15
Take another look at the host network
Show IPs
ip addr
Look at NAT rules
sudo iptables -t nat -L -n

16. copyright 2015
Multi container apps

17. copyright 2015 17
A typical 3 tier app
HTTPS (tcp:443)
Web Server
(tcp:4567)
App Server
(tcp:3306)
DB Server

18. copyright 2015 18
Launch a 3 tier app with links
Create the directory for persistent data
sudo mkdir -p /data/mysql
Start the database
sudo docker run -d -p 3306:3306 --name todomvc_db
-v /data/mysql:/var/lib/mysql cpswan/todomvc.mysql
Start the app server
sudo docker run -d -p 4567:4567 --name todomvc_app
--link todomvc_db:db cpswan/todomvc.sinatra
Start the web server
sudo docker run -d -p 443:443 --name todomvc_ssl
--link todomvc_app:app cpswan/todomvc.ssl

19. copyright 2015 19
Look at how the links work
Get a shell in the app container
sudo docker exec -it $(sudo docker ps |
grep sinatra | cut -c1-12) bash
Take a look at the app using ENV variable
head /opt/sinatra-ToDoMVC-docker/app.rb
exit
Source is at:
https://github.com/cpswan/sinatra-ToDoMVC/blob/docker/app.rb

20. copyright 2015 20
Look at how the links work pt.2
Get a shell in the ssl container
sudo docker exec -it $(sudo docker ps |
grep ssl | cut -c1-12) bash
ENV variable has been hard coded into config
tail /etc/nginx/nginx.conf
The launch script uses a template to fetch ENV vars
cat /etc/nginx/upstream.template
exit
Source is at:
https://github.com/cpswan/dockerToDoMVC/blob/master/NginxSSL/start_nginx.sh

21. copyright 2015 21
Another quick look at iptables
Look at NAT rules
sudo iptables -t nat -L -n
Look at DOCKER chain
sudo iptables -L

22. copyright 2015 22
Docker Compose
Install Docker Compose
sudo apt-get install -y python-pip
sudo pip install -U docker-compose
Download and view example file
wget http://is.gd/onugdc -O docker-compose.yml
cat docker-compose.yml

23. copyright 2015 23
Bring up the demo app again
Restart Docker to clear out containers
sudo service docker restart
Invoke Docker compose (in background)
sudo docker-compose up &
List Docker processes
sudo docker ps

24. copyright 2015
Docker networking modes

25. copyright 2015 25
--net=host
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d

26. copyright 2015 26
--net=host example
Start a container
sudo docker run -d --net=host cpswan/hello_onug
Use it
curl localhost:8080

27. copyright 2015 27
--net=container
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d

28. copyright 2015 28
--net=container example
Start containers
CON1=$(sudo docker run -d cpswan/todomvc.mysql)
sudo docker run --net=container:$CON1 –d cpswan/hello_onug
Get IP, show IP and use it
CON1IP=$(sudo docker inspect
--format='{{.NetworkSettings.IPAddress}}' $CON1)
echo $CON1IP && curl $CON1IP:8080

29. copyright 2015 29
--net=none
eth0
10.0.1.1
docker0
172.17.42.1
eth0
172.17.0.1
veth67ab
eth0
172.17.0.2
veth9c5d

30. copyright 2015 30
--net=none example
Clear up
sudo service docker restart
Start container
CON2=$(sudo docker run --net=none -d cpswan/hello_onug)
No IP!
sudo docker inspect
--format='{{.NetworkSettings.IPAddress}}' $CON2

31. copyright 2015
Pipework

32. copyright 2015 32
Get Pipework
Install Pipework
sudo wget http://is.gd/onugpw -O /usr/bin/pipework
sudo chmod +x /usr/bin/pipework

33. copyright 2015 33
Connect together some containers
Start another container
CON1=$(sudo docker run --net=none -d cpswan/todomvc.mysql)
Add first container to a bridge
sudo pipework br1 $CON1 192.168.1.1/24
Add second container to a bridge
sudo pipework br1 $CON2 192.168.1.2/24

34. copyright 2015 34
Test connectivity
Shell into first container
sudo docker exec -it $CON1 bash
Show address
ip addr
Connect to second container for Hello World
curl 192.168.1.2:8080
exit

35. copyright 2015
Connecting containers across VMs using
Open vSwitch

36. copyright 2015 36
Implement ODCA SDN UM #4
http://www.opendatacenteralliance.org/docs/software_defined_networking_master_usage_model_rev2.pdf

37. copyright 2015 37
Launch another instance
1) Return to console.aws.amazon.com
2) Right click on existing instance and Launch More Like This
3) Launch
4) View Instances and get IP addresses

38. copyright 2015 38
Install Docker and Pipework on 2nd instance
Connect:
ssh -i my_key.pem ubuntu@public_ip
Install Docker
wget -qO- https://get.docker.com/ | sh
Install Pipework
sudo wget http://is.gd/onugpw -O /usr/bin/pipework
sudo chmod +x /usr/bin/pipework

39. copyright 2015 39
Install OVS on both instances
Install OVS
sudo apt-get install -y openvswitch-switch

40. copyright 2015 40
Connect instances together via OVS
On first instance
sudo ovs-vsctl add-br ovsbr0
sudo ovs-vsctl add-port ovsbr0 gre1 -- set interface
gre1 type=gre options:remote_ip=private_IP_instance2
On second instance
sudo ovs-vsctl add-br ovsbr0
sudo ovs-vsctl add-port ovsbr0 gre2 -- set interface
gre2 type=gre options:remote_ip=private_IP_instance1

41. copyright 2015 41
Test connectivity between VMs
On second instance
sudo pipework ovsbr0 $(sudo docker run --net=none
-d cpswan/hello_onug) 192.168.2.2/24
On first instance
CON1=$(sudo docker run --net=none -d cpswan/todomvc.mysql)
sudo pipework ovsbr0 $CON1 192.168.2.1/24
sudo docker exec -it $CON1 bash
curl 192.168.2.2:8080
exit

42. copyright 2015
Containerised network
application services

43. copyright 2015 43
Run Network App Svcs
Run container with HAProxy and Nginx:
NAS=$(sudo docker run -d -p 80:80 -p 443:443
-p 4433:4433 cpswan/net-app-svcs)
Add another HelloWorld for it to load balance over
sudo pipework ovsbr0 $(sudo docker run --net=none
-d cpswan/hello_onug) 192.168.2.3/24
Add the NAS container to the OVS bridge
sudo pipework ovsbr0 $NAS 192.168.2.4/24

44. copyright 2015 44
Open up AWS Security Groups
1) Return to console.aws.amazon.com
2) Click on Security Groups, launch-wizard-1, inbound, Edit
3) Add HTTP, HTTPS and Custom TCP Rule for 4433
4) Save

45. copyright 2015 45
Load balancer in action
Browse to http://public_ip
Browse to http://public_ip/haproxy?stats and sign
in with:
Username: us3r
Password: pa55Word

46. copyright 2015 46
TLS termination in action
Browse to https://public_ip
Accept browser security warnings
Bring up certificate information

47. copyright 2015 47
WAF in action
Browse to https://public_ip:4433
Accept browser security warnings
Browse to https://public_dns_address:4433

48. copyright 2015 48
Take a look at config
Get a shell on the NAS container
sudo docker exec -it $NAS bash
Inspect HAProxy config
more /etc/haproxy/haproxy.cfg
Inspect Nginx config
more /etc/nginx/nginx.conf
Source code is at http://is.gd/onugnas

49. copyright 2015
Review

50. copyright 2015
Review
50
• Default Docker Network
• Multi container apps
• Networking modes
• Pipework
• Connecting containers across VMs using
Open vSwitch.
• Using containers for application network services such
as proxies, load balancers and for TLS termination

51. copyright 2015
Further reading

52. copyright 2015
Take a look at
52
• Docker Network Configuration
https://docs.docker.com/articles/networking/
• Weave
https://github.com/weaveworks/weave
• Flocker
https://github.com/ClusterHQ/flocker
• Socketplane
https://github.com/socketplane/socketplane
• Tenus
https://github.com/milosgajdos83/tenus
• Project Calico
https://github.com/Metaswitch/calico

53. copyright 2015
Don’t forget to shut down AWS instances!