This document summarizes a presentation about achieving container networking without overlays by routing container IP addresses directly in the datacenter network. The presentation argues this approach reduces complexity, increases reliability, and avoids vendor lock-in. It then demonstrates routing containers in a spine-leaf fabric topology using Docker, Quagga routing on servers, and dynamic routing protocols. Container IPs are distributed as host routes throughout the network, providing connectivity, isolation, capacity scaling and high availability.

1. Copyright © Apstra, Inc. - 2016 - All Rights Reserved
Jeremy Schulman
@nwkautomaniac
The Datacenter Network
You Wish You Had
It's Yours for the Taking

2. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
15 min Talk
25 min Demo
5 min Q & A
Agenda
#NoBuzzwords #NoSalesPitch #NoJargon #NoBS
Jeremy Schulman
20 years in networking
Head of Customer Enablement
@ApstraInc
● 10 yrs Software/Eng distributed systems
● 10 yrs Field Systems/Eng
● Open source contributor
network automation (est. 2013)

3. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Inspiration for this Talk
" There is no such thing as container networking,
there is only networking. "
-- Kelsey Hightower

4. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
What You'll See Today
How every container IP address can be routed as any
other host in the datacenter network
Use different IP prefixes for different app
types to achieve traffic isolation
INTERNET
app
back-end
app
front-end

5. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Why is this Approach Important?
Ah-Ha !
Ping!
Cha-Ching !
Developers
DevOps
Network
applications
workload
infrastructure
network infrastructure
and services
scale app features
scale workload capacity
scale network capacity
Deploy
Operate + Grow Business
Design / Build
Maintain Service Uptime

6. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Why is this Approach Important?
"Less is More"
Reason Developer / DevOps Network
Reduce complexity / magic No container network overlays No datacenter fabric overlays
Use what you know
Keep using containers and
container management tools
Keep using standard routing
and network troubleshooting
tools
No vendor lock-in No container networking lock-in
No network hardware lock-in
No network OS lock-in
Attain situational awareness Quickly determine if network is the cause of application issues
Reduce Complexity • Increase Reliability

7. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
The Network We Wish We Had
IP Connectivity
Capacity Scaling
Traffic Isolation
Always Available
Expectations

8. Copyright © Apstra, Inc. - 2016 - All Rights Reserved
Complexity is the Mind Killer
The network is an interrelated collection of
distributed devices and protocols.
Software you didn't write, but have to troubleshoot.
The Problem: The Network We Do Have

9. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Troubleshooting,
as explained by the Telephone Game
Message In ! Message Out ?!
Imagine each person speaks two different languages …
Now imagine you need to figure out where things go wrong ...
You need to speak *all* the languages *all* the time

10. Copyright © Apstra, Inc. - 2016 - All Rights Reserved
Simplified Modern Datacenter
● Use one dynamic routing protocol for IP reachability and isolation
● Operate the same routing protocol everywhere, starting at servers
● Route containers as hosts, visible "citizens" in the network
● Leverage emerging automation technologies to manage operations
The Solution: The Network We Can Have

11. Copyright © Apstra, Inc. - 2016 - All Rights Reserved
The Awesome Demo
Running on My Laptop

12. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Technology Showcase
● Docker networking
ipvlan in l3-mode
● Docker Swarm with ETCD
● Docker IPAM plugin
● Container IP host route
injection
● Cumulus Linux network OS
● Deployed in a L3
Spine-Leaf Clos Fabric
● Dynamic Routing
● Cumulus VX / VirtualBox
● Universal Network Service
Management
● Datacenter L3 architecture
● Spine / Leaf Clos
● Server / Quagga

13. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Back-End
Front-End
Datacenter Router
Datacenter Fabric
2 Spine x 3 Leaf
IP Clos Design
Apstra
AOS-Server
Docker
Swarm
203.10.15 / 24
9.1.0.0 / 16
Container IP-Pools
INTERNET

14. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Container Route Distribution
eth0
datacenter
apps
swarm
node
Quagga
ipam
plugin
container
ip address
monitor
docker
socket
Docker Swarm
Controller
IPAM Server
Infrastructure Servers
1. Routing on the Host (Quagga)
2. Container IPs attached on eth0
3. Quagga peers routes with Leaf
4. Container IPs seen as /32 routes

15. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
Datacenter Route Distribution
Internet
Servers L3 Clos Fabric Router
BGP
7
1
2
3
5
6
app
back-end
app
front-end

16. Copyright © Apstra, Inc. - 2016 - All Rights ReservedCopyright © Apstra, Inc. - 2016 - All Rights Reserved
L3 Clos Fabric RouterServers
Managing IP Routing
71 3 6
Apstra AOS-Server
manages network services
BGP, LLDP, interfaces, etc.

17. Copyright © Apstra, Inc. - 2016 - All Rights Reserved
The Awesome Demo

18. Copyright © Apstra, Inc. - 2016 - All Rights Reserved
Jeremy Schulman
@nwkautomaniac
Thank You!