Puppet Camp London 2015

1. Automating the Network
1
Matt Peterson
Office of the CTO
Cumulus Networks
PuppetCamp London – November 3rd, 2015

2. whois!
Puppet’ing since 0.24 (2008)
Added CL support in 2013

3. Rocket Turtle?
§ Cumulus Networks®

4. Why {Net}DevOps?
cumulusnetworks.com 4
greater operational insight
Networking is the last infrastructure silo

5. {Net}DevOps
cumulusnetworks.com 5
Leverage common DevOps tenants within Networking
§ Configuration management (today’s focus)
§ Make repetitive tasks predictable
§ Enforce policy to desired state - naming schemes,
software versions, … great audit trail mechanism
§ Infrastructure as code
§ Describe physical & logical components in human &
machine readable formats

6. {Net}DevOps
cumulusnetworks.com 6
§ Reactive to infrastructure as a whole
§ ie: L3 Clos architecture, single spine switch fans fail,
automatically adjust IGP cost to route around
§ Consistency (sometimes viewed as transparency)
§ Source of truth (across entire org, if possible), Git is
popular in compute circles
§ Common monitoring & escalation systems

7. A brief history of DevOps
§ …well, almost.
cumulusnetworks.com 7

8. Traditional network automation
§ Top-down efforts
§ SNMP
§ YANG/NETCONF
§ OpenFlow
§ Ansible
§ Screen-scraping
§ Difficult to integrate with the rest of the Ops stack
cumulusnetworks.com 8

9. “Whitebox switch” or bare metal networking
§ Treat as a server, preforms like a switch (or router)
• Kernel networking state sync to ASIC (and vice versa)
• ifconfig, ethtool, isc-dhcp … apt-get pkgname!
§ Consistent tooling across compute & networking
• CLI is usually bash, no walled garden
• Automation friendly from day 1
§ Choice on HW & SW suppliers
• Same as bare metal computing is today
• Applies to costly optics & cabling too!
routing bridging ops
Cumulus Linux
hardware

10. Modern network automation
cumulusnetworks.com 10

11. Modern network automation
§ Zero Touch Provisioning (ZTP)
§ Normally triggered by DHCP
§ Runs a provisioning script
§ Performs personalization
§ Install & enable Puppet
§ Puppet takes over after Cumulus install
cumulusnetworks.com 11

12. ZTP with DHCP example dhcp.conf
§ ddns-­‐update-­‐style
none;
§ default-­‐lease-­‐time
4320;
§ max-­‐lease-­‐time
8640;
§ authoritative;
§ option
cumulus-­‐provision-­‐url
code
239
=
text;
§ subnet
192.168.0.0
netmask
255.255.255.0
{
§
range
192.168.0.100
192.168.0.200;
§
option
routers
192.168.0.1;
§
option
domain-­‐name-­‐servers
192.168.0.1;
§
option
domain-­‐name
"lab.mycompany.com";
§
option
cumulus-­‐provision-­‐url
"http://192.168.0.2/provision.sh";
§ }

13. ZTP with DHCP example dhcp.conf
cumulusnetworks.com 13
dhclient.conf
option
cumulus-­‐provision-­‐url
code
239
=
text;
Exit hook
1. #!/bin/bash
2. #
Copyright
2015
Cumulus
Networks,
inc
all
rights
reserved
3. if
[
!
-­‐z
$new_cumulus_provision_url
]
4. then
5.
/usr/lib/cumulus/autoprovision
-­‐-­‐log
syslog
-­‐-­‐url
$new_cumulus_provision_url
&
6. fi

14. ZTP example with Puppet
1
#!/bin/bash
2
3
function
error()
{
4
echo
-­‐e
"e[0;33mERROR:
Provisioning
failed
running
$BASH_COMMAND
at
line
$BASH_LINENO
of
$(basename
$0)
e[0m"
>&2
5
exit
1
6
}
7
trap
error
ERR
8
#
Allow
Cumulus
testing
repo
9
sed
-­‐i
/etc/apt/sources.list
-­‐e
's/
^#s*(deb.*testing.*)$/1/g'
10
11
#
Upgrade
and
install
Puppet
12
apt-­‐get
update
-­‐y
13
apt-­‐get
upgrade
-­‐y
14
apt-­‐get
install
puppet
-­‐y
15
16
echo
"Configuring
puppet"
|
wall
-­‐n
17
sed
-­‐i
/etc/default/puppet
-­‐e
's/START=no/
START=yes/'
18
19
service
puppet
restart
20
21
#
CUMULUS-­‐AUTOPROVISIONING
22
23
exit
0

15. § Debian based, so we can do everything via. Puppet
§ Interface configuration (including Bonds & Bridges)
§ Routing software (Quagga – BGP/OSPF) configuration
§ Users, authentication, logging, packages, monitoring…
Puppet

16. Managing network interfaces
cumulusnetworks.com 16
auto
swp1
iface
swp1
address
192.168.1.10
netmask
255.255.255.0
mtu
9000
auto
swp2
iface
swp2
address
172.16.2.21
mtu
9000
…
auto
swp48
iface
swp48
address
10.1.1.10
netmask
255.255.254.0
mtu
9000

17. Managing network interfaces
cumulusnetworks.com 17
$swp1
=
{
'name'
=>
'swp1',
'address'
=>
'192.168.1.10',
'netmask'
=>
'255.255.255.0'
}
$swp2
=
{
'name'
=>
'swp2',
'address'
=>
'172.16.2.21'
}
$swp48
=
{
'name'
=>
'swp48',
'address'
=>
'10.1.1.10',
'netmask'
=>
'255.255.254.0'
}
$ifaces
=
[
$swp1,
$swp2,
$swp48
]
file
{
'/etc/network/interfaces':
ensure
=>
file,
content
=>
template('interfaces.erb')
}
<%
@ifaces.each
do
|iface|
-­‐%>
auto
<%=
iface['name']
%>
iface
<%=
iface['name']
%>
address
<%=
iface['address']
%>
<%
if
iface.has_key?('netmask')
-­‐%>
netmask
<%=
iface['netmask']
%>
<%
end
-­‐%>
mtu
9000
<%
end
%>

18. Managing network interfaces
cumulusnetworks.com 18
1. auto
lo
2. iface
lo
inet
loopback
3. <%
if
@int_loopback
-­‐%>
4.
address
<%=
int_loopback
%>
5.
netmask
255.255.255.255
6. <%
end
-­‐%>
7. auto
eth0
8. iface
eth0
inet
dhcp
9. <%
if
@int_unnumbered
-­‐%>
10. #
unnumbered
interfaces
11. <%
@int_unnumbered.each
do
|val|
-­‐%>
12. auto
<%=
val
%>
13. iface
<%=
val
%>
inet
static
14.
address
<%=
int_loopback
%>
15.
netmask
255.255.255.255
16. <%
end
-­‐%>
17. <%
else
%>
18. #
no
unnumbered
interfaces
19. <%
end
-­‐%>
20. <%
if
@int_layer3
%>
21. #
l3
interfaces
22. <%
int_layer3.each_pair
do
|key,
value_hash|
%>
23. auto
<%=
key
%>
24. iface
<%=
key
%>
inet
static
25.
address
<%=
value_hash['address']
%>
26.
netmask
<%=
value_hash['netmask']
%>
27. <%
end
%>
28. <%
else
%>
29. #
no
l3
interfaces
30. <%
end
%>
31. <%
if
int_bridges
%>
32. #
bridges
33. <%
int_bridges.each_pair
do
|key,
value_hash|
%>
34. auto
<%=
key
%>
35. iface
<%=
key
%>
inet
static
36.
address
<%=
value_hash['address']
%>
37.
netmask
<%=
value_hash['netmask']
%>
38.
bridge_ports
<%
value_hash['members'].each
do
|val|
-­‐
%><%=
val%>
<%
end
-­‐%>
39. <%
end
%>
40. <%
else
%>
41. #
no
bridges
42. <%
end
%>

19. Managing network interfaces
§ Everything in one file
§ Pros
§ Simple to implement and understand
§ Standard (no surprises)
§ Cons
§ Not simple if you have complex configurations
§ If one interface changes everything changes
cumulusnetworks.com 19

20. Managing network interfaces
cumulusnetworks.com 20
etc
└──
network
├──
interfaces
└──
interfaces.d
├──
eth0
├──
lo
├──
swp1
├──
swp2
├──
...
└──
swp48
auto
swp48
iface
swp48
address
10.1.1.10
netmask
255.255.254.0
mtu
9000
source
/etc/network/interfaces.d/*

21. Managing network interfaces
§ Idempotentency
§ notify
=>
Service['networking']
§ Change one interface, everything is reloaded
§ ifupdown2
§ Part of Cumulus Linux, optional in Debian, Ubuntu
§ ifreload
–a
cumulusnetworks.com 21

22. Managing network interfaces
cumulusnetworks.com 22
define
iface
{
$id
=
$name["name"]
$address
=
$name["address"]
$netmask
=
$name["netmask"]
file
{
"/etc/network/interfaces.d/$id":
ensure
=>
file,
content
=>
template("iface.erb"),
require
=>
File["/etc/network/interfaces.d"],
notify
=>
Exec["reload_$id"],
}
exec
{
"reload_$id":
command
=>
"ifdown
$id;ifup
$id",
path
=>
["/sbin"],
refreshonly
=>
true,
}
}
iface{
$ifaces:
}

23. Cumulus Linux Puppet Modules
cumulusnetworks.com 23
cumulus_interface
{'swp1':
ipv4
=>
'192.168.1.10/24',
}
cumulus_interface
{'swp2':
ipv4
=>
'172.16.2.21',
}
cumulus_interface
{'swp48':
ipv4
=>
'10.1.1.10/24',
}

24. Bridges & Bonds
cumulusnetworks.com 24
cumulus_bond
{'uplink':
slaves
=>
['swp10-­‐11']
}
cumulus_bridge
{
'br0':
ports
=>
['swp1-­‐2']
ipv4
=>
['10.1.1.1/24']
ipv6
=>
['2001:db8:abcd::/48']
alias_name
=>
'LXC
bridge'
mtu
=>
9000
}

25. Managing Layer 3
§ Quagga
§ OSPF, ECMP, BGP
§ Focused on an interactive CLI
• Makes automation a little awkward
§ Graceful reload is experimental
§ Other Open Source
§ Bird
§ OpenBGPD
§ ExaBGP
cumulusnetworks.com 25

26. Firewalls & ACLs
§ Firewalls
§ UFW rules are dead easy
• UFW Puppet modules exist
§ RedHat users are also covered
• Lokkit modules exist
§ IPTables
§ Loads of IPTables modules to choose from
• If you need it!
cumulusnetworks.com 26

27. Testing
§ Network Infrastructure as Code
§ It’s code!
§ Acceptance testing
§ Many vendors provide a virtual machine
§ You can connect them virtually into topologies
cumulusnetworks.com 27

28. Testing
cumulusnetworks.com 28

29. Testing
§ Vagrant
§ Serverspec
§ BATS, Cucumber, behave?
§ Beaker
§ GNS3
cumulusnetworks.com 29

30. Summary
§ Expressing networks as human (ie: virtual diagram) and machine
(YAML) = powerful (see PTM as an example)
§ HW appliances of all shapes are turning into
pure SW plays (some begrudgingly)
§ {Net}DevOps transformation has begun in networking
• Please help these men and women share your journey!

31. © 2014 Cumulus Networks. CUMULUS, the Cumulus Logo, CUMULUS NETWORKS, and the Rocket Turtle Logo (the “Marks”) are trademarks and service marks of
Cumulus Networks, Inc. in the U.S. and other countries. You are not permitted to use the Marks without the prior written consent of Cumulus Networks. The registered
trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis. All other marks are used
under fair use or license from their respective owners.
§ Thank You!
cumulusnetworks.com 31
Matt Peterson
matt@cumulusnetworks.com
@dorkmatt