1.
Stefan Vallin, Ph. D.
Principal Engineer
29 June 2016
Use Things You Can Program,
and Program the Things You Use
DevOps for
Networking

2.
Walls of Confusion
New and Changed
Requirements
Users
Product Mgr
New/Changed App Code
Configuration Change
Application
Developers
Network
Engineers
Operations

3.
What is DevOps?
http://dev.mobify.com/blog/devops-101-best-practices/
“The cool-kids way of
stringing stuff together with
shell scripts.
It exists because we can
now wrap things in shell
scripts that we used to only
dream of; the world is now
programmable at a much
larger scale, and we have
many new tools and
techniques for taking
advantage of it.”
Takes a new or
enhanced feature all
the way to production—
everyone is adding
value to the customer
A method that stresses
communication, collaboration,
integration, automation,
and cooperation across
organizational units to
deliver features quickly
and efficiently.
Based on lean and
agile principles
DevOps is the marrying
of process, infrastructure,
and product
New and Changed
Requirements
Customers
Users
New/ChangedApp Code
Configuration Change
Application
Developers
Network
Engineers
Operations

4.
Why Should You Care?
Better Customer Experience
Faster Better
Increased capacity to innovate
Let a small team multiply
their effectiveness
Faster time to value
Fewer failures
Recover faster from failures
More predictable behavior

5.
Why Should You Care?
We want
more change!
Dev
“
”
We want
less change!
Ops
“
”
You can have both!

6.
Network Automation
Historically DevOps
Tools: GIT, Python, NETCONF
• An IT/OSS project
• Waterfall
• Few network domain-experts
• Driven from the network
• Agile
• Designed and implemented by network
engineers with a DevOps mentality

7.
DevOps
Underpinning: Automation
People, Process
Culture
Practices Tools

8.
• Sharing
• Collaboration
• Learning
• Understanding other roles
• Understanding business
• No blame-culture
People, Process, Culture
Source: IBM: “DevOps for Dummies”
People, Process
Culture

9.
Across Roles
Product
Owners
DevOps
Team
AT&T Domain 2.0:Product Owners
Business
People
People, Process
Culture
Everyone need to change-
-not just network engineers
- Meet in same practices
- Meet in same tools
- Learn and educate
- More hands-on

10.
• Hypothesis-driven
• Slices that can be implemented in a
week
• Teams have a good understanding
of the flow of work from the
business all the way through to
customers
• Visibility
• Regularly seek customer feedback,
and incorporate this feedback into
the design of their products.
Lean Product Management
Source: Puppe Labs: “2016 State of DevOps Report”

11.
DevOps Networking Skills
Culture
Networking
Software
Engineering
Education ?
“ There remains much to do before this
vision can be implemented, including
pivots from networking craft to software
engineering, and from carrier operations
models to cloud “DevOps” models. ”
AT&T Domain 2.0:
People, Process
Culture

12.
• An increased knowledge of other IT disciplines
• More knowledge of the business
• More understanding of applications
• More emphasis on programming
Changes in Network
Professionals’ Skills
Stallings: Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud
People, Process
Culture

13.
DevOps
People, Process
Culture
Practices Tools

14.
• Infrastructure as code
• Automation
• Deploy with repeatable, reliable processes
• Self-service environments
• Automated environment for de-provisioning
• Automated recovery roll-back and roll-forward
• Hypothesis-driven development
• Dev and ops team shall use same tools
• Develop and test against production-like systems
• Amplify feedback loops
• Continuous integration
• Automated testing
• Continuous deployment
• Availability monitoring
• Monitor and validate operational quality
• Release management
• App performance monitoring
• Load-testing and auto-scale
Practices

15.
Before After
Before and After
Tools: Ticketing, Excel, Device CLI Tools: GIT, Python, NETCONF
• Operator role picks up ticket
in ITSM tool
• Implements the configuration
change manually or pseudo-
automatically
Cut and paste snippets
• Closes ticket
• Users provision their network
services directly
• DevOps role implements automation
• What do I do manually?
• How can I automate?
• How can I validate it?
• How do I handle failures?

16.
So DevOps is just scripting?
More than a shell script in:
~/my-cool-automation-scripts
Distribution, Execution, Monitoring, Error-Recovery, Logging,
Validation, Versioning, Feature Management, No-Hands

17.
Open Source, Open APIs, OpenFlow,
Open vSwitch, OpenDaylight, OpenConfig…
• Things can be said about all of the above,
but clear values coming out of it
• Standard APIs like NETCONF
• Scripting on the devices
• Linux-based devices, the shell prompt
• Networking devices are now open for
programming
Enabler: The Open Networking Trend
~ Use Things You
Can Program,
and Program the
Things You Use ~
Stallings: Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud

18.
Deep-Dive: DevOps is
about Managing Change
—
What is Changing?

19.
Separation of Concerns, Change life-cycles
Run-time fine-grained configuration
changes for individual services
• Modify QoS policy
• Add e-mail user
• Modify firewall rule
• Add backend to load-balancer
Network
Services
VNFs/VMs
Applications
Platforms
Backbone /
DC Network
Short
Lifecycle
Long
Lifecycle
Do Not
Blur/Mix
Make resources available to
provide services
• Install
• Start
• Stop
• Upgrade
• Provision L2 network in data center
connecting VNFs

20.
Nature of Configuration Changes
• Static “Golden Configs”
• Configuration
templates with variable
substitution
• Configuration
templates with loops,
if-statements
• Scripted configuration
based on stateServices
VNFs/VMs
Applications
Platforms
Backbone /
DC Network
Short
Lifecycle
Long
Lifecycle
Do Not
Blur/Mix
Different requirements for tools

21.
Comparison: Changes in Different Domains
Examples Changes Change Rate
Applications Latest code-base for web-shop In-house developed code.
Software changes
Daily
IT-Platforms New patch on WordPress Server
New Database Server
External Software releases and
patches.
Weekly
mysql config file Config file contents
Network Functions New VPN
Add leg to VPN
Add new VAS to VPN
Change QoS params for VPN
Configuration changes Thousands
per day
New device OS version New and upgraded devices Quarterly

22.
DevOps
People, Process
Culture
Practices Tools

23.
Typical Tools
Services
Applications
VNFs
VMs
Platforms
Backbone /
DC Network
Short
Lifecycle
Long
Lifecycle
• Service Orchestrators
• Scripts
• Vendor tools and UIs
• And the below
• Deployment:
• Chef, Puppet
• Juju charms, Heat
Templates, TOSCA
• Code and static configs
• GIT
• Build & Test
• Jenkins

24.
Application Centric Stack
Applications
Stitched
VMs
• IT Tools like Chef, Puppet,
TOSCA
• IT Tools like Chef, Puppet,
TOSCA
• Fine-grained configuration
tools for networking might be
needed

25.
Network Centric Stack
Network
Services
Virtual
Network
VNFs
• Tools like Chef, Puppet,
TOSCA not optimal
• Service-Orchestrators
• IT Tools like Chef, Puppet, TOSCA
• BUT, exposed networking
• AND Fine-tuned for packet-forwarding

26.
DevOps
People, Process
Culture
Practices Tools

27.
• Winner in Service Agility:
• Business Innovation and Product Owners in close co-op with DevOps team
• Data-Models and Running Code
• Sign of lagging behind
• Organization controlled by
“Architects in PowerPoint”
• OSS-projects
• Internal “purchase-orders”
Summary
Start creating your DevOps culture now

28.
Material and Reading
• John Willis, ”The Convergence of DevOps”
• http://itrevolution.com/the-convergence-of-devops/
• Kyle Young, Mobify: “DevOps 101: Best Practices for Optimizing and Automating
Your Infrastructure"
• http://dev.mobify.com/blog/devops-101-best-practices/
• David Tesar, “DevOps Practices”
• http://www.itproguy.com/devops-practices/
• Adrian Cockcroft: “Ops, DevOps and PaaS (NoOps) at Netflix”
• http://perfcap.blogspot.com/2012/03/ops-devops-and-noops-at-netflix.html
• Ashton, Metzler & Associates: “The Changing Role of the IT & Network Professional”
• http://www.ashtonmetzler.com/Quali%20Fifth%20Paper%20V2.0.pdf
• Lori MacVittie:”Network Engineers: Don’t Fear the Code”
• http://www.networkcomputing.com/careers/network-engineers-dont-fear-code/1347793692

29.
Material and Reading
• IBM: “DevOps for Dummies”
• https://www.ibm.com/marketing/iwm/iwm/web/signup.do?source=swg-rtl-sd-wp&S_PKG=ov18162
• Barry O’Reilly: “How to Implement Hypothesis-Driven Development”
• https://www.thoughtworks.com/insights/blog/how-implement-hypothesis-driven-development
• Julien Stroheker: “DevOps: Where do I start ? Cheat Sheet”
• https://blogs.technet.microsoft.com/juliens/2016/02/14/devops-where-do-i-start-cheat-sheet/
• Stefan Vallin and Caroline Chappel, Light Reading Upskill: “Virtualization, Automation”
• http://www.lightreading.com/virtualization-automation/l/d-id/722320
• Stallings: Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud
• Puppet, “State of DevOps Report”
• https://puppet.com/resources/white-paper/2016-state-of-devops-report