Speaker: Kaon Thana, The New York Times Abstract: The challenges and solutions of Infrastructure as Code (IaC) in the Classic Enterprise Network. As a small team of network engineers at the New York Times we manage many connectivity services such as a production data center, printing plant, corporate WAN, campus wired/wifi network and multi-cloud connectivity between AWS, GCP and Azure. In this talk I will discuss the challenges we faced over the last few years adopting automation practices in the brownfield and and the real-world solutions (with examples) we used to overcome the pitfalls.

1. Kaon Thana
Network Engineer
The New York Times
November, 2023
The challenges and solutions of Infrastructure as Code (IaC) in
the Classic Enterprise Network
Applying Platform Engineering Principles to
On-Premises Network Infrastructure

2. Network
Team
Connectivity
Campus Firewalls
Datacenter
Print Plant
Cloud Routing
Global Offices
VPN

3. Platform Engineering Principles
● Source Control
● Automation
● Infrastructure as Code
● Centralization
● Self-Service

4. Challenges
Vendor
Sprawl
Vendors
X
Models
X
Versions
Layer 0
Problem
| Environment
| Availability
| Regulations
Hidden
Work
| Invisible
tasks
| Institutional
knowledge
Fear
| Unknowns
| Fail at Scale
Entropy
| Unexpected
state changes
will happen

5. Solution: Condensed Execution Paths
| Arbitrary Python Jobs
| Crontabs
| Ad-Hoc Playbooks
| Access Control
| Centralized Execution
| Visibile
| Version Controlled
| Repeatable
| Manage Secrets

6. Solution: Condensed Execution Paths
GitHub + Drone Pipelines Ansible Tower Platform
“it only runs on my machine”
“I didn’t know this job exists”

7. Solution: Many-to-One
Autonomous
databases
Single relational
database
(Source of
Truth-iness)
Observability
Config
Operations

8. Golden Config
Jinja2 template
Conditional logic
Dynamic variables
“I forgot to disable that service”
“base_config_new_latest7.txt”

9. Driver Visibility
“let me pull up that excel doc
and get you the answer”
“is that new router
in monitoring?”

10. Solution: Digital Twin
prod
stage

11. Solution: Digital Twin
deploy validate
and test
“what is the expected behavior of this change?”

12. Solution: Digital Twin
self-service
learning labs
“I don’t know how this part of the network
operates so I can’t automate it”

13. Solution: Feedback Loops
source of
truth
running
state
fetch
set

14. Solution: Feedback Loop Example
“is this data accurate?”

15. Solution: Feedback Loop Example
“the provider made a change and this workflow
is broken now”

16. Recap
Platform Engineering is great
…but hard on-prem
Challenges
Solutions
Vendor
Sprawl
Layer 0
Problem
Hidden
Work
Fear Entropy
Condensed Paths
Many to One
Feedback Loops Digital Twins

17. Thank You!