Speaker: Damien Garros, OpsMill Abstract: As an industry we made a lot of progress adopting Automation to manage Network infrastructure in the last few years however only a few organizations have been able to reach the expert level and most are still trying to break the glass ceiling that comes with the solutions available right now. In this talk, we'll explore the current challenges that most organizations are facing to reach the expert level in automation. In a second part, we'll explore a new approach to building an Automation Platform with at its core a new generation of Source of Truth that brings the best of Data Modelings and Version Control together.

1. Your logo
here
Network Source of Truth and
Infrastructure as Code
Revisited
Damien Garros, OpsMill
Denver November 2023

2. About me : Damien Garros
@dgarros
damiengarros @damgarros
2
Co-Founder and CEO of
Focused on Infrastructure as Code, Automation & Observability
for 10+ years
Previously leading Technical Architecture at Network to Code

3. Agenda
● State of the Industry
● Analysis
● What can we improve ?
● Our vision of the future
3

4. Current
State of the Industry

5. Popular architecture to automate networks
Source
of Truth
Observability
Telemetry
SLA
Compliance
Reporting
Service
Catalogue
User
Interface
Data
Governance
Deployment
Automation
Infrastructure
Management
Platform
Network
Source of Truth
Network
Source of Truth
DB First
Git First
Infrastructure as Code
5

6. Database First / DB First
● Data stored in
traditional DB
● Schema / API / Query
engine
● Templates, Playbooks
and some data in Git
● API Integration with
other tools
● Deployment with CI/CD
Pipeline (optional)
Network
Source of Truth
Deploy Render
Transform
Data
Observability
CMDB
CI/CD
Pull
Data Sync
6

7. Infrastructure as Code - Git First
● Everything in Git
(YAML)
● All changes requires a
Pull Request
● Deployment with CI/CD
Pipeline
CI/CD
Deploy
Render
Transform
Network
Source of
Truth
7

8. The Power of Automation
8

9. Hype Cycle
9

10. Limitations of DB First approach
● No Version
Control
● Complicated to
maintain
● Can be hard to
extend the schema
● Partial
Integration with
CI/CD Pipeline
Network
Source of Truth
Deploy Render
Transform
Data
Observability
CMDB
CI/CD
Pull
Data Sync
10

11. Limitations of Git First approach
● No API
● Hard to manage the
data (no schema,
duplicate)
● Pipeline Slow and
Error Prone
CI/CD
Deploy
Render
Transform
Network
Source of
Truth
11

12. Different Capabilities per Solution
User Experience
External Integration
Schema & Query Engine
Isolated Environment
Peer Review
Complex Changes
-
-
DB First Git First
-
12

13. Schema & Query
Engine
13

14. A bit of history
2021
GraphQL
Simplify how to
interact with the data
2016
Netbox brings IPAM,
DCIM & Circuit data
together
2020
Schema
Extensibility
Circuit Maintenance
Device Lifecycle
BGP
Security
14

15. Infrastructure Graph
Benefits of an
Infrastructure Graph
● Documentation
● Reduce inconsistency /
errors
● Simplify Config
management
● Enrich Observability
Data
● Simplify
Troubleshooting, root
cause analysis
DCIM
Circuit
IPAM
Security
Routing
Session
Maintenance
Service
Design
Business
Technical /
Implementation
Models
Design /
Service /
Business
Models
Status
15

16. Knowledge Graphs in other Industries
Knowledge Graph : Definition
A knowledge graph is a
structured representation of
interconnected data that
captures relationships and
semantic meaning, facilitating
efficient data retrieval,
analysis, and inference.
ChatGPT 2023
Retail Industry Pharma
Cyber Security 16

17. Analysis

18. 18

19. Running Infrastructure
Keep The
Lights On
Deploy new services
to support the
business
Standard
Changes
Non Standard
Changes
Troubleshooting
Main
Missions
Tasks
19

20. Differents type of changes
Add Vlan
Standardized Changes
Drain Device Hardware Refresh Design Change
Add Service
Non-Standardized Changes
User Form
-
External Integration
Isolated Environment
Peer Review
Continuous Integration
-
-
-
-
-
20

21. Use Cases & Solutions don’t match
-
DB First Git First
Standardized
Changes
Non-Standardized
Changes
Troubleshooting
21

22. Business Requirements
UI
API
Schema & Query Engine
Version Control
Unified Storage
User Experience, everyone can use it
Easy integration, programmability
Data Integrity, Context
Reduces Risk, Traceability
Atomic changes, Reduces risk
22

23. What can we improve ?

24. What can we improve
● Flexible and Extensible Schema
● Version Control
● Unified Storage
● CI/CD Pipeline
24

25. Flexible &
Extensible
Schema

26. Schema Extensibility
● Every organization
has slightly
different
requirements
● Schema
Extensibility is
paramount
● Git is also popular
because it imposes
no restriction for
the schema
Circuit
Routing
Session
Maintenance
Business
Technical /
Implementation
Models
Design /
Service /
Business
Models
Status
DCIM
IPAM
Security
Service
Design
LB
26

27. Key Characteristics for a Knowledge Graph
● Graph Structure
● Interconnectedness
● Schema Driven
● Extensibility
● Inference and Reasoning
● Semantics and Context
● Query Engine
Graph Databases
brings
Flexible & Extensible
Schema together with
a powerful query
engine
27

28. Relational vs Graph
28

29. Version Control

30. Version Control & Git
Git is Awesome
● Branching / Diffing &
Merging
● History Tracking
● Collaboration / Peer Review
● Prepare Multiple Changes
Mandatory for complex changes
BUT
No query engine
No understanding of what
it stores
Only work with text files
Git is not a database
30

31. Data Version Control
● Several solutions
available to have version
control for your data
● Create a Branch on your
database !!
● Query each branch
individually
31

32. Nautobot + Dolt
32

33. Unified Storage

34. Tight Coupling challenges with DB First
● Complicated to upgrade
the Data Model
● Rendering and the
Transformation are
tightly coupled with
the internal data
model / API
● Transformation &
Rendering have a lot
of business logic
Data
Deploy
Render Transform
Internal data model
CI / CD
Source of Truth
Device Specific language
Tight Coupling
34

35. Git First has a unified storage
Git First / Infrastructure as
Code makes it easy to manage
changes because :
● Data
● Transformation
● Rendering
can be managed together
CI/CD
Deploy
Render
Transform
Network
Source of
Truth
Unified Storage
One PR
One Pipeline
One Review process
Atomic Change 35

36. Redefine the limit of the Source of Truth
Manage Data, Transformation
and Rendering together
Transformation & Rendering
have a lot of business logic and
should be part of the Source of
Truth
Data
Deploy
Render
Transform
Device Specific language
CI
CD
Source of Truth
Copyright @2023 OpsMill
36

37. CI/CD Pipeline

38. CI/CD pipelines aren’t
designed for
Infrastructure Management
Git doesn’t understand what has
changed, it only sees files
38

39. CI/CD pipelines for infrastructure
management
Copyright @2023 OpsMill
Entity to manage
Data structure in Yaml
Frequence of Deployment
Application Infrastructure
1 Application 100s devices
No Yes
Seldom Frequent
39

40. Limitations of CI/CD pipelines for
infrastructure management
Copyright @2023 OpsMill
Validation Deployment
Usually we validate
everything for all changes
just to be sure.
Difficult to deploy
something quickly
No workflow possible
Partial deployment difficult
to manage
40

41. Our vision of the future

42. Infrahub
A new generation of Source of Truth
Copyright @2023 OpsMill
Unified Storage w/ Version Control
Infrahub is a new type
of Source of Truth,
built from the ground
up for infrastructure
management & automation
Data Files
Config Rendering
Artifact
Peer Review
CI Pipeline
User Defined Schema
UI, API, GraphQL
42

43. Infrahub Unified Storage
Copyright @2023 OpsMill
Unified Storage w/
Version Control
Data
Files
User Defined
Schema
Dynamic and
flexible Schema
Immutable and
Version Control
Graph
Branches
Diff
Merge
Built on
a Graph Database
and a temporal Graph
Standard Git
Integrated with the
Graph
Templates
Scripts
CI Pipelines
43

44. Questions?

45. Thank You