This document discusses the evolution of Swisscom's network analytics capabilities from 2015 to the present. It begins with nationwide outages at Swisscom that revealed deficiencies in network visibility. It then summarizes the growth of Swisscom's data collection and analytics platforms over time, starting from early adopters in 2015-2016 to becoming mainstream by 2020-2022. It highlights key milestones like onboarding new platforms, scaling data ingestion rates, and expanding use cases from troubleshooting to anomaly detection and closed-loop network operations.

1. SwisscomNetworkAnalytics
DataMeshArchitecture
15.10.2022,ThomasGraf& MarcoTollini
Picture:Apollo8, December24th1968

2. 2
NationwideNetworkOutageseverywhere
Increasingin impact andduration- hintingNetworkVisibilitydeficiencies

3. 3
The customerknowsbeforeSwisscomthat
there is serviceinterruption.
Unableto recognizeimpactand rootcause
when configurationalor operational
networkchangesoccur.
Swisscomsuffersreputationdamage.
We need to worktogetherto mediate.
«
«
Markus Reber
Head of Networks at Swisscom

4. 4
At IETF only9.85% of the activitiesare
relatedto networkautomationand
monitoring.
We are still usingprotocolsdesigned40
yearsago to managenetworks.
IP networkprotocolsare not made to
exposemetricsfor analytics. IPFIXand BGP
monitoringprotocolare the rareexception.
«
«
Thomas Graf
Distinguished Network Engineer
and Network Analytics Architect at Swisscom

5. “ It is our duty to recognize service interruption
before our customer does.
Why do we still often fail to be first ? “
5

6. 6
Swisscom Big Data onboarded,
Meerkat Anomaly Detection Feasibility
10 active users. 9 platforms. 87 nodes. 250'000
metrics per seconds.
2017-2018
2019
2020
BGP Monitoring Protocol and YANG Push
IETF Engagement started
40 active users. 17 platforms. 233 nodes.
1'200'000 metrics per second.
Pivot Migration, Druid Scale Out,
Unyte IETF colaboration established
160 active users. 34 platforms. 2500 nodes.
3'000'000 metrics per second. Active probing with
1'500'000 broadband subscribers.
Flow Aggregation Proof of Concept
Internet Distribution Core and TV 2.0
2015-2016
Early
adopters
Early
majority
Late
majority Laggards
Platform onboarding
Change verification and troubleshooting
Capacity management
and trend detection
Anomaly detection
IETF vendor, operator and
university colaboration
Network visualization
DaisyNetworkAnalyticsTransformsSwisscomDevOpsMindset
Fromdevicemonitoringto networkanalyticswith closedloop operation
2021 Taking over end to end Daisy Chain Responsibility
215 active users. 40 platforms. 2700 nodes.
20'000'000 metrics per second. Active probing
with >1'500'000 broadband subscribers.
Key Points
> From bottom up to mainstream. From IETF to Swisscom DevOps teams.
> From network verification and troubleshooting to visualization
with anomaly detection and SLO reporting
> From capacity management to trend detection
> From network automation to closed loop operation
SLO Reporting
2022 L3 VPN Anomaly Detection and
Network Visualization Proof of Concept
400 active users. 47 platforms. 7000 nodes.
25'000'000 metrics per second.

7. 7
2ndGeneration
3rdGeneration
current
Data lake
Big data ecosystem
Kappa
Adds streaming for
real-time data
Proprietary
Enterprise Data Warehouse
1stGeneration
EvolvingBig Dataarchitecture
Domainoriented,like networks
4thGeneration
next-step
Data Mesh
Distributed and organized
in domains.
Data Infra as a Platform
Operational
Delivery Platform
Analytical
Data Platform
Analytical
Data Plane
Operational
Data Plane
Domain A Domain B Domain C
Federated Computentional
Governance for global interoparabiity
Data Product as a Architectual Quantum
Serve
Collect
Publish
Serve
Collect
Publish
Serve
Collect
Publish
From Principles to Logical Architecture

8. 8
Products
• Verification and Troubleshooting enables change and
incident management.
• Visualization makes routing and peering topologies
accessible to humans.
• Capacity Management enables proactivity for key
performance metrics..
• Anomaly Detection automates incident management.
Alerts users to important events with contexts.
• Service Level Objective reports delay and loss for a
time period.
• Trend Detection automates capacity management.
Alerts users early before running out of capacity.
• Closed Loop Operation validates network
orchestration. Controlled configuration deployments.
DomainOwnership
NetworkAnalyticsas a product
Forwarding
Plane
Control
Plane
Device
Topology
Collect
Transform and
Aggregates
Analytical
Data Plane
Operational
Data Plane
Publish
Alerts and
Reports
Serve
Normalize and
Correlates

9. 9
Data Collectionwith NetworkTelemetry
Structuredmetricsenableinformeddecision-making
Network Telemetry:
> A data collection framework
where the network device
pushes its metrics to Big
Data. Defined in RFC 9232.
Data Modelling:
> Key for Big Data correlation
to understand and react in
the right context
> Are interface drops bad?
> How should we react?
Forwarding Plane
Data Models
How customers are
using our network
and services. Active
and passive delay
measurement
Control Plane
Data Models
How networks are
provisioned and
redundancy adjusts to
topology
Topology
Data Models
How logical and
physical network
devices are connected
with each other and
carry load
Swisscom Service
Service Models
Translates between what customers wishes and intend which should be fulfilled
Realitity
vs.
Intent
Thor LC ID
54654
BGP
Community
64497:12220
VRF, Interface
Config

10. Kappareal-timestreaming
How do you eat an elephant?Pieceby piece
1. Pushing metrics to collectors.
2. Aggregate or directly ingest into topics.
3. Buffers, consolidate and forward.
4. Process and re-ingest.
5. Import for midterm storage.
6. Access and observe metrics.
7. Are informed about events and possible
service impact.
Network Device Human/Machine
Data Collection Analytics
Message Broker Data Storage
Data Processing Message Broker
1
2
3
4
7
6
5
3

11. From Networkto Alert Event
Observemultipleperspectivesat different times
1. A single link down result in multiple device topology,
control-plane and forwarding-plane events being exposed
at different times.
2. Determine which interfaces and BGP peerings are being
used first and then observe state. Observe BGP
withdrawals and updates, traffic drop spikes and missing
traffic. Generate multiple concerns.
3. Calculate for each observation a concern score between 0
and 1. The higher, the more probable the changes
impacted forwarding.
4. Unify several concerns for one VPN connectivity service to
one alert identifier.
Network Event
Observation Strategy
Concern Scoring
Alert Unification
2
3
4
T1
T2 T3
1

12. 12
L3 VPN NetworkAnomalyDetection
Networksare deterministic– customerspartially
Analytical Perspectives
Monitors the network service and
wherever it is congested or not.
> BGP updates and withdrawals.
> UDP vs. TCP missing traffic.
> Interface state changes.
Network Events
1. VPN orange lost connectivity.
VPN blue lost redundancy.
2. VPN blue lost connectivity.
Key Point
> AI/ML requires network intent and
network modelled data to deliver
dependable results.

13. “ Without network visibility,
no informed decisions can be made. “
13

14. NetworkAnalyticsTransformedSwisscomMediaReporting
Whynetworksand data mesh needto become one

15. Transitionto SegmentRouting
From MPLS over MPLS-SRto SRv6
Segment Routing reduces the amount of routing protocols, simplifies forwarding-plane
monitoring while enabling traffic engineering with closed loop and increase scale.
Inter-AS Core
HCC
HCC Spine
MPLS P
HCC Leaf
Inter-AS ASBR
Inter-AS ASBR
MPLS P
Inter-AS
MPLS P
HCC Leaf
Inter-AS ASBR
Cloud Inter-AS
MPLS PE
IS-IS SR
BGP IPv4 Labeled Unicast
HCC RR
Endpoint NH-Self NH-Unchanged NH-Self NH-Self Endpoint
Inter-AS PE
BGP IPv6 Unicast (Phase 3)
MPLS SR Domain
Phase 1 Q4 2020
MPLS SR Domain
Phase 2 Q2-4 2021
IS-IS LDP

16. 16
337'920PacketsDropped
Successfullymigratedto a 3 labelstack

17. 17
At 17:39 prefixes from
Facebook BGP ASN 32934
where withdrawn. Outbound
traffic steadily increased
twofold until 20:20. Inbound
traffic decreased by 85%.
Between 19:25 and 00:51, BGP
updates and withdrawals
where received.
At 00:41 traffic rate restored
to normal.
FacebookIncident October4/5th
The Swisscomperspective

18. “ The solution comes with innovators.
That's why Swisscom cooperates at IETF with
network operators, vendors and universities. “
18

19. Collaborationfor tomorrowsNetworkAnalytics
Text
Text
Text
Text
Text
Text
Imply
Imply Druid
Swisscom
Network Operator
Huawei
Network Vendor
NTT
Network Operator
INSA Lyon
University
Cisco
Network Vendor
ETH Zürich
University Text
Confluent
ApacheKafka

20. • Support for Local RIB in BGP Monitoring Protocol
https://datatracker.ietf.org/doc/draft-ietf-grow-bmp-local-rib
YANGDatastoresenablesClosedLoop Operation
Automateddata correlation– what else?
Automated networks can only run with a common data model. A digital twin YANG data store enables a
comparison between intend and reality. Schema preservation enables closed loop operation. Closed Loop is
like an autopilot on an airplane. We need to understand what the flight envelope is to keep the airplane
within. Without, we crash.
YANG is a data modelling language which will
not only transform how we managed our
networks; it will transform also how we
manage our services.
News: 17 industry leading colleagues from 4
network operators, 2 network and 3 analytics
providers, and 3 universities commit on a
project to integrate YANG and CBOR into
data mesh. Starts November 2022.
Conceptual Tree - Network Configuration
Conceptual Tree - Network State
Conceptual Tree - Network Configuration
Conceptual Tree - Network State
Network Configuration
Netconf <edit-config>
Network State
YANG Push
YANG Data Store
on Big Data Lake
YANG Data Store
on Network Device
Digital Twin

21. When Data Meshand Networkbecomeone
A simple, scalableapproach toYANG push
Simplify YANG push network data
collection with high scale and low
impact. Suited for nowadays distributed
forwarding systems.
Preserve YANG data model schema
definition throughout the data
processing chain.
Enable automated data correlation
among device, forwarding-plane and
control-plane.
An HTTPS-based Transport for YANG
Notifications
https://datatracker.ietf.org/doc/html/draf
t-ietf-netconf-https-notif
UDP-based Transport for Configured
Subscriptions
https://datatracker.ietf.org/doc/draft-
unyte-netconf-udp-notif
Subscription to Distributed Notifications
https://datatracker.ietf.org/doc/draft-
unyte-netconf-distributed-notif
Conceptual Tree - Network Configuration
Conceptual Tree - Network State
YANG Model
YANG Model
YANG Model
JSON/CBOR
Schema
ID
REST API
Get Schema
Message broker
YANG Schema Registry
On Big Data lake
YANG Data Store
On Big Data Lake
JSON/CBOR
Schema ID
YANG push
notification message
YANG Push
Data Collection
Netconf
<get-schema>
Parse YANG notification
message header and
maintain schema id to YANG
model and version mapping.

22. • Support for Adj-RIB-Out in BGP Monitoring Protocol
https://tools.ietf.org/html/rfc8671
• Support for Local RIB in BGP Monitoring Protocol
https://datatracker.ietf.org/doc/html/rfc9069
BMP Coveringall RIB's
Extendsmuch neededRIB coverage
BGP route exposure without BMP is a challenge of
the first order:
> Only best path is exposed (missing best-external and ECMP
routes)
> Next-hop attribute not preserved all the time
> Filtering between RIB's not visible
Adj-RIB-Outan RFC since November 2019. Local RIB since
February 2022. Juniper, Huawei and Nokia have public
releases available supporting both. Cisco has test code
available but haven't released yet.
BGP Peer-A
Adj-Rib-In Pre Policy
BGP Peer-A
Adj-Rib-In Post Policy
Static, Connected,
IGP Redistribution
Post Policy
Peer-A In Policy
BGP Peer-B
Adj-Rib-In Pre Policy
BGP Peer-B
Adj-Rib-In Post Policy
Peer-B In Policy
Local-Rib Pre Policy
BGP Peer-C
Adj-Rib-Out Pre Policy
BGP Peer-C
Adj-Rib-Out Post Policy
Peer-A Out Policy
BGP Peer-D
Adj-Rib-Out Pre Policy
BGP Peer-D
Adj-Rib-Out Post Policy
Peer-B Out Policy
Fib
Table Policy

23. • Support for Enterprise-specific TLVs in the BGP Monitoring Protocol
https://tools.ietf.org/html/draft-lucente-grow-bmp-tlv-ebit
• BMP Extension for Path Marking TLV
https://tools.ietf.org/html/draft-cppy-grow-bmp-path-marking-tlv
BMP with extendedTLV support
BringsvisibilityintoFIB'sandroute-policies
Knowing all the routes in all the RIB's brings the new
challenge
> That we don't know how they are being used in the FIB/RIB
(which one is best, best-external, ECMP, backup)
> That we don't know which route-policy
permitted/denied/changedwhich prefix/attribute
For IETF 110 Hackathon, IETF lab network with Big Data
integration has been further extendedto collaborate
developmentresearch with ETHZ, INSA, Cisco, Huawei and
pmacct (open source data-collection by Paolo Lucente).
BGP Peer-A
Adj-Rib-In Pre Policy
BGP Peer-A
Adj-Rib-In Post Policy
Static, Connected,
IGP Redistribution
Post Policy
Peer-A In Policy
BGP Peer-B
Adj-Rib-In Pre Policy
BGP Peer-B
Adj-Rib-In Post Policy
Peer-B In Policy
Local-Rib Pre Policy
BGP Peer-C
Adj-Rib-Out Pre Policy
BGP Peer-C
Adj-Rib-Out Post Policy
Peer-A Out Policy
BGP Peer-D
Adj-Rib-Out Pre Policy
BGP Peer-D
Adj-Rib-Out Post Policy
Peer-B Out Policy
Fib
Table Policy
• BGP Route Policy and Attribute Trace Using BMP
https://tools.ietf.org/html/draft-xu-grow-bmp-route-policy-attr-trace
• TLV support for BMP Route Monitoring and Peer Down Messages
https://tools.ietf.org/html/draft-ietf-grow-bmp-tlv

24. Export of MPLS Segment Routing Label Type Information in IPFIX
https://datatracker.ietf.org/doc/html/rfc9160
Export of Segment Routing IPv6 Information in IPFIX
https://datatracker.ietf.org/doc/html/draft-tgraf-opsawg-ipfix-srv6-srh
Export of Forwarding Path Delay in IPFIX
https://datatracker.ietf.org/doc/html/draft-tgraf-opsawg-ipfix-inband-telemetry
IPFIX CoveringSegmentRouting
For MPLS-SR, SRv6 and On-path Delay
SRv6 is commonly standardized, network vendors implementations are available and
network operators are at various stages in their deployments, missing data-plane visibility
though.
Segment Routing coverage in IPFIX brings visibility for:
> Which routing protocol provided the label or IPv6 Segment in the SR domain.
> The active Segmentwhere the packet is forwarded to in the SRv6 Domain.
> The SegmentList where the packet is going to be forwarded throughout the SRv6 Domain.
> The Endpoint Behavior describing how the packet is being forwarded in the SRv6 Domain.
> The Min, Max and Average On-path delay at each hop in the SR domain.
Node based
Flow Aggregation
Apache Kafka
Message Broker
Timeseries DB
Pmacct
Data Collection
IOAM
nodes
Data-collection based
Flow Aggregation
Message Broker based
Consolidation
Data Base
Join

25. 25
ETH Zürich,EduardBachmakov– Master Thesis
OperatingSystem-LevelLoad Distributionfor NetworkTelemetryData Collection
From network data collection
load distribution with
Anycast and ECMP on the
network to SO_REUSEPORT
with in the Linux network
kernel.
Describes current load
distribution challenges and
extends SO_REUSEPORT with
cutome eBPF code.
Running code on github at
https://github.com/insa-
unyte/ebpf-loadbalancer
https://www.research-collection.ethz.ch/handle/20.500.11850/507440

26. 26
ETH Zürich,LivioSgier– MasterThesis
VisualizingBGP RIB Changeswith BMPand IPFIX
From motivation over
network topology, to data
collection, data processing,
data storage to visualization.
Describes current state with
challenges and future
outlook..
Running code on github at
https://github.com/lsgier/BM
P-Visualization
https://www.research-collection.ethz.ch/handle/20.500.11850/451910

27. 27
IETF 114/MWC2022 – NetworkAnalyticsDevelopment
IPv6 Forum,SRv6 Data PlaneVisibility
5x BMP drafts and 1 RFC at
GROW working group.
Bringing RIB and route-policy
dimensions into BMP and
increase scale.
2x YANG push drafts at
NETCONF working group.
2x IPFIX Segment Routing
On-path delay draft and 1
RFC at OPSAWG working
group.
Network Anomaly Detection
code development.
YANG push udp-notif open-
source running code.
https://www.linkedin.com/pulse/network-analytics-
ietf-development-mwc-2022-thomas-graf/
https://www.linkedin.com/pulse/ietf-114-network-
analytics-bmp-ipfix-yang-push-thomas-graf/

28. Internship (Step In)
Requirements
✓ During or after university
education with a fascination for
the ICT world
What awaits you
✓ Indeendent work in an exciting project
area
✓ Contribute & implement your own ideas
✓ Personal & professional development
Employment information
✓ 40 - 100%
✓ 3 - 12 months
✓ Start: anytime

29. Trainee
Requirements
✓ Master's degree that was completed no longer than 12
months ago
✓ Fascination & affinity for the ICT world
✓ Very communicative, able to learn & willing to perform
What awaits you
✓ 2-3 project assignments in different areas of the company
✓ Participation in workshops & building of a large network
✓ Professional mentoring & support by our ambassadors
Employment Information
✓ 100%
✓ 12 months
✓ Possibility of permanant employment
✓ Start: every October

30. IT Trainee
Requirements
✓ Bachelor's or Master's degree in computer science or
technology, which was completed no longer than 12
months ago
✓ Very communicative, able to learn & willing to perform
What awaits you
✓ 2-3 project assignments in different areas of the company
✓ Participation in workshops & building of a large network
✓ Professional mentoring & support by our ambassadors
Employment information
✓ 60-100%
✓ 12-18 months
✓ Possibility of permanant employment
✓ Start: anytime

31. Junior Position
Requirements
✓ University degree with a fascination for the ICT world
and/or
✓ Initial experience in the professionals world and/or
✓ Career changers
What awaits you
✓ Broad responsibility in a junior position
✓ Contribute your own experience & skills
✓ Personal & professional development
Anstellungsinformationen
✓ 50-100%
✓ Permanent contract
✓ Start: anytime

32. 32
Contact information
Swisscom
Daisy Network Analytics
Thomas Graf
Binzring 17
8045 Zürich
Email thomas.graf@swisscom.com
Marco Tollini
Binzring 17
8045 Zürich
Email marco.tollini1@swisscom.com