This presentation will discuss recent research on current service assurance systems and how they relate to the industry’s most widely discussed technology topics today – Big Data, Artificial Intelligence, Machine Learning, and Deep Learning. We also introduce the concept of Small Data – using active testing and monitoring to provide direct answers to the relevant service quality questions and as the prerequisite high quality data inputs for effective Big Data and Machine Learning processes. Presented at Network Virtualization Europe, Madrid, May 23 2018

1. Control the Hype
Stefan Vallin, PhD
Director, Product Strategy, Netrounds
Service Assurance as it relates to Big
Data, Artificial Intelligence, and
Machine Learning

2. 2 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Netrounds At a Glance
Leader in Automated Active Test and Monitoring
Netrounds actively verifies that provisioned
services work when delivered and continue
to work throughout their lifetimes
“My service must work when I get it”
“If I have a problem, find it and fix it – fast!”
1
2 Proactively monitor quality from end user point of view
Automatically test services at time of launch

3. 3 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Waves of Hype or the Boring Scientist – Which Side
are You On?

4. 4 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
The AI Mystery Box – What is it?
Generic AI
magic
Lots of low
level data
Any and all
answers
The Believer
Any
question
The Ironic
Device
telemetry
Customer
experience
Any OSS/BSS
question
AI
The Scientific
Relevant
high quality
data
Specific
answer
Specific
question
f(x)
Specific AI
Service
assurance

5. 5 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
AI, ML & DL are Well-defined Mathematical Methods
Acknowledgement:
BI Nordic
Machine Learning Algorithm Category Examples of Algorithms
Classification predicts classes
Example: Filtering out relevant alarms
Supervised learning algorithms:
§ Decision trees
§ Bayes
§ Support vector machine
§ Neural Networks
§ kNN
Regression tests predict a continuous
target variable
Example: Prediction of link utilization
Supervised learning algorithms:
§ Logistic regression
§ kNN
Clustering classifies inputs into
categories by analyzing similarities
among the inputs
Example: Alarm grouping/correlation
Unsupervised learning algorithms:
§ K-means
§ Gaussian
§ DBSCAN
f(x)
The
algorithms
are not new

6. 6 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Or…

7. 7 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Which Data Sources Can Answer Which Questions?
Which devices
are faulty?
Where is there over
utilization?
What is the status
of my
infrastructure?
Which
services may
be affected?
What is the quality of
video and voice services
at the edge?
What is the TCP
throughput?
I just configured a new
service – does it work?What is the
mix of
protocols &
services?
Performance
Management
(PM)
Service
Impact
(SI)
Fault
Management
(FM)
Passive
Probing
Active Testing
and Monitoring
How many
authentications
failed?
Management plane Data Plane

8. 8 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Do Not Try This at Home…
Which devices
are faulty?
Where is there over
utilization?
What is the status
of my
infrastructure?
Which
services may
be affected?
What is the
mix of
protocols &
services?
Performance
Management
(PM)
Service
Impact
(SI)
Fault
Management
(FM)
Passive
Probing
How many
authentications
failed?
Management plane Data Plane
f(x)
What is the quality of
video and voice services
at the edge?
What is the TCP
throughput?
I just configured a new
service – does it work?
QuestionInput
?

9. 9 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
The Need for Active Service Monitoring
Descriptive Analytics –
What happened?
Diagnostic Analytics –
Why did it happen?
Predictive Analytics –
What will happen next?
Prescriptive Analytics –
What should be done to
prevent?Services
Devices
VNFs
Passive device telemetry – “big data”
Active service telemetry – “small data”

10. 10 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Example: Predictive Analytics Requires Relevant
KPIs as Input
L2-L7 active
service monitoring
data
Service Provider
IP Core
Predict network
performance problems
(RTP-jitter)
Specific
Answer
Predicted RTP jitter 1 hour ahead of time
Real RTP jitter
f(x)
Netrounds and Elastisys

11. 11 © 2018 Netrounds and/or its affiliates. All right reserved. Netrounds Confidential.
Active Monitoring is
needed here
For some questions,
Active Monitoring alone
provides the answer
Summary
The Scientific Version
Relevant
high quality
data
Specific
answer
Specific
question
f(x)
Specific AI
Can also
be small
data