Network performance optimisation using high-fidelity measures

Network
Performance Optimisation
How Communications Service Providers (CSPs)
can create new value
from quality attenuation analytics
© Predictable Network Solutions 2013

PREDICTABLE
NETWORK
SOLUTIONS

The only network performance science company in the world.
• New mathematical performance measurement and analysis techniques.
• Performance assessment methodology.
• World’s first network contention management solution.

Dr Neil Davies Co-founder and Chief Scientist
Ex: University of Bristol (23 years).
Former technical head of joint university/research institute (SRF/PACT).

Peter Thompson CTO
Ex: GoS Networks, U4EA, SGS-Thomson, INMOS & Universities of Bristol, Warwick and Cambridge.
Authority on technical and commercial issues of converged networking.

Martin Geddes Associate Director of Business Development
Ex: BT, Telco 2.0, Sprint, Oracle, Oxford University.
Thought leader on the future of the telecommunications industry.

Presentation Outline
• CSPs are seeking to increase their profitability and
return on assets.
• Predictable Network Solutions Ltd has the capability to
support optimisation beyond traditional approaches
to network data analytics.
– This capability is built around a robust scientific method.

• CSPs can benefit greatly from enhancing the fidelity of
their measurements of critical aspects of network
performance.
– Standard techniques fail to capture enough resolution.

• We have the missing leading-edge measurement
capabilities that all CSPs need.

3

The need to manage to the right metrics

CSPS’ QOE AND COST DILEMMA


4

What are the network optimisation
goals of every CSP?
Commercial

Technical

The CSP’s revenue is ultimately bounded
by the value perceived by the final end
user.
• User value is derived from
applications delivering fit-forpurpose outcomes (FFPOs).
• Users value consistency

CSPs need to make bad user experiences
sufficiently rare, at affordable cost.
• This creates a balancing act: running
the network too hot vs too cold.

– The absence of failures of service
– Bad experiences must be rare

•

Every CSP’s goal is to maximise the
value of FFPOs (i.e. QoE) at the
minimum input cost.

– For this they need to have good
proxies for QoE.

•

A good proxy is one that directly
relates to the delivered QoE…
– …that can also be measured,
managed and predicted…
– …and must also have low operational
cost to gather.


5

Network performance measures
Average
Single
Point

Offered Load
and Utilisation
(mean values only)

Today’s key CSP
QoE proxy.
Is it a good one?

Might there be some
important details about
traffic conditions that
are lost? (Yes!)

No! Reporting the
number of packets on a
1Gb/s Ethernet link
every five minutes is
like counting cars on a
six-lane highway for two
years!

6

Need distributions, not averages:
Same bandwidth, different QoE
Comparison between two LLU broadband providers to same location in the UK.

‘Bandwidth’ is an
average. It fails to
capture this
non-stationarity.

SAME ‘BANDWIDTH’

1/3 THE VALUE

The difference between these ISPs is the distribution of loss and delay.
The one on right has 1/3 the capability of the left for carrying POTS-quality VoIP.


7

Utilisation is a poor proxy for QoE

The data
CSPs use:
bandwidth

The data
CSPs need:
strong QoE
proxy


This is (the first
publishable) evidence
comparing utilisation
with a direct QoE
measurement.
This is a well-run and
well-managed
network. Our
engagements with
CSPs have shown this
to be a common
phenomenon.

8

High load,
but no QoE
breach

Overprovisioning
just wastes
money

Low load
(<0.01%),
but QoE
breach


Overprovisioning
doesn’t solve
your QoE
problem

9

The CSP QoE and cost problem
Commercial

Technical

The failure to appropriately measure
QoE means there are unmanaged
hazards in the current supply chains.
• These hazards can and do mature
into application and network
failures.
• FFPOs are dropping, and cost per
FFPO is rising.

In-life management costs increase due
to the inability to manage the QoE
hazards, which appear as ‘faults’. So:
• CSPs turn to arbitrary traffic
management to shed load which, in
turn, increases tension between
customers, legislators and CSPs;
• Or, CSPs regress to previous planning
and design ratios by capping access
speeds due to continuing failure;
• Or, stationarity continues to
decrease, reducing FFPOs and QoE,
which leads to less value-in-use and
tarnishes every CSP’s reputation.

– This leads to premature upgrades,
compared to the original capacity
plan.

•

Return on assets continues to drop…
– …so CSP share prices fall.


10

Service Quality

The CSP investment ‘cycle of doom’

QoE declines
faster than the
capacity plan
predicts
Time

Undepreciated Asset Value

Rising load makes
service quality fall,
forcing upgrades

Failure of technology to keep
up with ever rising demand
forces shorter upgrade cycles

Upgrade before
previous
investment
amortised


Death via
unserviceable
debt load 11

Time

All analytic approaches are limited by the fidelity of their inputs

HOW TO OBTAIN PERFORMANCE
DATA WITH REAL VALUE?

12

FFPOs require bounded
‘quality attenuation’ (∆Q)
Median time to complete HTTP transfer in seconds

Different QoE
implies
different
bounds on ∆Q

One-way loss rate (%)

Need to
manage
network to a
QoE goal

We care
about both
loss and
delay

One-way delay (ms)

ΔQ accumulates along a path
Example: 3G round-trip cross-sectional analysis

We want visibility
of how each
network element
contributes to ΔQ

(No service)


Average
Single
Point

Offered Load
and Utilisation
(mean values only)
PLUS

To get loss and delay
plus path decomposition
we need multi-point
measurements
(and not just multiple
single-point
measurements)

Multiple
Delay and Loss
Point (mean and variance)


15

There is no ‘quality’
in averaged measurements

CSPs need highfidelity data to
see fast-varying
QoE effects

AVERAGE DELAY

∆Q for 16kbit offered load at a busy international 3G location

16

FFPOs require strict bounds
on loss and delay
HTTP time to complete in seconds (95th percentile)

Just a few users
falling over the
‘cliff’ generates
churn, even if the
average user is OK

One-way delay (ms)

One-way loss rate (%)

CSPs need to
manage their
delivery to
avoid these QoE
‘cliffs’


Average
Single
Point

Offered Load
and Utilisation PLUS
(mean values only)

Multiple
Delay and Loss
Point (mean and variance)


Distribution

Arrival Patterns

Capturing the
‘outliers’ of QoE
means we need
the distribution of
packet arrival
patterns.
18

Average
Single
Point

Multiple
Point

Distribution

The data
CSPs use

When you capture
distributions via
multi-point
measurements you
get the strong QoE
proxy data you need.


The data
CSPs
need

19

How to measure the right things with a robust scientific method

EXPLOITING HIGH-FIDELITY
MEASUREMENTS

20

High-fidelity data capture
is the key enabler
Commercial

Technical

CSPs want to set a price floor for their
services, and differentiate via network
quality.
• This increases the focus on getting
the trade-off between cost and QoE
right.
• Current network management
approaches focus on making the
average experience better.

QoE depends on ∆Q…and nothing else.
• QoE certainly does not depend on
averages or peak bandwidths.

– The key is making bad experiences
rare.

Performance data needs to enable CSPs
to directly manage the cost/QoE tradeoff.

– Average or peak measures like
‘bandwidth’ at best allow CSPs to
manage cost vs performance.

•

The current capture processes lose
critical information that impacts QoE.
– CSPs don’t measure ∆Q directly.
– Current approaches try to
compensate by gathering more and
more data, the volume of which itself
degrades the network quality!


21

Average

Distribution

Single
Point

Limited predictive
power

Temporal
predictive power
(and localised assurance)

Multiple
Point

Spatial predictive
power

ΔQ
Temporal and spatial
predictive power


22

Average
Single
Point

Multiple
Point

Distribution

Limited predictive power
Temporal
predictive power

LOW FIDELITY
LOW VALUE

Spatial
predictive power

Represents all that can
be known about a system
(by observation)


HIGH FIDELITY
HIGH VALUE

23

NetHealthCheck™ Process
Our service
that embodies
these ideas

Inject low-rate
test streams
Measure test
streams at
multiple points

Analyse
measurements
to obtain
distributions
Understand
QoE/cost
tradeoff

24

Example client outcomes
1. Major UK mobile network operator
• Was in 2nd/3rd place in its market (depending on location) for HTTP
download key performance indicator (KPI).
• NetHealthCheck™ enabled a 100% improvement in this KPI without
any additional capital expenditure.
• Placed MNO as definitive 1st in the market.

2. BT Operate
• Applied to delivery of wholesale broadband services…
– …on a mature, highly-optimised, well-managed network.

• Revealed flexibility to optimise planning rules.
• Potential for 30% increase in utilisation of key resources.
• Estimated savings value of £2.3M.

25

NetHealthCheck™ Benefits
Structural
capacity
optimisation:
10% - 30%

+

Scheduling
optimisation:
25% - 75%

=

QoE
improvement
50% - 100%

These all
generate
‘slack’ to…

…sweat assets
to optimise CAPEX:
get ‘free’ growth.

…improve QoE at no cost:
for all customers, or specific groups.


26

For more information
Visit our website for detailed
case studies, presentations and white papers
www.pnsol.com
Contact us
info@pnsol.com


27

Network performance optimisation using high-fidelity measures

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Network performance optimisation using high-fidelity measures

Similar to Network performance optimisation using high-fidelity measures (20)

More from Martin Geddes

More from Martin Geddes (20)

Recently uploaded

Recently uploaded (20)

Network performance optimisation using high-fidelity measures