PNSol's CTO gives a presentation about the Overture Testbed, a facility to evaluate application sensitivity to network quality attenuation.

1. © Predictable Network Solutions Ltd 2017 www.pnsol.com
The Overture Testbed for
Design and Deployment Planning
Presentation at Scientific Network Management for Cloud Computing

2. © Predictable Network Solutions Ltd 2017
2
www.pnsol.com
Applications are distributed computations
• Computational components must
exchange information via the network
• All information is delayed by the
network; some is lost
• The characteristics of that delay and
loss affect the application performance
• Network capacity is no longer the
limiting factor for many applications
‘Capacity’ is the load at which delays
and losses become very large
Delay is the price we pay for being
distributed
Loss is the price we pay for using
statistically shared infrastructure
User experience is becoming more
and more dependent on network
quality characteristics

3. © Predictable Network Solutions Ltd 2017
3
www.pnsol.com
Overture ∆Q testbed
Reproduce full distribution
of delay and loss
Control upstream and
downstream separately
Measure full distribution of
application load

4. © Predictable Network Solutions Ltd 2017
4
www.pnsol.com
Reproducing network behaviour (gross)
VDSL speed via WiFi
DSL speed
DSL emulated over VDSL speed

5. © Predictable Network Solutions Ltd 2017
5
www.pnsol.com
Reproducing network
behaviour (fine)
Emulated network
Observed real-world

6. © Predictable Network Solutions Ltd 2017
6
www.pnsol.com
Contemporaneous A/B testing
Test against the same
server at the same time
Connect two identical
clients via different
network conditions
Connect two identical
clients via different
network conditions
Via measurable
parameter (e.g. PESQ)
or subjectively (e.g.
MOS)
If server is the issue it will
show up in both clients

7. © Predictable Network Solutions Ltd 2017
7
www.pnsol.com
Remote A/B testing
Overture
testbed
Application
server
Application
client A
Upstream network emulation
Downstream network emulation
∆Q_up_A
∆Q_down_ACompare
application
outcomes
Application
client B
Upstream network emulation
Downstream network emulation
∆Q_up_B
∆Q_down_B
Internet
Added ∆Q (A/B) Measured ∆Q
End-to-end ∆Q (A/B)

8. © Predictable Network Solutions Ltd 2017
8
www.pnsol.com
Cloud A/B testing
Overture
testbed
Application
server
Application
client A
Upstream network emulation
Downstream network emulation
∆Q_up_A
∆Q_down_ACompare
application
outcomes
Application
client B
Upstream network emulation
Downstream network emulation
∆Q_up_B
∆Q_down_B
Added ∆Q (A/B)Measured ∆Q
End-to-end ∆Q (A/B)
Internet
Cloud
platform

9. © Predictable Network Solutions Ltd 2017
9
www.pnsol.com
Uses – application developer
• Real-world performance awareness; taking the product “out-of-the-
lab” without leaving the lab
• App optimisation for different conditions
• Test apps to destruction
• Security threat analysis
• Performance regression testing

10. © Predictable Network Solutions Ltd 2017 www.pnsol.com
10
Service provider
• Trial effect of different network
configuration options
• In the lab
• In-line for groups of test customers
• Emulate benefit of ‘edge
computing’
• Different latency for ‘cloud’ and
‘fog’ locations
Regulator
• A/B-testing for regulatory use and
net neutrality understanding:
1. A is the line without sharing with a
heavy user, B is one with such
sharing;
2. A a heavy user in ‘neutral’
situation, B is heavy user with XYZ
measures to reduce impact on
network performance of others.
Uses – network service provider/regulator

11. © Predictable Network Solutions Ltd 2017 www.pnsol.com
www.pnsol.com
info@pnsol.com
For further information: