The document summarizes a presentation about using dedicated crowd sourcing to test mobile network quality of experience in real time during the 2012 London Olympics. It describes challenges in testing at the Olympics, how a dedicated crowd with test devices was deployed across the UK to continuously test networks, and how Epitiro's solution provided real-time data and reporting through a smartphone app and cloud backend to meet testing needs for the major event.

1. Good Evening
Welcome to Telecoms Evangelist evening
sponsored by Arrows Group
1Wednesday, 26 September 12

2. Quality of Experience
London 2012
We needed to automate our QoE in a live & complex
environment, challenges with access...
with the demand for realtime relevant feedback to aid
decision making...
•High Profile Global Event.
•One chance to get it right...
•Service Management demand for live
information
2Wednesday, 26 September 12

3. Video
3Wednesday, 26 September 12

4. I would like to play you a video,
reminding us all why our mobiles are so
important to our lives..
4Wednesday, 26 September 12

5. The Challenge...
• We need to provide Customer
Experience testing at key venues
across UK.
• The Solution had to provide a
comparison of all the operators,
near realtime
• Old methods would normally
mean up to a weeks delay from
the test taking place to getting the
report.
• Crowd sourcing is great, but is too
random, and relies on a large
number people testing randomly.
• What we needed was a new
approach...
5Wednesday, 26 September 12

6. The origin’s of an idea...
• The idea for realtime CET was
based on our internal crowd
sourced system,
• A system that we have
pioneered for a number years...
• But with Crowd sourcing... It is
difficult to achieve a consistent
relevant testing regime.
• What we needed was a
dedicated crowd...A crowd
which would be happy to work
24/7 and stand around all day.
• So that was it ‘Dedicated Crowd
Sourcing’ was the answer...
6Wednesday, 26 September 12

7. The basic requirements
• We needed a real product, so that we
could use to deliver an end2end
solution...
• Realtime App-based measurement
system...
• We needed an app that could provide
both voice & data testing in one unit...
• A flexible deployment strategy...
• Simple API driven reporting system
for a web-portal, Spreadsheets & an
App..
• See the results in realtime...
• An interface that would allow full
diagnostics when required...
JSON%api%Dashboard%
Coverage% layer% should% provide%
a% clear% view% of% all% signal%
strength% data% collected% over%
the%measurement%period.% %And%
provide% a% clear% uninterrupted%
a n d% a n i m a t e d% z o o m i n g%
experience%
Very%Simple%network%
dashboard%based%on%
the%displayed%areas…%%
Providing%a%simple%
view%of%Signal%level,%
Download%&%Upload%
Speed,%Ping%&%IP%
success%rate.%
The% legend,% clean% simple%
legend.% % This% should% a%
simple% and% show% poor% to%
good%coverage%in%a%simple%
linear%flow…%
Combined%Network%Coverage%
S i m p l e%
G o o g l e%
s t y l e%
navigaLon%
controls%%
CCSR%
98.2%%
97.5%%
95.3%%
97.6%%
97.6%%
97.6%%
The% Map% must% have% a% clear% and%
unambiguous%Ltle%
Logo% Title%area%
Operator%
logo%
Operator%ranking%table%
Including%Signal,%download%speed,%
upload%speed,%Ping,%webpage%download%
speed,%Voice%&%Data%CompleLon%Success%
rate%&%Sample%size%
7Wednesday, 26 September 12

8. A Cloud based solution...
Smartphone testing app
Backend server in Cloud
JSON api to enable reporting
on mobile devices
Root cause analysis tool
JSON api to enable
realtime excel reporting
8Wednesday, 26 September 12

9. Comparison Of Existing systems
Data Voice JSON Realtime Cost
Metrico
iTestNet
TEMS
Epitiro
Yes Yes Yes Yes High
Yes No Yes Yes Free
Yes Yes No No Mid
Yes Yes Yes Yes Low
The only solution to meet all our requirements was Epitiro
9Wednesday, 26 September 12

10. Dashboard & JSON api Specification provided
• We provided a detailed
specification for both the
API and the web Portal...
• Writing a test API interface
to ensure that the product
would interface with our
internal tools when it was
delivered...
• We wanted to support web-
portal, excel spreadsheet,
and apps which would all run
off the same JSON API...
JSON%api%Dashboard%
Coverage% layer% should% provide%
a% clear% view% of% all% signal%
strength% data% collected% over%
the%measurement%period.% %And%
provide% a% clear% uninterrupted%
a n d% a n i m a t e d% z o o m i n g%
experience%
Very%Simple%network%
dashboard%based%on%
the%displayed%areas…%%
Providing%a%simple%
view%of%Signal%level,%
Download%&%Upload%
Speed,%Ping%&%IP%
success%rate.%
The% legend,% clean% simple%
legend.% % This% should% a%
simple% and% show% poor% to%
good%coverage%in%a%simple%
linear%flow…%
Combined%Network%Coverage%
S i m p l e%
G o o g l e%
s t y l e%
navigaLon%
controls%%
CCSR%
98.2%%
97.5%%
95.3%%
97.6%%
97.6%%
97.6%%
The% Map% must% have% a% clear% and%
unambiguous%Ltle%
Logo% Title%area%
Operator%
logo%
Operator%ranking%table%
Including%Signal,%download%speed,%
upload%speed,%Ping,%webpage%download%
speed,%Voice%&%Data%CompleLon%Success%
rate%&%Sample%size%
10Wednesday, 26 September 12

11. Creating Reports
• JSON RESTful API provides single click
reporting...
• No training necessary...
• Delivery to multiple sources without
complicated database SOA infrastructure.
Back%Pack%2%–%Olympics%and%Paralympics%athletes'%parade%
11Wednesday, 26 September 12

12. Creating a Coverage Map
• 4 lines of HTML...
• 2 java script functions...
• Gets you a beautiful map
with coverage...
https://{..domain..}/tile.pl?tilex=511&tiley=340&zoom=10&operator=-1&bearer=ALL&?rand=2012-09-26:09
12Wednesday, 26 September 12

13. 72 Thousand
Miles driven during the last 16 weeks
13Wednesday, 26 September 12

14. 152 test units
Deployed across UK venue’s, cities and motorways
14Wednesday, 26 September 12

15. 16.3 Million
Network Tests made during in the last 16 weeksThat’s 107 Thousand test per handset
15Wednesday, 26 September 12

16. Demo
16Wednesday, 26 September 12

17. ats-group.co.uk
Tom Bryer
Managing Director
Est. 1992
17Wednesday, 26 September 12

18. London 2012 Games
Benchmarking the Mobile
Operators
Working With:
Est.
1992
18Wednesday, 26 September 12

19. During the Summer of 2012 – ATS&RO pioneered the CET bike!
Giving unprecedented access to London during busy times of the day..
19Wednesday, 26 September 12

20. From the initial concept… To Reality…
Production Engineering from ATS…
20Wednesday, 26 September 12

21. Creating the static units for the London 2012 Games…
21Wednesday, 26 September 12

22. Intelligent Communication Solutions
• Took delivery of 20 static benchmark units
on Friday morning
• Work very hard to get to the key locations
in the Olympic park
• Deadline for installation within the Olympic
park was Sunday evening
• All boxes were installed, activated &
commissioned on time
Rigging and Installation Services
Deploying the static units for the London 2012 Games…
22Wednesday, 26 September 12

23. 152 test units deployed across the UK…
23Wednesday, 26 September 12

24. Our
Benchmark
Test
Kit
Range…
Walk,
able
to
provide
access
to
key
areas
Sta'c,
24
by
7
at
key
locaDons
Car,
great
for
all
round
balance
of
TesDng
and
Mileage
Cycle,
provide
great
access
to
urban
areas
Bike,
great
for
central
London
or
other
urban
areas
24Wednesday, 26 September 12

25. Motorbike test unit…
25Wednesday, 26 September 12

26. Static Test unit…
26Wednesday, 26 September 12

27. Backpack test unit
27Wednesday, 26 September 12

28. Car Test Unit…
28Wednesday, 26 September 12

29. Currently Working With:
We
are
so
Impressed
with
realTime
CET…
We
have
set-­‐up
a
new
company
to
drive
this
forward:
29Wednesday, 26 September 12

30. On the way to completing a UK coverage map
Encouraging Mobile choice
30Wednesday, 26 September 12

31. Gavin Johns
Chief Executive Officer
Est. 2000
31Wednesday, 26 September 12

32. www.epitiro.com
Gavin Johns
CEO, Epitiro
iPQTM
Real-Time Insight
Epitiro Group Limited
10-11 Raleigh Walk, Waterfront 2000, Brigantine Place, Cardiff, CF10 4LN
Registered in England and Wales no: 3940403
info@epitiro.com | www.epitiro.com
32Wednesday, 26 September 12

33. www.epitiro.com
§ Epitiro provides leading Mobile Operators with network
monitoring systems to ensure they can measure and
manage Quality of Experience
§ Real-time insight of QoE helps address service-affecting
issues in their infancy
§ We help operators achieve;
§ Understand service
§ Reduced Support Calls
§ Higher Customer Satisfaction / Reduced Churn
§ Established 2000, Cardiff
‘Quality of Experience’
Monitoring Solutions for Mobile Operators
About Epitiro
ipQTM
Epitiro Group Limited
10-11 Raleigh Walk, Waterfront 2000, Brigantine Place, Cardiff, CF10 4LN
Registered in England and Wales no: 3940403
info@epitiro.com | www.epitiro.com
33Wednesday, 26 September 12

34. www.epitiro.com
Our Clients
34Wednesday, 26 September 12

35. www.epitiro.com
“Does it work when I want it?”
Challenges for Mobile Operators
Coverage
Performance
ReliabilityCapacity
Capability “Does it perform as I expect?”
• Voice, text and Data
“Is there service where I need it?”
Availability of Voice and Data
Services 24/7
Does the Network
support the latest
technology
How does the network cope at
times of peak demand
Coverage - Capacity – Capability - Performance - Reliability
The Five Pillars of Customer Experience…
“Can I make a call when I want to, for
as long as I want to?”
35Wednesday, 26 September 12

36. www.epitiro.com
36
Product Overview
ipQTM Block Diagram
36Wednesday, 26 September 12

37. www.epitiro.com
• Overview
– Targeted measurement of mobile network – based on
cell, bearer etc.
– Test Scheduling event based, time based, push-to-
test
– Active & Passive testing
• Device Support
– Android 2.1+ and iOS 4.0+
– Consolidated approach for future device support with
RIM and Windows
• Network and Bearer support
– 2G, 3G UMTS and CDMA, 4G WiMAX and LTE
– Wi-Fi
AT50M Software Probe Test App
AT50M
Software
Client
OTA Android Smartphones
37Wednesday, 26 September 12

38. www.epitiro.com
ipQ Analytics
Filter just the data you want to see
CeX Dashboard’s in Real Time
Drill-down to investigate
handset performance
See handset performance by location
38Wednesday, 26 September 12

39. www.epitiro.com
One Click shows top
KPIs
One-Click Dashboards
Click on any legend
to add/remove from
view
Click on any point
to see detailed
information
Line Chart View…
39Wednesday, 26 September 12

40. www.epitiro.com
Map Views
Click on a Pin to see the
test details of any
smartphone within the tile
See Any KPI by heat
map
Click on a tile to
zoom in and see Pin
locations of
smartphone tests
See Coverage and KPIs on a Map
40Wednesday, 26 September 12

41. www.epitiro.com
Test Management
Create unlimited
versions of KPI Tests
Extensive control over the
configuration of each Test
Easily make the Test
available for any
probe
A Full Configurable Client from the Server
41Wednesday, 26 September 12

42. www.epitiro.com
§ Programmable warning and critical thresholds
means is instantly alerted when baseline thresholds
are breached
§ Full access to testing allows to control the frequency
and the number of KPIs
§ Benchmarking competitor MNOs highlights
opportunities to improve service..
§ Instantly see the impact of any changes on the
network…
In Summary
Benefits of iPQ…
42Wednesday, 26 September 12

43. www.epitiro.com
§ Real-Time Reporting Interface Mobile Operators
§ Full ‘On-Demand’ Management of data collection
probe testing
§ Support Services to ensure installation, configuration,
maintenance and further data analysis
ipQTM Performance Monitoring Solution
ipQTM
Epitiro Group Limited
10-11 Raleigh Walk, Waterfront 2000, Brigantine Place, Cardiff, CF10 4LN
Registered in England and Wales no: 3940403
info@epitiro.com | www.epitiro.com
Product and Services
43Wednesday, 26 September 12