The document discusses the challenges faced by Indian OTT players regarding revenue per subscriber and diverse content consumption preferences, alongside a projected growth in the market. It highlights the importance of accurately measuring and controlling various experience KPIs for optimizing streaming performance. The analysis includes performance metrics for different OTT applications, indicating the need for improved optimization and monitoring to enhance user experience.

1. 1
Right measures and a quick benchmark of “Hyper
scale” Indian players
Nagu Gopalakrishnan
Director & Principal consultant, Automation & QoE
Solutions
nagu@tataelxsi.com

2. Award winning UX and design team
studios in London, Bangalore, Pune
DesignTechnology
6000+ technologists, engineers
Global R&D centers and labs

3. Content maybeking,Customer is the
kingmaker.
S C A LE & D I V ER S I T Y C H A LLEN G ES
Extremelydiversecontent consumption
preferences and varying last mileconnectivity
R EV EN U E P ER S U BS C I BER
DOMOREWITHLESS. Customer basemay
behigh,But revenueper subscriber is not on
par with western world. 5Bn+USDprojected
by2023. Not to mention churn.
Extremelydiversecontent consumption
preferences and varying last mileconnectivity
R EV EN U E P ER S U BS C I BER
DOMOREWITHLESS. Customer basemay
behigh,But revenueper subscriber is not on
par with western world. 5Bn+USDprojected
by2023. Not to mention churn.
D EFI N I N G EX P ER I EN C E K P I S
QoEshould beleading and lagging.
ExperienceKPIs areto controlled and not
just monitored.
Content maybeking,Customer is the
kingmaker.
S C A LE & D I V ER S I T Y C H A LLEN G ES
Extremelydiversecontent consumption
preferences and varying last mileconnectivity
R EV EN U E P ER S U BS C I BER
DOMOREWITHLESS. Customer basemay
behigh,But revenueper subscriber is not on
par with western world. 5Bn+USDprojected
by2023. Not to mention churn.
SomeExperienceKPIs measured and benchmarked
15 0 + M I LLI O N C U S T O M ER S
Content maybeking,Customer is the
kingmaker.
S C A LE & D I V ER S I T Y C H A LLEN G ES
Extremelydiversecontent consumption
preferences and varying last mileconnectivity
Quick Look at Indian Market

4. Performance monitoring of OTT Apps
OTT performance monitoring covers two aspects
Performance of video
streams
Performance of Applications
or application streaming

5. State and status of the art today
Streaming
performance
App performance
Challenges
• Mostly agent based and hence often
a lagging indicator.
• Often proactive control as a CI/CD
gate does not exist
• Issues often blamed on to last mile
connectivity
• Environment simulation is often not
scalable
Challenges
• Often differing dev, test and deployment
environment
• Heavy on browser dependent testing and
light on environment dependencies
• Big screen experiences are often
overlooked
• Automated performance measure in a CI
environment is often not used
Scale of testing is often limited to very few real
devices

6. What are the right measures and how do we
measure?

7. “Experience” Issues Trends across OTT
markets
40%
Never
triaged
Not reproducible
category
35%
Blame Game
Blamed on
Network/wifi/CDN
issues
25%
Triaged
and in backlog
Trend focuses on OTT bugs from the field across 30 different video players.
Monitoring is always a lagging indicator
Often consumed by Marketing, Operations and Strategy teams

8. Enabling experiences across the spectrum

9. Factors contributing Experience
Streaming conditions Network conditions End device conditions
Encoder related
Codec Related
Streamer Related
Packager Related
Others….
WAN Latency
Connectivity Medium
CDN Related
Others..
Device Type & Settings
Device Configuration
Player implementation
App implementation
Battery %
OS Video handling
Others…
What can we quantify, which can be
controlled?

10. Life cycle of a IP video stream
Criticality of highly accurate performance measures – Measured using
FalconEye HPAPIs

11. Benchmarking three OTT players
50Mil+ downloads 100Mil+ downloads 50Mil+ downloads
AVOD, SVOD AVOD, SVOD AVOD, SVOD

12. 0
2
4
6
8
10
12
400kbps 800kbps 3000kbps 4000kbps
TIME(SECONDS)
BANDWIDTH
Video Start up Time
HOTSTAR Zee5 SonyLiv
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
400kbps 800kbps 3000kbps 4000kbps
Blackframes
HOTSTAR Zee5 SonyLiv
0
0.005
0.01
0.015
0.02
0.025
400kbps 800kbps 3000kbps 4000kbps
Buffering Ratio
HOTSTAR Zee5 SonyLiv
Hotstar Zee5 SonyLiv
AVG BW=91827, RES=250x140 BW=200387, RES=180x144 AVG BW=206334, RES=256x144
AVG BW=144235, RES=320x180 BW=600822, RES=300x240 AVG BW=313261, RES=384x216
AVG BW=276158,RES=320x180, BW=799218, RES=450x360 AVG BW=463751, RES=512x288
AVG BW=370963,RES=416x234, BW=1197145, RES=600x480 AVG BW=651625, RES=640x360
AVG BW=551837, RES=640x360 BW=2485460, RES=900x720 AVG BW=967215, RES=960x540
AVG BW=777657
RES=720x404
AVG BW=1287316,
RES=1280x720
AVG BW=1358156,
RES=1280x720
AVG BW=2072762
RES=1280x720
AVG BW=2758412
RES=1920x1080,
ABR Functionality

13. So what does those graphs say?
More focused on lower
bandwidths
Poor player performance
Needs optimisation for
higher bandwidths
High spend on CDN and
storage costs
Higher the bandwidth better the
performance
Needs finetuning for
streaming performance
Focus on content quality
Inconsistent player
performance
Poor streaming
performance

14. 14
• App Launch Time
• Guide / Metadata
profiling
• Network Connect
profiling
• Navigation profiling
• Transitions
profiling
• Session Error Rates
• Minimum Time for
Stable Usage
Network QoE
Video QoE
Player QoE
App QoE
• Video Start Time
• Video Start Failure
Rate / Frequency
• Video Buffering Ratio
/ Frequency / Count /
Duration
• Video Playback
Failure Rate /
Frequency
• Channel Change
Time
• Video Restart Time
• Video Freeze
• Audio Freeze
• Macroblocking
• Pixelation
• Blurriness
• Video Blackouts
• Single Color
Freeze
• Video MOS
• Video Bitrate
• Audio Bitrate
• Manifest Request
• Download Rate
• Bandwidth Throttle
Performance
Measure
• Buffer Fill Measure
• Player ABR efficiency
KPIs for lifecycle of IP Video apps

15. Not just monitor, but control experiences

16. nagu@tataelxsi.com
Enabling OTT services to control content
consumption experiences