Twitch/MMSys'20 grand challenge on near-second latency bitrate adaptation algorithms

1. Online Learning for Low-latency Streaming
Grand Challenge on Adaptation Algorithms for Near-Second Latency
Theo Karagkioules, Rufael Mekuria, Dirk Griffioen, Arjen Wagenaar
Unified Streaming, Amsterdam, Netherlands

2. Low-latency adaptive streaming

3. Motivation
HTTP Adaptive Streaming (HAS) is now the dominant source of data traffic.
• It is projected to reach 82% of the global Internet traffic, by 20221
• OTT services (Netflix, YouTube etc.) account for > 50% of the global ⇓ traffic2
• Cord-cutters and live applications are expedited → rise of low-latency streaming
Some current challenges of bitrate adaptation:
• Channel instability (physical phenomena)
• QoE management
• Reliable low-latency video streaming (1-5 s) . Need for robust, lightweight and
universal solutions ← our contribution
1
Cisco Visual Networking Index, “Forecast and Trends, 2017-2022” - Cisco ’19
2
Global Internet Phenomena - Sandvine 19’
Low-latency adaptive streaming 1 / 11

4. Adaptive streaming media format
Low-latency bitrate adaptation is facilitated by:
• Multiple qualities={resolutions, encoding rates} served from a regular origin.
• By requiring commodity web infrastructure → operational costs ↓
• Fragments - smaller (independently decodable) files
• Independent fragment requests (multiple servers) maintain session state
• Data segmentation → finer granularity of bitrate selection → channel utilization ↑
• A client initially fetches an XML file, which contains content information
• Control logic, a.k.a a bitrate adaptation algorithm→ bitrate indicates quality
• Logic that selects the appropriate encoding bitrate ∀ fragments, given network
conditions
• Essentially matches the streaming rate to the channel rate, via means of adaptation
• CMAF in combination with the supported chunked transfer mechanism of HTTP 1.1
and beyond, have set the stage for low-latency streaming
Low-latency adaptive streaming 2 / 11

5. Bitrate adaptation via learning

6. Motivation
Insights from earlier research (MMSys ’17)3:
• Existing schemes → parameter tuning according to network or app. scenario
• Difficulty in generalizing well beyond a certain scope of usage
• Optimal adaptation over fluctuating channels remains a challenging operation
• Low-latency bitrate adaptation still relies on throughput estimation
To overcome this limitation → resort to learning or control theory
• Practical implementation may be hindered
• by energy-demanding architectures (Deep Learning)
• or by the complexity of exploring the complete optimization space (MDPs)
Proposal: Novel adaptation algorithm based on Online Convex Optimization (OCO)
• OCO emerged in 2003 (Zinkevich) as a very effective online learning framework
• Does not rely on estimations (throughput). Only on historical values.
• “Model-free”, as no assumption for the statistical model of the channel is required
• Independent of any parameter selection concerning the streaming environment
• It provides tractable feasibility and performance guarantees
3
T. Karagkioules et al. A Comparative Case Study of HTTP Adaptive Streaming Algorithms in Mobile Networks
Bitrate adaptation via learning 3 / 11

7. Problem definition
An adaptation algorithm is an optimization solution with the objective of maximizing
the video bitrate, while ensuring stable and continuous low-latency streaming.
However, the application of OCO in HAS is not a straightforward task
• OCO requires convex decision space and constraints
• HAS has a discrete decision space (set of qualities for every fragment)
• Instantaneous state-dependent constraints (finite-sized buffer)
Modelling assumptions:
• We model the adaptive streaming client by a learning agent
• Agent minimizes latency, s.t. scheduling constraints of the buffer
• We model the channel rate evolution by an adversary
• Adversary decides the cost of each decision only after it has been taken
• Adversarial setting is general enough to include any potential time-variant distribution
• Learning agent minimizes an adversarial loss function, unknown at decision time.
We fulfill the OCO requirement (convex decision set and constraints):
1. Agent decides video quality, according to a probability distribution for the
appropriate bitrate, given network and latency conditions
2. Relaxation to unbounded buffer → adheres to time-averaging constraints
Bitrate adaptation via learning 4 / 11

8. Relaxations
Relaxation 1: Convexification of decision set by randomization
∀ fragment t ∈ {1, . . . , T}, the client selects quality xt ∈ X = {1, . . . N}
• upon bitrate indication rxt ∈ {r1, . . . , rN} of the bitrate adaptation algorithm
Consider the probability simplex:
Ω = {ω ∈ RN
: ω ≥ 0 ∧ ∥ω∥1 = 1}
→ Instead of xt, client learns the optimal probability ωt = (ωt,n)n=1,...,N of picking xt ∈ X
Relaxation 2: Convexification of buffer constraints
Buffer evolution:
Bt+1 =










Bt − ρt
Playback rate
Seg. size≈V·rxt
St,xt
Ct
Throughput
⇓ time










+
+ V
Seg. duration
−
Inter-request delay
∆t
keeps Bt+1<Bmax
∈ [0, Bmax]
→ We treat the buffer as an infinite queue, i.e. Bt ∈ (−∞, ∞). ∆t is never imposed
• We allow instantaneous violation of the buffer budget
• But utilize a penalty to maintain 0 < Bt < Bmax on average
Bitrate adaptation via learning 5 / 11

9. Problem formulation
Consider the following functions =⇒ random processes:
˜ft(xt) ≜ V −
ρtSt,xt
Ct
Loss
=⇒ ft(ωt) ≜ E V −
ρtSt,xt
Ct
˜gt(xt)
Underflow
≜
ρtSt,xt
Ct
adversary
− V
Bt>0 on avg.
=⇒ gt(ωt) ≜ E
ρtSt,xt
Ct
− V
Given the loss function and constraints above, we formulate the constrained OCO:
min
ω∈Ω
T
t=1
ft(ω) s.t.
T
t=1
gt(ω) ≤ 0
At every decision epoch t the following events occur in succession:
(a) Agent computes ωt ∈ Ω → xt ∈ arg minx∈X |rx − N
n=1 ωt,nrn|
(b) Adversary decides jointly Ct, ρt, and ˜ft(xt). ˜gt(xt) → actual buffer displacement
(c) Feedback is provided to the agent
Bitrate adaptation via learning 6 / 11

10. OCO solution for adaptation (Learn2Adapt)
Challenge in adversarial problems: gt(ωt) are unknown at decision time
Solution: prediction around ωt−1 evaluated at ωt (Taylor)
ˆgt(ωt) ≜ gt−1(ωt−1) + ⟨∇gt−1(ωt−1), ωt − ωt−1⟩
We combine the objective and the constraint function in a regularized Lagrangian:
Lt(ω, Q(t)) = Q(t)ˆgt(ω)
Buffer deviation
+ VL
ˆft(ω)
Latency
+ α||ωt − ωt−1||2
Smoothness
• VL and α are convergence parameters
• ||ωt − ωt−1||2 is a regularization term that smooths the decisions
• Q(t) is a multiplier → accumulates the constraint deviations:
Q(t + 1) = [Q(t) + ˆgt(ωt)]+
,
→ We have accounted for latency, smoothness, and stalling, in terms of QoE provisions
Bitrate adaptation via learning 7 / 11

11. Learn2Adapt-LowLatency (L2A-LL)
→ Learn2Adapt-LL takes a step in the direction of the sub-gradient of the Lagrangian
Algorithm 1 Learn2Adapt-LowLatency (L2A-LL)
Initialize: Q(1) = 0, ω0 ∈ S
Parameters: cautiousness parameter VL, step size α
1: for all t ∈ {1, 2, . . . , T} do
2: ωt = projΩ ωt−1 −
VL∇ft−1(ωt−1)+Q(t)∇gt−1(ωt−1)
2α
3: Q(t + 1) = [Q(t) + ˆgt(ωt)]+
4: end for
Bitrate adaptation via learning 8 / 11

12. Implementation in dash.js
Open source:
Learn2Adapt-LowLatency (L2A-LL) has been implemented in ‘dash.js’
It is is publicly available for experimentation at Unified Streaming GitHub.
Experimental setup:
5 network profiles: Cascade, Intra-Cascade, Spike, Slow and Fast jitters.
Video sequence (encoded at R = 0.3, 0.6, 1.0 Mbps) organized in fragments
(V = 0.5s)
We allow a variable ‘catch-up’ playback rate pt but left its selection to dash.js.
Implementation location:
Modified dash.js instance now includes L2A-LL + all server and orchestration nodes.
The implemented instance of L2A-LL in dash.js exists in:
‘Learn2Adapt-LowLatency/dash.js/src/streaming/rules/abr/L2ARule.js’
All required player configuration can be found in:
‘Learn2Adapt-LowLatency/dash.js/samples/low-latency/index.html’.
Bitrate adaptation via learning 9 / 11

13. Experimental evaluation
Table 1: Preliminary experimental results for L2A-LL
Network profile Avg Bitrate (Mbps) Avg Buffer length (s) Latency (s) Stall duration (s) Num. Switches
Cascade 0.58 0.46 1.7 28 10
Intra-Cascade 0.35 0.4 2.5 38 7
Spike 0.59 0.4 1.5 6 3
Slow jitter 0.33 0.47 1.3 7 4
Fast jitter 0.33 0.57 1.2 0.7 2
General remarks:
• Our algorithmic approach allows to adjust QoE priorities
• Modular Lagrangian accounts for: latency, stalling and smoothness
• Of course is all a matter of trade-offs
• L2A-LL achieves relatively high avg. bitrate, and low latency (<2 s)
• L2A-LL adheres to theoretical performance guarantees (Theorem 4.14)
• Verified experimentally (also in the paper)
4
T. Karagkioules et al. Online learning for low-latency bitrate adaptation, MMSys ’20 - Challenges
Bitrate adaptation via learning 10 / 11

14. Summary
Learn2Adapt-LowLatency:
• is a novel bitrate adaptation algorithm, based on online learning
• does not require modifications according to application type
• requires no statistical assumptions for the channel
• performs well in a wide spectrum of possible scenarios
• robust, due to design principle; its ability to learn
• allows parameter selection according to QoE prioritization
• facilitates effective bitrate adaptation in low-latency mode when combined with
CMAF and chunked transfer
All these properties are significantly relevant to the field of modern HAS
• where OTT providers and broadcasters are continuously expanding their services
• to include more: diverse user classes, network scenarios and streaming applications
Bitrate adaptation via learning 11 / 11

15. For any questions, feel free to contact me at:
theo@unified-streaming.com
Thank you.
Bitrate adaptation via learning 11 / 11