Abstract: Video streaming today accounts for more than 80% of global internet traffic, with demands projected to rise further due to UHD, 8K, VR, and immersive media applications. While newer codecs like AV1 and VVC deliver better compression efficiency, their increased computational complexity significantly raises energy consumption in encoding, storage, transmission, and decoding. This creates a fundamental tension between maintaining high Quality of Experience (QoE) for users and ensuring sustainable, energy-efficient streaming. This talk presents a holistic framework for energy-aware adaptive streaming that addresses inefficiencies across the entire pipeline. First, a Video Complexity Analyzer (VCA) provides lightweight content features to guide efficient encoding. Second, convex-hull–based optimization enables Pareto-efficient bitrate ladder construction. Third, multi-codec bitrate ladder pruning (MCBE) reduces redundancy across AVC, HEVC, AV1, and VVC, achieving major savings in storage and transmission. Fourth, ML-driven encoding parameter selection balances quality and energy by predicting VMAF and power usage. Finally, a Relative Decoding Energy Index (RDEI) provides device-independent modeling of decoding costs, enabling client-side adaptation. Together, these methods reduce encoding energy by 46%, storage by 95%, transmission by 78%, and decoding by 15%, all while maintaining QoE within just-noticeable-difference thresholds. The framework paves the way toward scalable, sustainable, and net-zero streaming ecosystems.

1. Energy-Efficient Video Coding for HTTP Adaptive
Streaming
—
Dr. Vignesh V Menon
Postdoctoral Researcher
Video Communication and Applications Dept., Fraunhofer HHI, Germany

2. MHV’24
Agenda
29/09/2025
Slide 2
● Motivation & Context
● Background & Challenges
● Contributions
● Unified Framework
● Discussion & Outlook
● Conclusion

3. Motivation & Context

4. MHV’24
Why Energy-Efficient Video Streaming?
Slide 4
Video dominates internet usage
● Over 80% of global internet traffic is video (Cisco, Sandvine)
App category traffic share (2022)
● Video alone accounts for nearly 66% of total data volume
Bandwidth demands are growing
● Emerging applications (UHD, 8K, VR) need 100–500 Mbps
29/09/2025
[1] Cisco, “Cisco visual networking index: Forecast and methodology, 2017–2022 (White Paper),” 2019.
[2] Ericsson, Ericsson Mobility Report: November 2020, Nov. 2020. [Online]. Available:
https://www.ericsson.com/assets/local/reports-papers/mobility-report/documents/2020/november-2020-ericsson-mobility-report.pdf

5. Background & Challenges

6. MHV’24
Adaptive Video Streaming Fundamentals
29/09/2025
Slide 6
● Video is split into short segments (e.g., 4s)
● Each segment is encoded at multiple qualities (resolution,
bitrate)
● Client selects the most suitable version at runtime
● Selection is driven by real-time feedback (bandwidth,
buffer)
● Protocols: MPEG-DASH, HLS, CMAF
Client
Encoding
Server
Limitations of conventional ladders
● Often fixed or heuristic-based
● Do not consider energy consumption (encoding, storage,
decoding)
● May deliver redundant or inefficient representations
Thomas Stockhammer. 2011. Dynamic adaptive streaming over HTTP --: standards and design principles. In Proceedings of the second annual ACM conference on Multimedia systems (MMSys '11).

7. MHV’24
Challenges and Trade-offs
29/09/2025
Slide 7
User Expectations
● High quality video
○ 1080p/4K HDR at 60fps is becoming the default.
● Low latency & smooth playback
○ No stalls, fast startup, seamless switching.
● Device portability
○ Streaming on-the-go (mobile, tablets, VR).
● Long battery life
○ Users expect hours of streaming without charging.
T. Zinner, O. Abboud, O. Hohlfeld, T. Hossfeld, P. Tran-Gia (2010). Towards QoE Management for Scalable Video Streaming.
S. Lederer, C. Müller, and C. Timmerer. 2012. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd Multimedia Systems Conference (MMSys '12).

8. MHV’24
Why Energy Efficiency is Critical in VVC
29/09/2025
Slide 8
● Newer codec implementations are more efficient, but need more
CPU cycles to decode.
○ Higher computational load → thermal throttling, degraded QoE
○ Massive streaming demand → higher carbon footprint
● Energy-per-frame increases with complexity → direct impact on
battery.
● Alliance for Open Media Video 1 (AV1) and Versatile Video Coding
(VVC) offer better compression, but decoding cost can outweigh
savings in low-power devices.
● Need frameworks that optimize rate–quality–energy trade-offs.
Table: Averaged BD-BR savings depending on codec and resolution [1]. Table: Averaged BD-PSNR savings depending on codec and resolution [1].
[1] Uhrina, Miroslav, Lukas Sevcik, Juraj Bienik, and Lenka Smatanova. 2024. "Performance Comparison of VVC, AV1, HEVC, and AVC for High Resolutions" Electronics 13, no. 5: 953.
https://doi.org/10.3390/electronics13050953

9. Contributions

10. MHV’24
Green Video Complexity Analysis (VCA): Features
29/09/2025
Slide 10
Goal: Low-complexity, accurate features for real-time
streaming decisions
We use seven DCT-energy-based features extracted
using Video Complexity Analyzer (VCA) [1,2]:
● average texture energy (EY),
● average gradient of the luma texture energy (h)
● average luma brightness (LY),
● average chroma texture energy of U and V
channels (EU and EV)
● average chroma brightness of U and V channels
(LU and LV) .
1. V. V. Menon, C. Feldmann, K. Schoeffmann, M. Ghanbari, and C. Timmerer. 2023. Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming. In Proceedings of the First International ACM
Green Multimedia Systems Workshop (GMSys 2023). Association for Computing Machinery, New York, NY, USA, 259–264. https://doi.org/10.1145/3593908.3593942
2. V. V. Menon, C. Feldmann, H. Amirpour, M. Ghanbari, and C. Timmerer. 2022. VCA: video complexity analyzer. In Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22). Association for
Computing Machinery, New York, NY, USA, 259–264. https://doi.org/10.1145/3524273.3532896
https://github.com/cd-athena/VCA

11. MHV’24
Green Video Complexity Analysis (VCA): Optimizations & Results
29/09/2025
Slide 11
Optimizations in VCA:
● Multi-threading & SIMD acceleration
● Low-pass DCT for speedup
Results:
● Analyzes UHD videos at 292 fps
● 97% lower energy vs. SITI (state-of-the-art)
Enables fast per-title encoding & energy-efficient bitrate ladder estimation

12. MHV’24
Motivation for convex-hull estimation
29/09/2025
Slide 12
● The convex hull is where the encoding point achieves “Pareto efficiency”.
● Convex-hull estimation provide a dynamic and adaptive means to optimize
(bitrate/ resolution/ framerate…) selections.
● By dynamically adjusting the bitrate-resolution pairs in response to the
video content complexity and coding algorithms, these methods achieve
an optimal trade-off between computational efficiency and visual fidelity in
the face of the increased intricacies associated with advanced codecs like
VVC [1,2].
Figure: Conceptual plot to depict the bitrate-quality relationship for
any video source encoded at different resolutions. Source: [3]
[1] B. Bross, Y. Wang, Y. Ye, S. Liu, J. Chen, G. Sullivan, and J. Ohm. (2021). Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video
Technology. 31. 3736-3764. 10.1109/TCSVT.2021.3101953.
[2] R. Kaafarani et al., “Evaluation Of Bitrate Ladders For Versatile Video Coder,” in 2021 International Conference on Visual Communications and Image Processing (VCIP), 2021, pp. 1–5.
[3] A. Aaron, Z. Li, M. Manohara, J.D. Cock, D. Ronca, "Per-title encode optimization." The Netflix Techblog (2015).

13. MHV’24
Balancing Rate, Quality, and Decoding Complexity
29/09/2025
Slide 13
Rate-XPSNR curves and decoding times of example Inter4K sequences using VVenC /
VVdeC.
Quality-Rate-Time points for Inter4K across six spatial resolutions.
Trade-Off Visualization
● Higher resolutions and lower QP values provide better
video quality but increase bitrate and decoding time.
● Lower resolutions and higher QP values reduce bitrate
and decoding time but compromise video quality.
A. Katsenou, V. V. Menon, A. Wieckowski, B. Bross and D. Marpe, "Decoding Complexity-Rate-Quality Pareto-Front for Adaptive VVC Streaming," 2024 IEEE International Conference on Visual Communications and Image
Processing (VCIP), Tokyo, Japan, 2024, pp. 1-5, doi: 10.1109/VCIP63160.2024.10849881.

14. MHV’24
Algorithm for RQT-PF Estimation
29/09/2025
Slide 14
A. Katsenou, V. V. Menon, A. Wieckowski, B. Bross and D. Marpe, "Decoding Complexity-Rate-Quality Pareto-Front for Adaptive VVC Streaming," 2024 IEEE International Conference on Visual Communications and Image
Processing (VCIP), Tokyo, Japan, 2024, pp. 1-5, doi: 10.1109/VCIP63160.2024.10849881.
● Multi-Objective Optimization
○ Objective: Minimize bitrate and decoding time while
maximizing quality.
○ Optimization is performed in the Rate-Quality-Time (RQT)
space to identify Pareto-optimal points.
● Composite Metric
○ Define a composite metric M as a combination of decoding
time and bitrate:
■ M = α * log(τD) + (1 - α) * log(b), where α controls the
importance of decoding time vs. bitrate.
○ Goal: Minimize M and maximize quality simultaneously.

15. MHV’24
Example Results
29/09/2025
Slide 15
A. Katsenou, V. V. Menon, A. Wieckowski, B. Bross and D. Marpe, "Decoding Complexity-Rate-Quality Pareto-Front for Adaptive VVC Streaming," 2024 IEEE International Conference on Visual Communications and Image
Processing (VCIP), Tokyo, Japan, 2024, pp. 1-5, doi: 10.1109/VCIP63160.2024.10849881.

16. MHV’24
Reducing Redundancy Across Codecs
Problem: Multi-Codec Ladders are Energy-Heavy
29/09/2025
Slide 16
● Multiple codecs in use: AVC, HEVC, AV1, VVC
● Services maintain separate ladders per codec
○ Initially, streaming services used AVC for wider device
compatibility.
○ As newer devices with HEVC and AV1 support becomes
prevalent, HEVC and AV1-encoded bitrate ladder
representations are introduced
○ Recent years have developed new formats such as VVC,
EVC, and LCEVC
● Leads to:
○ High encoding energy (repeated per codec)
○ Storage overhead (redundant versions), and
○ Transmission waste (extra representations)
V. V. Menon, R. Farahani, P. T. Rajendran, S. Afzal, K. Schoeffmann and C. Timmerer, "Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming," 2023 IEEE International
Conference on Visual Communications and Image Processing (VCIP), Jeju, Korea, Republic of, 2023, pp. 1-5, doi: 10.1109/VCIP59821.2023.10402699.

17. MHV’24
Reducing Redundancy Across Codecs
29/09/2025
Slide 17
MCBE Approach:
● Predict perceptual quality (VMAF) using Random Forest models
● Eliminate redundant representations:
○ New-gen codec worse than AVC at low bitrates
○ Representations within Just Noticeable Difference (JND)
threshold
V. V. Menon, R. Farahani, P. T. Rajendran, S. Afzal, K. Schoeffmann and C. Timmerer, "Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming," 2023 IEEE International
Conference on Visual Communications and Image Processing (VCIP), Jeju, Korea, Republic of, 2023, pp. 1-5, doi: 10.1109/VCIP59821.2023.10402699.

18. MHV’24
Reducing Redundancy Across Codecs
29/09/2025
Slide 18
Results
● Encoding energy ↓ 56%
● Storage energy ↓ 95%
● Transmission energy ↓ 78%
Efficient multi-codec deployment without QoE loss
V. V. Menon, P. T. Rajendran, C. Feldmann, K. Schoeffmann, M. Ghanbari and C. Timmerer, "JND-Aware Two-Pass Per-Title Encoding Scheme for Adaptive Live Streaming," in IEEE Transactions on
Circuits and Systems for Video Technology, vol. 34, no. 2, pp. 1281-1294, Feb. 2024, doi: 10.1109/TCSVT.2023.3290725.
V. V. Menon, R. Farahani, P. T. Rajendran, S. Afzal, K. Schoeffmann and C. Timmerer, "Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming," 2023 IEEE International
Conference on Visual Communications and Image Processing (VCIP), Jeju, Korea, Republic of, 2023, pp. 1-5, doi: 10.1109/VCIP59821.2023.10402699.

19. MHV’24
ML-Driven Encoding Parameter Selection
Problem: Energy–Quality Trade-off in Encoding
29/09/2025
Slide 19
● Encoders like x265/VVenC have many parameters:
○ Resolution, QP, framerate, presets
● Challenge:
○ Higher quality → higher energy consumption
○ Energy reduction → potential QoE drop
● Existing approaches lack joint optimization of energy & quality
Z. Azimi, R. Farahani, V. V. Menon, C. Timmerer and R. Prodan, "Towards ML-Driven Video Encoding Parameter Selection for Quality and Energy Optimization," 2024 16th International Conference
on Quality of Multimedia Experience (QoMEX), Karlshamn, Sweden, 2024, pp. 80-83, doi: 10.1109/QoMEX61742.2024.10598278.

20. MHV’24
ML-Driven Encoding Parameter Selection
Approach
29/09/2025
Slide 20
● XGBoost models trained on video complexity + encoding settings
● Predict:
○ VMAF (quality)
○ Energy consumption
● XAI analysis: identify most influential parameters
● Decision agent: balances energy vs. quality via weighting function

21. MHV’24
ML-Driven Encoding Parameter Selection
Results: Energy Savings with ML Guidance
29/09/2025
Slide 21
● Encoder energy consumption ↓ 46%
● Quality drop < 1 JND (~4–5 VMAF points)
● Sustainable trade-off:
○ Significant energy reduction
○ No perceptible QoE loss
Z. Azimi, R. Farahani, V. V. Menon, C. Timmerer and R. Prodan, "Towards ML-Driven Video Encoding Parameter Selection for Quality and Energy Optimization," 2024 16th International Conference
on Quality of Multimedia Experience (QoMEX), Karlshamn, Sweden, 2024, pp. 80-83, doi: 10.1109/QoMEX61742.2024.10598278.

22. MHV’24
Decoding Energy Modeling – RDEI
Problem: Decoding Energy is Hardware-Dependent
29/09/2025
Slide 22
● High decoding complexity in VVC → more device energy use
● Existing energy models:
○ Tied to specific hardware/architecture
○ Poor generalization across devices
● Streaming providers need a portable, codec-level model
R. Farahani, V. V. Menon and C. Timmerer, "Machine Learning-Based Decoding Energy Modeling for VVC Streaming," 2025 IEEE International Conference on Image Processing (ICIP), Anchorage, AK,
USA, 2025, pp. 2671-2676, doi: 10.1109/ICIP55913.2025.11084324.

23. MHV’24
Decoding Energy Modeling – RDEI
Approach: Relative Decoding Energy Index (RDEI)
29/09/2025
Slide 23
● Introduce RDEI – Relative Decoding Energy Index
● Normalizes decoding energy against a baseline
● Makes predictions comparable across devices
● ML Models used:
○ Random Forest,
○ XGBoost,
○ Linear Regression,
○ Neural Networks
● Trained on 1000 VVC streams with varied content & QPs
R. Farahani, V. V. Menon and C. Timmerer, "Machine Learning-Based Decoding Energy Modeling for VVC Streaming," 2025 IEEE International Conference on Image Processing (ICIP), Anchorage, AK,
USA, 2025, pp. 2671-2676, doi: 10.1109/ICIP55913.2025.11084324.

24. MHV’24
Decoding Energy Modeling – RDEI
Results: Accurate & Portable Decoding Energy Model
29/09/2025
Slide 24
● RDEI predictions robust across hardware platforms
● High prediction accuracy (R² > 0.9)
● Enables energy-aware adaptation decisions at the client side
● Facilitates cross-platform sustainable streaming
R. Farahani, V. V. Menon and C. Timmerer, "Machine Learning-Based Decoding Energy Modeling for VVC Streaming," 2025 IEEE International Conference on Image Processing (ICIP), Anchorage, AK,
USA, 2025, pp. 2671-2676, doi: 10.1109/ICIP55913.2025.11084324.

25. Unified Framework

26. MHV’24
Problem: Fragmented Energy-Aware Solutions
29/09/2025
Slide 26
● Existing methods target individual stages:
○ Encoding → parameter tuning
○ Ladder design → Pareto optimization
○ Delivery → multi-codec pruning
○ Decoding → energy modeling
● Lack of a holistic framework that unifies them

27. MHV’24
Approach: Unified Energy-Aware Streaming Framework
29/09/2025
Slide 27
● End-to-End Design integrating:
○ VCA → content complexity analysis
○ Pareto optimization → rate–quality–complexity trade-offs
○ MCBE → cross-codec redundancy pruning
○ ML-driven encoding → energy–quality parameter tuning
○ RDEI → decoding energy-aware client adaptation
● Works at both server and client sides

28. MHV’24
Results: Toward Sustainable VVC Streaming
29/09/2025
Slide 28
● Encoding energy ↓ 46%
● Storage energy ↓ 95%
● Transmission energy ↓ 78%
● Decoding energy ↓ 15% (via optimized ladders)
● Maintains QoE (within JND thresholds)

29. Discussion & Outlook

30. MHV’24
Impact on Real-World Streaming Ecosystems
29/09/2025
Slide 30
● Sustainability & Green Streaming
○ Lower energy use → reduced carbon footprint
● Better User Experience
○ Lower decoding energy → longer device battery life
○ Stable QoE even under resource constraints
● Cost Savings for Providers
○ Less storage & transmission overhead
○ Reduced encoder workload at scale
● Standardization Potential
○ Fits into DASH/HLS adaptive streaming frameworks
○ Relevant for industry consortia (MPEG, DASH-IF, CTA WAVE)

31. MHV’24
Next Steps in Energy-Efficient Streaming
29/09/2025
Slide 31
● Live Streaming Integration
○ Extend energy-aware methods to real-time encoding
● AI-Guided Per-Title Encoding
○ Deep learning for automatic parameter tuning
○ Adaptive across diverse content types at scale
● XR & Metaverse Applications
○ Energy–quality trade-offs critical for VR/AR streaming
○ Frame interpolation & super-resolution for immersive media
● Device-Aware Adaptation
○ Tailor bitrate ladders and adaptation to device energy profiles
● Toward Net-Zero Streaming
○ Combine with renewable-powered CDNs & green data centers

32. Conclusions

33. MHV’24
Key Takeaways
29/09/2025
Slide 33
● Energy–Quality trade-off is central for sustainable VVC streaming
● Holistic framework integrates encoding, ladder design, and decoding
● Achieves large energy savings:
○ Encoding ↓ 46%, Storage ↓ 95%, Transmission ↓ 78%, Decoding ↓ 15%
● Maintains Quality of Experience (within JND thresholds)
● Toward green, scalable adaptive streaming

34. Thank you for your attention
— Dr. Vignesh V Menon
Email: vignesh.menon@hhi.fraunhofer.de
https://www.linkedin.com/in/vignesh-v-menon/