Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming

Since video accounts for the majority of today’s internet traffic, the popularity of HTTP Adaptive Streaming (HAS) is increasing steadily. In HAS, each video is encoded at multiple bitrates and spatial resolutions (i.e., representations) to adapt to a heterogeneity of network conditions, device characteristics, and end-user preferences. Most of the streaming services utilize cloud-based encoding techniques which enable a fully parallel encoding process to speed up the encoding and consequently to reduce the overall time complexity. State-of-the-art approaches further improve the encoding process by utilizing encoder analysis information from already encoded representation(s) to improve the encoding time complexity of the remaining representations. In this paper, we investigate various multi-encoding algorithms (i.e., multi-rate and multi-resolution) and propose novel multi-encoding algorithms for large-scale HTTP Adaptive Streaming deployments. Experimental results demonstrate that the proposed multi-encoding algorithm optimized for the highest compression efficiency reduces the overall encoding time by 39% with a 1.5% bitrate increase compared to stand-alone encodings. Its optimized version for the highest time savings reduces the overall encoding time by 50% with a 2.6% bitrate increase compared to stand-alone encodings.

  • Be the first to comment

  • Be the first to like this

Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming

  1. 1. Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming Vignesh V Menon, Hadi Amirpour, Christian Timmerer, Mohammad Ghanbari Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), University of Klagenfurt, Austria Date: 02 July 2021 Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 1
  2. 2. Outline 1 Introduction 2 State-of-the-art: Multi-Rate Encoding 3 State-of-the-art: Multi-Encoding 4 Efficient Multi-Rate Encoding 5 Efficient Multi-Encoding 6 Experimental Evaluation 7 Conclusions and Future Directions Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 2
  3. 3. Introduction Introduction Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 3
  4. 4. Introduction Introduction Background of HTTP Adaptive Streaming (HAS)1 Source: https://bitmovin.com/adaptive-streaming/ Why Adaptive Streaming? Adapt for a wide range of devices Adapt for a broad set of Internet speeds What HAS does? Each source video is split into segments Encoded at multiple bitrates, resolutions, and codecs Delivered to the client based on the device capability, network speed etc. 1 A. Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1 (2019), pp. 562–585. doi: 10.1109/COMST.2018.2862938. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 4
  5. 5. Introduction Introduction Encoding time of HAS representations Relative encoding time of the representations normalized to encoding time of the 2160p 25Mbps representation. 500 1000 1500 2000 3000 4500 5800 7000 11600 16800 20000 25000 Bitrate (in kbps) 0 20 40 60 80 100 Relative Time Complexity (in percentage) 540p 1080p 2160p As resolution doubles, encoding time complexity doubles! Many encoder analysis decisions are redundant across the representations. Multi-rate: Exploit this redundancy across representations of a resolution. Multi-resolution: Exploit this redundancy across resolutions. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 5
  6. 6. State-of-the-art: Multi-Rate Encoding State-of-the-art: Multi-Rate Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 6
  7. 7. State-of-the-art: Multi-Rate Encoding Related Works: Multi-Rate Encoding Single-bound2 and Double-bound3 for CU depth estimation 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (a) Single-bound for CU depth estimation 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (b) Double-bound for CU depth estimation Motivation: CUs generally tend to have higher depth values (smaller CU sizes) in higher bitrate representations. CU depth information from the highest bitrate representation is used as the upper bound of the CU depth estimation of dependent representations. In double-bound, additionally, CU depth information from the lowest bitrate representation is used as the lower bound of the CU depth estimation of dependent representations. 2 D. Schroeder, P. Rehm, and E. Steinbach. “Block structure reuse for multi-rate high efficiency video coding”. In: 2015 IEEE International Conference on Image Processing (ICIP). Sept. 2015, pp. 3972–3976. 3 H. Amirpour et al. “Fast Multi-rate Encoding for Adaptive HTTP Streaming”. In: 2020 Data Compression Conference (DCC). 2020, pp. 358–358. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 7
  8. 8. State-of-the-art: Multi-Rate Encoding Related Works: Multi-Rate Encoding Integrated x265 mode-1 and x265 mode-24 Integrated x265 mode-1 Force the optimal decisions made in the highest bitrate representation the slice-type scene-cut decisions the quadtree structure prediction modes reference indices Integrated x265 mode-2 Reuse all CUs and PUs from the highest bitrate representation without additional RDO. They also employ static refinement techniques defined in x265. If the reference depth is d, it is re- evaluated by using the RDO against the cost of splitting the CU (i.e., depth d + 1). The motion vectors from the highest bi- trate representation are used as Motion vector predictors (MVPs). 4 K. Goswami et al. “Adaptive Multi-Resolution Encoding for ABR Streaming”. In: 2018 25th IEEE International Conference on Image Processing (ICIP). 2018, pp. 1008–1012. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 8
  9. 9. State-of-the-art: Multi-Encoding State-of-the-art: Multi-Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 9
  10. 10. State-of-the-art: Multi-Encoding Related Work: Multi-Encoding 1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps 1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps 3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps Scaled CU,PU reuse + refinement Figure: Encoder analysis sharing method in x265 Algorithm.5 5 Aruna Mathesawaran et al. “Open source framework for reduced-complexity multi-rate HEVC encoding”. In: Applications of Digital Image Processing XLIII. ed. by Andrew G. Tescher and Touradj Ebrahimi. Vol. 11510. International Society for Optics and Photonics. SPIE, 2020, pp. 461 –471. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 10
  11. 11. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 11
  12. 12. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Prediction Mode Heuristics Prediction Mode Heuristics If the CU depth chosen for the dependent representations is the same as the reference represen- tation, the following heuristics are proposed. If Skip mode was chosen in the highest bitrate representation, RDO is evaluated for only Merge/Skip modes. If 2Nx2N mode was chosen in the highest bitrate representation, skip RDO for AMP modes. If inter-prediction mode was chosen in the highest bitrate representation, skip RDO for intra-prediction modes. In double-bound for CU estimation, if the intra-prediction mode was chosen for the highest and lowest bitrate representations, RDO is evaluated for only intra-prediction modes in the intermediate representations. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 12
  13. 13. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Motion Estimation Heuristics Motion Estimation Heuristics If the CU depth and PU chosen for the dependent representations is the same as the reference representation, the following heuristics are proposed. Force the same reference frame as that of the highest bitrate representation. The Motion Vector Predictor (MVP) is set to be the Motion Vector (MV) of the highest bitrate representation. In double-bound for CU estimation, the motion search range is decreased to a smaller window if the MV of the highest bitrate representation and the MV of the lowest bitrate representation are close to each other. The search range is determined to be the maximum difference between the x coordinates and y coordinates of the MVs. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 13
  14. 14. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Proposed Multi-rate Algorithm-1 and Algorithm-2 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (a) Proposed Multi-rate Algorithm-1 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (b) Proposed Multi-rate Algorithm-2 − > denotes analysis sharing using single-bound for CU depth estimation and proposed heuristics. − > denotes analysis sharing using double-bound for CU depth estimation and proposed heuristics. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 14
  15. 15. Efficient Multi-Encoding Efficient Multi-Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 15
  16. 16. Efficient Multi-Encoding Efficient Multi-Encoding Proposed Multi-Encoding Algorithm-1 1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps 1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps 3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps Scaling + Single-bound Figure: Encoder analysis sharing method in Proposed Multi-Encoding Algorithm-1. Multi-rate Approach: Proposed Multi-rate Algorithm-2 Multi-resolution Approach: scaled CU information from highest bitrate representation of the previous resolution is used as lower bound of CU depth estimation process. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 16
  17. 17. Efficient Multi-Encoding Efficient Multi-Encoding Proposed Multi-Encoding Algorithm-2 1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps 1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps 3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps Scaling + Single-bound Figure: Encoder analysis sharing method in Proposed Multi-Encoding Algorithm-2. Multi-rate Approach: Proposed Multi-rate Algorithm-2 Multi-resolution Approach: scaled CU information from lowest bitrate representation of the previous resolution is used as lower bound of CU depth estimation process. More rigid bound compared to Multi-encoding Algorithm-1. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 17
  18. 18. Experimental Evaluation Experimental Evaluation Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 18
  19. 19. Experimental Evaluation Experimental Evaluation Test Setup Algorithms presented here are tested using x265 v3.46 with the veryslow preset (highest quality preset practically usable of x265). Test PC: Dual-processor server with Intel Xeon Gold 5218R (80 cores, frequency at 2.10 GHz). Test Sequences: (i) 4 JVET test sequences7 and (ii) 3 professionally produced UltraHD HDR cinematic content.8 The lowest resolution is 960 × 540, intermediate resolution is 1920 × 1080, and the highest resolution is 3840 × 2160. The lower resolution sources were generated from the original video source by applying bi-cubic scaling using FFmpeg.9 6 MulticoreWare Inc. x265 HEVC Encoder/H.265 Video Codec. url: http://x265.org/. 7 Jill Boyce et al. JVET-J1010: JVET common test conditions and software reference configurations. July 2018. 8 M. H. Pinson. “The Consumer Digital Video Library [Best of the Web]”. In: IEEE Signal Processing Magazine 30.4 (2013), pp. 172–174. doi: 10.1109/MSP.2013.2258265. 9 FFmpeg. FFmpeg Documentation. url: https://ffmpeg.org/ffmpeg.html. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 19
  20. 20. Experimental Evaluation Experimental Evaluation Test Setup Table: Bitrate representations Bi,j (in kbps) based on HLS authoring specification10 Layer (i) Resolution j = 1 j = 2 j = 3 j = 4 1 960x540 2000 1500 1000 500 2 1920x1080 7000 5800 4500 3000 3 3840x2160 25000 20000 16800 11600 Performance evaluation Metrics: ∆T : overall encoding time saving BDRP : Average difference in bitrate with respect to reference encode to maintain the same PSNR BDRV : Average difference in bitrate with respect to reference encode to maintain the same VMAF BDRP ∆T : Trade-off between BDR and ∆T. 10 https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 20
  21. 21. Experimental Evaluation Experimental Evaluation Multi-rate Encoding Table: Results for the multi-rate algorithms Method ∆T BDRP BDRP ∆T BDRV BDRV ∆T Single-bound for CU estimation 14.50% -0.59% -4.39% 0.10% 1.08% Double-bound for CU estimation 26.37% 0.73% 3.44% 1.25% 5.54% x265 Algorithm-1 18.61% 1.67% 9.52% 1.56% 8.85% x265 Algorithm-2 55.11% 8.42% 15.62% 8.72% 16.10% Multi-rate Algorithm-1 (ours) 26.50% -0.28% -0.84% 0.43% 2.31% Multi-rate Algorithm-2 (ours) 37.37% 1.06% 3.36% 1.57% 4.88% Multi-rate Algorithm-1 improves ∆T of Single-bound for CU estimation by 12% with neg- ligible increase in BDR. Multi-rate Algorithm-2 improves ∆T of Double-bound for CU estimation by 11% with negligible increase in BDR. Better BDR ∆T metric Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 21
  22. 22. Experimental Evaluation Experimental Evaluation Multi-Encoding Algorithm ∆T BDRP BDRV State-of-the-art 80.05% 13.53% 9.59% Multi-encoding Algorithm-1 39.72% 2.32% 1.55% Multi-encoding Algorithm-2 50.90% 3.45% 2.63% State-of-the-art Multi-encoding Algorithm-1 Multi-encoding Algorithm-2 0 2 4 6 8 10 12 14 16 BDR/ T (in percentage) BDRP/ T BDRV/ T State-of-the-art algorithm yields the highest ∆T. Multi-encoding Algorithm-1 has the lowest increase in BDR and the lowest BDR ∆T metric. Multi-encoding Algorithm-2 improves ∆T of Multi-encoding Algorithm-1 by 11% with neg- ligible increase in BDR. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 22
  23. 23. Conclusions and Future Directions Conclusions and Future Directions Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 23
  24. 24. Conclusions and Future Directions Conclusions and Future Directions Conclusions Presented efficient multi-encoding algorithms for HTTP Adaptive Streaming, tested with the open-source x265 HEVC encoder. Multi-rate Encoding: Proposed heuristics improved the encoding speed of the single-bound and double-bound for CU estimation by an average of 12% and 11%, respectively with a negligible increase in BD-Rate. BDR ∆T metric for proposed algorithms is better compared to the state-of-the-art algorithms. Multi-encoding: Proposed two algorithms that extended the proposed multi-rate encoding algorithms across resolutions. BDR ∆T metric for proposed algorithms are better compared to the state-of-the-art algorithm. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 24
  25. 25. Conclusions and Future Directions Conclusions and Future Directions Future Directions The encoding speedup can be improved by parallelizing the execution of encoding instances. The dependent representations can be started after a delay of one frame execution time of the reference representation. Further work is required to extend the algorithms to resolutions, which are non-integer power-of-2 multiples of each other. Further work is required to investigate more recent codecs, e.g., VVC. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 25
  26. 26. Conclusions and Future Directions Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Hadi Amirpour (hadi.amirpourazarian@aau.at) Christian Timmerer (Christian.Timmerer@aau.at) Mohammad Ghanbari (ghan@essex.ac.uk) Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 26

    Be the first to comment

Since video accounts for the majority of today’s internet traffic, the popularity of HTTP Adaptive Streaming (HAS) is increasing steadily. In HAS, each video is encoded at multiple bitrates and spatial resolutions (i.e., representations) to adapt to a heterogeneity of network conditions, device characteristics, and end-user preferences. Most of the streaming services utilize cloud-based encoding techniques which enable a fully parallel encoding process to speed up the encoding and consequently to reduce the overall time complexity. State-of-the-art approaches further improve the encoding process by utilizing encoder analysis information from already encoded representation(s) to improve the encoding time complexity of the remaining representations. In this paper, we investigate various multi-encoding algorithms (i.e., multi-rate and multi-resolution) and propose novel multi-encoding algorithms for large-scale HTTP Adaptive Streaming deployments. Experimental results demonstrate that the proposed multi-encoding algorithm optimized for the highest compression efficiency reduces the overall encoding time by 39% with a 1.5% bitrate increase compared to stand-alone encodings. Its optimized version for the highest time savings reduces the overall encoding time by 50% with a 2.6% bitrate increase compared to stand-alone encodings.

Views

Total views

637

On Slideshare

0

From embeds

0

Number of embeds

454

Actions

Downloads

1

Shares

0

Comments

0

Likes

0

×