Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP

905 views

Published on

Real-time entertainment services such as streaming audio-visual content deployed over the open, unmanaged Internet account now for more than 70% during peak periods. More and more such bandwidth hungry applications and services are proposed like immersive media services such as virtual reality and, specifically omnidirectional/360-degree videos. The adaptive streaming of omnidirectional video over HTTP imposes an important challenge on today’s video delivery infrastructures which calls for dedicated, thoroughly designed techniques for content generation, delivery, and consumption.
This paper describes the usage of tiles — as specified within modern video codecs such HEVC/H.265 and VP9 — enabling bandwidth efficient adaptive streaming of omnidirectional video over HTTP and we define various streaming strategies. Therefore, the parameters and characteristics of a dataset for omnidirectional video are proposed and exemplary instantiated to evaluate various aspects of such an ecosystem, namely bitrate overhead, bandwidth requirements, and quality aspects in terms of viewport PSNR. The results indicate bitrate savings from 40% (in a realistic scenario with recorded head movements from real users) up to 65% (in an ideal scenario with a centered/fixed viewport) and serve as a baseline and guidelines for advanced techniques including the outline of a research roadmap for the near future.

Published in: Technology

Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP

  1. 1. Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP Design, Implementation, and Evaluation Priv.-Doz. Dr. Christian Timmerer Alpen-Adria-Universität Klagenfurt (AAU) w Faculty of Technical Sciences (TEWI) w Department of Information Technology (ITEC) w Multimedia Communication (MMC) w Sensory Experience Lab (SELab) http://blog.timmerer.com w http://selab.itec.aau.at/ w http://dash.itec.aau.at w christian.timmerer@itec.aau.at Chief Innovation Officer (CIO) at bitmovin GmbH http://www.bitmovin.com w christian.timmerer@bitmovin.com ACM MMSys 2017, Taipei, Taiwan, June 20, 2017 http://www.slideshare.net/christian.timmerer
  2. 2. Outline • Introduction • Omnidirectional Video (ODV) Streaming • Tile-based Adaptive Streaming • Evaluations • Conclusions and Future Work June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 2
  3. 3. Importance of Multimedia Delivery • Real-time entertainment: Streaming video and audio; >70% of Internet traffic at peak periods (forecast: 2021 >80%) • Popular services: YouTube (17.53%), Netflix (35.15%), Amazon Video (4.26%), Hulu (2.68%); all delivered over-the-top (OTT); huge potential for mobile access! • More people now subscribe to Netflix (50.85M) than cable TV (48.61M) in the US (Q1 2017) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 3 Global Internet Phenomena Report: 2016 https://www.businessinsider.com/netflix-more-popular-than-cable-us-chart-2017-6/
  4. 4. Over-The-Top – Adaptive Media Streaming • In a nutshell… June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 4 Adaptation logic is within the client, not normatively specified by the standard, subject to research and development
  5. 5. June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 5 Example Platform/Infrastructure https://bitmovin.com/
  6. 6. What is Omnidirectional Video (ODV)? June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 6 Capturing Devices • Stitching, Projection Formats • Encoding, Encryption, Encapsulation • Storage, Content Distribution, Delivery • Processing, Decoding, Rendering, … Consumer Devices
  7. 7. Adaptive Streaming of ODV Content June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 7 https://bitmovin.com/mpeg-vr-one-ring-rule Projection format: Equirectangular projection Coding format: AVC File format: ISOBMFF, M2TS Delivery format: DASH, HLS Web clients: HTML5, MSE, WebVR https://multimediacommunication.blogspot.co.at/2017/03/360-degree-live-adaptive-streaming-with.html
  8. 8. Tile-based Adaptive Streaming June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 8 Tile 1 Tile 2 Tile 3 Tile 4 Tile 5 Tile 6 Tile 7 Tile 8 Tile 9 Tile 10 Tile 11 Tile 12 Tile 13 Tile 14 Tile 15 Tile 16 Tile 17 Tile 18 Tile 19 Tile 20 Tile 21 Tile 22 Tile 23 Tile 24 Encoding & Packaging Tile 1 Tile 2 Tile 3 Tile 4 Tile 5 Tile 6 Tile 7 Tile 8 Tile 9 Tile 10 Tile 11 Tile 12 Tile 13 Tile 14 Tile 15 Tile 16 Tile 17 Tile 18 Tile 19 Tile 20 Tile 21 Tile 22 Tile 23 Tile 24 Tile 1 Tile 2 Tile 3 Tile 4 Tile 5 Tile 6 Tile 7 Tile 8 Tile 9 Tile 10 Tile 11 Tile 12 Tile 13 Tile 14 Tile 15 Tile 16 Tile 17 Tile 18 Tile 19 Tile 20 Tile 21 Tile 22 Tile 23 Tile 24 Tile 1 Tile 2 Tile 3 Tile 4 Tile 5 Tile 6 Tile 7 Tile 8 Tile 9 Tile 10 Tile 11 Tile 12 Tile 13 Tile 14 Tile 15 Tile 16 Tile 17 Tile 18 Tile 19 Tile 20 Tile 21 Tile 22 Tile 23 Tile 24 Delivery MPEG-HEVC/H.265 Tiles in ISOBMFF Adaptive Player Tile 1 Tile 2 Tile 3 Tile 4 Tile 5 Tile 6 Tile 7 Tile 8 Tile 9 Tile 10 Tile 11 Tile 12 Tile 13 Tile 14 Tile 15 Tile 16 Tile 17 Tile 18 Tile 19 Tile 20 Tile 21 Tile 22 Tile 23 Tile 24 Tile-based streaming of VR/360° content with MPEG-DASH SRD … … … … Adaptive Streaming using MPEG-DASH SRD Head Mounted Displays Browsers, Smart (Mobile) Devices (Stereo) 2D, (Stereo) 3D
  9. 9. Tile-based Streaming Options June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 9
  10. 10. ODV Quality Metrics: S-PSNR June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 10 M. Yu, H. Lakshman, and B. Girod. 2015. A Framework to Evaluate Omnidirectional Video Coding Schemes. In 2015 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 31–36. DOI:http://dx.doi.org/10.1109/ISMAR.2015.12
  11. 11. ODV Quality Metrics: V-PSNR June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 11 M. Yu, H. Lakshman, and B. Girod. 2015. A Framework to Evaluate Omnidirectional Video Coding Schemes. In 2015 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 31–36. DOI:http://dx.doi.org/10.1109/ISMAR.2015.12
  12. 12. Preliminary Evaluation Results • Bitrate overhead due to various tiling patterns • Bandwidth requirements • Quality (viewport PSNR) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 12
  13. 13. Dataset • Segment length / Intra period – 1s (tiled content) vs 1, 2, 4s (monolithic content) • Tiling pattern (columns × rows): 1×1, (i.e., tiles monolithic), 3×2, 5×3, 6×4, and 8×5 • Resolution: 1920×960, 3840×1920 and 7680×3840 • Map projection: equirectangular format • Quantization parameter: QP={22,27,32,37,42} • Head motion recordings for V-PSNR evaluation June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 13
  14. 14. Bitrate Overhead due to Tiling June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 14 500 1000 1500 2000 2500 384042444648 Bitrate [kbps] Y−PSNR[dB] ● ● ● ● ● ● ● 1x1 tiles 3x2 tiles 5x3 tiles 6x4 tiles 8x5 tiles Tile Overhead for Resolution: 1920x960 Sequence: AssassinsCreed 2000 4000 6000 8000 30323436384042 Bitrate [kbps] Y−PSNR[dB] ● ● ● ● ● ● ● 1x1 tiles 3x2 tiles 5x3 tiles 6x4 tiles 8x5 tiles Tile Overhead for Resolution: 1920x960 Sequence: ExploreTheWorld
  15. 15. Bandwidth Requirements (1) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 15
  16. 16. Bandwidth Requirements (2) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 16
  17. 17. Bandwidth Requirements (3) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 17
  18. 18. Viewport PSNR (1) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 18
  19. 19. Viewport PSNR (2) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 19
  20. 20. Conclusions • System architecture and encoding options for tile- based streaming • 6 × 4 provides the best tradeoff – View-port selection flexibility – Bitrate overhead – Bandwidth requirements • Tile-based streaming offers bitrate savings up to 65% • Viewport PSNR: bitrate savings up to 40% June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 20
  21. 21. Roadmap for Future Work (1) • Publicly available dataset enabling reproducible research in this area • Other codecs supporting tiles (incl. decoding options) • Investigation of advanced adaptation/streaming strategies • Incorporation of machine learning techniques to predict head movements (possibly including saliency maps, eye tracking in combination with HMDs) June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 21
  22. 22. Roadmap for Future Work (2) • Usage of different projection formats reducing the limitations of the equirectangular project format • Utilization of HTTP/2 push mechanisms as requesting each tile individually drastically increases the number of HTTP requests which may impact the overall streaming performance • Subjective quality assessments as a prerequisite for QoE models including various/different tilling patterns June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 22
  23. 23. Thank you for your attention June 20, 2017 Dr. Timmerer [AAU/Bitmovin] 23 ... questions, comments, etc. are welcome … Priv.-Doz. Dipl.-Ing. Dr. Christian Timmerer Associate Professor Alpen-Adria-Universität Klagenfurt, Department of Information Technology (ITEC) Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA christian.timmerer@itec.uni-klu.ac.at http://blog.timmerer.com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Christian Timmerer

×