Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective QoE Evaluation

358 views

Published on

Omnidirectional video (ODV) streaming applica- tions are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile- based streaming which allows to have higher quality within the user’s viewport while quality outside the user’s viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is character- ized by high reproducibility since relevant media delivery aspects (including the user’s head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective QoE Evaluation

  1. 1. Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective QoE Evaluation Raimund Schatz AIT Austrian Institute of Technology 11th International Conference on Quality of Multimedia Experience (QoMEX 2019) June 5-7, 2019, Berlin Christian Timmerer Alpen-Adria-Universität Klagenfurt / Bitmovin Inc. Anatoliy Zabrovskiy Alpen-Adria-Universität Klagenfurt
  2. 2. § QoE of Omnidirectional Video (ODV, aka 360º Movie) Streaming § Immersive panorama video surrounding the user § User can turn head to change gaze direction § ODV streaming = content class with dramatically increasing popularity § Cisco VNI: globally, virtual–reality traffic will increase 61-fold between 2015 and 2020 Focus of this Talk & Motivation 208.06.19 © AIT Immersive Media VR Responsive VR ODV MR AR
  3. 3. Motivation § Current deployments of ODV: technically straightforward … Drawback: viewport-agnostic! à High storage & bandwidth requirements à Impaired QoE (under constrained conditions)
  4. 4. Tile-based Streaming to the Rescue! Mario Graf, Christian Timmerer, and Christopher Mueller. 2017. Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP: Design, Implementation, and Evaluation. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys'17). ACM, New York, NY, USA, 261-271. DOI: https://doi.org/10.1145/3083187.3084016 https://multimediacommunication.blogspot.com/2017/04/acm-mmsys17-special-session-paper.html
  5. 5. Tile-based Streaming, The Challenge: Lots of Knobs & Levers … § Tiling Patterns? § Encoding Quality Levels? § Delivery Strategy? § … àWANTED: Valid Ground Truths & Guidelines based on subjective ODV Quality Studies! à This Talk: Results (subjective & objective) from a Joint ODV QoE Study by AIT & AAU 508.06.19
  6. 6. The Study 608.06.19
  7. 7. Research Questions RQ1: How does full vs. partial delivery impact ODV streaming QoE? Is there a clear user preference? RQ2: What is the QoE impact of different ODV tile quality encoding levels? (RQ2) RQ3: Do camera movement (static vs. moving) or head turning speed exert an influence on quality perception? 708.06.19
  8. 8. Subjective Lab Test: SRCs & HRCs § SRCs: Two 8k high-quality source clips (duration: 30s) § HRCs, Factors: § Delivery: Full vs. Partial § Head turn: Slow vs. Fast § Tile encoding quality: QP 46, 32, 22 808.06.19
  9. 9. SRCs: Source Clips (8k) 908.06.19 URLs: http://medialab.sjtu.edu.cn/vr8K/index.html https://www.itu.int/en/ITU-T/studygroups/2017-2020/16/Pages/video/jctvc.aspx
  10. 10. HRCs: Full vs. Partial Delivery 1008.06.19 Gray Tiles
  11. 11. HRCs: Head-Turn § Head turning movement in the middle of the clip § 90 deg to the right § Fast (1s) vs. Slow (4s) § Challenge: slight changes in head movement à huge differences in tile- based streaming behavior à Solution: render FoV into PVS! 1108.06.19
  12. 12. Processing Pipeline: from 360 SRCs to 2D PVSes 1308.06.19
  13. 13. Subjective Lab Test: Setup § Tiled ODV QoE Lab Study Sessions in Summer 2018 § Setup: Large 4k screen (65“, SONY) viewed at 1,5m distance, ratings via tablet 1408.06.19 Similar setup used in standardization, e,g, Versatile Video Coding (VVC)
  14. 14. Subjective Lab Test: Protocol 1508.06.19 Ratings provided on table using the open source software http://www.thefragebogen.de
  15. 15. Test Subjects § N=35 § 21 male, 14 female § 50% had (some) VR experience § Mixed education / backgrounds § Age: mean 32, median 33, range: 22-50 yrs 1608.06.19 nafi 4.0
  16. 16. The Results 1708.06.19
  17. 17. Subjective Ratings: MOS [0-100] 18 § Delivery: Partial Delivery à visibly low QoE § Tile Encoding Quality: QP46 = significant QoE drop, but only very little difference between QP22 & 32 QP: 46 32 22 | 46 ….
  18. 18. Subjective Ratings: Acceptability 19 § Acceptability Ratings: confirm MOS results § In particular: Partial Delivery = really unacceptable Full Delivery Partial Delivery QP: 46 32 22 | 46 ….
  19. 19. Mixed Model ANOVA Results (Full Delivery only) 2008.06.19 Head movement Content / Cam movement
  20. 20. Objective Metrics: wPSNR, SSIM and VMAF for FoV PVS § Low sensitivity of SSIM w.r.t. full vs. partial delivery § Little impact of content / cam movement speed § Consistent impact of head movement speed (but not pronounced, too) 2108.06.19 Results for segments 3-4-5 (i.e. head turn part only): More objective analysis results à see the paper!
  21. 21. Findings & Conclusions § RQ1 (Full vs partial delivery): § Avoid partial delivery (or find better visual workarounds), since unacceptable for end users § RQ2 (Impact of tile encoding quality): § Sufficient to use QP 32 for tile encoding for practical deployments as saturation kicks in here § RQ3 (Impact of camera movement and head turn speed): § Camera movement masking low encoding quality (known) § Head turn speed: only weak impact, interaction with content/cam movement (+ for static, - for dynamic scene) § Objective Evaluation: § Largely in line with subjective results, with some exceptions depending on metric and impairment 2208.06.19
  22. 22. Issues, Challenges & Outlook § Several limitations: 2D Display used (not HMD), limited number of SRCs, head-motion patterns used, etc. § Further research needed! § Future studies featuring 2D Display vs. HMD viewing impact, larger variety of content, head-motions, etc. § Also: tile-based rendering pipeline = technically challenging, lots of tweaking & error-checking required à joint efforts required 2308.06.19
  23. 23. Thank you for your attention! Raimund Schatz AIT Austrian Institute of Technology GmbH Donau-City-Str. 1, 1220 Vienna raimund.schatz@ait.ac.at This work was supported in part by the Austrian Research Promotion Agency (FFG) under the Next Generation Video Streaming project “PROMETHEUS. Christian Timmerer Alpen-Adria-Universität Klagenfurt / Bitmovin Inc.
  24. 24. Objective Metrics: Metrics Calculated for Whole 8k Content
  25. 25. Questionnaire 2608.06.19

×