SlideShare a Scribd company logo

Immersive Video Delivery: From Omnidirectional Video to Holography

Alpen-Adria-Universität
Alpen-Adria-Universität
Alpen-Adria-UniversitätAssociate Professor at Alpen-Adria-Universität

Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.

Immersive Video Delivery: From Omnidirectional Video to Holography

1 of 26
Download to read offline
Immersive Video Delivery: From
Omnidirectional Video to Holography
Christian Timmerer, Univ.-Prof. at AAU, Director at CD Lab ATHENA
Klagenfurt, Austria
Acknowledgments: Jeroen van der Hooft, Hadi Amirpour, Maria Torres Vega,
Yago Sanchez, Raimund Schatz, Thomas Schierl
March 14, 2023
1
“In the first half of 2022, video accounted
for a hefty 65.93% of total volume over
the Internet. That’s a 24% increase over
H1 2021.”
The Global Internet Phenomena Report 2023
2
Source: https://twitter.com/StarWarsAUNZ/status/657841954139471872
Presenter
Christian Timmerer
Univ.-Prof. at Alpen-Adria-Universität Klagenfurt
Director CD Lab ATHENA
CIO | Head of Research and Standardization at Bitmovin
3
3
2003: MSc CS (Dipl.-Ing.)
2006: PhD CS (Dr.-techn.)
2012: Co-founded Bitmovin
2014: Habilitation (Priv.-Doz.) & Assoc. Prof.
2016: Dep. Director @ ITEC/AAU
2019: Director @ ATHENA
2022: Univ.-Prof. for Multimedia Systems
Web: http://timmerer.com/
Bitmovin MPEG-DASH
4
4
● Introduction
● Background
● Immersive Video Delivery Chain
● 3DoF Omnidirectional Video
● 6DoF Volumetric Video
● 6DoF Imagery Video
● Outlook and Open Challenges
Agenda
5
Motivation
Sources: * Sandvine Global Internet
Phenomena (January 2023). **
Cisco Annual Internet Report
(2018–2023) White Paper (March
2020)
6
6
Video streaming is dominating today’s Internet traffic
● 2022: 65.93%; Netflix 13.74%, YouTube 10.51%, Disney+ 4.2%,
Tik Tok 3.55%, Amazon Prime 2.67%*
● Video and other applications continue to be of enormous
demand in today’s home, but there will be significant
bandwidth demands with the application requirements
of the future**

Recommended

HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?Alpen-Adria-Universität
 
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingMMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingJesus Aguilar
 
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingDanieleLorenzi6
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)Alpen-Adria-Universität
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsAlpen-Adria-Universität
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningAlpen-Adria-Universität
 
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsHow to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsAlpen-Adria-Universität
 

More Related Content

Similar to Immersive Video Delivery: From Omnidirectional Video to Holography

Paper id 2120148
Paper id 2120148Paper id 2120148
Paper id 2120148IJRAT
 
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive StreamingMachine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningEkrem Çetinkaya
 
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentMinh Nguyen
 
MPEG-Immersive 3DoF+: 360 Video Streaming for Virtual Reality
MPEG-Immersive 3DoF+: 360 Video Streaming for Virtual RealityMPEG-Immersive 3DoF+: 360 Video Streaming for Virtual Reality
MPEG-Immersive 3DoF+: 360 Video Streaming for Virtual Realitymcslgachon
 
Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfVignesh V Menon
 
LwTE: Light-weight Transcoding at the Edge
LwTE: Light-weight Transcoding at the EdgeLwTE: Light-weight Transcoding at the Edge
LwTE: Light-weight Transcoding at the EdgeAlpen-Adria-Universität
 
An Overview on Multimedia Transcoding Techniques on Streaming Digital Contents
An Overview on Multimedia Transcoding Techniques on Streaming Digital ContentsAn Overview on Multimedia Transcoding Techniques on Streaming Digital Contents
An Overview on Multimedia Transcoding Techniques on Streaming Digital Contentsidescitation
 
Thesis arpan pal_gisfi
Thesis arpan pal_gisfiThesis arpan pal_gisfi
Thesis arpan pal_gisfiArpan Pal
 
IEEE_ICC'23_SARENA.pdf
IEEE_ICC'23_SARENA.pdfIEEE_ICC'23_SARENA.pdf
IEEE_ICC'23_SARENA.pdfReza Farahani
 
Flexible Transport of 3D Videos over Networks
Flexible Transport of 3D Videos over NetworksFlexible Transport of 3D Videos over Networks
Flexible Transport of 3D Videos over NetworksAhmed Hamza
 
IEEEGlobecom'22-OL-RICHTER.pdf
IEEEGlobecom'22-OL-RICHTER.pdfIEEEGlobecom'22-OL-RICHTER.pdf
IEEEGlobecom'22-OL-RICHTER.pdfReza Farahani
 
TB-Survey-2020.pdf
TB-Survey-2020.pdfTB-Survey-2020.pdf
TB-Survey-2020.pdfssuser50a5ec
 
Evaluation of bandwidth performance for interactive spherical video
Evaluation of bandwidth performance for interactive spherical videoEvaluation of bandwidth performance for interactive spherical video
Evaluation of bandwidth performance for interactive spherical videoAlpen-Adria-Universität
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...Minh Nguyen
 
IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...
IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...
IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...Reza Farahani
 

Similar to Immersive Video Delivery: From Omnidirectional Video to Holography (20)

Paper id 2120148
Paper id 2120148Paper id 2120148
Paper id 2120148
 
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive StreamingMachine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
 
MPEG-Immersive 3DoF+: 360 Video Streaming for Virtual Reality
MPEG-Immersive 3DoF+: 360 Video Streaming for Virtual RealityMPEG-Immersive 3DoF+: 360 Video Streaming for Virtual Reality
MPEG-Immersive 3DoF+: 360 Video Streaming for Virtual Reality
 
Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive Streaming
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
LwTE: Light-weight Transcoding at the Edge
LwTE: Light-weight Transcoding at the EdgeLwTE: Light-weight Transcoding at the Edge
LwTE: Light-weight Transcoding at the Edge
 
An Overview on Multimedia Transcoding Techniques on Streaming Digital Contents
An Overview on Multimedia Transcoding Techniques on Streaming Digital ContentsAn Overview on Multimedia Transcoding Techniques on Streaming Digital Contents
An Overview on Multimedia Transcoding Techniques on Streaming Digital Contents
 
Thesis arpan pal_gisfi
Thesis arpan pal_gisfiThesis arpan pal_gisfi
Thesis arpan pal_gisfi
 
IEEE_ICC'23_SARENA.pdf
IEEE_ICC'23_SARENA.pdfIEEE_ICC'23_SARENA.pdf
IEEE_ICC'23_SARENA.pdf
 
904072
904072904072
904072
 
Flexible Transport of 3D Videos over Networks
Flexible Transport of 3D Videos over NetworksFlexible Transport of 3D Videos over Networks
Flexible Transport of 3D Videos over Networks
 
IEEEGlobecom'22-OL-RICHTER.pdf
IEEEGlobecom'22-OL-RICHTER.pdfIEEEGlobecom'22-OL-RICHTER.pdf
IEEEGlobecom'22-OL-RICHTER.pdf
 
TB-Survey-2020.pdf
TB-Survey-2020.pdfTB-Survey-2020.pdf
TB-Survey-2020.pdf
 
Evaluation of bandwidth performance for interactive spherical video
Evaluation of bandwidth performance for interactive spherical videoEvaluation of bandwidth performance for interactive spherical video
Evaluation of bandwidth performance for interactive spherical video
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
 
MPEG-DASH open source tools and cloud services
MPEG-DASH open source tools and cloud servicesMPEG-DASH open source tools and cloud services
MPEG-DASH open source tools and cloud services
 
IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...
IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...
IEEE ICC'22_ LEADER_ A Collaborative Edge- and SDN-Assisted Framework for HTT...
 
What’s new in MPEG?
What’s new in MPEG?What’s new in MPEG?
What’s new in MPEG?
 

More from Alpen-Adria-Universität

Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...Alpen-Adria-Universität
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...Alpen-Adria-Universität
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Alpen-Adria-Universität
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Alpen-Adria-Universität
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamAlpen-Adria-Universität
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingAlpen-Adria-Universität
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentAlpen-Adria-Universität
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...Alpen-Adria-Universität
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesAlpen-Adria-Universität
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Alpen-Adria-Universität
 
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...Alpen-Adria-Universität
 
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumMPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumAlpen-Adria-Universität
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsOTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsAlpen-Adria-Universität
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingAlpen-Adria-Universität
 
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingOPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingAlpen-Adria-Universität
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingPerceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingAlpen-Adria-Universität
 

More from Alpen-Adria-Universität (20)

Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
 
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
 
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumMPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
 
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsOTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
 
An Introduction to OMNeT++ 6.0
An Introduction to OMNeT++ 6.0An Introduction to OMNeT++ 6.0
An Introduction to OMNeT++ 6.0
 
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingOPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingPerceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
 

Recently uploaded

"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys Vasyliev"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys VasylievFwdays
 
Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?MENGSAYLOEM1
 
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfOTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfPaige Cruz
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Adrian Sanabria
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docxLeveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docxVotarikari Shravan
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
From Challenger to Champion: How SpiraPlan Outperforms JIRA+Plugins
From Challenger to Champion: How SpiraPlan Outperforms JIRA+PluginsFrom Challenger to Champion: How SpiraPlan Outperforms JIRA+Plugins
From Challenger to Champion: How SpiraPlan Outperforms JIRA+PluginsInflectra
 
Q1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXL
Q1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXLQ1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXL
Q1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXLMemory Fabric Forum
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupMemory Fabric Forum
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfDomotica daVinci
 
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Umar Saif
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, GoogleISPMAIndia
 
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEDNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEandreiandasan
 
LF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIELF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIEDanBrown980551
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!XfilesPro
 
H3 Platform CXL Solution_Memory Fabric Forum.pptx
H3 Platform CXL Solution_Memory Fabric Forum.pptxH3 Platform CXL Solution_Memory Fabric Forum.pptx
H3 Platform CXL Solution_Memory Fabric Forum.pptxMemory Fabric Forum
 
Zi-Stick UBS Dongle ZIgbee from Aeotec manual
Zi-Stick UBS Dongle ZIgbee from  Aeotec manualZi-Stick UBS Dongle ZIgbee from  Aeotec manual
Zi-Stick UBS Dongle ZIgbee from Aeotec manualDomotica daVinci
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stackSummit
 

Recently uploaded (20)

"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys Vasyliev"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys Vasyliev
 
Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?
 
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdfOTel Orientation_ How to Train Teams (OTel in Practice).pdf
OTel Orientation_ How to Train Teams (OTel in Practice).pdf
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docxLeveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
From Challenger to Champion: How SpiraPlan Outperforms JIRA+Plugins
From Challenger to Champion: How SpiraPlan Outperforms JIRA+PluginsFrom Challenger to Champion: How SpiraPlan Outperforms JIRA+Plugins
From Challenger to Champion: How SpiraPlan Outperforms JIRA+Plugins
 
Q1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXL
Q1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXLQ1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXL
Q1 Memory Fabric Forum: Memory Processor Interface 2023, Focus on CXL
 
Q1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product LineupQ1 Memory Fabric Forum: SMART CXL Product Lineup
Q1 Memory Fabric Forum: SMART CXL Product Lineup
 
Curtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdfCurtain Module Manual Zigbee Neo CS01-1C.pdf
Curtain Module Manual Zigbee Neo CS01-1C.pdf
 
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
 
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFEDNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
DNA LIGASE BIOTECHNOLOGY BIOLOGY STUDY OF LIFE
 
LF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIELF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIE
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
 
H3 Platform CXL Solution_Memory Fabric Forum.pptx
H3 Platform CXL Solution_Memory Fabric Forum.pptxH3 Platform CXL Solution_Memory Fabric Forum.pptx
H3 Platform CXL Solution_Memory Fabric Forum.pptx
 
Zi-Stick UBS Dongle ZIgbee from Aeotec manual
Zi-Stick UBS Dongle ZIgbee from  Aeotec manualZi-Stick UBS Dongle ZIgbee from  Aeotec manual
Zi-Stick UBS Dongle ZIgbee from Aeotec manual
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stack
 

Immersive Video Delivery: From Omnidirectional Video to Holography

  • 1. Immersive Video Delivery: From Omnidirectional Video to Holography Christian Timmerer, Univ.-Prof. at AAU, Director at CD Lab ATHENA Klagenfurt, Austria Acknowledgments: Jeroen van der Hooft, Hadi Amirpour, Maria Torres Vega, Yago Sanchez, Raimund Schatz, Thomas Schierl March 14, 2023 1
  • 2. “In the first half of 2022, video accounted for a hefty 65.93% of total volume over the Internet. That’s a 24% increase over H1 2021.” The Global Internet Phenomena Report 2023 2 Source: https://twitter.com/StarWarsAUNZ/status/657841954139471872
  • 3. Presenter Christian Timmerer Univ.-Prof. at Alpen-Adria-Universität Klagenfurt Director CD Lab ATHENA CIO | Head of Research and Standardization at Bitmovin 3 3 2003: MSc CS (Dipl.-Ing.) 2006: PhD CS (Dr.-techn.) 2012: Co-founded Bitmovin 2014: Habilitation (Priv.-Doz.) & Assoc. Prof. 2016: Dep. Director @ ITEC/AAU 2019: Director @ ATHENA 2022: Univ.-Prof. for Multimedia Systems Web: http://timmerer.com/ Bitmovin MPEG-DASH
  • 4. 4 4
  • 5. ● Introduction ● Background ● Immersive Video Delivery Chain ● 3DoF Omnidirectional Video ● 6DoF Volumetric Video ● 6DoF Imagery Video ● Outlook and Open Challenges Agenda 5
  • 6. Motivation Sources: * Sandvine Global Internet Phenomena (January 2023). ** Cisco Annual Internet Report (2018–2023) White Paper (March 2020) 6 6 Video streaming is dominating today’s Internet traffic ● 2022: 65.93%; Netflix 13.74%, YouTube 10.51%, Disney+ 4.2%, Tik Tok 3.55%, Amazon Prime 2.67%* ● Video and other applications continue to be of enormous demand in today’s home, but there will be significant bandwidth demands with the application requirements of the future**
  • 7. HTTP Adaptive Streaming 101 Adaptation logic is within the client, not normatively specified by a standard, subject to research and development 7 7 Client
  • 8. Multimedia Systems Challenges and Tradeoffs 8 8 Basic figure by Klara Nahrstedt, University of Illinois at Urbana–Champaign, IEEE MIPR 2018
  • 9. “Application-oriented basic research” to address current and future research and deployment challenges of HAS and emerging streaming methods ATHENA – Adaptive Streaming over HTTP and Emerging Networked Multimedia Services Content Provisioning Content Delivery Content Consumption End-to-End Aspects ● Video encoding for HAS ● Quality-aware encoding ● Learning-based encoding ● Multi-codec HAS ● Edge computing ● Information CDN/SDN⇿clients ● Netw. assistance for/by clients ● Utility evaluation ● Bitrate adaptation schemes ● Playback improvements ● Context and user awareness ● Quality of Experience (QoE) studies ● Application/transport layer enhancements ● Quality of Experience (QoE) models ● Low-latency HAS ● Learning-based HAS https://athena.itec.aau.at/ 9 9 Funding:
  • 10. ATHENA Selected Open Source Tools 10 10 LLL-CAdViSE ● Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation ● DASH, HLS, CTE, CMAF, ITU-T P.1203 https://athena.itec.aau.at/ Video Complexity Analyzer v2.0 ● Spatial and temporal complexity metric ● Support for HDR ● Low-pass DCT optimizations
  • 11. ● Immersion “the fact of becoming completely involved in something” https://dictionary. cambridge.org/dictionary/english/immersion ● Immersive Video Deeper viewer involvement in the content – be it in a story, a game, or a remote conversation ● 3DoF vs. 6DoF Six mechanical degrees of freedom vs. rotational motion only: pitch, yaw, and roll Background on Immersive Video (1) 11 11 Source: https://en.wikipedia.org/wiki/ Six_degrees_of_freedom
  • 12. ● Omnidirectional Video 1843 – Joseph Puchberger of Retz, Austria patents first 150-degree camera Today – handheld devices capable of live streaming ● Volumetric Video Objects: point-clouds and meshes ● Light Fields and Holography Image-based solutions for immersive video ● Application domains Healthcare, education, entertainment, remote conferencing Background on Immersive Video (2) 12 12
  • 14. Immersive Delivery Chain (2) 14 14 End-to-End Latency Source: https://www.wowza.com/low-latency
  • 15. Capturing ● Multiple cameras mounted on a sphere Camera views are stitched together into a single sphere: mirror-based, depth- aware stitching, and depth-enabled light field rendering Processing and Compression ● Sphere-to-plane projection mapping Equirectangular projection (ERP) ● Viewport-adaptive coding and transmission Pyramid, truncated pyramid, multi-resolution CMP 3DoF Omnidirectional Video (1) 15 15
  • 16. Transmission: VoD ● Viewport-(In)dependent delivery Agnostic to the users viewing orientation vs. viewport-dependent projections and tile-based encoding ● Components Rate adaptation, viewport prediction, saliency detection, and random access Transmission: Live ● Technical options Adopting HTTP Adaptive Streaming, parameter tuning, low-latency, and partially reliable delivery 3DoF Omnidirectional Video (2) 16 16
  • 17. In-network Optimizations ● Multi-access edge computing (MEC) Offload compute-intensive tasks to the edge Perception ● Quality of Experience The degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state. 3DoF Omnidirectional Video (3) 17 17
  • 18. Capturing and Processing ● Cameras Light detection and ranging (LiDAR)-based cameras Special camera rig setups with various devices Compression ● Point Clouds MPEG’s Video-based and Geometry-based Point Cloud Compression – https://mpeg-pcc.org/ ● Meshes For rendering only (otherwise point clouds) or meshes also for encoding and decoding 6DoF Volumetric Video (1) 18 18
  • 19. Transmission ● VoD and Live Viewport-aware object-based streaming ● In-Network Optimizations Cloud- or edge-enabled immersive media streaming Perception ● Point Clouds Predefined movement paths, emulated network conditions, and viewport prediction methods ● Meshes Meshes offer higher visual quality at larger bitrates than point clouds. At lower bitrates, however, point clouds outperform meshes. 6DoF Volumetric Video (2) 19 19
  • 20. Capturing ● Light Fields 𝐿(𝑢, 𝑣, 𝑠, 𝑡) – object & image planes Relatively easy to capture/process but with ample storage and bandwidth requirements ● Holograms Optically generated hologram (OGH) vs. computer-generated hologram (CGH) Compression ● Light Fields Transform-, prediction-, and learning-based compression (incl. frame interpolation) ● Holograms JPEG Pleno standard framework for representing new imaging modalities, such as texture-plus-depth, light field, point cloud, and holographic imaging 6DoF Imagery Video (1) 20 20
  • 21. Transmission ● Video On Demand Viewport scalability, quality scalability, spatial scalability, random access, and uniform quality distribution while maintaining high compression efficiency Rate adaptation; viewport prediction; saliency detection Rendering and Perception ● Quality Evaluation Light field multi-view images are converted into a pseudo-video and assessed by subjects vs. interactive setup to evaluate the quality of the light field 6DoF Imagery Video (2) 21 21
  • 22. Outlook and Open Challenges (1) 22 22
  • 23. Low-Latency Content Delivery ● Low-Latency Video Streaming Protocols LL-DASH, WebRTC, Media over QUIC (MOQ), 5G/6G Improved In-Network Solutions ● Software-Defined Networking Segment routing, Virtual Network Function (VNF), machine learning, media processing within the network/at the edge Scalable and Portable Capturing Devices ● High-Quality Volumetric Videos Professional camera setups to be properly calibrated, lighting is properly set, and potentially adapting to changes Outlook and Open Challenges (2) 23 23
  • 24. Increased Compression Performance ● MPEG Immersive Video (MIV) Video-based visual volumetric coding (V3C), bitstream format common with V-PCC – https://mpeg-miv.org/ Evaluation of the User’s Perception ● Less Intrusive Alternative Assessment Methods Behavioral, psycho-physiological (EEG), electrocardiogram, eye tracking, etc. Subjective Assessment Benchmark ● Analog to the Turing test known from the Artificial Intelligence (AI) domain Determine a system’s ability to provide an immersive experience that is indistinguishable from reality Outlook and Open Challenges (3) 24 24
  • 25. Thank you for your attention 25
  • 26. 26 26