SlideShare a Scribd company logo
Video Streaming: Then, Now, Future
Christian Timmerer, Professor at AAU, Director at CD Lab ATHENA
Klagenfurt, Austria
June 5, 2024
1
https://www.slideshare.net/christian.timmerer
● Introduction / motivation
● Early days of video streaming + selected projects
● Toward HTTP adaptive streaming + MPEG-DASH
● ATHENA project
● Emerging research topics
Agenda
2
Introduction / Motivation
Sources:
* Sandvine Global Internet Phenomena (January 2024).
** Cisco Annual Internet Report (2018–2023) White Paper (March 2020)
3
3
Video streaming is dominating today’s
Internet traffic
● 2024*: 68% (fixed) and 64% (mobile)
● Video on-demand: 54% / 57% – Live: 14% / 7%
● Main applications: YouTube, Netflix (>10%),
Tik Tok, Amazon Prime, Disney+ (<10%)
● Video and other applications continue to be of
enormous demand in today’s home, but there will
be significant bandwidth demands with the
application requirements of the future**
Multimedia Systems Challenges and Tradeoffs
4
4
Basic figure by Klara Nahrstedt, University of Illinois at Urbana–Champaign, IEEE MIPR 2018
Evolution of Video
5
5
Era of streaming
Yuriy Reznik, Christian Timmerer, 20 Years of Streaming in 20 Minutes, Mile-High Video
2020, https://athena.itec.aau.at/2020/11/20-years-of-streaming-in-20-minutes/
● 1993: Multicast Backbone (MBONE)
○ Virtual multicast network connecting several universities & ISPs
○ RTP-based video conferencing tool (vic) is used to stream videos
○ 1994 Rolling Stones concert – first major event streamed online
● 1995: RealAudio, 1997: RealVideo
○ First commercially successful mass-scale streaming system
○ Proprietary protocols, codecs: PNA, RealAudio, RealVideo
○ Worked over UDP, TCP, and HTTP (“cloaking” mode)
○ First major broadcast: 1995 Seattle Mariners vs New York
Yankees
● 1996: VDOnet, Vivo, NetShow, VXtreme, ...
○ Many vendors have competed in streaming space initially
○ Vivo & Xing have been acquired by Real, VXtreme by Microsoft
○ By 1998, 3 main vendors remained: Real, Microsoft and Apple
● 1998: RealSystem G2
○ First Adaptive BitRate (ABR) streaming system
Early Streaming Systems
6
6
Yuriy Reznik, Christian Timmerer, 20 Years of Streaming in 20 Minutes, Mile-High Video
2020, https://athena.itec.aau.at/2020/11/20-years-of-streaming-in-20-minutes/
1998: RealSystem G2, “SureStream” technology
● First commercially successful Adaptive BitRate (ABR) streaming system
● Encoder: Encoded streams: Player:
● References
○ B. Girod, et al, “Scalable codec architectures for Internet video-on-demand,” ACSSC, pp. 357 – 361, 1997.
○ G. Conklin, et al, “Video Coding for Streaming Media Delivery on the Internet," TCSVT, 11 (3), pp. 20-34, 2001
○ US Patents: 6314466, 6480541, 7075986, 7885340
First ABR Streaming System
7
7
Multi-rate
packaging option
Selection of
streams to include
Panel showing which
stream is selected
Yuriy Reznik, Christian Timmerer, 20 Years of Streaming in 20 Minutes, Mile-High Video
2020, https://athena.itec.aau.at/2020/11/20-years-of-streaming-in-20-minutes/
Real Time Streaming Protocol (RTSP) et al., Internet Streaming Media
Alliance (ISMA), 3GPP Packet-Switched Streaming Service (PSS)
Early Streaming Standards
8
8
Server
DESCRIBE rtsp://example.com/mov.test RTSP/1.0
Session Description Protocol
SETUP rtsp://example.com/mov.test/streamID=0 RTSP/1.0
3GPP-Adaptation:url=
“rtsp://example.com/mov.test/streamID=0”;size=20000;target-time=5000
3GPP-Link-Char: url=“rtsp://example.com/mov.test/streamID=0”; GBW=32
RTSP OK
Client
Time (s)
PLAY rtsp://example.com/mov.test RTSP/1.0
RTSP OK
Real-time Transport Protocol
RTCP Reports
Setup
Session
Streaming Client Buffer
Client Buffer
Model
● ADMITS (Adaptation in Distributed Multimedia IT Systems)
Media servers, meta-databases, proxies, routers, clients: which kind
of component can be best used for what kind of adaptation?
● IST-FP6: DANAE (Dynamic and Distributed Adaptation of
scalable multimedia coNtent in a context-Aware Environment)
MPEG-21-based adaptation framework: adaptation decisions –
usage environment – adapting multimedia content
● IST-FP6: ENTHRONE (End-to-End QoS through Integrated
Management of Content, Networks and Terminals)
ENTHRONE Integrated Management Supervisor (EIMS): considers all
stakeholder requirements – end-to-end management solution
ensuring QoS for the end user.
● MPEG-21 Multimedia Framework
Digital Item (Adaptation) – ISO/IEC 21000-7
Selected Projects at ITEC (1)
9
9
2002
-
2004
2004
-
2006
2006
-
2008
A. Vetro and C. Timmerer, "Digital Item Adaptation:
Overview of Standardization and Research Activities," in
IEEE Transactions on Multimedia, vol. 7, no. 3, pp. 418-426,
June 2005, doi: 10.1109/TMM.2005.846795.
● Consumption of multimedia content may also stimulate
other senses
Vision or audition
Olfaction, mechanoreception, equilibrioception,
thermoception, …
● Annotation with metadata providing so-called sensory
effects that steer appropriate devices capable of
rendering these effects
Excursus: Sensory Effects and MPEG-V
10
10
MPEG-V Building Blocks
Sensory Effect Description
Language (SEDL)
Sensory Effect Vocabulary
(SEV)
Sensory Effect Metadata
(SEM)
ISO/IEC 23005-3
G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver, "Mulsemedia:
State of the Art, Perspectives, and Challenges", ACM Trans.
Multimedia Comput. Commun. Appl. 11, 1s, Article 17
(September 2014), doi:10.1145/2617994.
● ICT-FP7: ALICANTE (Media Ecosystem
Deployment Through Ubiquitous
Content-Aware Network Environments)
Content-aware network processing will allow the
underlying infrastructure to identify, process, and
manipulate media streams and objects in real
time to maximize quality of service and
experience.
● Supports different streaming formats/protocols
RT(S)P, P2P, and HTTP
Selected Projects at ITEC (2)
11
11
M. Grafl, C. Timmerer, et al., "Scalable Media Coding Enabling
Content-Aware Networking," in IEEE MultiMedia, vol. 20, no. 2,
pp. 30-41, April-June 2013, doi: 10.1109/MMUL.2012.57.
2010
-
2013
Video Delivery over HTTP/TCP
12
12
○ Enables
playback while
still downloading
○ Server sends the
file as fast as
possible
Progressive
Download
○ Enables seeking
via media indexing
○ Server paces
transmission based
on encoding rate
Pseudo
Streaming
○ Content is divided
into short-duration
chunks
○ Enables live
streaming and ad
insertion
Chunked
Streaming
○ Multiple versions
of the content are
created
○ Enables to adapt
to network and
device conditions
Adaptive
Streaming
Progressive Download
13
13
HTTP Request
HTTP Response
Playback starts only after there
is several seconds of data in the
playback buffer
Download will continue as fast as
possible
Fetched content will be wasted if
the viewer clicks away
Can seek only throughout the
fetched content
Bing Wang, Jim Kurose, Prashant Shenoy, and Don Towsley. 2004. Multimedia streaming via TCP: an analytic performance study.
ACM International Conference on Multimedia (MM'04). DOI:https://doi.org/10.1145/1027527.1027735
“TCP generally provides good
streaming performance when
the achievable TCP
throughput is roughly twice
the media bitrate, with only a
few seconds of startup delay”
14
14
Streaming is cooler, more viewer friendly
15
15
Playback starts when there
is just few seconds of data
Download rate will match the encoding
bitrate and downloading pauses if the
player pauses → Less waste
Can seek to anywhere in
the entire content
L. De Cicco, S. Mascolo. An Experimental Investigation of the Akamai Adaptive Video Streaming. in Proc of USAB
2010, special session Interactive Multimedia Applications (WIMA), Klagenfurt, Austria, 3-4 November 2010, LCNS 6389,
pp. 447-464, Springer-Verlag, doi:10.1007/978-3-642-16607-5
Implements “pseudo streaming”
and mimics RTSP-style streaming
but via HTTP.
HTTP Adaptive Streaming 101
Adaptation logic is within the
client, not normatively specified
by a standard, subject to
research and development
16
16
Client
Bitrate Adaptation Schemes
17
17
Bitrate
Adaptation
Schemes
Client-based
Adaptation
Bandwidth
-based
Buffer-
based
Mixed
adaptation
Proprietary
solutions
MDP-based
Server-based
Adaptation
Network-
assisted
Adaptation
Hybrid
Adaptation
SDN-based
Server and
network-
assisted
A. Bentaleb, B. Taani, A. C. Begen, C. Timmerer and R. Zimmermann, "A Survey on Bitrate Adaptation Schemes for
Streaming Media Over HTTP," in IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 562-585, Firstquarter 2019.
doi: 10.1109/COMST.2018.2862938
MPD
Period id = 1
start = 0 s
Period id = 3
start = 300 s
Period id = 4
start = 850 s
Period id = 2
start = 100 s
Splicing of arbitrary
content like ads
MPEG DASH Data Model (ISO/IEC 23009-1)
Period id = 2
start = 100 s
Adaptation Set 0
subtitle turkish
Adaptation Set 2
audio english
Adaptation Set 3
audio german
Adaptation Set 1
video
Selection of
components/tracks
Adaptation Set 1
BaseURL=http://abr.rocks.com/
Representation 2
Rate = 1 Mbps
Representation 4
Rate = 3 Mbps
Representation 1
Rate = 500 Kbps
Representation 3
Rate = 2 Mbps
Selection of
representations
18
18
Representation 3
Rate = 2 Mbps
Resolution = 720p
Segment Info
Duration = 10 s
Template:
3/$Number$.mp4
Well-defined
media format
Segment Access
Initialization
Segment
http://abr.rocks.com/3/0.mp4
Media Segment 1
start = 0 s
http://abr.rocks.com/3/1.mp4
Media Segment 2
start = 10 s
http://abr.rocks.com/3/2.mp4
Chunks with
addresses and timing
19
19
“Application-oriented basic research” to address current and future research
and deployment challenges of HAS and emerging streaming methods
ATHENA – Adaptive Streaming over HTTP and
Emerging Networked Multimedia Services
Content Provisioning Content Delivery Content Consumption
End-to-End Aspects
● Video encoding for HAS
● Quality-aware encoding
● Learning-based encoding
● Multi-codec HAS
● Edge computing
● Information CDN/SDN⇿clients
● Netw. assistance for/by clients
● Utility evaluation
● Bitrate adaptation schemes
● Playback improvements
● Context and user awareness
● Quality of Experience (QoE) studies
● Application/transport layer enhancements
● Quality of Experience (QoE) models
● Low-latency HAS
● Learning-based HAS
https://athena.itec.aau.at/
20
20
Funding:
Hermann Hellwagner, Advances in Edge-Based and In-Network Media
Processing for Adaptive Video Streaming, Keynote at IEEE MIPR’23, Singapore.
Related
FFG-funded
projects:
AdvUHD-DASH,
PROMETHEUS,
APOLLO, GAIA
Problem Statement / Basic Idea for Live Streaming
● Static bitrate ladder
Pre-defined bitrate/resolution pairs for all contents
● Per-title encoding
Optimize bitrate ladder based on the content
● UHD / HFR content
Increase of spatial and temporal resolutions
● Live (is Life)[* Opus, 1985]
Perceptual-based bitrate/resolution/framerate
prediction algorithm based on content complexity
Video Complexity Analysis (1)
21
21
VoD
Live
* … https://www.youtube.com/watch?v=pATX-lV0VFk
State of the Art: ITU-T Spatial Information / Temporal Information (SI/TI)
● The correlation with ground truth spatial and temporal complexity is low
Video Complexity Analysis (2)
22
22
Sobel Filter
Video Complexity Analyzer (VCA)
● VCA introduces DCT-based spatial-temporal complexity features
● Spatial complexity correlation has increased, but temporal complexity
remains low
Video Complexity Analysis (3)
23
23
Vignesh V Menon, Christian Feldmann, Hadi Amirpour, Mohammad Ghanbari, and Christian Timmerer. 2022.
VCA: video complexity analyzer. In Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22). Association for
Computing Machinery, New York, NY, USA, 259–264. https://athena.itec.aau.at/2022/04/vca-video-complexity-analyzer/
Enhanced Video Complexity Analyzer (EVCA)
● EVCA refines the definition of the temporal complexity feature
● Temporal complexity correlation has increased
Video Complexity Analysis (4)
24
24
H. Amirpour, M. Ghasempour, L. Qu, W. Hamidouche, and C. Timmerer. 2024. EVCA: Enhanced Video Complexity
Analyzer. In Proceedings of the 15th ACM Multimedia Systems Conference (MMSys '24). Association for
Computing Machinery, New York, NY, USA, 285–291. https://doi.org/10.1145/3625468.3652171
Deep Video Complexity Analyzer (DeepVCA)
Video Complexity Analysis (5)
25
25
H. Amirpour, K. Schoeffmann, M. Ghanbari and C. Timmerer, "DeepVCA: Deep Video Complexity Analyzer,"
in IEEE Transactions on Circuits and Systems for Video Technology, doi: 10.1109/TCSVT.2024.3382366.
https://athena.itec.aau.at/2024/04/ieee-tcsvt-deepvca-deep-video-complexity-analyzer/
E-WISH: A policy driven ABR Algorithm
26
Energy Cost
+ 𝛅
[Data Cost]
Data Cost
Total Cost = ɑ
Segment bitrate
Throughput
[Buffer Cost]
Buffer Cost
+ β
Download time
Buffer occupancy
[Quality Cost]
Quality Cost
+ ɣ
Video quality
Distortion
Instability
Bitrate, Resolution, FPS (↑)
M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for
HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia
Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605
Technology Transfer
E-WISH: An Energy-aware ABR Algorithm
27
[Data Cost] [Quality Cost]
[Buffer Cost] [Energy Cost]
Data Cost Buffer Cost Quality Cost Energy Cost
Total Cost = ɑ + β + ɣ + 𝛅
Segment bitrate
Throughput
Download time
Buffer occupancy
Video quality
Distortion
Instability
Bitrate, Resolution, FPS
M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for
HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia
Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605
D.Lorenzi, M. Nguyen, F. Tashtarian, and C. Timmerer, “E-WISH: An Energy-aware ABR Algorithm For
Green HTTP Adaptive Video Streaming”, 3rd ACM Mile-High Video Conference (MHV '24),
https://doi.org/10.1145/3638036.3640802
Live Video Streaming Pipeline
28
LIVE
Origin Server
(Encoding & Packaging)
Live Camera Player
CDN Server
manifest.mpd
Bitrate Resolution
5 Mbps 1920x1080
2.8 Mbps 1280x720
1.1 Mbps 960x540
System Admin
Bitrate
Ladder
HTTP Request
HTTP Response
Manifest includes representations:
different versions of the content optimized for various
playback conditions, e.g., different bitrates and resolutions
Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX
Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
● Fixed bitrate ladder
○ Content-aware, e.g., Netflix’s per-title encoding
○ Context-aware, e.g., [Lebreton and Yamagishi 2023]
○ Agnostic of the content and context (proprietary)
ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming
Select representations
with bitrates that closely
match the desired bitrates
Players
Advertise a large
number of
representations
Encoded
segments
CDN
Server
Mega-Manifest
29
Origin Server
OTL
Process requests
and video qualities
and determine OTL
LIVE
Live
Camera
Utilize the OTL
OTL: Optimized Temporary Ladder
Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX
Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
Quality of Experience (QoE) ...
● “... is the degree of delight or annoyance of the user of an application or service.
It results from the fulfillment of his or her expectations with respect to the utility and / or
enjoyment of the application or service in the light of the user’s personality and current state.”1)
● … can be easily extended to various domains, e.g., immersive media experiences.2)
30
30
1)
P. Le Callet, S. Möller, A. Perkis, et al. QUALINET White Paper on
Definitions of Quality of Experience. European Network on Quality of
Experience in Multimedia Systems and Services (COST Action IC 1003). 2012.
2)
A. Perkis, C. Timmerer, et al. 2020. QUALINET White Paper on Definitions
of Immersive Media Experience (IMEx). arXiv:2007.07032 [cs.MM]
3)
J. van der Hooft, T. Wauters, F. De Turck, C. Timmerer, and H. Hellwagner.
“Towards 6DoF HTTP Adaptive Streaming Through Point Cloud
Compression”. 27th ACM Int’l. Conf. on Multimedia (MM'19). Oct. 2019.
4)
J. van der Hooft, M. T. Vega, C. Timmerer, A. C. Begen, F. De Turck and R.
Schatz, "Objective and Subjective QoE Evaluation for Adaptive Point
Cloud Streaming," 2020 Twelfth International Conference on Quality of
Multimedia Experience (QoMEX), Athlone, Ireland, 2020.
3) 4)
2010 - 2014 - now
Immersive Video Delivery:
From Omnidirectional Video to Holography
31
31
J. v. d. Hooft, H. Amirpour, M. Torres Vega, Y. Sanchez, R. Schatz; T. Schierl, C. Timmerer, "A Tutorial on
Immersive Video Delivery: From Omnidirectional Video to Holography," in in IEEE Communications
Surveys & Tutorials, vol. 25, no. 2, pp. 1336-1375, Secondquarter 2023, doi: 10.1109/COMST.2023.3263252.
● 3DoF Omnidirectional Video
● 6DoF Volumetric Video
● 6DoF Imagery Video
NeRV: Neural Representations for Videos, NeurIPS 2021
Deep Learning-Based Video Coding: A Review and a
Case Study, ACM CSUR 2020
● Video Coding
Learned video coding / E2E deep video coding
Future of Video Streaming
32
32
An End-to-End Learning Framework for Video
Compression, TPAMI 2021
● Video Streaming
DeepStream: Video Streaming Enhancements
using Compressed Deep Neural Networks
Deep learning-based video coding
Neural implicit representation for videos
DeepStream: TCSVT 2022
Generative AI for video streaming
ATHENA team
Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem
Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari,
Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer,
Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan,
Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, Yiying Wei
(Inter-)National Collaborators (selection)
● Prof. Ali C. Begen, Ozyegin University, Turkey
● Prof. Filip De Turck, Ghent University – imec, Belgium
● Dr. Jeroen van der Hooft, Ghent University – imec, Belgium
● Dr. Maria Torres Vega, Ghent University – imec, Belgium
● Dr. Raimund Schatz, AIT Austrian Institute of Technology, Austria
● Prof. Roger Zimmermann, NUS, Singapore
● Dr. Abdelhak Bentaleb, Concordia University, Canada
● Prof. Christine Guillemot, INRIA, Rennes, France
● Prof. Patrick Le Callet, Polytech Nantes-Institut Universitaire de France, France
● Prof. Junchen Jiang, University of Chicago, USA
● Dr. Sergey Gorinsky, IMDEA, Spain
Acknowledgments
33
33
https://athena.itec.aau.at/
Co-Authors (2003-2023; cf. FODOK)
Abdelhak Bentaleb, Adam T. Lindsay, Adam Wieckowski, Adithyan Ilangovan, Adlen Ksentini, Ahmed Mehaoua, Alan Bertoni, Alberto Leon Martin, Alessandro Floris, Alex
Chernilov, Ali C. Begen, Alireza Erfanian, Amy Reibman, Anastasia Henkel, Anastasios Kourtis, Anatoliy Zabrovskiy, Andrea Detti, Andreas Hutter, Andreas Sackl, Andreas Schorr,
Andrew Perkis, Anne-Lore Mevel, Anthony Vetro, Antonio Pinto, Armin Trattnig, Arthur Lugmayr, Asim Hameed, Aytac Azgin, Babak Taraghi, Barnabás Takacs, Barry Crabtree,
Bayan Taani, Benjamin Bross, Benjamin Rainer, Bernd Girod, Bernhard Feiten, Brahim Allan, Bruno Gardlo, Carsten Griwodz, Cathrine Lamy-Bergot, Cedric Westphal, Chris
Needham, Christian Feldmann, Christian Keimel, Christian R. Helmrich, Christian Raffelsberger, Christian Sieber, Christine Guillemot, Christopher Müller, Cise Midoglu, Claudio
Alberti, Costas Troulos, Cyril Concolato, Daniel Corujo, Daniel Gatica-Perez, Daniel Hoelbling-Inzko, Daniel Negru, Daniel Posch, Daniel Silhavy, Daniel Weinberger, Daniele
Lorenzi, Daniele Renzi, Darragh Egan, David Wortley, Davide Rogai, Deepak Chaudhary, Detlev Marpe, Devillers Sylvain, Dietmar Jannach, Doncel V. Rodriguez, Dragi Kimovski,
Ekrem Cetinkaya, Emanuele Quacchio, Emre Karsli, Eric Delfosse, et al., Eugen Borcoci, Eva Cheng, Evangelos Pallis, Evgeny Kuzmin, Evgeny Petrov, Fabrizio S. Rovati, Fan
Chen, Farzad Tashtarian, Fernando López, Fernando Pereira, Filip De Turck, Filippo Chiariglione, Francesc Sanahuja, Franz J. Hauck, Gabriel Panis, Gabriele Panis, Gabriella
Olmo, George Gardikis, George Xilouris, Gheorghita Ghinea, Giovanni Cordara, Gülnaziye Bingöl, Hadi Amirpour, Haichao Yao, Hamid Hadian, Hannaneh Barahouei Pasandi,
Haral Kosch, Harald Kosch, Harilaos. Koumaras, Harold Vincent Poor, He Xu, Hemanth Kumar Ravuri, Henrique Guilhon de Castro, Hermann Hellwagner, Hongjie He, Hui Lu, Ian
Burnett, Ingo KofIer, Ingo Wolf, Ioannis Kompatsiaris, Irene Viola, Ismail Djama, Jaime Delgado, Jean Gelissen, Jean Le Feuvre, Jean-Charles Point, Jeroen van der Hooft, Jesus
Aguilar Armijo, Jianping Wang, Jim Hurley, Jingwen Zhu, Joachim Seidl, Johannes Jaborning, Johannes Liegl, John Barrett, John R. Smith, Jonathan Ishmael, Joost Geurts, Jörg
Heuer, Jörg Pöcher, Jose Martínez, Karel Fliegel, Karsten Grüneberg, Karsten Müller, Katharina Gassner, Keith Mitchell, Klaus Leopold, Klaus Schöffmann, Laszlo Szobonya,
Leonardo Peroni, Li Fang, Lingfeng Qu, Lu Yu, Luca Celetto, Luigi Atzori, Mamadou Sidibé, Marco Mattavelli, Maria Teresa Andrade, Maria Torres Vega, Marie-Jose Montpetit,
Mario Graf, Mario Taschwer, Marius Preda, Mark Stuart, Markus Waltl, Marta Larson, Martin Hahn, Martin Prangl, Martin Smole, Mathias Wien, Matteo Maiero, Matthias Hirth,
Matthias Klusch, May Lim, Michael Eberhard, Michael Grafl, Michael Mackay, Michael Ransburg, Miguel Gómez, Mike Buckham, Minh Nguyen, Mohammad Ghanbari, Mohammad
Hosseini, Mohammad Shojafar, Muawiyath Shujau, Myriam Amielh, Nakisa Shams, Narciso García, Narges Mehran, Negin Ghamsarian, Niall Murray, Nikos Zotos, Ningxiong Mao,
Ozgu Alay, Pablo Cesar, Pascal Frossard, Patrick Kapahnke, Patrick Le Callet, Pavel Korshunov, Pedro Carvalho, Pedro Souto, Peng Chen, Peter De Neve, Peter Lederer,
Peyman Sheikholharam Mashhadi, Philippe Hanhart, Phuoc Tran-Gia, Prajit T Rajendran, Prateek Agrawal, Radu Aurel Prodan, Raimund Schatz, Ralf Terlutter, Reinhard Grandl,
Reyhane Falanji, Reza Farahani, Reza Saeedinia, Reza Shokri Kalan, Ricardo Petrocco, Richard Marsh, Rik Van de Walle, Robby De Sutter, Roger Zimmermann, Roland
Kersche, Roland Mathá, Rongrong Ni, S. Omid Fatemi, Saad Harous, Sam Dutton, Sam Van Damme, Samira Afzal, Sandro Linder, Sean Brennan, Selina Zoe Haack, Serge
Lerouge, Sergey Gorinsky, Shivi Vats, Simone Ferlin-Reiter, Simone Milani, Simone Porcu, Stefan Lederer, Stefan Mangold, Stefan Petscharnig, Stefan Pham, Stefano Battista,
Stephen Davis, Stephen Gulliver, Steve Lerouge, Robby De Sutter, Sylvain Devillers, Tamer Nadeem, Teodora Guenkova-Luy, Theodore Zahariadis, Thomas DeMartini, Thomas
Frank, Thomas Gottron, Thomas Schierl, Thomas Stockhammer, Thomas Zinner, Tibor Szkaliczki, Tim Stevens, Tim Wauters, Tobias Hossfeld, Tommaso Melodia, Toufik Ahmed,
Touradj Ebrahimi, Vaios Koumaras, Venkata Phani Kumar Malladi, Victor Doncel, Vignesh Menon, Vishu Madaan, Vladislav Kashanskii, Wael Cherif, Weisi Lin, Wen Ji, Wenwu
Zhu, Will Shucheng Liu, Xiaoqi Cao, Xuemei Zhou, Yael Lapid, Yago Sanchez, Yaning Liu, Yao Zhao, Yaolin Yang, Yuan Yuan, Yuansong Qiao, Zahra Najafabadi Samani,
Zhengdao Zhan, Zhu Li
Acknowledgments
34
34
Current Staff at ITEC
Afzal Samira, Amirpour Hadi, Artioli Emanuele, Azimi Ourimi Zoha, Bauer Christian, Farahani
Reza, Ghasempour Mohammad, Harshina Kseniia, Hellwagner Hermann, Hoi Manuel Elias,
Horvath Kurt Klaus, Isak Selina, Kimovski Dragi, Lachini Armin, Lorenzi Daniele, Lux Mathias,
Nasirihaghighi Sahar, Prodan Radu Aurel, Qureshi Kamran, Rader Martin, Schniz Felix,
Schöffmann Klaus, Seyeddizaji Seyedehhaleh, Tashtarian Farzad, Tuček Tom, Uitz Sebastian,
Vats Shivi, Wei Yiying, Gliesche Anna Julia, Letter Margit, Messner Rudolf, Steinbacher
Martina, Taschwer Mario, Kowatsch Paul, Leitinger Claudia, Leopold Mario, Strobl Magdalena
+ Alumni: https://itec.aau.at/alumni/
Acknowledgments
35
35
Thank you for your attention
36

More Related Content

Similar to Video Streaming: Then, Now, and in the Future

Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive Streaming
Alpen-Adria-Universität
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
Vignesh V Menon
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
Minh Nguyen
 
Online Video and Next-gen Storage
Online Video and Next-gen StorageOnline Video and Next-gen Storage
Online Video and Next-gen Storage
Jim Kaskade
 
11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...
11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...
11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...
Alexander Decker
 
Performance evaluation of mpeg 4 video transmission over ip-networks
Performance evaluation of mpeg 4 video transmission over ip-networksPerformance evaluation of mpeg 4 video transmission over ip-networks
Performance evaluation of mpeg 4 video transmission over ip-networks
Alexander Decker
 
PEMWN'21 - ANGELA
PEMWN'21 - ANGELAPEMWN'21 - ANGELA
PEMWN'21 - ANGELA
Jesus Aguilar
 
Digital Industry Standards
Digital Industry StandardsDigital Industry Standards
Digital Industry Standards
Chuck Gary
 
SDI to IP 2110 Transition Part 1
SDI to IP 2110 Transition Part 1SDI to IP 2110 Transition Part 1
SDI to IP 2110 Transition Part 1
Dr. Mohieddin Moradi
 
Prashant Resume
Prashant ResumePrashant Resume
Prashant Resume
Prashant Desai
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Alpen-Adria-Universität
 
USuurey_Presentation__CollaborativeHASSystems.pdf
USuurey_Presentation__CollaborativeHASSystems.pdfUSuurey_Presentation__CollaborativeHASSystems.pdf
USuurey_Presentation__CollaborativeHASSystems.pdf
Reza Farahani
 
chapter15.ppt
chapter15.pptchapter15.ppt
chapter15.ppt
AdityaAggarwal93
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Alpen-Adria-Universität
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
Alpen-Adria-Universität
 
(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...
(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...
(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...
Naoki Shibata
 
MMSys'21 DS- RezaFarahani.pdf
MMSys'21 DS- RezaFarahani.pdfMMSys'21 DS- RezaFarahani.pdf
MMSys'21 DS- RezaFarahani.pdf
Reza Farahani
 
proposal
proposalproposal
proposal
Videoguy
 
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingMMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
Jesus Aguilar
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
Alpen-Adria-Universität
 

Similar to Video Streaming: Then, Now, and in the Future (20)

Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive Streaming
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
 
Online Video and Next-gen Storage
Online Video and Next-gen StorageOnline Video and Next-gen Storage
Online Video and Next-gen Storage
 
11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...
11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...
11.performance evaluation of mpeg 0004www.iiste.org call for-paper video tran...
 
Performance evaluation of mpeg 4 video transmission over ip-networks
Performance evaluation of mpeg 4 video transmission over ip-networksPerformance evaluation of mpeg 4 video transmission over ip-networks
Performance evaluation of mpeg 4 video transmission over ip-networks
 
PEMWN'21 - ANGELA
PEMWN'21 - ANGELAPEMWN'21 - ANGELA
PEMWN'21 - ANGELA
 
Digital Industry Standards
Digital Industry StandardsDigital Industry Standards
Digital Industry Standards
 
SDI to IP 2110 Transition Part 1
SDI to IP 2110 Transition Part 1SDI to IP 2110 Transition Part 1
SDI to IP 2110 Transition Part 1
 
Prashant Resume
Prashant ResumePrashant Resume
Prashant Resume
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
 
USuurey_Presentation__CollaborativeHASSystems.pdf
USuurey_Presentation__CollaborativeHASSystems.pdfUSuurey_Presentation__CollaborativeHASSystems.pdf
USuurey_Presentation__CollaborativeHASSystems.pdf
 
chapter15.ppt
chapter15.pptchapter15.ppt
chapter15.ppt
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
 
(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...
(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...
(Slides) P2P video broadcast based on per-peer transcoding and its evaluatio...
 
MMSys'21 DS- RezaFarahani.pdf
MMSys'21 DS- RezaFarahani.pdfMMSys'21 DS- RezaFarahani.pdf
MMSys'21 DS- RezaFarahani.pdf
 
proposal
proposalproposal
proposal
 
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streamingMMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
MMSys'21 - Multi-access edge computing for adaptive bitrate video streaming
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
 

More from Alpen-Adria-Universität

Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instancesVEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
Alpen-Adria-Universität
 
GREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video ProcessingGREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video Processing
Alpen-Adria-Universität
 
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Alpen-Adria-Universität
 
VEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission PredictionVEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission Prediction
Alpen-Adria-Universität
 
Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Alpen-Adria-Universität
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Alpen-Adria-Universität
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Alpen-Adria-Universität
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Alpen-Adria-Universität
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Alpen-Adria-Universität
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
Alpen-Adria-Universität
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Alpen-Adria-Universität
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
Alpen-Adria-Universität
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Alpen-Adria-Universität
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Alpen-Adria-Universität
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Alpen-Adria-Universität
 
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Alpen-Adria-Universität
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
Alpen-Adria-Universität
 

More from Alpen-Adria-Universität (20)

Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instancesVEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
 
GREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video ProcessingGREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video Processing
 
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
 
VEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission PredictionVEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission Prediction
 
Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
 

Recently uploaded

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 

Recently uploaded (20)

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 

Video Streaming: Then, Now, and in the Future

  • 1. Video Streaming: Then, Now, Future Christian Timmerer, Professor at AAU, Director at CD Lab ATHENA Klagenfurt, Austria June 5, 2024 1 https://www.slideshare.net/christian.timmerer
  • 2. ● Introduction / motivation ● Early days of video streaming + selected projects ● Toward HTTP adaptive streaming + MPEG-DASH ● ATHENA project ● Emerging research topics Agenda 2
  • 3. Introduction / Motivation Sources: * Sandvine Global Internet Phenomena (January 2024). ** Cisco Annual Internet Report (2018–2023) White Paper (March 2020) 3 3 Video streaming is dominating today’s Internet traffic ● 2024*: 68% (fixed) and 64% (mobile) ● Video on-demand: 54% / 57% – Live: 14% / 7% ● Main applications: YouTube, Netflix (>10%), Tik Tok, Amazon Prime, Disney+ (<10%) ● Video and other applications continue to be of enormous demand in today’s home, but there will be significant bandwidth demands with the application requirements of the future**
  • 4. Multimedia Systems Challenges and Tradeoffs 4 4 Basic figure by Klara Nahrstedt, University of Illinois at Urbana–Champaign, IEEE MIPR 2018
  • 5. Evolution of Video 5 5 Era of streaming Yuriy Reznik, Christian Timmerer, 20 Years of Streaming in 20 Minutes, Mile-High Video 2020, https://athena.itec.aau.at/2020/11/20-years-of-streaming-in-20-minutes/
  • 6. ● 1993: Multicast Backbone (MBONE) ○ Virtual multicast network connecting several universities & ISPs ○ RTP-based video conferencing tool (vic) is used to stream videos ○ 1994 Rolling Stones concert – first major event streamed online ● 1995: RealAudio, 1997: RealVideo ○ First commercially successful mass-scale streaming system ○ Proprietary protocols, codecs: PNA, RealAudio, RealVideo ○ Worked over UDP, TCP, and HTTP (“cloaking” mode) ○ First major broadcast: 1995 Seattle Mariners vs New York Yankees ● 1996: VDOnet, Vivo, NetShow, VXtreme, ... ○ Many vendors have competed in streaming space initially ○ Vivo & Xing have been acquired by Real, VXtreme by Microsoft ○ By 1998, 3 main vendors remained: Real, Microsoft and Apple ● 1998: RealSystem G2 ○ First Adaptive BitRate (ABR) streaming system Early Streaming Systems 6 6 Yuriy Reznik, Christian Timmerer, 20 Years of Streaming in 20 Minutes, Mile-High Video 2020, https://athena.itec.aau.at/2020/11/20-years-of-streaming-in-20-minutes/
  • 7. 1998: RealSystem G2, “SureStream” technology ● First commercially successful Adaptive BitRate (ABR) streaming system ● Encoder: Encoded streams: Player: ● References ○ B. Girod, et al, “Scalable codec architectures for Internet video-on-demand,” ACSSC, pp. 357 – 361, 1997. ○ G. Conklin, et al, “Video Coding for Streaming Media Delivery on the Internet," TCSVT, 11 (3), pp. 20-34, 2001 ○ US Patents: 6314466, 6480541, 7075986, 7885340 First ABR Streaming System 7 7 Multi-rate packaging option Selection of streams to include Panel showing which stream is selected Yuriy Reznik, Christian Timmerer, 20 Years of Streaming in 20 Minutes, Mile-High Video 2020, https://athena.itec.aau.at/2020/11/20-years-of-streaming-in-20-minutes/
  • 8. Real Time Streaming Protocol (RTSP) et al., Internet Streaming Media Alliance (ISMA), 3GPP Packet-Switched Streaming Service (PSS) Early Streaming Standards 8 8 Server DESCRIBE rtsp://example.com/mov.test RTSP/1.0 Session Description Protocol SETUP rtsp://example.com/mov.test/streamID=0 RTSP/1.0 3GPP-Adaptation:url= “rtsp://example.com/mov.test/streamID=0”;size=20000;target-time=5000 3GPP-Link-Char: url=“rtsp://example.com/mov.test/streamID=0”; GBW=32 RTSP OK Client Time (s) PLAY rtsp://example.com/mov.test RTSP/1.0 RTSP OK Real-time Transport Protocol RTCP Reports Setup Session Streaming Client Buffer Client Buffer Model
  • 9. ● ADMITS (Adaptation in Distributed Multimedia IT Systems) Media servers, meta-databases, proxies, routers, clients: which kind of component can be best used for what kind of adaptation? ● IST-FP6: DANAE (Dynamic and Distributed Adaptation of scalable multimedia coNtent in a context-Aware Environment) MPEG-21-based adaptation framework: adaptation decisions – usage environment – adapting multimedia content ● IST-FP6: ENTHRONE (End-to-End QoS through Integrated Management of Content, Networks and Terminals) ENTHRONE Integrated Management Supervisor (EIMS): considers all stakeholder requirements – end-to-end management solution ensuring QoS for the end user. ● MPEG-21 Multimedia Framework Digital Item (Adaptation) – ISO/IEC 21000-7 Selected Projects at ITEC (1) 9 9 2002 - 2004 2004 - 2006 2006 - 2008 A. Vetro and C. Timmerer, "Digital Item Adaptation: Overview of Standardization and Research Activities," in IEEE Transactions on Multimedia, vol. 7, no. 3, pp. 418-426, June 2005, doi: 10.1109/TMM.2005.846795.
  • 10. ● Consumption of multimedia content may also stimulate other senses Vision or audition Olfaction, mechanoreception, equilibrioception, thermoception, … ● Annotation with metadata providing so-called sensory effects that steer appropriate devices capable of rendering these effects Excursus: Sensory Effects and MPEG-V 10 10 MPEG-V Building Blocks Sensory Effect Description Language (SEDL) Sensory Effect Vocabulary (SEV) Sensory Effect Metadata (SEM) ISO/IEC 23005-3 G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver, "Mulsemedia: State of the Art, Perspectives, and Challenges", ACM Trans. Multimedia Comput. Commun. Appl. 11, 1s, Article 17 (September 2014), doi:10.1145/2617994.
  • 11. ● ICT-FP7: ALICANTE (Media Ecosystem Deployment Through Ubiquitous Content-Aware Network Environments) Content-aware network processing will allow the underlying infrastructure to identify, process, and manipulate media streams and objects in real time to maximize quality of service and experience. ● Supports different streaming formats/protocols RT(S)P, P2P, and HTTP Selected Projects at ITEC (2) 11 11 M. Grafl, C. Timmerer, et al., "Scalable Media Coding Enabling Content-Aware Networking," in IEEE MultiMedia, vol. 20, no. 2, pp. 30-41, April-June 2013, doi: 10.1109/MMUL.2012.57. 2010 - 2013
  • 12. Video Delivery over HTTP/TCP 12 12 ○ Enables playback while still downloading ○ Server sends the file as fast as possible Progressive Download ○ Enables seeking via media indexing ○ Server paces transmission based on encoding rate Pseudo Streaming ○ Content is divided into short-duration chunks ○ Enables live streaming and ad insertion Chunked Streaming ○ Multiple versions of the content are created ○ Enables to adapt to network and device conditions Adaptive Streaming
  • 13. Progressive Download 13 13 HTTP Request HTTP Response Playback starts only after there is several seconds of data in the playback buffer Download will continue as fast as possible Fetched content will be wasted if the viewer clicks away Can seek only throughout the fetched content Bing Wang, Jim Kurose, Prashant Shenoy, and Don Towsley. 2004. Multimedia streaming via TCP: an analytic performance study. ACM International Conference on Multimedia (MM'04). DOI:https://doi.org/10.1145/1027527.1027735 “TCP generally provides good streaming performance when the achievable TCP throughput is roughly twice the media bitrate, with only a few seconds of startup delay”
  • 14. 14 14
  • 15. Streaming is cooler, more viewer friendly 15 15 Playback starts when there is just few seconds of data Download rate will match the encoding bitrate and downloading pauses if the player pauses → Less waste Can seek to anywhere in the entire content L. De Cicco, S. Mascolo. An Experimental Investigation of the Akamai Adaptive Video Streaming. in Proc of USAB 2010, special session Interactive Multimedia Applications (WIMA), Klagenfurt, Austria, 3-4 November 2010, LCNS 6389, pp. 447-464, Springer-Verlag, doi:10.1007/978-3-642-16607-5 Implements “pseudo streaming” and mimics RTSP-style streaming but via HTTP.
  • 16. HTTP Adaptive Streaming 101 Adaptation logic is within the client, not normatively specified by a standard, subject to research and development 16 16 Client
  • 17. Bitrate Adaptation Schemes 17 17 Bitrate Adaptation Schemes Client-based Adaptation Bandwidth -based Buffer- based Mixed adaptation Proprietary solutions MDP-based Server-based Adaptation Network- assisted Adaptation Hybrid Adaptation SDN-based Server and network- assisted A. Bentaleb, B. Taani, A. C. Begen, C. Timmerer and R. Zimmermann, "A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP," in IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 562-585, Firstquarter 2019. doi: 10.1109/COMST.2018.2862938
  • 18. MPD Period id = 1 start = 0 s Period id = 3 start = 300 s Period id = 4 start = 850 s Period id = 2 start = 100 s Splicing of arbitrary content like ads MPEG DASH Data Model (ISO/IEC 23009-1) Period id = 2 start = 100 s Adaptation Set 0 subtitle turkish Adaptation Set 2 audio english Adaptation Set 3 audio german Adaptation Set 1 video Selection of components/tracks Adaptation Set 1 BaseURL=http://abr.rocks.com/ Representation 2 Rate = 1 Mbps Representation 4 Rate = 3 Mbps Representation 1 Rate = 500 Kbps Representation 3 Rate = 2 Mbps Selection of representations 18 18 Representation 3 Rate = 2 Mbps Resolution = 720p Segment Info Duration = 10 s Template: 3/$Number$.mp4 Well-defined media format Segment Access Initialization Segment http://abr.rocks.com/3/0.mp4 Media Segment 1 start = 0 s http://abr.rocks.com/3/1.mp4 Media Segment 2 start = 10 s http://abr.rocks.com/3/2.mp4 Chunks with addresses and timing
  • 19. 19 19
  • 20. “Application-oriented basic research” to address current and future research and deployment challenges of HAS and emerging streaming methods ATHENA – Adaptive Streaming over HTTP and Emerging Networked Multimedia Services Content Provisioning Content Delivery Content Consumption End-to-End Aspects ● Video encoding for HAS ● Quality-aware encoding ● Learning-based encoding ● Multi-codec HAS ● Edge computing ● Information CDN/SDN⇿clients ● Netw. assistance for/by clients ● Utility evaluation ● Bitrate adaptation schemes ● Playback improvements ● Context and user awareness ● Quality of Experience (QoE) studies ● Application/transport layer enhancements ● Quality of Experience (QoE) models ● Low-latency HAS ● Learning-based HAS https://athena.itec.aau.at/ 20 20 Funding: Hermann Hellwagner, Advances in Edge-Based and In-Network Media Processing for Adaptive Video Streaming, Keynote at IEEE MIPR’23, Singapore. Related FFG-funded projects: AdvUHD-DASH, PROMETHEUS, APOLLO, GAIA
  • 21. Problem Statement / Basic Idea for Live Streaming ● Static bitrate ladder Pre-defined bitrate/resolution pairs for all contents ● Per-title encoding Optimize bitrate ladder based on the content ● UHD / HFR content Increase of spatial and temporal resolutions ● Live (is Life)[* Opus, 1985] Perceptual-based bitrate/resolution/framerate prediction algorithm based on content complexity Video Complexity Analysis (1) 21 21 VoD Live * … https://www.youtube.com/watch?v=pATX-lV0VFk
  • 22. State of the Art: ITU-T Spatial Information / Temporal Information (SI/TI) ● The correlation with ground truth spatial and temporal complexity is low Video Complexity Analysis (2) 22 22 Sobel Filter
  • 23. Video Complexity Analyzer (VCA) ● VCA introduces DCT-based spatial-temporal complexity features ● Spatial complexity correlation has increased, but temporal complexity remains low Video Complexity Analysis (3) 23 23 Vignesh V Menon, Christian Feldmann, Hadi Amirpour, Mohammad Ghanbari, and Christian Timmerer. 2022. VCA: video complexity analyzer. In Proceedings of the 13th ACM Multimedia Systems Conference (MMSys '22). Association for Computing Machinery, New York, NY, USA, 259–264. https://athena.itec.aau.at/2022/04/vca-video-complexity-analyzer/
  • 24. Enhanced Video Complexity Analyzer (EVCA) ● EVCA refines the definition of the temporal complexity feature ● Temporal complexity correlation has increased Video Complexity Analysis (4) 24 24 H. Amirpour, M. Ghasempour, L. Qu, W. Hamidouche, and C. Timmerer. 2024. EVCA: Enhanced Video Complexity Analyzer. In Proceedings of the 15th ACM Multimedia Systems Conference (MMSys '24). Association for Computing Machinery, New York, NY, USA, 285–291. https://doi.org/10.1145/3625468.3652171
  • 25. Deep Video Complexity Analyzer (DeepVCA) Video Complexity Analysis (5) 25 25 H. Amirpour, K. Schoeffmann, M. Ghanbari and C. Timmerer, "DeepVCA: Deep Video Complexity Analyzer," in IEEE Transactions on Circuits and Systems for Video Technology, doi: 10.1109/TCSVT.2024.3382366. https://athena.itec.aau.at/2024/04/ieee-tcsvt-deepvca-deep-video-complexity-analyzer/
  • 26. E-WISH: A policy driven ABR Algorithm 26 Energy Cost + 𝛅 [Data Cost] Data Cost Total Cost = ɑ Segment bitrate Throughput [Buffer Cost] Buffer Cost + β Download time Buffer occupancy [Quality Cost] Quality Cost + ɣ Video quality Distortion Instability Bitrate, Resolution, FPS (↑) M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605 Technology Transfer
  • 27. E-WISH: An Energy-aware ABR Algorithm 27 [Data Cost] [Quality Cost] [Buffer Cost] [Energy Cost] Data Cost Buffer Cost Quality Cost Energy Cost Total Cost = ɑ + β + ɣ + 𝛅 Segment bitrate Throughput Download time Buffer occupancy Video quality Distortion Instability Bitrate, Resolution, FPS M. Nguyen, E. Çetinkaya, H. Hellwagner and C. Timmerer, "WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile Devices," 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), doi: 10.1109/MMSP53017.2021.9733605 D.Lorenzi, M. Nguyen, F. Tashtarian, and C. Timmerer, “E-WISH: An Energy-aware ABR Algorithm For Green HTTP Adaptive Video Streaming”, 3rd ACM Mile-High Video Conference (MHV '24), https://doi.org/10.1145/3638036.3640802
  • 28. Live Video Streaming Pipeline 28 LIVE Origin Server (Encoding & Packaging) Live Camera Player CDN Server manifest.mpd Bitrate Resolution 5 Mbps 1920x1080 2.8 Mbps 1280x720 1.1 Mbps 960x540 System Admin Bitrate Ladder HTTP Request HTTP Response Manifest includes representations: different versions of the content optimized for various playback conditions, e.g., different bitrates and resolutions Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024 ● Fixed bitrate ladder ○ Content-aware, e.g., Netflix’s per-title encoding ○ Context-aware, e.g., [Lebreton and Yamagishi 2023] ○ Agnostic of the content and context (proprietary)
  • 29. ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming Select representations with bitrates that closely match the desired bitrates Players Advertise a large number of representations Encoded segments CDN Server Mega-Manifest 29 Origin Server OTL Process requests and video qualities and determine OTL LIVE Live Camera Utilize the OTL OTL: Optimized Temporary Ladder Tashtarian et al., "ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming", 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), Santa Clara, CA, 2024
  • 30. Quality of Experience (QoE) ... ● “... is the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state.”1) ● … can be easily extended to various domains, e.g., immersive media experiences.2) 30 30 1) P. Le Callet, S. Möller, A. Perkis, et al. QUALINET White Paper on Definitions of Quality of Experience. European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003). 2012. 2) A. Perkis, C. Timmerer, et al. 2020. QUALINET White Paper on Definitions of Immersive Media Experience (IMEx). arXiv:2007.07032 [cs.MM] 3) J. van der Hooft, T. Wauters, F. De Turck, C. Timmerer, and H. Hellwagner. “Towards 6DoF HTTP Adaptive Streaming Through Point Cloud Compression”. 27th ACM Int’l. Conf. on Multimedia (MM'19). Oct. 2019. 4) J. van der Hooft, M. T. Vega, C. Timmerer, A. C. Begen, F. De Turck and R. Schatz, "Objective and Subjective QoE Evaluation for Adaptive Point Cloud Streaming," 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), Athlone, Ireland, 2020. 3) 4) 2010 - 2014 - now
  • 31. Immersive Video Delivery: From Omnidirectional Video to Holography 31 31 J. v. d. Hooft, H. Amirpour, M. Torres Vega, Y. Sanchez, R. Schatz; T. Schierl, C. Timmerer, "A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography," in in IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1336-1375, Secondquarter 2023, doi: 10.1109/COMST.2023.3263252. ● 3DoF Omnidirectional Video ● 6DoF Volumetric Video ● 6DoF Imagery Video
  • 32. NeRV: Neural Representations for Videos, NeurIPS 2021 Deep Learning-Based Video Coding: A Review and a Case Study, ACM CSUR 2020 ● Video Coding Learned video coding / E2E deep video coding Future of Video Streaming 32 32 An End-to-End Learning Framework for Video Compression, TPAMI 2021 ● Video Streaming DeepStream: Video Streaming Enhancements using Compressed Deep Neural Networks Deep learning-based video coding Neural implicit representation for videos DeepStream: TCSVT 2022 Generative AI for video streaming
  • 33. ATHENA team Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari, Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer, Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan, Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, Yiying Wei (Inter-)National Collaborators (selection) ● Prof. Ali C. Begen, Ozyegin University, Turkey ● Prof. Filip De Turck, Ghent University – imec, Belgium ● Dr. Jeroen van der Hooft, Ghent University – imec, Belgium ● Dr. Maria Torres Vega, Ghent University – imec, Belgium ● Dr. Raimund Schatz, AIT Austrian Institute of Technology, Austria ● Prof. Roger Zimmermann, NUS, Singapore ● Dr. Abdelhak Bentaleb, Concordia University, Canada ● Prof. Christine Guillemot, INRIA, Rennes, France ● Prof. Patrick Le Callet, Polytech Nantes-Institut Universitaire de France, France ● Prof. Junchen Jiang, University of Chicago, USA ● Dr. Sergey Gorinsky, IMDEA, Spain Acknowledgments 33 33 https://athena.itec.aau.at/
  • 34. Co-Authors (2003-2023; cf. FODOK) Abdelhak Bentaleb, Adam T. Lindsay, Adam Wieckowski, Adithyan Ilangovan, Adlen Ksentini, Ahmed Mehaoua, Alan Bertoni, Alberto Leon Martin, Alessandro Floris, Alex Chernilov, Ali C. Begen, Alireza Erfanian, Amy Reibman, Anastasia Henkel, Anastasios Kourtis, Anatoliy Zabrovskiy, Andrea Detti, Andreas Hutter, Andreas Sackl, Andreas Schorr, Andrew Perkis, Anne-Lore Mevel, Anthony Vetro, Antonio Pinto, Armin Trattnig, Arthur Lugmayr, Asim Hameed, Aytac Azgin, Babak Taraghi, Barnabás Takacs, Barry Crabtree, Bayan Taani, Benjamin Bross, Benjamin Rainer, Bernd Girod, Bernhard Feiten, Brahim Allan, Bruno Gardlo, Carsten Griwodz, Cathrine Lamy-Bergot, Cedric Westphal, Chris Needham, Christian Feldmann, Christian Keimel, Christian R. Helmrich, Christian Raffelsberger, Christian Sieber, Christine Guillemot, Christopher Müller, Cise Midoglu, Claudio Alberti, Costas Troulos, Cyril Concolato, Daniel Corujo, Daniel Gatica-Perez, Daniel Hoelbling-Inzko, Daniel Negru, Daniel Posch, Daniel Silhavy, Daniel Weinberger, Daniele Lorenzi, Daniele Renzi, Darragh Egan, David Wortley, Davide Rogai, Deepak Chaudhary, Detlev Marpe, Devillers Sylvain, Dietmar Jannach, Doncel V. Rodriguez, Dragi Kimovski, Ekrem Cetinkaya, Emanuele Quacchio, Emre Karsli, Eric Delfosse, et al., Eugen Borcoci, Eva Cheng, Evangelos Pallis, Evgeny Kuzmin, Evgeny Petrov, Fabrizio S. Rovati, Fan Chen, Farzad Tashtarian, Fernando López, Fernando Pereira, Filip De Turck, Filippo Chiariglione, Francesc Sanahuja, Franz J. Hauck, Gabriel Panis, Gabriele Panis, Gabriella Olmo, George Gardikis, George Xilouris, Gheorghita Ghinea, Giovanni Cordara, Gülnaziye Bingöl, Hadi Amirpour, Haichao Yao, Hamid Hadian, Hannaneh Barahouei Pasandi, Haral Kosch, Harald Kosch, Harilaos. Koumaras, Harold Vincent Poor, He Xu, Hemanth Kumar Ravuri, Henrique Guilhon de Castro, Hermann Hellwagner, Hongjie He, Hui Lu, Ian Burnett, Ingo KofIer, Ingo Wolf, Ioannis Kompatsiaris, Irene Viola, Ismail Djama, Jaime Delgado, Jean Gelissen, Jean Le Feuvre, Jean-Charles Point, Jeroen van der Hooft, Jesus Aguilar Armijo, Jianping Wang, Jim Hurley, Jingwen Zhu, Joachim Seidl, Johannes Jaborning, Johannes Liegl, John Barrett, John R. Smith, Jonathan Ishmael, Joost Geurts, Jörg Heuer, Jörg Pöcher, Jose Martínez, Karel Fliegel, Karsten Grüneberg, Karsten Müller, Katharina Gassner, Keith Mitchell, Klaus Leopold, Klaus Schöffmann, Laszlo Szobonya, Leonardo Peroni, Li Fang, Lingfeng Qu, Lu Yu, Luca Celetto, Luigi Atzori, Mamadou Sidibé, Marco Mattavelli, Maria Teresa Andrade, Maria Torres Vega, Marie-Jose Montpetit, Mario Graf, Mario Taschwer, Marius Preda, Mark Stuart, Markus Waltl, Marta Larson, Martin Hahn, Martin Prangl, Martin Smole, Mathias Wien, Matteo Maiero, Matthias Hirth, Matthias Klusch, May Lim, Michael Eberhard, Michael Grafl, Michael Mackay, Michael Ransburg, Miguel Gómez, Mike Buckham, Minh Nguyen, Mohammad Ghanbari, Mohammad Hosseini, Mohammad Shojafar, Muawiyath Shujau, Myriam Amielh, Nakisa Shams, Narciso García, Narges Mehran, Negin Ghamsarian, Niall Murray, Nikos Zotos, Ningxiong Mao, Ozgu Alay, Pablo Cesar, Pascal Frossard, Patrick Kapahnke, Patrick Le Callet, Pavel Korshunov, Pedro Carvalho, Pedro Souto, Peng Chen, Peter De Neve, Peter Lederer, Peyman Sheikholharam Mashhadi, Philippe Hanhart, Phuoc Tran-Gia, Prajit T Rajendran, Prateek Agrawal, Radu Aurel Prodan, Raimund Schatz, Ralf Terlutter, Reinhard Grandl, Reyhane Falanji, Reza Farahani, Reza Saeedinia, Reza Shokri Kalan, Ricardo Petrocco, Richard Marsh, Rik Van de Walle, Robby De Sutter, Roger Zimmermann, Roland Kersche, Roland Mathá, Rongrong Ni, S. Omid Fatemi, Saad Harous, Sam Dutton, Sam Van Damme, Samira Afzal, Sandro Linder, Sean Brennan, Selina Zoe Haack, Serge Lerouge, Sergey Gorinsky, Shivi Vats, Simone Ferlin-Reiter, Simone Milani, Simone Porcu, Stefan Lederer, Stefan Mangold, Stefan Petscharnig, Stefan Pham, Stefano Battista, Stephen Davis, Stephen Gulliver, Steve Lerouge, Robby De Sutter, Sylvain Devillers, Tamer Nadeem, Teodora Guenkova-Luy, Theodore Zahariadis, Thomas DeMartini, Thomas Frank, Thomas Gottron, Thomas Schierl, Thomas Stockhammer, Thomas Zinner, Tibor Szkaliczki, Tim Stevens, Tim Wauters, Tobias Hossfeld, Tommaso Melodia, Toufik Ahmed, Touradj Ebrahimi, Vaios Koumaras, Venkata Phani Kumar Malladi, Victor Doncel, Vignesh Menon, Vishu Madaan, Vladislav Kashanskii, Wael Cherif, Weisi Lin, Wen Ji, Wenwu Zhu, Will Shucheng Liu, Xiaoqi Cao, Xuemei Zhou, Yael Lapid, Yago Sanchez, Yaning Liu, Yao Zhao, Yaolin Yang, Yuan Yuan, Yuansong Qiao, Zahra Najafabadi Samani, Zhengdao Zhan, Zhu Li Acknowledgments 34 34
  • 35. Current Staff at ITEC Afzal Samira, Amirpour Hadi, Artioli Emanuele, Azimi Ourimi Zoha, Bauer Christian, Farahani Reza, Ghasempour Mohammad, Harshina Kseniia, Hellwagner Hermann, Hoi Manuel Elias, Horvath Kurt Klaus, Isak Selina, Kimovski Dragi, Lachini Armin, Lorenzi Daniele, Lux Mathias, Nasirihaghighi Sahar, Prodan Radu Aurel, Qureshi Kamran, Rader Martin, Schniz Felix, Schöffmann Klaus, Seyeddizaji Seyedehhaleh, Tashtarian Farzad, Tuček Tom, Uitz Sebastian, Vats Shivi, Wei Yiying, Gliesche Anna Julia, Letter Margit, Messner Rudolf, Steinbacher Martina, Taschwer Mario, Kowatsch Paul, Leitinger Claudia, Leopold Mario, Strobl Magdalena + Alumni: https://itec.aau.at/alumni/ Acknowledgments 35 35
  • 36. Thank you for your attention 36