Published in APNOMS'17 on October 2017.
Watching 360◦ videos using Head-Mounted Display (HMD) allows users to only see a part of the whole 360◦ videos. With this feature, tiled videos become a potential solution for aggressively reducing the required bandwidth for 360◦ video streaming, turning it into a reality in cellular networks. In this paper, we design several experiments for quantifying the performance of tile-based 360◦ video streaming over a real cellular network on our campus. In particular, we empirically investigate the impacts of tile streaming over 4G networks, such as coding efficiency, bandwidth saving, and scalability.
Our experiments lead to interesting findings, for example, (i) only streaming the tiles viewed by the viewer achieves bitrate reduction by up to 80% and (ii) the coding efficiency of 3x3 tiled videos may be higher than non-tiled videos at higher bitrates.
We believe this work will stimulate more studies in the emerging area of mobile AR/VR (Augmented Reality and Virtual Reality) over 4G networks.
This document discusses optimizing 360-degree video streaming to head-mounted virtual reality. It covers challenges like existing codecs only supporting 2D videos and 360 videos having wider views than conventional videos. Approaches proposed include fixation prediction to avoid streaming unwatched parts, QoE modeling designed for 360 videos to improve user experience, and an adaptive streaming platform to select and transmit tiles based on fixation prediction while allocating bitrates based on the QoE model. Part I discusses fixation prediction including using neural networks trained on viewing features. Part II covers QoE modeling, noting limitations of existing metrics and factors that affect QoE like content and bitrates. It constructs a logarithmic linear QoE model. Part III outlines an
Fixation Prediction for 360° Video Streaming in Head-Mounted Virtual RealityWen-Chih Lo
Published in NOSSDAV'17 on June 2017.
We study the problem of predicting the Field-of-Views (FoVs) of viewers watching 360° videos using commodity Head-Mounted Displays (HMDs). Existing solutions either use the viewer's current orientation to approximate the FoVs in the future, or extrapolate future FoVs using the historical orientations and dead-reckoning algorithms. In this paper, we develop fixation prediction networks that concurrently leverage sensor- and content-related features to predict the viewer fixation in the future, which is quite different from the solutions in the literature. The sensor-related features include HMD orientations, while the content-related features include image saliency maps and motion maps. We build a 360° video streaming testbed to HMDs, and recruit twenty-five viewers to watch ten 360° videos. We then train and validate two design alternatives of our proposed networks, which allows us to identify the better-performing design with the optimal parameter settings.
Trace-driven simulation results show the merits of our proposed fixation prediction networks compared to the existing solutions, including: (i) lower consumed bandwidth, (ii) shorter initial buffering time, and (iii) short running time.
360° Video Viewing Dataset in Head-Mounted Virtual RealityWen-Chih Lo
Published in MMSys'17 on June 2017.
360° videos and Head-Mounted Displays (HMDs) are getting increasingly popular. However, streaming 360° videos to HMDs is challenging. This is because only video content in viewers' Field-of-Views (FoVs) is rendered, and thus sending complete 360° videos wastes resources, including network bandwidth, storage space, and processing power. Optimizing the 360° video streaming to HMDs is, however, highly data and viewer dependent, and thus dictates real datasets. However, to our best knowledge, such datasets are not available in the literature. In this paper, we present our datasets of both content data (such as image saliency maps and motion maps derived from 360° videos) and sensor data (such as viewer head positions and orientations derived from HMD sensors). We put extra efforts to align the content and sensor data using the timestamps in the raw log files.
The resulting datasets can be used by researchers, engineers, and hobbyists to either optimize existing 360° video streaming applications (like rate-distortion optimization) and novel applications (like crowd-driven camera movements). We believe that our dataset will stimulate more research activities along this exciting new research direction.
This document summarizes test results of Ceph QoS throttling mechanisms. It finds that:
1) BlueStore adaptive throttling can achieve a desired 1:1:10:10:10 weight distribution but reduces performance by 57%. Lower throttling reduces performance less but harms weight quality.
2) Outstanding IO adaptive throttling reduces performance by 80% to achieve the desired weights. It can also guarantee reservations with an 89% performance reduction.
3) The per-pool QoS with dmClock faces failures when workload queue depths are low or multiple threads access the queue simultaneously, breaking the weight phase. Larger block sizes also impact consistency.
Vijayendra Shamanna from SanDisk presented on optimizing the Ceph distributed storage system for all-flash architectures. Some key points:
1) Ceph is an open-source distributed storage system that provides file, block, and object storage interfaces. It operates by spreading data across multiple commodity servers and disks for high performance and reliability.
2) SanDisk has optimized various aspects of Ceph's software architecture and components like the messenger layer, OSD request processing, and filestore to improve performance on all-flash systems.
3) Testing showed the optimized Ceph configuration delivering over 200,000 IOPS and low latency with random 8K reads on an all-flash setup.
The document discusses the history and development of adaptive bitrate (ABR) streaming and its relationship to content delivery networks (CDNs). It makes three key points:
1. While ABR streaming aims to provide multiple encoded versions of content, this conflicts with CDNs which work best when caching a smaller number of objects. Having many versions increases cache misses and midgress traffic.
2. The disconnect between ABR and CDNs can be modeled mathematically. Deploying multiple encodings of the same content increases the CDN cache miss probability by a factor related to the usage probabilities of each encoding.
3. The decision to deploy a new encoding standard like HEVC or delivery format like CMAF
The document summarizes an upcoming webinar on new developments in MPEG standards. It will discuss Versatile Video Coding (VVC), MPEG-H 3D Audio Baseline Profile, video-based point cloud compression (V-PCC), and MPEG Immersive Video (MIV). The webinar will provide overviews of each standard and their applications, as well as results from recent verification tests that evaluated subjective quality and performance. Speakers will include leaders from MPEG working groups and the Joint Video Experts Team.
This document discusses optimizing 360-degree video streaming to head-mounted virtual reality. It covers challenges like existing codecs only supporting 2D videos and 360 videos having wider views than conventional videos. Approaches proposed include fixation prediction to avoid streaming unwatched parts, QoE modeling designed for 360 videos to improve user experience, and an adaptive streaming platform to select and transmit tiles based on fixation prediction while allocating bitrates based on the QoE model. Part I discusses fixation prediction including using neural networks trained on viewing features. Part II covers QoE modeling, noting limitations of existing metrics and factors that affect QoE like content and bitrates. It constructs a logarithmic linear QoE model. Part III outlines an
Fixation Prediction for 360° Video Streaming in Head-Mounted Virtual RealityWen-Chih Lo
Published in NOSSDAV'17 on June 2017.
We study the problem of predicting the Field-of-Views (FoVs) of viewers watching 360° videos using commodity Head-Mounted Displays (HMDs). Existing solutions either use the viewer's current orientation to approximate the FoVs in the future, or extrapolate future FoVs using the historical orientations and dead-reckoning algorithms. In this paper, we develop fixation prediction networks that concurrently leverage sensor- and content-related features to predict the viewer fixation in the future, which is quite different from the solutions in the literature. The sensor-related features include HMD orientations, while the content-related features include image saliency maps and motion maps. We build a 360° video streaming testbed to HMDs, and recruit twenty-five viewers to watch ten 360° videos. We then train and validate two design alternatives of our proposed networks, which allows us to identify the better-performing design with the optimal parameter settings.
Trace-driven simulation results show the merits of our proposed fixation prediction networks compared to the existing solutions, including: (i) lower consumed bandwidth, (ii) shorter initial buffering time, and (iii) short running time.
360° Video Viewing Dataset in Head-Mounted Virtual RealityWen-Chih Lo
Published in MMSys'17 on June 2017.
360° videos and Head-Mounted Displays (HMDs) are getting increasingly popular. However, streaming 360° videos to HMDs is challenging. This is because only video content in viewers' Field-of-Views (FoVs) is rendered, and thus sending complete 360° videos wastes resources, including network bandwidth, storage space, and processing power. Optimizing the 360° video streaming to HMDs is, however, highly data and viewer dependent, and thus dictates real datasets. However, to our best knowledge, such datasets are not available in the literature. In this paper, we present our datasets of both content data (such as image saliency maps and motion maps derived from 360° videos) and sensor data (such as viewer head positions and orientations derived from HMD sensors). We put extra efforts to align the content and sensor data using the timestamps in the raw log files.
The resulting datasets can be used by researchers, engineers, and hobbyists to either optimize existing 360° video streaming applications (like rate-distortion optimization) and novel applications (like crowd-driven camera movements). We believe that our dataset will stimulate more research activities along this exciting new research direction.
This document summarizes test results of Ceph QoS throttling mechanisms. It finds that:
1) BlueStore adaptive throttling can achieve a desired 1:1:10:10:10 weight distribution but reduces performance by 57%. Lower throttling reduces performance less but harms weight quality.
2) Outstanding IO adaptive throttling reduces performance by 80% to achieve the desired weights. It can also guarantee reservations with an 89% performance reduction.
3) The per-pool QoS with dmClock faces failures when workload queue depths are low or multiple threads access the queue simultaneously, breaking the weight phase. Larger block sizes also impact consistency.
Vijayendra Shamanna from SanDisk presented on optimizing the Ceph distributed storage system for all-flash architectures. Some key points:
1) Ceph is an open-source distributed storage system that provides file, block, and object storage interfaces. It operates by spreading data across multiple commodity servers and disks for high performance and reliability.
2) SanDisk has optimized various aspects of Ceph's software architecture and components like the messenger layer, OSD request processing, and filestore to improve performance on all-flash systems.
3) Testing showed the optimized Ceph configuration delivering over 200,000 IOPS and low latency with random 8K reads on an all-flash setup.
The document discusses the history and development of adaptive bitrate (ABR) streaming and its relationship to content delivery networks (CDNs). It makes three key points:
1. While ABR streaming aims to provide multiple encoded versions of content, this conflicts with CDNs which work best when caching a smaller number of objects. Having many versions increases cache misses and midgress traffic.
2. The disconnect between ABR and CDNs can be modeled mathematically. Deploying multiple encodings of the same content increases the CDN cache miss probability by a factor related to the usage probabilities of each encoding.
3. The decision to deploy a new encoding standard like HEVC or delivery format like CMAF
The document summarizes an upcoming webinar on new developments in MPEG standards. It will discuss Versatile Video Coding (VVC), MPEG-H 3D Audio Baseline Profile, video-based point cloud compression (V-PCC), and MPEG Immersive Video (MIV). The webinar will provide overviews of each standard and their applications, as well as results from recent verification tests that evaluated subjective quality and performance. Speakers will include leaders from MPEG working groups and the Joint Video Experts Team.
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...Alpen-Adria-Universität
HTTP Adaptive Streaming(HAS) is the most common approach for delivering video content over the Internet. Therequirement to encode the same content at different quality levels(i.e., representations) in HAS is a challenging problem for content providers. Fast multirate encoding approaches try to accelerate this process by reusing information from previously encoded representations. In this paper, we propose to use convolutional neural networks (CNNs) to speed up the encoding of multiple representations with a specific focus on parallel encoding. In parallel encoding, the overall time-complexity is limited to the maximum time-complexity of one of the representations that are encoded in parallel. Therefore, instead of reducing the time-complexity for all representations, the highest time-complexities are reduced. Experimental results show that FaME-ML achieves significant time-complexity savings in parallel encoding scenarios(41%in average) with a slight increase in bitrate and quality degradation compared to the HEVC reference software.
The media landscape changes significantly over the last few years by new content formats, new service offerings, additional consumption devices and new monetization models. Think of Netflix, DAZN, Mediatheks, mobile devices, interactive content, smart TVs, Virtual and Augmented Reality, and so on. Many of these efforts have been realized by a limited usage of standards, but are standards irrelevant? Secondly, more and more services are enabled by latest mobile compute platforms enabling new services and experiences. This presentation will provide an overview some of these trends and will motivate the development of global interop standards. Specific aspects will include the move of linear TV services to the Internet (both mobile and fixed) as well recent advances on Extended Reality and immersive media trends.
Lessons Learned Running a Container Cloud on Apache Hadoop YARNBillie Rinaldi
The document discusses lessons learned from running a container cloud on Apache Hadoop YARN. Some key points include:
- Running over 5.8 million containers and 1.1 million tests on the container cloud with improved hardware utilization and test speed.
- Challenges involved issues like IP management, Docker storage drivers causing kernel panics, user namespacing limitations, and image management.
- Solutions involved allocating IP addresses, testing storage configurations, running containers as a common user, and managing a private image registry for cleanup.
Seattle Video Tech: The Future of SSAI on OTT DevicesDavid Sayed
The document discusses server-side ad insertion (SSAI) techniques for over-the-top (OTT) devices. It finds that while streaming boxes generally support SSAI well, smart TVs and set-top boxes often have limited support. Older or less capable devices may require fallback strategies like removing ads or DRM. Upcoming standards could improve SSAI compatibility going forward, but content providers are advised to focus efforts on recent devices rather than trying to support all devices universally.
Streaming media has evolved significantly over the past 20 years. Early systems in the 1990s used proprietary protocols over UDP and later included pre-roll buffers and adaptive bitrate techniques. Standards like RTSP, 3GPP, and ISMA provided interoperability but relied on complex server implementations. The shift to HTTP in the 2000s simplified delivery using progressive download and then adaptive streaming formats like HLS, DASH, and CMAF that divide media into short segments. These standards separate the media format from the delivery method, enabling delivery via HTTP while supporting features like DRM and playback across different devices and networks.
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/ceva/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit-siegel
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Yair Siegel, Director of Segment Marketing at CEVA, presents the "Fast Deployment of Low-power Deep Learning on CEVA Vision Processors" tutorial at the May 2016 Embedded Vision Summit.
Image recognition capabilities enabled by deep learning are benefitting more and more applications, including automotive safety, surveillance and drones. This is driving a shift towards running neural networks inside embedded devices. But, there are numerous challenges in squeezing deep learning into resource-limited devices. This presentation details a fast path for taking a neural network from research into an embedded implementation on a CEVA vision processor core, making use of CEVA’s neural network software framework. Siegel explains how the CEVA framework integrates with existing deep learning development environments like Caffe, and how it can be used to create low-power embedded systems with neural network capabilities.
With the recent surge in Internet multimedia traffic, the enhancement and improvement of media players, specifically DASH media players happened at an incredible rate. DASH Media players take advantage of adapting a media stream to the network fluctuations by continuously monitoring the network and making decisions in near real-time. The performance of algorithms that are in charge of making such decisions was often difficult to be evaluated and objectively assessed.
CAdViSE provides a Cloud-based Adaptive Video Streaming Evaluation framework for the automated testing of adaptive media players. In this talk, I will introduce the CAdViSE framework, its application, and propose the benefits and advantages that it can bring to every web-based media player development pipeline. To demonstrate the power of CAdViSE in evaluating Adaptive Bitrate (ABR) algorithms I will exhibit its capabilities when combined with objective Quality of Experience (QoE) models. For this talk, my team at Bitmovin/ATHENA has selected the ITU-T P.1203 (mode 1) model in order to execute experiments and calculate the Mean Opinion Score (MOS), and better understand the behavior of a set of well-known ABR algorithms in a real-life setting. The talk will display how we tested and deployed our framework using a modular architecture into a cloud infrastructure. This method yields a massive growth to the number of concurrent experiments and the number of media players that can be evaluated and compared at the same time, thus enabling maximum potential scalability. In my team’s most recent experiments, we used Amazon Web Services (AWS) for demonstration purposes. Another awesome feature of CAdViSE that will be discussed here is the ability to shape the test network with endless network profiles. To do so, we used a fluctuation network profile and a real LTE network trace based on the recorded internet usage of a bicycle commuter in Belgium.
CAdViSE produces comprehensive logs for each media streaming experimental session. These logs can then be applied against different goals, such as objective evaluation to stitch back media segments and conduct subjective evaluations afterwards. In addition, startup delays, stall events, and other media streaming defects can be imitated exactly as they happened during the experimental streaming sessions.
Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective Qo...Alpen-Adria-Universität
Omnidirectional video (ODV) streaming applica- tions are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile- based streaming which allows to have higher quality within the user’s viewport while quality outside the user’s viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is character- ized by high reproducibility since relevant media delivery aspects (including the user’s head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.
On Optimizing Resource Utilization in AVC-based Real-time Video StreamingAlpen-Adria-Universität
Real-time video streaming traffic and related applications have witnessed significant growth in recent years. However, this has been accompanied by some challenging issues, predominantly resource utilization. IP multicasting, as a solution to this problem, suffers from many problems. Using scalable video coding could not gain wide adoption in the industry, due to reduced compression efficiency and additional computational complexity. The emerging software-defined networking (SDN)and network function virtualization (NFV) paradigms enable re-searchers to cope with IP multicasting issues in novel ways. In this paper, by leveraging the SDN and NFV concepts, we introduce a cost-aware approach to provide advanced video coding (AVC)-based real-time video streaming services in the network. In this study, we use two types of virtualized network functions (VNFs): virtual reverse proxy (VRP) and virtual transcoder (VTF)functions. At the edge of the network, VRPs are responsible for collecting clients’ requests and sending them to an SDN controller. Then, executing a mixed-integer linear program (MILP) determines an optimal multicast tree from an appropriate set of video source servers to the optimal group of transcoders. The desired video is sent over the multicast tree. The VTFs transcode the received video segments and stream to the requested VRPs over unicast paths. To mitigate the time complexity of the proposed MILPmodel, we propose a heuristic algorithm that determines a near-optimal solution in a reasonable amount of time. Using theMiniNet emulator, we evaluate the proposed approach and show it achieves better performance in terms of cost and resource utilization in comparison with traditional multicast and unicast approaches.
- The document outlines requirements and a proposed solution for live streaming and video on demand for online lectures using AWS services like MediaLive, MediaPackage, S3, and CloudFront.
- The solution aims to support 20-25 concurrent live streams with low latency, provide video on demand within an hour, and leverage CDN edge servers for content delivery.
- The proposed architecture includes using OBS to push live streams to RTMP endpoints, encoding streams in multiple resolutions and bitrates with MediaLive, storing HLS manifests and segments in S3, and delivering content via CloudFront with MediaPackage.
Description of Microsoft Silverlight technology.
Advantages over "standard streaming", download and progressive download methods.
Silverlight session description and analysis using wireshark
HEVC (High Efficiency Video Coding) is an new video compression standard that can provide around a 50% reduction in bitrate compared to existing standards like H.264. It was first released in 2013 and is designed to improve the efficiency of encoding ultra-high definition (UHD) video formats. HEVC will help enable the distribution and viewing of 4K and higher resolution video by reducing the storage and bandwidth needs for UHD content. It is expected to initially see adoption for over-the-top streaming applications and help increase the number of HD streams that can be delivered by cable and IPTV operators. Widespread adoption of live UHD video will still take time as distribution infrastructure and content creation capabilities evolve to
Managing Transition to HEVC/VP9/AV1 with Multi-Codec StreamingBitmovin Inc
Video streaming is in transition towards the next generation of video codecs, offering to double the quality while lowering the required bandwidth. As the successor crown to the ubiquitous AVC/H.264 is still up for grabs, major content providers and device manufacturers are throwing their weights behind competing formats - HEVC/VP9/AV1 - leading to market fragmentation, specifically within web environments. To deal with this challenge, OTT services need to support multiple codecs in an efficient way. In this presentation, we will discuss how to evaluate the benefits and the tradeoffs of embracing these next generation compression technologies in your media workflow.
Encoding Video for the Web - Webinar from ReelSEO.comMark Robertson ⏩
The document summarizes an online presentation about encoding video for the web. The presentation covered topics like video compression, codecs, containers, bit rates, and tools for encoding. Speakers demonstrated how to create high-quality H.264 encoding settings using free and open source tools like x264 and Handbrake. They provided examples of encoding presets for different purposes and platforms. The presentation concluded with a question and answer session.
Lync Server 2013: Network Quality considerations in LAN, WAN and Wi-FiStåle Hansen
This document discusses network quality considerations for Lync 2013 voice over IP (VoIP) in local area networks (LANs), wide area networks (WANs), and wireless networks (Wi-Fi). It defines good voice quality as when users do not notice issues. Key VoIP metrics that should be monitored include latency, packet loss, and jitter. Codec choices and available bandwidth impact quality. The document provides bandwidth recommendations and demonstrates tools for planning voice and video capacity. It also covers call admission control, quality of service tagging, wireless access point deployment best practices, and wireless network optimizations from vendors like Aruba, Meru, and Cisco to improve voice quality over Wi-Fi.
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...Amazon Web Services Korea
AWS presented techniques for minimizing video bitrates including using the AV1 codec and quality variable bitrate encoding. They also discussed cloud-native media workflows using AWS services like MediaConnect, MediaLive, and MediaConvert. MediaConnect updates included new entitlement features for syndicating live content between accounts. MediaLive additions focused on statistical multiplexing. MediaConvert gained accelerated transcoding. Other services like MediaPackage and low latency delivery were also covered.
VidOvation is a technology provider that has 50 years of experience delivering video and data communication systems over various network types. It specializes in IPTV and digital signage solutions. VidOvation's systems offer advantages like better quality, lower total cost of ownership, and more flexibility compared to other TV broadcast solutions. Its solutions can be used for applications such as corporate communications, distance learning, digital libraries, and business TV.
An Overview of High Efficiency Video Codec HEVC (H.265)Varun Ravi
The document provides an overview of the High Efficiency Video Coding (HEVC) H.265 standard. It discusses the need for improved video compression standards due to increasing video content and limited bandwidth. HEVC was developed to meet this need by providing around 50% better compression over its predecessor H.264 while still maintaining high video quality. The document describes the various techniques used in HEVC such as improved block partitioning, transform sizes, prediction modes, and entropy coding that help achieve its compression gains. Both hardware and software implementations of HEVC decoders and encoders are discussed.
Video coding standards define bitstream structures and decoding methods for video compression. Popular standards include MPEG-1/2/4 and H.264/HEVC developed by ISO/IEC and ITU-T. Standards are developed through identification of requirements, algorithm development, selection of core techniques, validation testing, and publication. They enable interoperability and future decoding of emerging standards. [/SUMMARY]
This document provides an overview and summary of Arecont Vision products and technology. Some key points:
- Arecont Vision offers a range of megapixel camera products including dual sensor cameras that can capture both color and black and white video.
- Their cameras utilize H.264 compression for high quality video with reduced bandwidth and storage requirements compared to older compression standards.
- They also offer panoramic 180 degree field of view cameras and dual mode cameras that can capture both high resolution stills and 1080p video.
- Features like region of interest, image cropping, and digital PTZ allow more efficient use of bandwidth and storage for surveillance applications.
- The document discusses Arecont Vision's
This document discusses challenges with synchronizing distributed access (DA) network rollouts and video headend upgrades, and proposes a solution. It notes that DA rollouts are putting pressure on video headends to upgrade, but some DA rollouts are delayed. It also notes that existing broadcast equipment may need upgrading regardless of DA. The proposed solution is to upgrade broadcast equipment individually now using a technology that can serve both traditional and DA networks through an advanced EDGE QAM with R-PHY outputs and Video Core. This allows the equipment to be fully prepared for the shift to DA once rollout projects are completed, without compromising broadcast quality of service.
FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Le...Alpen-Adria-Universität
HTTP Adaptive Streaming(HAS) is the most common approach for delivering video content over the Internet. Therequirement to encode the same content at different quality levels(i.e., representations) in HAS is a challenging problem for content providers. Fast multirate encoding approaches try to accelerate this process by reusing information from previously encoded representations. In this paper, we propose to use convolutional neural networks (CNNs) to speed up the encoding of multiple representations with a specific focus on parallel encoding. In parallel encoding, the overall time-complexity is limited to the maximum time-complexity of one of the representations that are encoded in parallel. Therefore, instead of reducing the time-complexity for all representations, the highest time-complexities are reduced. Experimental results show that FaME-ML achieves significant time-complexity savings in parallel encoding scenarios(41%in average) with a slight increase in bitrate and quality degradation compared to the HEVC reference software.
The media landscape changes significantly over the last few years by new content formats, new service offerings, additional consumption devices and new monetization models. Think of Netflix, DAZN, Mediatheks, mobile devices, interactive content, smart TVs, Virtual and Augmented Reality, and so on. Many of these efforts have been realized by a limited usage of standards, but are standards irrelevant? Secondly, more and more services are enabled by latest mobile compute platforms enabling new services and experiences. This presentation will provide an overview some of these trends and will motivate the development of global interop standards. Specific aspects will include the move of linear TV services to the Internet (both mobile and fixed) as well recent advances on Extended Reality and immersive media trends.
Lessons Learned Running a Container Cloud on Apache Hadoop YARNBillie Rinaldi
The document discusses lessons learned from running a container cloud on Apache Hadoop YARN. Some key points include:
- Running over 5.8 million containers and 1.1 million tests on the container cloud with improved hardware utilization and test speed.
- Challenges involved issues like IP management, Docker storage drivers causing kernel panics, user namespacing limitations, and image management.
- Solutions involved allocating IP addresses, testing storage configurations, running containers as a common user, and managing a private image registry for cleanup.
Seattle Video Tech: The Future of SSAI on OTT DevicesDavid Sayed
The document discusses server-side ad insertion (SSAI) techniques for over-the-top (OTT) devices. It finds that while streaming boxes generally support SSAI well, smart TVs and set-top boxes often have limited support. Older or less capable devices may require fallback strategies like removing ads or DRM. Upcoming standards could improve SSAI compatibility going forward, but content providers are advised to focus efforts on recent devices rather than trying to support all devices universally.
Streaming media has evolved significantly over the past 20 years. Early systems in the 1990s used proprietary protocols over UDP and later included pre-roll buffers and adaptive bitrate techniques. Standards like RTSP, 3GPP, and ISMA provided interoperability but relied on complex server implementations. The shift to HTTP in the 2000s simplified delivery using progressive download and then adaptive streaming formats like HLS, DASH, and CMAF that divide media into short segments. These standards separate the media format from the delivery method, enabling delivery via HTTP while supporting features like DRM and playback across different devices and networks.
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/ceva/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit-siegel
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Yair Siegel, Director of Segment Marketing at CEVA, presents the "Fast Deployment of Low-power Deep Learning on CEVA Vision Processors" tutorial at the May 2016 Embedded Vision Summit.
Image recognition capabilities enabled by deep learning are benefitting more and more applications, including automotive safety, surveillance and drones. This is driving a shift towards running neural networks inside embedded devices. But, there are numerous challenges in squeezing deep learning into resource-limited devices. This presentation details a fast path for taking a neural network from research into an embedded implementation on a CEVA vision processor core, making use of CEVA’s neural network software framework. Siegel explains how the CEVA framework integrates with existing deep learning development environments like Caffe, and how it can be used to create low-power embedded systems with neural network capabilities.
With the recent surge in Internet multimedia traffic, the enhancement and improvement of media players, specifically DASH media players happened at an incredible rate. DASH Media players take advantage of adapting a media stream to the network fluctuations by continuously monitoring the network and making decisions in near real-time. The performance of algorithms that are in charge of making such decisions was often difficult to be evaluated and objectively assessed.
CAdViSE provides a Cloud-based Adaptive Video Streaming Evaluation framework for the automated testing of adaptive media players. In this talk, I will introduce the CAdViSE framework, its application, and propose the benefits and advantages that it can bring to every web-based media player development pipeline. To demonstrate the power of CAdViSE in evaluating Adaptive Bitrate (ABR) algorithms I will exhibit its capabilities when combined with objective Quality of Experience (QoE) models. For this talk, my team at Bitmovin/ATHENA has selected the ITU-T P.1203 (mode 1) model in order to execute experiments and calculate the Mean Opinion Score (MOS), and better understand the behavior of a set of well-known ABR algorithms in a real-life setting. The talk will display how we tested and deployed our framework using a modular architecture into a cloud infrastructure. This method yields a massive growth to the number of concurrent experiments and the number of media players that can be evaluated and compared at the same time, thus enabling maximum potential scalability. In my team’s most recent experiments, we used Amazon Web Services (AWS) for demonstration purposes. Another awesome feature of CAdViSE that will be discussed here is the ability to shape the test network with endless network profiles. To do so, we used a fluctuation network profile and a real LTE network trace based on the recorded internet usage of a bicycle commuter in Belgium.
CAdViSE produces comprehensive logs for each media streaming experimental session. These logs can then be applied against different goals, such as objective evaluation to stitch back media segments and conduct subjective evaluations afterwards. In addition, startup delays, stall events, and other media streaming defects can be imitated exactly as they happened during the experimental streaming sessions.
Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective Qo...Alpen-Adria-Universität
Omnidirectional video (ODV) streaming applica- tions are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile- based streaming which allows to have higher quality within the user’s viewport while quality outside the user’s viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is character- ized by high reproducibility since relevant media delivery aspects (including the user’s head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.
On Optimizing Resource Utilization in AVC-based Real-time Video StreamingAlpen-Adria-Universität
Real-time video streaming traffic and related applications have witnessed significant growth in recent years. However, this has been accompanied by some challenging issues, predominantly resource utilization. IP multicasting, as a solution to this problem, suffers from many problems. Using scalable video coding could not gain wide adoption in the industry, due to reduced compression efficiency and additional computational complexity. The emerging software-defined networking (SDN)and network function virtualization (NFV) paradigms enable re-searchers to cope with IP multicasting issues in novel ways. In this paper, by leveraging the SDN and NFV concepts, we introduce a cost-aware approach to provide advanced video coding (AVC)-based real-time video streaming services in the network. In this study, we use two types of virtualized network functions (VNFs): virtual reverse proxy (VRP) and virtual transcoder (VTF)functions. At the edge of the network, VRPs are responsible for collecting clients’ requests and sending them to an SDN controller. Then, executing a mixed-integer linear program (MILP) determines an optimal multicast tree from an appropriate set of video source servers to the optimal group of transcoders. The desired video is sent over the multicast tree. The VTFs transcode the received video segments and stream to the requested VRPs over unicast paths. To mitigate the time complexity of the proposed MILPmodel, we propose a heuristic algorithm that determines a near-optimal solution in a reasonable amount of time. Using theMiniNet emulator, we evaluate the proposed approach and show it achieves better performance in terms of cost and resource utilization in comparison with traditional multicast and unicast approaches.
- The document outlines requirements and a proposed solution for live streaming and video on demand for online lectures using AWS services like MediaLive, MediaPackage, S3, and CloudFront.
- The solution aims to support 20-25 concurrent live streams with low latency, provide video on demand within an hour, and leverage CDN edge servers for content delivery.
- The proposed architecture includes using OBS to push live streams to RTMP endpoints, encoding streams in multiple resolutions and bitrates with MediaLive, storing HLS manifests and segments in S3, and delivering content via CloudFront with MediaPackage.
Description of Microsoft Silverlight technology.
Advantages over "standard streaming", download and progressive download methods.
Silverlight session description and analysis using wireshark
HEVC (High Efficiency Video Coding) is an new video compression standard that can provide around a 50% reduction in bitrate compared to existing standards like H.264. It was first released in 2013 and is designed to improve the efficiency of encoding ultra-high definition (UHD) video formats. HEVC will help enable the distribution and viewing of 4K and higher resolution video by reducing the storage and bandwidth needs for UHD content. It is expected to initially see adoption for over-the-top streaming applications and help increase the number of HD streams that can be delivered by cable and IPTV operators. Widespread adoption of live UHD video will still take time as distribution infrastructure and content creation capabilities evolve to
Managing Transition to HEVC/VP9/AV1 with Multi-Codec StreamingBitmovin Inc
Video streaming is in transition towards the next generation of video codecs, offering to double the quality while lowering the required bandwidth. As the successor crown to the ubiquitous AVC/H.264 is still up for grabs, major content providers and device manufacturers are throwing their weights behind competing formats - HEVC/VP9/AV1 - leading to market fragmentation, specifically within web environments. To deal with this challenge, OTT services need to support multiple codecs in an efficient way. In this presentation, we will discuss how to evaluate the benefits and the tradeoffs of embracing these next generation compression technologies in your media workflow.
Encoding Video for the Web - Webinar from ReelSEO.comMark Robertson ⏩
The document summarizes an online presentation about encoding video for the web. The presentation covered topics like video compression, codecs, containers, bit rates, and tools for encoding. Speakers demonstrated how to create high-quality H.264 encoding settings using free and open source tools like x264 and Handbrake. They provided examples of encoding presets for different purposes and platforms. The presentation concluded with a question and answer session.
Lync Server 2013: Network Quality considerations in LAN, WAN and Wi-FiStåle Hansen
This document discusses network quality considerations for Lync 2013 voice over IP (VoIP) in local area networks (LANs), wide area networks (WANs), and wireless networks (Wi-Fi). It defines good voice quality as when users do not notice issues. Key VoIP metrics that should be monitored include latency, packet loss, and jitter. Codec choices and available bandwidth impact quality. The document provides bandwidth recommendations and demonstrates tools for planning voice and video capacity. It also covers call admission control, quality of service tagging, wireless access point deployment best practices, and wireless network optimizations from vendors like Aruba, Meru, and Cisco to improve voice quality over Wi-Fi.
[AWS Media Symposium 2019] AWS Media Services Innovation - Christer Whitehorn...Amazon Web Services Korea
AWS presented techniques for minimizing video bitrates including using the AV1 codec and quality variable bitrate encoding. They also discussed cloud-native media workflows using AWS services like MediaConnect, MediaLive, and MediaConvert. MediaConnect updates included new entitlement features for syndicating live content between accounts. MediaLive additions focused on statistical multiplexing. MediaConvert gained accelerated transcoding. Other services like MediaPackage and low latency delivery were also covered.
VidOvation is a technology provider that has 50 years of experience delivering video and data communication systems over various network types. It specializes in IPTV and digital signage solutions. VidOvation's systems offer advantages like better quality, lower total cost of ownership, and more flexibility compared to other TV broadcast solutions. Its solutions can be used for applications such as corporate communications, distance learning, digital libraries, and business TV.
An Overview of High Efficiency Video Codec HEVC (H.265)Varun Ravi
The document provides an overview of the High Efficiency Video Coding (HEVC) H.265 standard. It discusses the need for improved video compression standards due to increasing video content and limited bandwidth. HEVC was developed to meet this need by providing around 50% better compression over its predecessor H.264 while still maintaining high video quality. The document describes the various techniques used in HEVC such as improved block partitioning, transform sizes, prediction modes, and entropy coding that help achieve its compression gains. Both hardware and software implementations of HEVC decoders and encoders are discussed.
Video coding standards define bitstream structures and decoding methods for video compression. Popular standards include MPEG-1/2/4 and H.264/HEVC developed by ISO/IEC and ITU-T. Standards are developed through identification of requirements, algorithm development, selection of core techniques, validation testing, and publication. They enable interoperability and future decoding of emerging standards. [/SUMMARY]
This document provides an overview and summary of Arecont Vision products and technology. Some key points:
- Arecont Vision offers a range of megapixel camera products including dual sensor cameras that can capture both color and black and white video.
- Their cameras utilize H.264 compression for high quality video with reduced bandwidth and storage requirements compared to older compression standards.
- They also offer panoramic 180 degree field of view cameras and dual mode cameras that can capture both high resolution stills and 1080p video.
- Features like region of interest, image cropping, and digital PTZ allow more efficient use of bandwidth and storage for surveillance applications.
- The document discusses Arecont Vision's
This document discusses challenges with synchronizing distributed access (DA) network rollouts and video headend upgrades, and proposes a solution. It notes that DA rollouts are putting pressure on video headends to upgrade, but some DA rollouts are delayed. It also notes that existing broadcast equipment may need upgrading regardless of DA. The proposed solution is to upgrade broadcast equipment individually now using a technology that can serve both traditional and DA networks through an advanced EDGE QAM with R-PHY outputs and Video Core. This allows the equipment to be fully prepared for the shift to DA once rollout projects are completed, without compromising broadcast quality of service.
This document discusses streaming video to HTML. It begins with introducing the speaker and their background in building video applications. The rest of the document covers: an overview of video traffic on the internet; how HTTP adaptive streaming works; the different streaming protocols like HLS, Smooth Streaming, and DASH; browser support for streaming; MediaSource Extensions; what DASH is and how it works; building a DASH player using dash.js; and resources for learning more about streaming video.
This document discusses a proof-of-concept for live streaming of video and subtitles within a browser. It uses HTTP Live Streaming to segment video into MP4 files and subtitles into WebVTT text files. A JavaScript player fetches and synchronizes the segments, displaying video using HTML5 video and subtitles via HTML track elements. While functional, the demo has limitations around rendering performance with dense subtitles, lack of seeking support, and HTML5 constraints on cue loading.
Introduction material for 360 Video. This includes Multimedia Pipeline and Rendering pipeline for playback. Comparison of projections for 360 video rendering.
This document discusses challenges and potential solutions for adaptive streaming of 360-degree video. It outlines several main challenges: the new spherical geometry requires new quality metrics; large video sizes are difficult to store, deliver and display; low-latency is needed over bandwidth-limited networks; and uncertainty around which portions users will view. Possible solutions proposed include a geometry-aware quality metric, determining popular representations to store, and navigation-aware adaptive streaming based on predicted user viewpoints. Overall the document argues that optimizing 360-degree video streaming requires understanding user navigation on the spherical domain in order to efficiently deliver high-quality content.
i-Cast - product I built once. Still around. Awesome.Lennart Hagberg
The document discusses standards for delivering digital video content globally to 150 countries and 200 locations. It outlines requirements for digital video delivery including being built on standards, optimized for IE6, scalable and flexible, and delivering content in HD and SD formats using H.264 video codec and AAC audio codec. The workflow involves content being uploaded and stored, then transcoded into different formats and distributed through various channels like websites, RSS feeds, and downloads.
This document describes the serverless media workflow built by the speaker for adding video capabilities to Vingle. It discusses ingesting videos from clients into S3 using transfer acceleration, processing videos with AWS services like Elastic Transcoder and Elemental MediaConvert, and delivering videos via S3 and CloudFront. The architecture evolved over two versions to support features like parallel encoding and bypassing processing when possible to improve performance and reduce costs.
Similar to Performance Measurements of 360◦ Video Streaming to Head-Mounted Displays Over Live 4G Cellular Networks (20)
Kalyan chart satta matka guessing resultsanammadhu484
MATKASATTABOSS.COM IS INDIA'S MOST TRUSTED NO.1 WEBSITE. WE PROVIDE YOU EXACT GUESSING OF THE MATKA RESULT BY OUR TOP GUESSER, MATKASATTABOSS.COM ALWAYS PROVIDES EXACT AND FAST MATKA RESULTS. PLAY SATTA MATKA AND BECOME SATTA KING BY THE HELP OF MATKASATTABOSS.COM. INDIA'S TOP SATTA MATKA MARKET AND THEIR FAST MATKA RESULTS. GET ALL THE RESULTS AND WIN MONEY BY PERFECT KALYAN MATKA TIPS , MATKA GUESSING BY OUR TOP GUESSER AND KALYAN RAJSHREE RAJYOG SWASTIK NATRAAJ BANGLORE BIRLA RAJDHANI MILAN TIME BAZAAR MATKA CHART .
11June 2024. An online pre-engagement session was organized on Tuesday June 11 to introduce the Science Policy Lab approach and the main components of the conceptual framework.
About 40 experts from around the globe gathered online for a pre-engagement session, paving the way for the first SASi-SPi Science Policy Lab event scheduled for June 18-19, 2024 in Malmö. The session presented the objectives for the upcoming Science Policy Lab (S-PoL), which featured a role-playing game designed to simulate stakeholder interactions and policy interventions for food systems transitions. Participants called for the sharing of meeting materials and continued collaboration, reflecting a strong commitment to advancing towards sustainable agrifood systems.
1.) Introduction
Our Movement is not new; it is the same as it was for Freedom, Justice, and Equality since we were labeled as slaves. However, this movement at its core must entail economics.
2.) Historical Context
This is the same movement because none of the previous movements, such as boycotts, were ever completed. For some, maybe, but for the most part, it’s just a place to keep your stable until you’re ready to assimilate them into your system. The rest of the crabs are left in the world’s worst parts, begging for scraps.
3.) Economic Empowerment
Our Movement aims to show that it is indeed possible for the less fortunate to establish their economic system. Everyone else – Caucasian, Asian, Mexican, Israeli, Jews, etc. – has their systems, and they all set up and usurp money from the less fortunate. So, the less fortunate buy from every one of them, yet none of them buy from the less fortunate. Moreover, the less fortunate really don’t have anything to sell.
4.) Collaboration with Organizations
Our Movement will demonstrate how organizations such as the National Association for the Advancement of Colored People, National Urban League, Black Lives Matter, and others can assist in creating a much more indestructible Black Wall Street.
5.) Vision for the Future
Our Movement will not settle for less than those who came before us and stopped before the rights were equal. The economy, jobs, healthcare, education, housing, incarceration – everything is unfair, and what isn’t is rigged for the less fortunate to fail, as evidenced in society.
6.) Call to Action
Our movement has started and implemented everything needed for the advancement of the economic system. There are positions for only those who understand the importance of this movement, as failure to address it will continue the degradation of the people deemed less fortunate.
No, this isn’t Noah’s Ark, nor am I a Prophet. I’m just a man who wrote a couple of books, created a magnificent website: http://www.thearkproject.llc, and who truly hopes to try and initiate a truly sustainable economic system for deprived people. We may not all have the same beliefs, but if our methods are tried, tested, and proven, we can come together and help others. My website: http://www.thearkproject.llc is very informative and considerably controversial. Please check it out, and if you are afraid, leave immediately; it’s no place for cowards. The last Prophet said: “Whoever among you sees an evil action, then let him change it with his hand [by taking action]; if he cannot, then with his tongue [by speaking out]; and if he cannot, then, with his heart – and that is the weakest of faith.” [Sahih Muslim] If we all, or even some of us, did this, there would be significant change. We are able to witness it on small and grand scales, for example, from climate control to business partnerships. I encourage, invite, and challenge you all to support me by visiting my website.
Public Art Is (Re)connection: people, heritage and spacesMarta Pucciarelli
Keynote speech at the Public Art Inside Out Symposium, 7-8 May 2024, organized by Getty Conservation Center and MUDEC in Milan. “Public art is (re)connection” is co-authored with Princess Marilyn Douala Bell.
Gamify it until you make it Improving Agile Development and Operations with ...Ben Linders
So many challenges, so little time. While we’re busy developing software and keeping it operational, we also need to sharpen the saw, but how? Gamification can be a way to look at how you’re doing and find out where to improve. It’s a great way to have everyone involved and get the best out of people.
In this presentation, Ben Linders will show how playing games with the DevOps coaching cards can help to explore your current development and deployment (DevOps) practices and decide as a team what to improve or experiment with.
The games that we play are based on an engagement model. Instead of imposing change, the games enable people to pull in ideas for change and apply those in a way that best suits their collective needs.
By playing games, you can learn from each other. Teams can use games, exercises, and coaching cards to discuss values, principles, and practices, and share their experiences and learnings.
Different game formats can be used to share experiences on DevOps principles and practices and explore how they can be applied effectively. This presentation provides an overview of playing formats and will inspire you to come up with your own formats.
Performance Measurements of 360◦ Video Streaming to Head-Mounted Displays Over Live 4G Cellular Networks
1. Performance Measurements
of 360◦ Video Streaming to
Head-Mounted Displays Over
Live 4G Cellular Networks
Wen-Chih Lo, Ching-Ling Fan,
Shou-Cheng Yen, and Cheng-Hsin Hsu
Department of Computer Science
National Tsing Hua University
Hsin-Chu, Taiwan
2. Introduction
• Using conventional displays to watch broadcast
live events ⇒ passive experience
1360 videos provide immersive experience
4. Mobile VR
• Mobile HMDs
• Samsung Gear VR, Carl Zeiss VR One Plus, Google
Cardboard, even our smartphone
• Experience VR anywhere
• YouTube, Facebook, Discovery, vTime
3
5. Streaming 360° Videos
• 4k resolution in HMD requires 12k resolution for
the whole 360° videos (≈ 135 Mbps in HEVC )
• extremely large file size ⇒ insufficient bandwidth
4Images source: 360Heros
6. Opportunities and Solution
Approaches
• The HMD viewer only gets to see a small part of
the whole 360˚ video (< 1/3 )
• The viewer actively changes the viewing
orientation when rotating his/her head.
⇒ Only stream the current Field-of-View (FoV) of
the viewer
5
FoV
7. • 360° video is split into tiles of sub-videos (spatial)
and independently encoded
• Only the tiles overlapped with the viewer’s FoV are
streamed to the client
Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
α
β
θ
FoV
0°
1 1 1 1
1 1 1 1
1 1 1
0 0 0 0 0 1
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0
0 0
011 1 1 1
1 1 1 1
1 1 1
1 1
6
8. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
7
Seg. 1
Seg. 3
Seg. 2
Low-quality High-quality
9. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
8
Seg. 1
Seg. 3
Seg. 2
Low-quality High-quality
Selected tiles
Available
Bandwidth
Time
Viewer Orientation
0°, 0°
10. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
9
Seg. 3
Seg. 2
Low-quality High-quality
Selected tiles
Available
Bandwidth
Time
Viewer Orientation
-15°, 75°
11. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
10
Seg. 3
Low-quality High-quality
Selected tiles
Available
Bandwidth
Time
Viewer Orientation
89°, 0°
12. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
11
Seg. 1
Seg. 3
Seg. 2
Low-quality High-quality
Selected tiles
Available
Bandwidth
Time
Viewer Orientation
0°, 0°
13. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
12
Seg. 3
Seg. 2
Low-quality High-quality
Selected tiles
Available
Bandwidth
Time
Viewer Orientation
-15°, 75°
14. Tiling with MPEG Dynamic Adaptive
Streaming over HTTP (DASH)
• Tiles are split into temporal segments
• tiles and qualities can change in every segments
13
Seg. 3
Low-quality High-quality
Selected tiles
Available
Bandwidth
Time
Viewer Orientation
89°, 0°
15. Before Streaming 360° Videos
over 4G Networks
• Tiling with MPEG DASH -> reduce bandwidth
• Questions to answer
• How tile size affects the streaming system performance?
• How much bandwidth saving can we get by selectively
requesting useful tiles?
• How many users can be supported in one 4G cell?
14
16. Contributions
• We design and implement an end-to-end 360°
video streaming system to Head Mounted Displays
• We evaluate our system's performance over a real
4G cellular network to answer the three questions
• We collect and share (upon request) the dataset
collected with our system
15
17. Overview of the Tile-Based
Streaming Platform
16
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Internet
Raw Video HMD
HTTP Request
HTTP Response
HEVC
Encoder
MPEG
DASH
MPD
18. Overview of the Tile-Based
Streaming Platform
17
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
• 360° Video Server
HEVC
Encoder
MPEG
DASH
MPD
Internet
HTTP Request
HTTP Response
19. Overview of the Tile-Based
Streaming Platform
18
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
HEVC
Encoder
MPEG
DASH
MPD
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
21. The Pre-Processing Procedure
20
• Split the videos into tiles of sub-videos
• Encode the tiles using motion-constrained HEVC encoder with
different bitrates (qualities)
Input Video Tiling
Motion-
Constrained
HEVC Encoder
DASH with
Multiple Bitrates
Encode tiles into
different qualities
22. The Pre-Processing Procedure
21
• Split the videos into tiles of sub-videos
• Encode the tiles using motion-constrained HEVC encoder with
different bitrates (qualities)
• constrain the tiles encoding so that each tile only refers to the same tiles
in previous or future frames -> avoid decoding glitches
Input Video Tiling
Motion-
Constrained
HEVC Encoder
DASH with
Multiple Bitrates
Encode tiles into
different qualities
23. The Pre-Processing Procedure
22
• Split the videos into tiles of sub-videos
• Encode the tiles using motion-constrained HEVC encoder with
different bitrates (qualities)
• constrain the tiles encoding so that each tile only refers to the same tiles
in previous or future frames -> avoid decoding glitches
• adapt to network condition
Input Video Tiling
Motion-
Constrained
HEVC Encoder
DASH with
Multiple Bitrates
Encode tiles into
different qualities
24. The Pre-Processing Procedure
23
• Split the videos into tiles of sub-videos
• Encode the tiles using motion-constrained HEVC encoder with
different bitrates (qualities)
• Encapsulate tiles into single HEVC bitstream
Input Video Tiling
Motion-
Constrained
HEVC Encoder
DASH with
Multiple Bitrates
Encode tiles into
different qualities
HEVC
Bitstream
Encapsulation
25. The Pre-Processing Procedure
24
• Split the videos into tiles of sub-videos
• Encode the tiles using motion-constrained HEVC encoder with
different bitrates (qualities)
• Encapsulate tiles into single HEVC bitstream
• Integrate with DASH for spatial index generation (MPD and SRD)
Input Video Tiling
Motion-
Constrained
HEVC Encoder
DASH with
Multiple Bitrates
Encode tiles into
different qualities
HEVC
Bitstream
Encapsulation
Base track + Tile tracks
MPD Generator with
SRD Information
MPD File
Representation…..........
BaseURL….....................
Segmentbase................
Segment Representations
and URLs
Spatial Information of Tiles
26. Overview of the Tile-Based
Streaming Platform
25
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
• HTTP Server
HEVC
Encoder
MPEG
DASH
MPD
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
27. Overview of the Tile-Based
Streaming Platform
26
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
HEVC
Encoder
MPEG
DASH
MPD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
• HTTP Server
• Client with HMD
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
28. Overview of the Tile-Based
Streaming Platform
27
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
HEVC
Encoder
MPEG
DASH
MPD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
• HTTP Server
• Client with HMD
• Tile selector
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
29. Overview of the Tile-Based
Streaming Platform
28
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
HEVC
Encoder
MPEG
DASH
MPD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
• HTTP Server
• Client with HMD
• Tile selector
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
30. Overview of the Tile-Based
Streaming Platform
29
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
HEVC
Encoder
MPEG
DASH
MPD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
• HTTP Server
• Client with HMD
• Tile selector
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
MPD
Viewer Orientation
Network Condition
31. Overview of the Tile-Based
Streaming Platform
30
360° Video Server
Pre-Processing HTTP Server
Client with HMD
Tile
Selector
HEVC
Decoder
Raw Video HMD
HEVC
Encoder
MPEG
DASH
MPD
• 360° Video Server
• HEVC[1] encoder
• MPEG DASH[2] content generator
• HTTP Server
• Client with HMD
• Tile selector
• HEVC decoder
Internet
HTTP Request
HTTP Response
[1] G. Sullivan et al. "Overview of the high efficiency video coding (HEVC) standard." Sullivan, Gary J., et al. "Overview of the high
efficiency video coding (HEVC) standard." IEEE Transactions on circuits and systems for video technology 22 (12), 2012, 1649-1668.
[2] ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)
Client with HMD
Viewer Orientation
Network Condition
32. Testbed
• Server
• Kvazaar[1] HEVC encoder
• MP4Box[2] DASH content generator
• Apache HTTP server[3]
• Client
• Oculus Rift DK2[4]
• MP4Client[5]
• 4G LTE Network
• RBS 6601 base station
• HUAWEI E3267 4G dongle
31
[1] Kavazaar, an opensource HEVC encoder. https://github.com/ultravideo/kvazaar
[2] MP4Box , a multimedia packager. https://gpac.wp.imt.fr/mp4box/
[3] Apache HTTP server. https://httpd.apache.org/
[4] Oculus Rift Development Kit 2 (DK2). https://www3.oculus.com/en-us/dk2/
[5] MP4Client, an opensource multimedia player. https://gpac.wp.imt.fr/player/
33. • Number of tiles: {1x1, 3x3, 5x5, 7x7, 9x9}
• DASH segment length: {1, 4, 10} secs
• Bitrates: {3, 6, 12} Mbps
• Viewer’s FoV is randomly chosen from dataset[1]
• Perform 3 times and report the average
Experiment Setup
32[1] W. Lo, C. Fan, J. Lee, C. Huang, K. Chen, and C. Hsu, “360◦ video viewing dataset in head-mounted Virtual
Reality,” in Proc. of ACM MMSys’17
Client with HMDServer
… …
34. Measurement Design
• How tile size affects the streaming system
performance?
• How much bandwidth saving can we get by
selectively requesting useful tiles?
• How many users can be supported in one cell?
33
35. Measurement Design
• How tile size affects the streaming system
performance?
• vary the number of tiles and the bitrates
• How much bandwidth saving can we get by
selectively requesting useful tiles?
• How many users can be supported in one cell?
34
36. The Number of Tiles ↑
⇒ Coding Efficiency ↓
• Due to motion constraints among tiles
35
37. Number of Tiles ↑, Protocol
Overhead ↑
• The majority of the streamed tiles are videos
• The protocol overhead is always less than 3%
36
initial header files and MPD
files
generated during streaming
38. More Tiles Consume More Time
to Download
• Reasons
• Sequentially download
• Tile encapsulation overhead
37
39. Measurement Design
• How tile size affects the streaming system
performance?
• vary the number of tiles and the bitrates
• How much bandwidth saving can we get by
selectively requesting useful tiles?
• modify the client to only request the tiles that will be
watched by viewers
• How many users can be supported in one cell?
38
40. The Bandwidth Saving of FoV-
Based Streaming
• Skipping tiles based on viewer’s FoV saves the
bandwidth by up to 80%
39
80%
41. More Tiles Incurs Minor Video
Quality Drop
• Reduces the amount of data and incurs minor
video quality drop
40
[1]
[1] M. Yu, H. Lakshman, and B. Girod, “A framework to evaluate omnidirectional video coding
schemes,” in Proc. of IEEE International Symposium on Mixed and Augmented Reality (ISMAR’15)
80%
42. Measurement Design
• How tile size affects the streaming system
performance?
• vary the number of tiles and the bitrates
• How much bandwidth saving can we get by
selectively requesting useful tiles?
• modify the client to only request the tiles that will be
watched by viewers
• How many users can be supported in one cell?
• repeat the above experiments but with more clients
41
43. Scalability
• Our streaming testbed can support at least 3 clients
• More number of tiles -> smaller average bandwidth
42
44. Conclusion
• We design measurement experiments to quantify
the performance of VR streaming over cellular
networks
• We build a streaming testbed and conduct
extensive experiments using real user trace[1]
• FoV-based video streaming can save up to 80%
in bandwidth consumption
• More tiles suffer from lower coding efficiency
and late segments
43
[1] W. Lo, C. Fan, J. Lee, C. Huang, K. Chen, and C. Hsu, “360◦ video viewing dataset in head-mounted Virtual
Reality,” in Proc. of ACM MMSys’17
45. • This work can be extended to optimal bitrate
allocation for mobile AR/VR systems with HMDs
Future Work
44