The document discusses the VP8 video codec, including its history, patent situation, applications, and technical details. It was developed by On2 Technologies and later acquired by Google, who open sourced it. While royalty-free, its patent situation is unclear. It is being adopted for HTML5 video through the WebM project and is supported by many browsers, hardware vendors, and libraries. Key technical aspects covered include its color space, block-based encoding, intra-frame prediction methods, motion estimation, and motion vector types.
ELC-NA 2020: War story - Using mainline linux for an Android TV bspNeil Armstrong
The document summarizes the process of using mainline Linux for an Android TV BSP on new Amlogic SoCs. It discusses:
- Android's history with mainline Linux and recent efforts to use a common kernel
- The need to push SoC support upstream and backport patches between Android and mainline trees
- Challenges with hardware abstraction layers originally written for proprietary kernels
- Issues integrating the Mali GPU using the drm-hwcomposer HAL
- Supporting multiple Android boot flows in a single codebase
- Other integration problems with audio and WiFi HALs, and display modes
- That Android is becoming more mainline-friendly but full integration remains a long process.
Shuah Khan from Samsung Open Source Group presented on sharing media resources through the Media Controller API. The talk discussed challenges with media drivers accessing shared resources unaware. It introduced the Managed Media Controller API to create a common locking mechanism. Examples were provided of ALSA and au0828 drivers using it to share a USB TV device's tuner through entity and graph management.
BUD17-104: Scripting Languages in IoT: Challenges and ApproachesLinaro
"Session ID: BUD17-104
Session Name: Scripting Languages in IoT: Challenges and Approaches - BUD17-104
Speaker: Paul Sokolovsky,
Track: LITE
★ Session Summary ★
Scripting languages is hot emerging topic in IoT. They allow easy learnability and rapid prototyping and further benefits (like production use) as they evolve. This session compares approaches of MicroPython and JerryScript/Zephyr.js projects and gives status update on their Zephyr RTOS ports.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/bud17/bud17-104/
Presentation: https://www.slideshare.net/linaroorg/bud17104-scripting-languages-in-iot-challenges-and-approaches
Video: https://youtu.be/lIO8QL2SRuU
---------------------------------------------------
★ Event Details ★
Linaro Connect Budapest 2017 (BUD17)
6-10 March 2017
Corinthia Hotel, Budapest,
Erzsébet krt. 43-49,
1073 Hungary
---------------------------------------------------
Keyword: IoT, scripting languages, Zephyr, LITE, Paul Sokolovsky,
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961"
This document outlines the structure and programming of the PIC16F84A microcontroller. It begins with an introduction to microcontrollers and their components, including the CPU, RAM, EEPROM, and I/O units. It then discusses the specifications of the PIC16F84A, including its 4MHz oscillator, 2mA current, 5V voltage, and pin descriptions. The document concludes by covering the software requirements of MPLAB and Mikroc, the hardware requirements of a PC and programmer, and programming languages used including C and Assembly languages. The PIC16F84A is programmed using Mikroc/MPLAB and a burner program.
The document discusses various 2-D orthogonal and unitary transforms that can be used to represent digital images, including:
1. The discrete Fourier transform (DFT) which transforms an image into the frequency domain and has properties like energy conservation and fast computation via FFT.
2. The discrete cosine transform (DCT) which has good energy compaction properties and is close to the optimal Karhunen-Loeve transform.
3. The discrete sine transform (DST) which is real, symmetric, and orthogonal like the DCT.
4. The Hadamard transform which uses only ±1 values and has a fast computation, and the Haar transform which is a simpler wavelet transform
The document discusses DCT/IDCT concepts and applications. It provides an introduction to DCT and IDCT, explaining that they are used widely in video and audio compression. It describes the DCT and IDCT functions and how they work to transform signals between spatial and frequency domains. Examples of one-dimensional and two-dimensional DCT/IDCT equations are also given. Finally, common applications of DCT/IDCT compression techniques are listed, such as in DVD players, cable TV, graphics cards, and medical imaging systems.
Reaching the multimedia web from embedded platforms with WPEWebkitIgalia
Nowadays the Web is one of the primary ways for multimedia content consumption
and real-time communication (through WebRTC). During this talk Philippe will
present the WPEWebKit web-engine that has been deployed on a wide range of
embedded platforms and how you can add it to your own Linux-based embedded
device. WPEWebKit is the official WebKit upstream port for embedded platforms.
For multimedia playback and real-time communication it heavily relies on the
GStreamer multimedia framework. Philippe will give an overview of the W3C
specifications supported by WPEWebKit. WPEWebKit products have been deployed in
various embedded environments and hardware platforms. Philippe will focus on
i.MX platforms, outlining the steps required to enable WPEWebKit in Yocto-based
BSPs. WPEWebKit can also be used in server-side innovative ways, such as
dynamic HTML/JS/CSS powered video overlaying. Philippe will present this
use-case, detailing how live video streams can be augmented with overlays.
GstWPE is a GStreamer plugin embedding a WPEWebKit WebView, allowing to inject
a live audio/video representation of any Web page into a GStreamer pipeline.
Both GPU-based hardware-accelerated and software rasterisers runtimes are
supported.
(c) Embedded Linux Conference - North America (ELC-NA 2021)
September 27-30, 2021
Hyatt Regency Seattle | Seattle, Washington + Virtual
https://events.linuxfoundation.org/embedded-linux-conference-north-america/
ELC-NA 2020: War story - Using mainline linux for an Android TV bspNeil Armstrong
The document summarizes the process of using mainline Linux for an Android TV BSP on new Amlogic SoCs. It discusses:
- Android's history with mainline Linux and recent efforts to use a common kernel
- The need to push SoC support upstream and backport patches between Android and mainline trees
- Challenges with hardware abstraction layers originally written for proprietary kernels
- Issues integrating the Mali GPU using the drm-hwcomposer HAL
- Supporting multiple Android boot flows in a single codebase
- Other integration problems with audio and WiFi HALs, and display modes
- That Android is becoming more mainline-friendly but full integration remains a long process.
Shuah Khan from Samsung Open Source Group presented on sharing media resources through the Media Controller API. The talk discussed challenges with media drivers accessing shared resources unaware. It introduced the Managed Media Controller API to create a common locking mechanism. Examples were provided of ALSA and au0828 drivers using it to share a USB TV device's tuner through entity and graph management.
BUD17-104: Scripting Languages in IoT: Challenges and ApproachesLinaro
"Session ID: BUD17-104
Session Name: Scripting Languages in IoT: Challenges and Approaches - BUD17-104
Speaker: Paul Sokolovsky,
Track: LITE
★ Session Summary ★
Scripting languages is hot emerging topic in IoT. They allow easy learnability and rapid prototyping and further benefits (like production use) as they evolve. This session compares approaches of MicroPython and JerryScript/Zephyr.js projects and gives status update on their Zephyr RTOS ports.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/bud17/bud17-104/
Presentation: https://www.slideshare.net/linaroorg/bud17104-scripting-languages-in-iot-challenges-and-approaches
Video: https://youtu.be/lIO8QL2SRuU
---------------------------------------------------
★ Event Details ★
Linaro Connect Budapest 2017 (BUD17)
6-10 March 2017
Corinthia Hotel, Budapest,
Erzsébet krt. 43-49,
1073 Hungary
---------------------------------------------------
Keyword: IoT, scripting languages, Zephyr, LITE, Paul Sokolovsky,
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961"
This document outlines the structure and programming of the PIC16F84A microcontroller. It begins with an introduction to microcontrollers and their components, including the CPU, RAM, EEPROM, and I/O units. It then discusses the specifications of the PIC16F84A, including its 4MHz oscillator, 2mA current, 5V voltage, and pin descriptions. The document concludes by covering the software requirements of MPLAB and Mikroc, the hardware requirements of a PC and programmer, and programming languages used including C and Assembly languages. The PIC16F84A is programmed using Mikroc/MPLAB and a burner program.
The document discusses various 2-D orthogonal and unitary transforms that can be used to represent digital images, including:
1. The discrete Fourier transform (DFT) which transforms an image into the frequency domain and has properties like energy conservation and fast computation via FFT.
2. The discrete cosine transform (DCT) which has good energy compaction properties and is close to the optimal Karhunen-Loeve transform.
3. The discrete sine transform (DST) which is real, symmetric, and orthogonal like the DCT.
4. The Hadamard transform which uses only ±1 values and has a fast computation, and the Haar transform which is a simpler wavelet transform
The document discusses DCT/IDCT concepts and applications. It provides an introduction to DCT and IDCT, explaining that they are used widely in video and audio compression. It describes the DCT and IDCT functions and how they work to transform signals between spatial and frequency domains. Examples of one-dimensional and two-dimensional DCT/IDCT equations are also given. Finally, common applications of DCT/IDCT compression techniques are listed, such as in DVD players, cable TV, graphics cards, and medical imaging systems.
Reaching the multimedia web from embedded platforms with WPEWebkitIgalia
Nowadays the Web is one of the primary ways for multimedia content consumption
and real-time communication (through WebRTC). During this talk Philippe will
present the WPEWebKit web-engine that has been deployed on a wide range of
embedded platforms and how you can add it to your own Linux-based embedded
device. WPEWebKit is the official WebKit upstream port for embedded platforms.
For multimedia playback and real-time communication it heavily relies on the
GStreamer multimedia framework. Philippe will give an overview of the W3C
specifications supported by WPEWebKit. WPEWebKit products have been deployed in
various embedded environments and hardware platforms. Philippe will focus on
i.MX platforms, outlining the steps required to enable WPEWebKit in Yocto-based
BSPs. WPEWebKit can also be used in server-side innovative ways, such as
dynamic HTML/JS/CSS powered video overlaying. Philippe will present this
use-case, detailing how live video streams can be augmented with overlays.
GstWPE is a GStreamer plugin embedding a WPEWebKit WebView, allowing to inject
a live audio/video representation of any Web page into a GStreamer pipeline.
Both GPU-based hardware-accelerated and software rasterisers runtimes are
supported.
(c) Embedded Linux Conference - North America (ELC-NA 2021)
September 27-30, 2021
Hyatt Regency Seattle | Seattle, Washington + Virtual
https://events.linuxfoundation.org/embedded-linux-conference-north-america/
Bitmovin AV1/VVC Presentation_Streaming Media East by Christian FeldmannBitmovin Inc
This document provides an overview and update on AV1 and VVC video coding standards. It summarizes the novel technical features of each, including AV1 improvements like overlapped block motion compensation and affine motion, and VVC features like triangle partitioning and decoder-side motion refinement. Performance results show VVC provides around 30% better compression than HM 16.20, while AV1 is around 20% better. Both standards are still in development and neither has wide adoption yet. VVC licensing is unknown while AV1 remains free and open.
This document discusses post-processing and rate distortion algorithms for the VP8 video codec. It first provides background on the need for post-processing algorithms to reduce blocking artifacts in compressed video, and for rate control algorithms to regulate bitrates and achieve high video quality within bandwidth constraints. It then summarizes existing in-loop deblocking filters and post-processing algorithms. A novel optimal post-processing/in-loop filtering algorithm is described that can achieve better performance than H.264/AVC or VP8 by computing optimal filter coefficients. Finally, a proposed rate distortion optimization algorithm for VP8 is discussed to improve its rate control and coding efficiency.
This document discusses GStreamer support and integration in WebKit. It covers:
1. The current integration of GStreamer for HTML5 audio and video playback, as well as WebAudio.
2. Plans for next-generation video rendering using GstGL instead of WebCore's internal video sink.
3. Support for adaptive streaming using Media Source Extensions and how GStreamer implements the "append" and playback pipelines.
4. Protected content playback support using Encrypted Media Extensions for key negotiation and decryption, including potential approaches for secure video rendering.
5. Progress on WebRTC support through the OpenWebRTC project.
GStreamer support in WebKit. What's new? (GStreamer Conference 2015)Igalia
This document discusses GStreamer support and integration in WebKit. It covers:
1. The current integration of GStreamer for HTML5 audio and video playback, as well as WebAudio.
2. Plans for next-generation video rendering using GstGL to leverage hardware acceleration.
3. Support for adaptive streaming using Media Source Extensions and Dash playback via GStreamer.
4. Encrypted media playback support via Encrypted Media Extensions for DRM protected content.
5. Progress on WebRTC integration for real-time communication capabilities.
George Grey, CEO of Linaro, discusses Linaro's mission of leading collaboration in the ARM ecosystem. He outlines trends like ARM expanding from mobile into other areas like sensors and data centers. Linaro's focus is on building shared open source software platforms to enable cross-vendor support and security updates. Future plans include reference platforms for IoT devices, gateways and digital homes to help products "just work" together and receive long term support.
LAS16-100K1: Welcome Keynote
Speakers: George Grey
Date: September 26, 2016
★ Session Description ★
George Grey, CEO of Linaro will welcome attendees to the conference and give an update on the latest projects taking place at Linaro.
★ Resources ★
Etherpad: pad.linaro.org/p/las16-100k1
Presentations & Videos: http://connect.linaro.org/resource/las16/las16-100k1/
★ Event Details ★
Linaro Connect Las Vegas 2016 – #LAS16
September 26-30, 2016
http://www.linaro.org
http://connect.linaro.org
Video Codecs and the Future by Vince PugliaDialogic Inc.
This document discusses video codecs and their role in web-based communications. It provides a brief history of codecs from analog to digital formats. It then focuses on codecs used for WebRTC, including VP8, H.264, VP9 and future developments. It explores the ongoing "game of chess" between codec developers as different browsers and companies support different options. Finally, it discusses efforts by the Alliance for Open Media and IETF to develop future open, high quality video codecs.
- The document describes the Multipilot 32, a flight controller board developed by LASER NAVIGATION.
- It has an ARM Cortex-M3 processor, 512KB flash memory, 64KB RAM, and interfaces for sensors, radios, motors and more.
- The board runs ChibiOS or VROS and supports projects like OpenPilot, AutoPilot and Paparazzi. It is compatible with many sensors, radios and brushless ESC/servos.
- Developers can use IDEs like CooCox, Eclipse, or commercial options, with libraries for sensor control, radio control and more.
- The Multipilot 32 is available for pre-
GeoVision : CCTV Solutions : GV-NVR Lite VS QNAP VS-2008 ProTSOLUTIONS
The document compares the GV-NVR System Lite NVR to the QNAP VS-2008 Pro NVR. It finds that the GV-NVR System Lite provides smoother live video, records video at full frame rates from multiple megapixel cameras, and makes exporting and playing back recorded video files faster and easier than the VS-2008 Pro. The GV-NVR System Lite also offers more flexible recording modes and supports the RTSP and PSIA protocols.
Embedded Recipes 2018 - Upstream multimedia on amlogic so cs from fiction t...Anne Nicolas
Inexpensive set-top boxes are everywhere and many of them are powered by AmLogic SoCs. These chips provide 4K H.265/VP9 video decoding and as of the last couple years, have a very good Mainline Linux and U-Boot support.
Unfortunately, most of the multimedia features, like Hardware Accelerated OpenGL ES, Video Decoding, Audio Playback, and Video Rendering were missing from Upstream projects like Linux, Mesa, Gstreamer or FFmpeg. Thankfully a growing community of independent developers have managed to get most of these features to work using upstream only source code.
Maxime will present the challenges and benefits to have Multimedia support using upstream projects and will detail the technical challenges to offer a complete Multimedia support on these Amlogic SoCs.
The audience is anyone interested in ARM SoCs, multimedia, upstream development process or learning more about the architecture of the AmLogic application processors.
Attendees can expect a presentation on the actual work of code upstreamed for the Amlogic SoCs, a multimedia overview and challenges on non-x86 platforms, an overview of the hardware architecture, and much more.
http://fr.droidcon.com/2014/agenda/
http://fr.droidcon.com/2014/agenda/detail?title=Porting+VLC+on+Android
VLC is a free and open source cross-platform multimedia player and framework that plays most multimedia files, discs and streams. VLC runs on most platforms and is developed by volunteers.
This talk will explain how we ported VLC to Android using the NDK, the issues we faced and how we solved most of them. It will also detail the future of VLC on the Android platform.
Speaker : Jean-Baptiste Kempf, VideoLan
Jean-Baptiste Kempf is the president of the VideoLAN non-profit organization and one of the main developers of VLC.
Jean-Baptiste led some of the ports of VLC on mobile devices, including the Android port, and is the current release manager of VLC.
The document discusses various topics related to video, including:
1. Video parameters such as frame rate, bitrate, compression techniques, scanning methods, and flicker effect.
2. Video compression techniques including intraframe and interframe compression, motion estimation, and motion compensation.
3. Popular video codecs and containers including MPEG, H.264, Theora, and containers like MP4, AVI, and Matroska.
4. Video hardware connections like HDMI, VGA, component video, and their specifications.
This document provides a summary of the AV1 ecosystem as of mid-2019, including:
- The release of optimized AV1 decoders like dav1d that improved decoding speed significantly.
- Growing adoption of AV1 in browsers like Firefox, Chrome, and Edge as well as video players like VLC.
- Continued development of open source AV1 encoders like SVT-AV1 and rav1e.
- Expanding support for AV1 in streaming formats and containers. YouTube paused their deployment of AV1 but it was being used for nearly 1 in 8 videos on the platform.
The document highlights how AV1 is helping to address the bandwidth constraints facing internet video traffic dominated by
Kernel Recipes 2018 - Is Video4Linux ready for all cutting-edge hardware? - E...Anne Nicolas
This document discusses whether the Video4Linux (V4L) APIs are ready to support cutting-edge hardware. It begins with an overview of the current V4L2 APIs and introduces new APIs being developed to support stateful and stateless codecs as well as fences and asynchronous USB video class (UVC). While progress has been made supporting new features, the document concludes that V4L is not fully ready yet for all modern hardware.
An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance fo...Tanya Vernitsky
Learn from codec and encoding experts at Bitmovin and Mozilla - the two companies behind the world's first AV1 playback with HTML5 - as we discover the cool new video tools in this royalty-free video codec from the Alliance for Open Media.
This webinar covers:
+ A bit about the history that led to AV1 and the current state of the codec
+ Discover the cool new tools in AV1 brings and compare performance relative to other codecs
+ Review what's already possible and what to expect next
An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance fo...Bitmovin Inc
Learn abut AV1 from codec and encoding experts at Bitmovin and Mozilla - the two companies behind the world's first AV1 playback with HTML5 - as we discover the cool new video tools in this royalty-free video codec from the Alliance for Open Media.
This webinar covers:
+ A bit about the history that led to AV1 and the current state of the codec
+ Discover the cool new tools in AV1 brings and compare performance relative to other codecs
+ Review what's already possible and what to expect next
This document summarizes a presentation about recent updates to the VLC media player. It notes that VLC 2.2.0 was released with new features like automatic rotation of vertically shot videos, extensions support, and resume playback. It also improved support for numerous codecs and formats. The presentation highlights VLC's growing popularity with over 2 billion downloads and its active development community of over 600 contributors.
Status of the Vulkan Video ecosystem – XDC 2023Igalia
The Vulkan Video extension has been in development since 2018, with the
intention of exposing video codec blocks as a new acceleration API. AVC and
HEVC are shipping with good support for decode, and nascent encode support. AV1
and VP9 are in the works.
While several APIs exist amid the convenience/control/portability tradeoff, the
Vulkan extension exposes a much greater level of fine-grained control,
portability and tight-integration that none of the existing solutions offer, at
the expense of convenience.
In this talk we will very briefly explore the API, cover the recent
developments of the extension, and discuss plans for future work. This talk
will provide background information for anyone interested in integrating codec
acceleration within their application.
(c) X.Org Developer's Conference (XDC) 2023
October 17-19, 2023
A Coruña (Spain)
https://indico.freedesktop.org/event/4/
The document describes the FLV (Flash Video) format for storing H.264 video. It outlines the FLV file structure including the FLV header, tags, and tag types for audio, video, and script data. Specifically, it details the structure of video tags for H.264, including the video tag header and encoding of AVCNALU and composition time. Supported video formats include H.263, Screen Video, VP6, and audio formats like MP3, PCM, AAC.
The document discusses the history and current state of video codecs and containers for web video. It describes how HTML5 initially adopted open standards like Theora but now supports multiple codecs. It introduces Google's new open VP8 codec and the WebM container as an open alternative to the proprietary H.264 codec. Comparisons show VP8 performs comparably to H.264 for most types of video.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Bitmovin AV1/VVC Presentation_Streaming Media East by Christian FeldmannBitmovin Inc
This document provides an overview and update on AV1 and VVC video coding standards. It summarizes the novel technical features of each, including AV1 improvements like overlapped block motion compensation and affine motion, and VVC features like triangle partitioning and decoder-side motion refinement. Performance results show VVC provides around 30% better compression than HM 16.20, while AV1 is around 20% better. Both standards are still in development and neither has wide adoption yet. VVC licensing is unknown while AV1 remains free and open.
This document discusses post-processing and rate distortion algorithms for the VP8 video codec. It first provides background on the need for post-processing algorithms to reduce blocking artifacts in compressed video, and for rate control algorithms to regulate bitrates and achieve high video quality within bandwidth constraints. It then summarizes existing in-loop deblocking filters and post-processing algorithms. A novel optimal post-processing/in-loop filtering algorithm is described that can achieve better performance than H.264/AVC or VP8 by computing optimal filter coefficients. Finally, a proposed rate distortion optimization algorithm for VP8 is discussed to improve its rate control and coding efficiency.
This document discusses GStreamer support and integration in WebKit. It covers:
1. The current integration of GStreamer for HTML5 audio and video playback, as well as WebAudio.
2. Plans for next-generation video rendering using GstGL instead of WebCore's internal video sink.
3. Support for adaptive streaming using Media Source Extensions and how GStreamer implements the "append" and playback pipelines.
4. Protected content playback support using Encrypted Media Extensions for key negotiation and decryption, including potential approaches for secure video rendering.
5. Progress on WebRTC support through the OpenWebRTC project.
GStreamer support in WebKit. What's new? (GStreamer Conference 2015)Igalia
This document discusses GStreamer support and integration in WebKit. It covers:
1. The current integration of GStreamer for HTML5 audio and video playback, as well as WebAudio.
2. Plans for next-generation video rendering using GstGL to leverage hardware acceleration.
3. Support for adaptive streaming using Media Source Extensions and Dash playback via GStreamer.
4. Encrypted media playback support via Encrypted Media Extensions for DRM protected content.
5. Progress on WebRTC integration for real-time communication capabilities.
George Grey, CEO of Linaro, discusses Linaro's mission of leading collaboration in the ARM ecosystem. He outlines trends like ARM expanding from mobile into other areas like sensors and data centers. Linaro's focus is on building shared open source software platforms to enable cross-vendor support and security updates. Future plans include reference platforms for IoT devices, gateways and digital homes to help products "just work" together and receive long term support.
LAS16-100K1: Welcome Keynote
Speakers: George Grey
Date: September 26, 2016
★ Session Description ★
George Grey, CEO of Linaro will welcome attendees to the conference and give an update on the latest projects taking place at Linaro.
★ Resources ★
Etherpad: pad.linaro.org/p/las16-100k1
Presentations & Videos: http://connect.linaro.org/resource/las16/las16-100k1/
★ Event Details ★
Linaro Connect Las Vegas 2016 – #LAS16
September 26-30, 2016
http://www.linaro.org
http://connect.linaro.org
Video Codecs and the Future by Vince PugliaDialogic Inc.
This document discusses video codecs and their role in web-based communications. It provides a brief history of codecs from analog to digital formats. It then focuses on codecs used for WebRTC, including VP8, H.264, VP9 and future developments. It explores the ongoing "game of chess" between codec developers as different browsers and companies support different options. Finally, it discusses efforts by the Alliance for Open Media and IETF to develop future open, high quality video codecs.
- The document describes the Multipilot 32, a flight controller board developed by LASER NAVIGATION.
- It has an ARM Cortex-M3 processor, 512KB flash memory, 64KB RAM, and interfaces for sensors, radios, motors and more.
- The board runs ChibiOS or VROS and supports projects like OpenPilot, AutoPilot and Paparazzi. It is compatible with many sensors, radios and brushless ESC/servos.
- Developers can use IDEs like CooCox, Eclipse, or commercial options, with libraries for sensor control, radio control and more.
- The Multipilot 32 is available for pre-
GeoVision : CCTV Solutions : GV-NVR Lite VS QNAP VS-2008 ProTSOLUTIONS
The document compares the GV-NVR System Lite NVR to the QNAP VS-2008 Pro NVR. It finds that the GV-NVR System Lite provides smoother live video, records video at full frame rates from multiple megapixel cameras, and makes exporting and playing back recorded video files faster and easier than the VS-2008 Pro. The GV-NVR System Lite also offers more flexible recording modes and supports the RTSP and PSIA protocols.
Embedded Recipes 2018 - Upstream multimedia on amlogic so cs from fiction t...Anne Nicolas
Inexpensive set-top boxes are everywhere and many of them are powered by AmLogic SoCs. These chips provide 4K H.265/VP9 video decoding and as of the last couple years, have a very good Mainline Linux and U-Boot support.
Unfortunately, most of the multimedia features, like Hardware Accelerated OpenGL ES, Video Decoding, Audio Playback, and Video Rendering were missing from Upstream projects like Linux, Mesa, Gstreamer or FFmpeg. Thankfully a growing community of independent developers have managed to get most of these features to work using upstream only source code.
Maxime will present the challenges and benefits to have Multimedia support using upstream projects and will detail the technical challenges to offer a complete Multimedia support on these Amlogic SoCs.
The audience is anyone interested in ARM SoCs, multimedia, upstream development process or learning more about the architecture of the AmLogic application processors.
Attendees can expect a presentation on the actual work of code upstreamed for the Amlogic SoCs, a multimedia overview and challenges on non-x86 platforms, an overview of the hardware architecture, and much more.
http://fr.droidcon.com/2014/agenda/
http://fr.droidcon.com/2014/agenda/detail?title=Porting+VLC+on+Android
VLC is a free and open source cross-platform multimedia player and framework that plays most multimedia files, discs and streams. VLC runs on most platforms and is developed by volunteers.
This talk will explain how we ported VLC to Android using the NDK, the issues we faced and how we solved most of them. It will also detail the future of VLC on the Android platform.
Speaker : Jean-Baptiste Kempf, VideoLan
Jean-Baptiste Kempf is the president of the VideoLAN non-profit organization and one of the main developers of VLC.
Jean-Baptiste led some of the ports of VLC on mobile devices, including the Android port, and is the current release manager of VLC.
The document discusses various topics related to video, including:
1. Video parameters such as frame rate, bitrate, compression techniques, scanning methods, and flicker effect.
2. Video compression techniques including intraframe and interframe compression, motion estimation, and motion compensation.
3. Popular video codecs and containers including MPEG, H.264, Theora, and containers like MP4, AVI, and Matroska.
4. Video hardware connections like HDMI, VGA, component video, and their specifications.
This document provides a summary of the AV1 ecosystem as of mid-2019, including:
- The release of optimized AV1 decoders like dav1d that improved decoding speed significantly.
- Growing adoption of AV1 in browsers like Firefox, Chrome, and Edge as well as video players like VLC.
- Continued development of open source AV1 encoders like SVT-AV1 and rav1e.
- Expanding support for AV1 in streaming formats and containers. YouTube paused their deployment of AV1 but it was being used for nearly 1 in 8 videos on the platform.
The document highlights how AV1 is helping to address the bandwidth constraints facing internet video traffic dominated by
Kernel Recipes 2018 - Is Video4Linux ready for all cutting-edge hardware? - E...Anne Nicolas
This document discusses whether the Video4Linux (V4L) APIs are ready to support cutting-edge hardware. It begins with an overview of the current V4L2 APIs and introduces new APIs being developed to support stateful and stateless codecs as well as fences and asynchronous USB video class (UVC). While progress has been made supporting new features, the document concludes that V4L is not fully ready yet for all modern hardware.
An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance fo...Tanya Vernitsky
Learn from codec and encoding experts at Bitmovin and Mozilla - the two companies behind the world's first AV1 playback with HTML5 - as we discover the cool new video tools in this royalty-free video codec from the Alliance for Open Media.
This webinar covers:
+ A bit about the history that led to AV1 and the current state of the codec
+ Discover the cool new tools in AV1 brings and compare performance relative to other codecs
+ Review what's already possible and what to expect next
An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance fo...Bitmovin Inc
Learn abut AV1 from codec and encoding experts at Bitmovin and Mozilla - the two companies behind the world's first AV1 playback with HTML5 - as we discover the cool new video tools in this royalty-free video codec from the Alliance for Open Media.
This webinar covers:
+ A bit about the history that led to AV1 and the current state of the codec
+ Discover the cool new tools in AV1 brings and compare performance relative to other codecs
+ Review what's already possible and what to expect next
This document summarizes a presentation about recent updates to the VLC media player. It notes that VLC 2.2.0 was released with new features like automatic rotation of vertically shot videos, extensions support, and resume playback. It also improved support for numerous codecs and formats. The presentation highlights VLC's growing popularity with over 2 billion downloads and its active development community of over 600 contributors.
Status of the Vulkan Video ecosystem – XDC 2023Igalia
The Vulkan Video extension has been in development since 2018, with the
intention of exposing video codec blocks as a new acceleration API. AVC and
HEVC are shipping with good support for decode, and nascent encode support. AV1
and VP9 are in the works.
While several APIs exist amid the convenience/control/portability tradeoff, the
Vulkan extension exposes a much greater level of fine-grained control,
portability and tight-integration that none of the existing solutions offer, at
the expense of convenience.
In this talk we will very briefly explore the API, cover the recent
developments of the extension, and discuss plans for future work. This talk
will provide background information for anyone interested in integrating codec
acceleration within their application.
(c) X.Org Developer's Conference (XDC) 2023
October 17-19, 2023
A Coruña (Spain)
https://indico.freedesktop.org/event/4/
The document describes the FLV (Flash Video) format for storing H.264 video. It outlines the FLV file structure including the FLV header, tags, and tag types for audio, video, and script data. Specifically, it details the structure of video tags for H.264, including the video tag header and encoding of AVCNALU and composition time. Supported video formats include H.263, Screen Video, VP6, and audio formats like MP3, PCM, AAC.
The document discusses the history and current state of video codecs and containers for web video. It describes how HTML5 initially adopted open standards like Theora but now supports multiple codecs. It introduces Google's new open VP8 codec and the WebM container as an open alternative to the proprietary H.264 codec. Comparisons show VP8 performs comparably to H.264 for most types of video.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfTechgropse Pvt.Ltd.
In this blog post, we'll delve into the intersection of AI and app development in Saudi Arabia, focusing on the food delivery sector. We'll explore how AI is revolutionizing the way Saudi consumers order food, how restaurants manage their operations, and how delivery partners navigate the bustling streets of cities like Riyadh, Jeddah, and Dammam. Through real-world case studies, we'll showcase how leading Saudi food delivery apps are leveraging AI to redefine convenience, personalization, and efficiency.
OpenID AuthZEN Interop Read Out - AuthorizationDavid Brossard
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
GraphRAG for Life Science to increase LLM accuracy
The VP8 Video Codec
1. The VP8 Video Codec
Multimedia Codecs
SS 2011
Thomas Maier <tm061@hdm-stuttgart.de>
Dominik Hübner <dh052@hdm-stuttgart.de>
Sven Pfleiderer <sp055@hdm-stuttgart.de>
2. Problem Definition
● No standardized codec for web video
● Currently used:
● H264: patent licensing royalties needed
● Theora: royalty free, outdated technology
● Heterogenous client hardware
● Bandwidth constraints
3. History
● On2 Technologies developed VP8
● Announced September 2008 to replace VP7
● Acquisition of On2 by Google early 2010
● Open letter from the Free Software Foundation
to Google demanding open sourcing of VP8
30.06.2011 The VP8 Video Codec 3
4. History
● Release of VP8 under a BSD-like license
● Launch of the WebM and WebP projects
● Faster VP8 decoder written by x264 developers
in July 2010
● RFC draft of bitstream guide submitted to IETF
(not as a standard) in January 2011
30.06.2011 The VP8 Video Codec 4
5. Patent Situation
● Patent situation unclear
● VP8 affects patents of h264
● Possible prior art by Nokia in ~2000
● MPEG LA announced a call for patents against
VP8
30.06.2011 The VP8 Video Codec 5
6. The WebM-Project
● Founded by Google in May 2010
● Royalty free media file format
● Open-sourced under a BSD-style license
● Optimized for the web
● Low computational complexity
● Simple container format
● Click and encode
30.06.2011 The VP8 Video Codec 6
7. The WebM-Project
● Container is a subset of Matroska
● VP8 for video
● Vorbis for audio
● *.webm extension
● Internet media types
● video/webm
● audio/webm
30.06.2011 The VP8 Video Codec 7
8. Web Video
● HTML5 video tag < video >
● Replacement for Flash and Silverlight
● Customizable video controls with CSS
● Scriptable with standardized JavaScript APIs
● No standardized video format
● h264
● VP8
● Theora
30.06.2011 The VP8 Video Codec 8
9. Application
Browser Theora H.264 VP8 WebM
Internet Manual 9.0 Manual
Explorer Install Install
Mozilla 3.5 No 4.0
Firefox
Google 3.0 Yes 6.0
Chrome (removed in
future)
Safari Manual 3.1 Manual
Install Install
Opera 10.50 No 10.60
Konquerer 4.4 Depends on Yes
QT
Epiphany 2.28 Depends on Depends on
GStreamer GStreamer
http://en.wikipedia.org/wiki/HTML5_video#Table
30.06.2011 The VP8 Video Codec 9
10. Application
● Youtube successively converts to VP8
● Flash support announced
● Important for DRM
● Skype 5.0
● Nvidia announced 3D support
30.06.2011 The VP8 Video Codec 10
11. Application
Tools and libraries
● GStreamer
● FFmpeg
● libvpx
● ffvp8
30.06.2011 The VP8 Video Codec 11
12. Application
Hardware support
● AMD
● ARM
● Broadcom
● MIPS
● Nvidia
● Texas Instruments
● Open IP for hardware decoders
30.06.2011 The VP8 Video Codec 12
20. Intra Frame Prediction
● Exploits spacial coherence of frames
● Uses already coded blocks within current frame
● Applies to macroblocks in an interframe as well
as to macroblocks in a key frame
● 16x16 luma and 8x8 chroma components are
predicted independently
30.06.2011 The VP8 Video Codec 20
21. Chroma Prediction Modes
● H_PRED
● V_PRED
● DC_PRED
● TM_PRED
30.06.2011 The VP8 Video Codec 21
22. H_PRED
● Horizontal Prediction
● Fills each pixel column with a copy of left
neighboring column (L)
● If current macroblock is on the left column, a
default value of 129 is assigned
30.06.2011 The VP8 Video Codec 22
23. H_PRED
L0
L L1
L2
L3
30.06.2011 The VP8 Video Codec 23
24. V_PRED
● Vertical Prediction
● Fills each pixel row with a copy of the row
above (A)
● If current macroblock is on the top column, a
default value of 127 is assigned
30.06.2011 The VP8 Video Codec 24
25. V_PRED
A0
A
A1 A2 A3
30.06.2011 The VP8 Video Codec 25
26. DC_PRED
● Fills each block with a single value
● This value is the average of the pixels left and
above of the block
● If block is on the top: The average of the left
pixels is used
● If block is on the left: The average of the above
pixels is used
● If block is on the left top corner: A constant
value of 128 is used
30.06.2011 The VP8 Video Codec 26
27. DC_PRED
AVG A0
A
A1 A2 A3
L0
L
L1
L2
L3
30.06.2011 The VP8 Video Codec 27
28. TM_PRED
● TrueMotion Prediction
● Uses above row A, left column L and a pixel P
which is above and left of the block
● Most used intra prediction mode
● X ij =Li A j −P
30.06.2011 The VP8 Video Codec 28
29. TM_PRED
P A0
A
A1 A2 A3
L0
L
L1
L2 X21
L3
X21 = L2 + A1 - P
30.06.2011 The VP8 Video Codec 29
31. Luma Prediction Modes
● Basically all chroma prediction modes
● With 16x16 macroblocks
● Additional B_PRED mode
30.06.2011 The VP8 Video Codec 31
32. B_PRED
● Splits 16x16 macroblock into 16 4x4 sub-blocks
● Each sub-block is independently predicted
● Ten available prediction modes for sub-blocks
30.06.2011 The VP8 Video Codec 32
33. B_PRED Modes
● B_DC_PRED: predict DC using row above and
column
● B_TM_PRED: propagate second differences a
la TM
● B_VE_PRED: predict rows using row above
● B_HE_PRED: predict columns using column to
the left
● B_LD_PRED: southwest (left and down) 45
degree diagonal prediction
30.06.2011 The VP8 Video Codec 33
34. B_PRED Modes
● B_RD_PRED: southeast (right and down)
● B_VR_PRED: SSE (vertical right) diagonal
● B_VL_PRED: SSW (vertical left)
● B_HD_PRED: ESE (horizontal down)
● B_HU_PRED: ENE (horizontal up)
30.06.2011 The VP8 Video Codec 34
36. Motion Estimation
● Determine motion vectors which transform one
frame to another
● Uses motion vectors for 16x16, 16x8, 8x16, 8x8
and 4x4 blocks
● Motion vectors from neighboring blocks can be
referenced
30.06.2011 The VP8 Video Codec 36
37. Motion Estimation
● Motion vector: Horizontal and vertical
displacement
● Only luma blocks are predicted, chroma blocks
are calculated from luma
● Resolution: 1/4 pixel for luma, 1/8 pixel for
chroma
● Chroma vectors are calculated by averaging
vectors from luma blocks
30.06.2011 The VP8 Video Codec 37
38. Motion Vector Types
● MV_NEAREST
● MV_NEAR
● MV_ZERO
● MV_NEW
● MV_SPLIT
30.06.2011 The VP8 Video Codec 38
39. MV_NEAREST
● Re-use non-zero motion vector of last decoded
block
30.06.2011 The VP8 Video Codec 39
40. MV_NEAR
● Re-use non-zero motion vector of second-to-
last decoded block
30.06.2011 The VP8 Video Codec 40
41. MV_ZERO
● Block has not moved
● Block is at the same position as in preceding
frame
30.06.2011 The VP8 Video Codec 41
42. MV_NEW
● New motion vector
● Mode followed by motion vector data
● Data is added to buffer of last encoded blocks
30.06.2011 The VP8 Video Codec 42
43. MV_SPLIT
● Use multiple motion vectors for a macroblock
● Macroblock can be split up into sub-blocks
● Each sub-block can have its own motion vector
● Useful when objects within a macroblock have
different motion characteristics
30.06.2011 The VP8 Video Codec 43
45. Motion Compensation
● Apply motion vectors to previous frame
● Generate a predicted frame
● Only difference between predicted and actual
frame needs to be transmitted
30.06.2011 The VP8 Video Codec 45
46. Sub-pixel Interpolation
● If “full pixel” motion vector, block is copied to
corresponding piece of the prediction buffer
● If at least one of the displacements affects sub-
pixels, missing pixels are synthesized by
horizontal and vertical interpolation
30.06.2011 The VP8 Video Codec 46
48. Inter Frame Prediction
Exploits the temporal coherence
between nearby frames
Components:
● Reference Frames
● Motion Vectors
30.06.2011 The VP8 Video Codec 48
49. Inter-Frame Types
● Key Frames
● Decoded without reference to other frames
● Provide seeking points
● Predicted Frames
● Decoding depends on all prior frames up to last
Key-Frame
● No usage of B-Frames
30.06.2011 The VP8 Video Codec 49
50. Prediction Frame Types
● Previous Frame
● Alternate Reference Frame
● Golden Reference Frame
● Each of these three types can be used for
prediction
30.06.2011 The VP8 Video Codec 50
51. Previous Frame
● Last fully decoded frame
● Updated with every shown frame
30.06.2011 The VP8 Video Codec 51
52. Alternate Reference Frame
● Fully decoded frame buffer
● Can be used for noise reduced prediction
● In combination with golden frames:
Compensate lack of B-frames
30.06.2011 The VP8 Video Codec 52
53. Golden Reference Frame
● Fully decoded image buffer
● Can be partially updated
● Can be used for error recovery
● Can be used to encode a cut between scenes
30.06.2011 The VP8 Video Codec 53
54. Updating Frame Buffers
● Key frame: Updates all three buffers
● Predicted frame: Flag for updating alternate or
golden frame buffer
30.06.2011 The VP8 Video Codec 54
58. Decorrelation
● Necessary for efficient entropy encoding
● Achieved with hybrid transformation
● Discrete Cosine Transformation
● Walsh-Hadamard Transformation
30.06.2011 The VP8 Video Codec 58
59. Transformation
Preparation for transfomation process: Divide
Macroblocks into Subblocks
16 4
16 * Y
4
Y
16
4
8 4* U
4
4
U/V
8
4* V
4
Frame Macroblock Subblock
30.06.2011 The VP8 Video Codec 59
60. Discrete Cosine Transformation
● 16 luma blocks / 4 + 4 chroma blocks
● Transform each block into spectral components
using the 2D - DCT
∣ ∣ ∣ ∣
255 0 255 0 510 195.1686 0 471.1786
255 0 255 0 DCT 0 0 0 0
255 0 255 0 0 0 0 0
255 0 255 0 0 0 0 0
Values based on dct2() function of Matlab
30.06.2011 The VP8 Video Codec 60
61. Transformation
The DC components of all subblocks are often
correlated among each other
∣ ∣ ∣ ∣
Macroblocks 85 85 85 85 340 0 0 0
85 85 85 85 DCT 0 0 0 0
85 85 85 85 0 0 0 0
85 85 85 85 0 0 0 0
∣ ∣ ∣ ∣
145 145 145 145 580 0 0 0
145 145 145 145 DCT 0 0 0 0
145 145 145 145 0 0 0 0
145 145 145 145 0 0 0 0
Values based on dct2() function of Matlab
30.06.2011 The VP8 Video Codec 61
62. Walsh-Hadamard Transformation
● Use the correlation of the DC components with a 2nd
order transformation
● The WHT works with a simple transformation matrix
→ Transformation is a matrix multiplication
∣ ∣ ∣ ∣
1 1 1 1 1 1 1 1
1 1 −1 −1 1 1 1 −1 −1
H= H=
1 −1 1 −1 4 1 −1 1 −1
1 −1 −1 1 1 −1 −1 1
Normalized Walsh-Hadamard matrix
30.06.2011 The VP8 Video Codec 62
66. Quantization
● Quantization of the transformation coefficients:
● Less data per coefficient
● More zeros!
● Scalar quantization
● Designed for quality range of
~30dB to ~45dB SNR
30.06.2011 The VP8 Video Codec 66
67. Quantization
For each frame different factors for:
● 1st order luma DC
1st order luma
● 1st order luma AC (DCT)
● 2nd order luma DC DC
nd
2 order luma
● 2nd order luma AC (WHT) AC
● Chroma DC DC
● Chroma AC AC Chroma
(DCT)
DC
AC
30.06.2011 The VP8 Video Codec 67
68. Quantization
● 128 quantization levels with given factors
● Quantization table for DC coefficients in Y1 planes
30.06.2011 The VP8 Video Codec 68
69. Quantization
Example: 1st order luma AC coefficients
● Quantization level: 3
→ Quantization factor from table: 6
● DC coefficient is ignored here
∣ ∣ ∣ ∣
−312 7 1 0 0 1 0 0
1 12 −5 2 1 0 2 −1 0
A= Q=round ∗ A =
2 −3 3 −1 6 0 −1 1 0
1 0 −2 1 0 0 0 0
30.06.2011 The VP8 Video Codec 69
70. Quantization
Adaptive Quantization
● Up to 4 different segments (q0-q3)
● Each segment with n macroblocks and its own
quantization parameter set
30.06.2011 The VP8 Video Codec 70
71. Quantization
The quantized coefficients are read in zig-zag
order
-19 1 0 0
0 2 -1 0
-1 0 0 0
0 0 0 0
vals=[−19, 1, 0,−1, 2, 0 ,0 ,−1 ,0 , 0, 0, 0, 0, 0, 0, 0]
30.06.2011 The VP8 Video Codec 71
75. Adaptive Loop Filtering
● VP8 has two filter modes
– Simple
– Normal
● Configuration in frame-header
● Two parameters
– loop_filter_level
– sharpness_level
30.06.2011 The VP8 Video Codec 75
76. Adaptive Loop Filtering
● Filter order per macroblock 1
1. Left macroblock edge
2. Vertical subblock edges
3. Macroblock edge at the top 3 2
4. Horizontal subblock edges
4
● Macroblock processing in
scan line order
30.06.2011 The VP8 Video Codec 76
77. Adaptive Loop Filtering
Filter segments
● n segments per edge 2
n = blocklength 4
● 2,4,6 or 8 taps wide
6
● Pixels before edge: px
8
● Pixels after edge: qx
p2 p1 p0 q0 q1 q2
30.06.2011 The VP8 Video Codec 77
78. Adaptive Loop Filtering
Simple Mode
● Segments 4 or 6 taps wide
● sharpness_level ignored
● Filter edge if total difference > threshold
● Threshold derived from loop_filter_level,
quantization level and other factors
30.06.2011 The VP8 Video Codec 78
80. Adaptive Loop Filtering
Normal Mode
● Segments 2,4,6 or 8 taps wide
● Different adjustments for different positions
● Different weightings for inner positions
30.06.2011 The VP8 Video Codec 80
81. Adaptive Loop Filtering
Adaptive?
Heavy Motion
→ Strong Filtering
No Motion
→
No Filtering
Low Motion
→
Slight Filtering
30.06.2011 The VP8 Video Codec 81
82. Adaptive Loop Filtering
SIMD processors aka Vector CPUs
● Loop filter optimized for SIMD operations
● Sources already implemented
30.06.2011 The VP8 Video Codec 82
83. Adaptive Loop Filtering
Problem: Dependencies between macroblocks
m Macroblocks
0 1
m
30.06.2011 The VP8 Video Codec 83
86. Frame Format
● Frames are divided in 3 partitions
● Uncompressed header chunk
● Macroblock coding modes and motion vectors
● Quantized transform coefficients
Frame
Partition 1 Partition 2
Header
30.06.2011 The VP8 Video Codec 86
87. Entropy Encoding
● Entropy coding minifies redundancy
● 2 steps
● Huffman Tree with a small alphabet
● Binary arithmetic coding
30.06.2011 The VP8 Video Codec 87
88. Entropy Encoding
● DCT and WHT coefficients are precoded to
tokens using a predefined tree structure
● Goal
● Reduce number of reads from raw binary stream
● Solution
● Create tokens for symbol values
● Minimize necessary reads for most frequent
symbols
30.06.2011 The VP8 Video Codec 88
89. Entropy Encoding
● Token types
● Single numbers
– Coefficient value
● 0, 1, 2, 3, 4
● Number ranges
– 6 ranges of coefficient values
● 5-6, 7-10, 11-18, 19-34, 35-66, 67-2048
● EOB (End Of Block)
– No more non-zeros values remaining in macroblock
30.06.2011 The VP8 Video Codec 89
90. Entropy Encoding
● How are these tokens created?
● Step 1: Read quantized DCT/WHT coefficients
from 4x4 sub-blocks
∣ ∣
187 0 0 0
2 0 0 0 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
1 0 0 0
0 0 0 0
30.06.2011 The VP8 Video Codec 90
91. Entropy Encoding
● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
Output:
30.06.2011 The VP8 Video Codec 91
92. Entropy Encoding
● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
Output: 11111111
30.06.2011 The VP8 Video Codec 92
93. Entropy Encoding
● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
Output: 11111111 10
30.06.2011 The VP8 Video Codec 93
94. Entropy Encoding
● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
Output: 11111111 10 1100
Why not 11100?
We can save 1 bit!
30.06.2011 The VP8 Video Codec 94
95. Entropy Encoding
● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
Output: 11111111 10 1100 110
30.06.2011 The VP8 Video Codec 95
96. Entropy Encoding
● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
Output: 11111111 10 1100 110 0
30.06.2011 The VP8 Video Codec 96
97. Entropy Encoding
● Restoring coefficients from value ranges
● Add some extra bits as offset from base of the
current range
Output: 11111111 10 1100 110 0
Range: 67 – 2048
Number: 187 Extra Bits: 11
Offset: 187 – 67 = 120 Binary Offset: 0000 0111 1000
New Output: 11111111 0000 0111 1000 10 1100 110 0
30.06.2011 The VP8 Video Codec 97
98. Entropy Encoding
● Binary arithmetic encoding
● Extra bits are encoded with pre-set, constant
probabilities
● Token probabilities reside in 96 probability tables
● Token bits are encoded with
– Default probabilities whenever keyframes are updated
– Regarding probability tables can be updated with each
new frame
30.06.2011 The VP8 Video Codec 98
99. Entropy Encoding
● Binary arithmetic encoding
● Token probability tables are chosen according to 3
contexts
– Plane (Y, U, V)
– Band (position of the coefficient)
– Local complexity (value of the preceding coefficient)
30.06.2011 The VP8 Video Codec 99
100. Entropy Encoding
● Binary arithmetic encoding
30.06.2011 The VP8 Video Codec 100
102. Parallel Processing
● Partition 2 (DCT/WHT coefficients) can be
divided in 8 sub-partitions
Frame
Partition 1 Partition 2
Header
Sub- Sub- Sub- Sub- Sub-
Partition 1 Partition 2 Partition 3 Partition ... Partition 8
30.06.2011 The VP8 Video Codec 102
103. Parallel Processing
● Partition 2 (DCT/WHT coefficients) can be
divided sub-partitions
● Support for up to 8 cores
Core 1
Core 2
Core 3
Core 4
30.06.2011 The VP8 Video Codec 103
108. Conclusions
● “Good enough” for web video
● Maybe new default choice for web video
● “Thereʼs no way in hell anyone could write a
decoder solely with this spec alone.” - x264
developer
● Patent situation still unclear
30.06.2011 The VP8 Video Codec 108