SlideShare a Scribd company logo
1 of 8
Download to read offline
Implementing a Vulkan decoder
Víctor Jáquez
Igalia
The 5th
Vulkan Developer Conference
Munich, Germany / February 7–9
Platinum Sponsors:
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
ººº
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
API Overview
 Semi-stateless
 Low level
 Fine grained
● Each driver implementation can operate differently
● Verbose
 Big specification
 Create a session parameters object
 Add new parameters
 Destroy session parameters object
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Stream Parameters Sets
This approach is very different from current parsers.
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Picture references (DPB)
 References and output coincide
 Distinct references and output sets
● references are a set of images
● references are in a single multi-
layered image
General use implementation should manage each case.
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Bitstream buffers
 Upload all slices
● Extra memory copies
● VK_EXT_external_memory_host
 Prepend codec start-code
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
Multiplanar Images
 VK_FORMAT_G8_B8R8_2PLANE_420_UNORM → NV12
● YUV420 2 planes in single image
 Currently, multiplane images are emulated with
separate images per-plane
GStreamer format mapping issue:
2 Vulkan image layouts = 1 GStreamer format
Implementing a Vulkan Video decoder
Víctor Jáquez, February 2023
GStreamer Capabilities
template
 At element registering time
● Query all possible decoding devices
● Query all profiles each device is
capable
● Query all color formats each
device/profile can output

More Related Content

Similar to Implementing a Vulkan Decoder

Similar to Implementing a Vulkan Decoder (20)

Vulkan Video in GStreamer
Vulkan Video in GStreamerVulkan Video in GStreamer
Vulkan Video in GStreamer
 
Spring Update | July 2023
Spring Update | July 2023Spring Update | July 2023
Spring Update | July 2023
 
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...
 
MOVED: RDK/WPE Port on DB410C - SFO17-206
MOVED: RDK/WPE Port on DB410C - SFO17-206MOVED: RDK/WPE Port on DB410C - SFO17-206
MOVED: RDK/WPE Port on DB410C - SFO17-206
 
Vulkan Update Japan Virtual Open House Feb 2021
Vulkan Update Japan Virtual Open House Feb 2021Vulkan Update Japan Virtual Open House Feb 2021
Vulkan Update Japan Virtual Open House Feb 2021
 
WSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on Containers
WSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on ContainersWSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on Containers
WSO2ConEU 2016 Tutorial - Deploying WSO2 Middleware on Containers
 
Deploying WSO2 Middleware on Containers
Deploying WSO2 Middleware on ContainersDeploying WSO2 Middleware on Containers
Deploying WSO2 Middleware on Containers
 
ENIB 2015-2016 - CAI Web - S01E01- La forge JavaScript
ENIB 2015-2016 - CAI Web - S01E01- La forge JavaScriptENIB 2015-2016 - CAI Web - S01E01- La forge JavaScript
ENIB 2015-2016 - CAI Web - S01E01- La forge JavaScript
 
Making your app soar without a container manifest
Making your app soar without a container manifestMaking your app soar without a container manifest
Making your app soar without a container manifest
 
8 Years of Open Drivers, including the State of Vulkan in Mesa
8 Years of Open Drivers, including the State of Vulkan in Mesa8 Years of Open Drivers, including the State of Vulkan in Mesa
8 Years of Open Drivers, including the State of Vulkan in Mesa
 
Environment management in a continuous delivery world (3)
Environment management in a continuous delivery world (3)Environment management in a continuous delivery world (3)
Environment management in a continuous delivery world (3)
 
ENIB 2015 2016 - CAI Web S02E03- Forge JS 1/4 - La forge JavaScript
ENIB 2015 2016 - CAI Web S02E03- Forge JS 1/4 - La forge JavaScriptENIB 2015 2016 - CAI Web S02E03- Forge JS 1/4 - La forge JavaScript
ENIB 2015 2016 - CAI Web S02E03- Forge JS 1/4 - La forge JavaScript
 
CI/CD Across Multiple Environments
CI/CD Across Multiple EnvironmentsCI/CD Across Multiple Environments
CI/CD Across Multiple Environments
 
Red Hat Forum Benelux 2015
Red Hat Forum Benelux 2015Red Hat Forum Benelux 2015
Red Hat Forum Benelux 2015
 
Pivotal Cloud Foundry 2.5: A First Look
Pivotal Cloud Foundry 2.5: A First LookPivotal Cloud Foundry 2.5: A First Look
Pivotal Cloud Foundry 2.5: A First Look
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kubernetes
 
[20200720]cloud native develoment - Nelson Lin
[20200720]cloud native develoment - Nelson Lin[20200720]cloud native develoment - Nelson Lin
[20200720]cloud native develoment - Nelson Lin
 
Silverlight
SilverlightSilverlight
Silverlight
 
GEF 5.0.0 - From a User's Perspective
GEF 5.0.0 - From a User's PerspectiveGEF 5.0.0 - From a User's Perspective
GEF 5.0.0 - From a User's Perspective
 
Docker primer and tips
Docker primer and tipsDocker primer and tips
Docker primer and tips
 

More from Igalia

Building End-user Applications on Embedded Devices with WPE
Building End-user Applications on Embedded Devices with WPEBuilding End-user Applications on Embedded Devices with WPE
Building End-user Applications on Embedded Devices with WPE
Igalia
 
Automated Testing for Web-based Systems on Embedded Devices
Automated Testing for Web-based Systems on Embedded DevicesAutomated Testing for Web-based Systems on Embedded Devices
Automated Testing for Web-based Systems on Embedded Devices
Igalia
 
Running JS via WASM faster with JIT
Running JS via WASM      faster with JITRunning JS via WASM      faster with JIT
Running JS via WASM faster with JIT
Igalia
 
Introducción a Mesa. Caso específico dos dispositivos Raspberry Pi por Igalia
Introducción a Mesa. Caso específico dos dispositivos Raspberry Pi por IgaliaIntroducción a Mesa. Caso específico dos dispositivos Raspberry Pi por Igalia
Introducción a Mesa. Caso específico dos dispositivos Raspberry Pi por Igalia
Igalia
 

More from Igalia (20)

A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Building End-user Applications on Embedded Devices with WPE
Building End-user Applications on Embedded Devices with WPEBuilding End-user Applications on Embedded Devices with WPE
Building End-user Applications on Embedded Devices with WPE
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Automated Testing for Web-based Systems on Embedded Devices
Automated Testing for Web-based Systems on Embedded DevicesAutomated Testing for Web-based Systems on Embedded Devices
Automated Testing for Web-based Systems on Embedded Devices
 
Embedding WPE WebKit - from Bring-up to Maintenance
Embedding WPE WebKit - from Bring-up to MaintenanceEmbedding WPE WebKit - from Bring-up to Maintenance
Embedding WPE WebKit - from Bring-up to Maintenance
 
Optimizing Scheduler for Linux Gaming.pdf
Optimizing Scheduler for Linux Gaming.pdfOptimizing Scheduler for Linux Gaming.pdf
Optimizing Scheduler for Linux Gaming.pdf
 
Running JS via WASM faster with JIT
Running JS via WASM      faster with JITRunning JS via WASM      faster with JIT
Running JS via WASM faster with JIT
 
To crash or not to crash: if you do, at least recover fast!
To crash or not to crash: if you do, at least recover fast!To crash or not to crash: if you do, at least recover fast!
To crash or not to crash: if you do, at least recover fast!
 
Introducción a Mesa. Caso específico dos dispositivos Raspberry Pi por Igalia
Introducción a Mesa. Caso específico dos dispositivos Raspberry Pi por IgaliaIntroducción a Mesa. Caso específico dos dispositivos Raspberry Pi por Igalia
Introducción a Mesa. Caso específico dos dispositivos Raspberry Pi por Igalia
 
2023 in Chimera Linux
2023 in Chimera                    Linux2023 in Chimera                    Linux
2023 in Chimera Linux
 
Building a Linux distro with LLVM
Building a Linux distro        with LLVMBuilding a Linux distro        with LLVM
Building a Linux distro with LLVM
 
turnip: Update on Open Source Vulkan Driver for Adreno GPUs
turnip: Update on Open Source Vulkan Driver for Adreno GPUsturnip: Update on Open Source Vulkan Driver for Adreno GPUs
turnip: Update on Open Source Vulkan Driver for Adreno GPUs
 
Graphics stack updates for Raspberry Pi devices
Graphics stack updates for Raspberry Pi devicesGraphics stack updates for Raspberry Pi devices
Graphics stack updates for Raspberry Pi devices
 
Delegated Compositing - Utilizing Wayland Protocols for Chromium on ChromeOS
Delegated Compositing - Utilizing Wayland Protocols for Chromium on ChromeOSDelegated Compositing - Utilizing Wayland Protocols for Chromium on ChromeOS
Delegated Compositing - Utilizing Wayland Protocols for Chromium on ChromeOS
 
MessageFormat: The future of i18n on the web
MessageFormat: The future of i18n on the webMessageFormat: The future of i18n on the web
MessageFormat: The future of i18n on the web
 
Replacing the geometry pipeline with mesh shaders
Replacing the geometry pipeline with mesh shadersReplacing the geometry pipeline with mesh shaders
Replacing the geometry pipeline with mesh shaders
 
I'm not an AMD expert, but...
I'm not an AMD expert, but...I'm not an AMD expert, but...
I'm not an AMD expert, but...
 
Status of Vulkan on Raspberry
Status of Vulkan on RaspberryStatus of Vulkan on Raspberry
Status of Vulkan on Raspberry
 
Enable hardware acceleration for GL applications without glamor on Xorg modes...
Enable hardware acceleration for GL applications without glamor on Xorg modes...Enable hardware acceleration for GL applications without glamor on Xorg modes...
Enable hardware acceleration for GL applications without glamor on Xorg modes...
 
Async page flip in DRM atomic API
Async page flip in DRM  atomic APIAsync page flip in DRM  atomic API
Async page flip in DRM atomic API
 

Recently uploaded

Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Lisi Hocke
 

Recently uploaded (20)

Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea Goulet
 
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
 
Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...
Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...
Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdf
 
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
Workshop -  Architecting Innovative Graph Applications- GraphSummit MilanWorkshop -  Architecting Innovative Graph Applications- GraphSummit Milan
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
 
Auto Affiliate AI Earns First Commission in 3 Hours..pdf
Auto Affiliate  AI Earns First Commission in 3 Hours..pdfAuto Affiliate  AI Earns First Commission in 3 Hours..pdf
Auto Affiliate AI Earns First Commission in 3 Hours..pdf
 
A Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfA Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdf
 
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
 
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
 
GraphSummit Milan - Visione e roadmap del prodotto Neo4j
GraphSummit Milan - Visione e roadmap del prodotto Neo4jGraphSummit Milan - Visione e roadmap del prodotto Neo4j
GraphSummit Milan - Visione e roadmap del prodotto Neo4j
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Era
 
Food Delivery Business App Development Guide 2024
Food Delivery Business App Development Guide 2024Food Delivery Business App Development Guide 2024
Food Delivery Business App Development Guide 2024
 
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaUNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST API
 
Weeding your micro service landscape.pdf
Weeding your micro service landscape.pdfWeeding your micro service landscape.pdf
Weeding your micro service landscape.pdf
 
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
 

Implementing a Vulkan Decoder

  • 1. Implementing a Vulkan decoder Víctor Jáquez Igalia The 5th Vulkan Developer Conference Munich, Germany / February 7–9 Platinum Sponsors:
  • 2. Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 ººº
  • 3. Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 API Overview  Semi-stateless  Low level  Fine grained ● Each driver implementation can operate differently ● Verbose  Big specification
  • 4.  Create a session parameters object  Add new parameters  Destroy session parameters object Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 Stream Parameters Sets This approach is very different from current parsers.
  • 5. Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 Picture references (DPB)  References and output coincide  Distinct references and output sets ● references are a set of images ● references are in a single multi- layered image General use implementation should manage each case.
  • 6. Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 Bitstream buffers  Upload all slices ● Extra memory copies ● VK_EXT_external_memory_host  Prepend codec start-code
  • 7. Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 Multiplanar Images  VK_FORMAT_G8_B8R8_2PLANE_420_UNORM → NV12 ● YUV420 2 planes in single image  Currently, multiplane images are emulated with separate images per-plane GStreamer format mapping issue: 2 Vulkan image layouts = 1 GStreamer format
  • 8. Implementing a Vulkan Video decoder Víctor Jáquez, February 2023 GStreamer Capabilities template  At element registering time ● Query all possible decoding devices ● Query all profiles each device is capable ● Query all color formats each device/profile can output