Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Cairosink, GPU Buffer Sharing
...and other random ramblings
Guillaume Emont - Igalia
guijemont@igalia.com
ongoing experiment, I may not have the best answers yet
Work sponsored by Samsung Research America
http://www.sisa.samsung.com/
I. Technologies, past and present
Cairo: 2D graphics
Cairogles: 2D graphics in OpenGL(ES)
support for EGL/GLX/WGL
Sharing video buffers: before
- Add a specific cap
(e.g. video/x-raw-gl)
- subclass GstBuffer
Sharing video buffers: nowadays
- GstContext
- GstVideoGLTextureUploadMeta
- GstAllocator and GstMemory
- GstContext, system to share a GL context and more between elements
- GstVideoGLTextureUploadMeta: a way to tell how to u...
Sharing video buffers: nowadays
"modern" gl(es) sink implementations
known to the author:
- eglglessink (and libgstegl)
- g...
II. Integrating cairogles and GStreamer
Many possibilities:
- xvimagesink + GstVideoOverlay
- eglglessink
- glimagesink
- make our own sink
xvimagesink with overlay: not so fast, I suspect it triggers a download to
CPU to get the image from xvimagesink's Drawabl...
First attempt:
a cairosink that does everything
Cairosink: status
- GstAllocator and GstMemory subclasses
- Use either OpenGL or OpenGLES
- Possibility to use PixelBuffer ...
allocator and memory not that useful as it's the only element using it. Main
advantage: _could_ allow to start uploading t...
Cairosink: simplistic benchmark
Synchronous / PBO / XV
XV: limited by refresh rate? Looks like its genuine limit (often below 60)
frames 720x400, RGB or Y...
Cairosink: difficulties (or rants)
- cairogles can be compiled with
either gl or glesv2 (not both)
- driver bugs and regress...
driver regression: GL+EGL used to work on my machine...
Desktop vs mobile GPU
- EGLMakeCurrent() is costly on many mobile GPUs/drivers
- bandwidth differences
context acquisition (makeCurrent) slowness is likely to counter the effects of asynchronous
direct upload
Cairosink: what's to do
- more flexibility on code path
- compatibility with other elements
(not our own memory/allocator)
we want to be able to tailor to various GPUs various "options": is it faster
to do a synchronous or asynchronous upload?
Cairosink: other ideas to explore
- libgstgl backend (ongoing experiment)
- wrapper around glimagesink
libgstgl issues:
mechanism to MakeCurrent() or save/restore states
share GMainContext? (all in one thread)
Thank you
Questions?
Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)
Upcoming SlideShare
Loading in …5
×

Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)

1,199 views

Published on

By Guillaume Emont.

GStreamer 1.0 introduced a new architecture that makes memory management easier. In particular, handling non-system memory and sharing it between elements is made easier.

While building a video sink for cairogles, Guillaume wanted to explore the possibility of sharing GPU memory (in his case, GL or GLES textures) between elements of a pipeline. In this talk, Guillaume will explain how he made the various parts fit together, and discuss the challenges that he has met and how he has worked on them.

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)

  1. 1. Cairosink, GPU Buffer Sharing ...and other random ramblings Guillaume Emont - Igalia guijemont@igalia.com
  2. 2. ongoing experiment, I may not have the best answers yet
  3. 3. Work sponsored by Samsung Research America http://www.sisa.samsung.com/
  4. 4. I. Technologies, past and present
  5. 5. Cairo: 2D graphics Cairogles: 2D graphics in OpenGL(ES)
  6. 6. support for EGL/GLX/WGL
  7. 7. Sharing video buffers: before - Add a specific cap (e.g. video/x-raw-gl) - subclass GstBuffer
  8. 8. Sharing video buffers: nowadays - GstContext - GstVideoGLTextureUploadMeta - GstAllocator and GstMemory
  9. 9. - GstContext, system to share a GL context and more between elements - GstVideoGLTextureUploadMeta: a way to tell how to upload a buffer to GPU - GstAllocator and GstMemory: cleaner way to handle non-CPU memory
  10. 10. Sharing video buffers: nowadays "modern" gl(es) sink implementations known to the author: - eglglessink (and libgstegl) - glimagesink
  11. 11. II. Integrating cairogles and GStreamer
  12. 12. Many possibilities: - xvimagesink + GstVideoOverlay - eglglessink - glimagesink - make our own sink
  13. 13. xvimagesink with overlay: not so fast, I suspect it triggers a download to CPU to get the image from xvimagesink's Drawable to a cairo-gl surface. eglglessink: promising, but obviously doesn't support openGL or GLX glimagesink started work on proper integration, likely will require addition of a few features there
  14. 14. First attempt: a cairosink that does everything
  15. 15. Cairosink: status - GstAllocator and GstMemory subclasses - Use either OpenGL or OpenGLES - Possibility to use PixelBuffer Objects (PBOs) (inefficiently)
  16. 16. allocator and memory not that useful as it's the only element using it. Main advantage: _could_ allow to start uploading the frame as soon as possible. PBOs in a nutshell: a way to asynchronously upload a frame to a texture. The upload is then done through the magic of DMA, without the CPU being involved. More on why cairosink is currently inefficient at using it later. FIXME: graph explaining PBO?
  17. 17. Cairosink: simplistic benchmark
  18. 18. Synchronous / PBO / XV XV: limited by refresh rate? Looks like its genuine limit (often below 60) frames 720x400, RGB or YUV
  19. 19. Cairosink: difficulties (or rants) - cairogles can be compiled with either gl or glesv2 (not both) - driver bugs and regressions - cairo only supports RGB - desktop vs mobile GPUs
  20. 20. driver regression: GL+EGL used to work on my machine...
  21. 21. Desktop vs mobile GPU - EGLMakeCurrent() is costly on many mobile GPUs/drivers - bandwidth differences
  22. 22. context acquisition (makeCurrent) slowness is likely to counter the effects of asynchronous direct upload
  23. 23. Cairosink: what's to do - more flexibility on code path - compatibility with other elements (not our own memory/allocator)
  24. 24. we want to be able to tailor to various GPUs various "options": is it faster to do a synchronous or asynchronous upload?
  25. 25. Cairosink: other ideas to explore - libgstgl backend (ongoing experiment) - wrapper around glimagesink
  26. 26. libgstgl issues: mechanism to MakeCurrent() or save/restore states share GMainContext? (all in one thread)
  27. 27. Thank you Questions?

×