Cairosink, GPU Buffer Sharing
...and other random ramblings
Guillaume Emont - Igalia
guijemont@igalia.com
ongoing experiment, I may not have the best answers yet
Work sponsored by Samsung Research America
http://www.sisa.samsung.com/
I. Technologies, past and present
Cairo: 2D graphics
Cairogles: 2D graphics in OpenGL(ES)
support for EGL/GLX/WGL
Sharing video buffers: before
- Add a specific cap
(e.g. video/x-raw-gl)
- subclass GstBuffer
Sharing video buffers: nowadays
- GstContext
- GstVideoGLTextureUploadMeta
- GstAllocator and GstMemory
- GstContext, system to share a GL context and more between elements
- GstVideoGLTextureUploadMeta: a way to tell how to u...
Sharing video buffers: nowadays
"modern" gl(es) sink implementations
known to the author:
- eglglessink (and libgstegl)
- g...
II. Integrating cairogles and GStreamer
Many possibilities:
- xvimagesink + GstVideoOverlay
- eglglessink
- glimagesink
- make our own sink
xvimagesink with overlay: not so fast, I suspect it triggers a download to
CPU to get the image from xvimagesink's Drawabl...
First attempt:
a cairosink that does everything
Cairosink: status
- GstAllocator and GstMemory subclasses
- Use either OpenGL or OpenGLES
- Possibility to use PixelBuffer ...
allocator and memory not that useful as it's the only element using it. Main
advantage: _could_ allow to start uploading t...
Cairosink: simplistic benchmark
Synchronous / PBO / XV
XV: limited by refresh rate? Looks like its genuine limit (often below 60)
frames 720x400, RGB or Y...
Cairosink: difficulties (or rants)
- cairogles can be compiled with
either gl or glesv2 (not both)
- driver bugs and regress...
driver regression: GL+EGL used to work on my machine...
Desktop vs mobile GPU
- EGLMakeCurrent() is costly on many mobile GPUs/drivers
- bandwidth differences
context acquisition (makeCurrent) slowness is likely to counter the effects of asynchronous
direct upload
Cairosink: what's to do
- more flexibility on code path
- compatibility with other elements
(not our own memory/allocator)
we want to be able to tailor to various GPUs various "options": is it faster
to do a synchronous or asynchronous upload?
Cairosink: other ideas to explore
- libgstgl backend (ongoing experiment)
- wrapper around glimagesink
libgstgl issues:
mechanism to MakeCurrent() or save/restore states
share GMainContext? (all in one thread)
Thank you
Questions?
Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)
Upcoming SlideShare
Loading in …5
×

Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)

891 views
732 views

Published on

By Guillaume Emont.

GStreamer 1.0 introduced a new architecture that makes memory management easier. In particular, handling non-system memory and sharing it between elements is made easier.

While building a video sink for cairogles, Guillaume wanted to explore the possibility of sharing GPU memory (in his case, GL or GLES textures) between elements of a pipeline. In this talk, Guillaume will explain how he made the various parts fit together, and discuss the challenges that he has met and how he has worked on them.

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
891
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Cairosink and GPU Buffer Sharing (GStreamer Conference 2013)

  1. 1. Cairosink, GPU Buffer Sharing ...and other random ramblings Guillaume Emont - Igalia guijemont@igalia.com
  2. 2. ongoing experiment, I may not have the best answers yet
  3. 3. Work sponsored by Samsung Research America http://www.sisa.samsung.com/
  4. 4. I. Technologies, past and present
  5. 5. Cairo: 2D graphics Cairogles: 2D graphics in OpenGL(ES)
  6. 6. support for EGL/GLX/WGL
  7. 7. Sharing video buffers: before - Add a specific cap (e.g. video/x-raw-gl) - subclass GstBuffer
  8. 8. Sharing video buffers: nowadays - GstContext - GstVideoGLTextureUploadMeta - GstAllocator and GstMemory
  9. 9. - GstContext, system to share a GL context and more between elements - GstVideoGLTextureUploadMeta: a way to tell how to upload a buffer to GPU - GstAllocator and GstMemory: cleaner way to handle non-CPU memory
  10. 10. Sharing video buffers: nowadays "modern" gl(es) sink implementations known to the author: - eglglessink (and libgstegl) - glimagesink
  11. 11. II. Integrating cairogles and GStreamer
  12. 12. Many possibilities: - xvimagesink + GstVideoOverlay - eglglessink - glimagesink - make our own sink
  13. 13. xvimagesink with overlay: not so fast, I suspect it triggers a download to CPU to get the image from xvimagesink's Drawable to a cairo-gl surface. eglglessink: promising, but obviously doesn't support openGL or GLX glimagesink started work on proper integration, likely will require addition of a few features there
  14. 14. First attempt: a cairosink that does everything
  15. 15. Cairosink: status - GstAllocator and GstMemory subclasses - Use either OpenGL or OpenGLES - Possibility to use PixelBuffer Objects (PBOs) (inefficiently)
  16. 16. allocator and memory not that useful as it's the only element using it. Main advantage: _could_ allow to start uploading the frame as soon as possible. PBOs in a nutshell: a way to asynchronously upload a frame to a texture. The upload is then done through the magic of DMA, without the CPU being involved. More on why cairosink is currently inefficient at using it later. FIXME: graph explaining PBO?
  17. 17. Cairosink: simplistic benchmark
  18. 18. Synchronous / PBO / XV XV: limited by refresh rate? Looks like its genuine limit (often below 60) frames 720x400, RGB or YUV
  19. 19. Cairosink: difficulties (or rants) - cairogles can be compiled with either gl or glesv2 (not both) - driver bugs and regressions - cairo only supports RGB - desktop vs mobile GPUs
  20. 20. driver regression: GL+EGL used to work on my machine...
  21. 21. Desktop vs mobile GPU - EGLMakeCurrent() is costly on many mobile GPUs/drivers - bandwidth differences
  22. 22. context acquisition (makeCurrent) slowness is likely to counter the effects of asynchronous direct upload
  23. 23. Cairosink: what's to do - more flexibility on code path - compatibility with other elements (not our own memory/allocator)
  24. 24. we want to be able to tailor to various GPUs various "options": is it faster to do a synchronous or asynchronous upload?
  25. 25. Cairosink: other ideas to explore - libgstgl backend (ongoing experiment) - wrapper around glimagesink
  26. 26. libgstgl issues: mechanism to MakeCurrent() or save/restore states share GMainContext? (all in one thread)
  27. 27. Thank you Questions?

×