GStreamer & SysLink
Víctor Manuel Jáquez Leal
2011-10-22 Sat

Víctor Manuel Jáquez Leal ()

GStreamer & SysLink

2011-10-2...
Outline
1

Introduction

2

Objective

3

SysLink

4

Hardware

5

GStreamer

6

Future

7

References

Víctor Manuel Jáqu...
Parallel computing

Serial computing is now dead.
Parallel computing is a revolution that is now upon us.
Although it star...
Why Parallel computing?

To achieve performance guarantee.
It generally leads to lower power consumption.

Víctor Manuel J...
Parallel computing challenges

Programming for serial computing is already dicult.
Programming for parallel computing will...
Parallel computing approaches

Symmetric Multiprocessing

Identical processors.
A single shared main
memory.
A single OS i...
Parallel computing approaches

Symmetric Multiprocessing

Identical processors.
A single shared main
memory.
A single OS i...
Symmetric Multiprocessing
SMP

It is the most common
conguration.
It has bandwidth issues.
It lacks of fault-tolerant
feat...
Asymmetric Multiprocessing
AMP

A problem in one kernel/CPU
will not aect the whole
system.
Applications must use a
dieren...
Another AMP Conguration

One master processor.
Multiple remote processors.
slave processors are treated as
devices (under ...
Inter-Processors Communication
IPC

An IPC mechanism is key for the AMP architecture.
Sharing memory regions between proce...
Examples of such multi-core SoCs

Texas Instruments's OMAP processors
Qualcomm's SnapDragon processors
Broadcom's SiByte f...
What is SysLink?

SysLink is the latest generation AMP/IPC driver developed by
Texas Instrument.
It provides mechanisms to...
Objective
This talk will describe the main SysLink components and how we
could map those concepts into GStreamer elements....
Objective
This talk will describe the main SysLink components and how we
could map those concepts into GStreamer elements....
Project

Gitorious repository page

https://gitorious.org/vj-pandaboard/syslink/
These slides are also here

https://gitor...
History of TI's AMP/IPC implementations (1/6)

OMAP2: Reference implementation.
OMAP3: DSP/Bridge
OMAP2/OMAP3/DaVinci: DSP...
History of TI's AMP/IPC implementations (2/6)

OMAP2: Reference implementation.
dsp-gateway by Nokia.

Full open source.

...
History of TI's AMP/IPC implementations (3/6)

OMAP3: DSP/Bridge
Full featured implementation.
Only supports DSP (C64x+).
...
History of TI's AMP/IPC implementations (4/6)

OMAP2/OMAP3/DaVinci: DSPLink
A stripped implementation of DSP/Bridge.
It wa...
History of TI's AMP/IPC implementations (5/6)

OMAP4: SysLink
Supports other types of cores (ARM Cortex-M3 and C64x)
Handl...
History of TI's AMP/IPC implementations (6/6)

OMAP4/OMAP5: rpmsg / remoteproc
Developed completely in upstream.

Víctor M...
Key features of SysLink
Messaging: . . .

Víctor Manuel Jáquez Leal ()

GStreamer  SysLink

2011-10-22 Sat

21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .

Víctor Manuel Jáquez Leal ()

GStreamer  SysLin...
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .

Víctor Manuel Jáquez Lea...
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .

...
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Z...
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Z...
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Z...
SysLink v2 architecture

Víctor Manuel Jáquez Leal ()

GStreamer  SysLink

2011-10-22 Sat

22 / 37
SysLink v2 components
MultiProc: Identies each available processor.

Víctor Manuel Jáquez Leal ()

GStreamer  SysLink

201...
SysLink v2 components
MultiProc: Identies each available processor.
SharedRegion: Handles shared memory areas.

Víctor Man...
SysLink v2 components
MultiProc: Identies each available processor.
SharedRegion: Handles shared memory areas.
Gate: Prote...
SysLink v2 components
MultiProc: Identies each available processor.
SharedRegion: Handles shared memory areas.
Gate: Prote...
SysLink v2 components
MultiProc: Identies each available processor.
SharedRegion: Handles shared memory areas.
Gate: Prote...
SysLink v2 components
MultiProc: Identies each available processor.
SharedRegion: Handles shared memory areas.
Gate: Prote...
SysLink v2 components
MultiProc: Identies each available processor.
SharedRegion: Handles shared memory areas.
Gate: Prote...
Remote Command Messaging
RCM

It handles remote function calls.
RCM client
It manages the allocation, sending, receiving o...
RCM Usage

Víctor Manuel Jáquez Leal ()

GStreamer  SysLink

2011-10-22 Sat

25 / 37
OMAP4430
Pandaboard

Dual-core ARM Cortex-A9 MPCore.
Dual-core ARM Cortex-M3.
DSP C64x (not much info).
3D graphic acceler...
Ducati
The two Cortex-M3 processors comprise Ducati.
Sys M3: runs the Notify driver.
App M3: executes the entire multimedi...
Distributed Codec Engine
DCE

libdce
It is library to remotely invoke the hardware accelerated codecs
on IVA-HD via SysLin...
Distributed OpenMAX
DOMX

It is a RPC shim for an IL client to use a component on Ducati
DOMX AppM3
It is asynchronous.
It...
DMM/TILER
It organizes video/image memory in a 2D fashion.
It facilitates

Also

Allocating.
Rotating
Freeing.
Mirroring
M...
Row Stride
The stride is the number of bytes from one row of pixels in
memory to the next row of pixels in memory.
If padd...
2D TILER Buer Pool

GstDucatiBuerPool (1D)
https://gitorious.org/gstreamer-omap/gstducati/blobs/master/src/gstducatibuerpo...
Sync vs Async output buers

DCE is sync
No problem here.

DOMX is async
How to handle it?
GstPad tastks and queue in a thr...
rpmsg / remoteproc

rpmsg
A virtio-based messaging bus that allows kernel drivers to
communicate with remote processors av...
Conclusion

Is it possible to live without OpenMAX?
How to handle GEM/TILER buers?
strides?
buer pool?
Asynchronous output...
Image credits
ASMP vs SMP
http://ohlandl.ipv7.net/CPU/ASMP_SMP.html
Who's Afraid of Asymmetric Multiprocessing?
http://rtc...
Thanks!

Questions?

vjaquez@igalia.com :)

Víctor Manuel Jáquez Leal ()

GStreamer  SysLink

2011-10-22 Sat

37 / 37
Upcoming SlideShare
Loading in …5
×

GStreamer and SysLink (GStreamer Conference 2011)

613 views
419 views

Published on

By Víctor Jáquez.

SysLink is a multi-core Inter-Processor Communication (IPC) software stack, pushed by Texas Instrument, where the developers could take advantage of asymmetric multiprocessing with several heterogeneous processors running different instances of operating system, whether Linux or any other flavor of RT OS.

In the realm of the multimedia processing, this mechanism offloads the CPU-intensive (de)codification tasks from the host processor, enhancing the user experience, particularly in embedded devices, where specialized processing units could be stored on SoCs.

This talk will describe the main SysLink components (Linux kernel API, RCM modules, etc.) and how we could map those concepts into GStreamer elements, providing hardware accelerated multimedia, but without the burden of OpenMAX. The target SoC is the OMAP4 using a panda-board.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
613
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

GStreamer and SysLink (GStreamer Conference 2011)

  1. 1. GStreamer & SysLink Víctor Manuel Jáquez Leal 2011-10-22 Sat Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 1 / 37
  2. 2. Outline 1 Introduction 2 Objective 3 SysLink 4 Hardware 5 GStreamer 6 Future 7 References Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 2 / 37
  3. 3. Parallel computing Serial computing is now dead. Parallel computing is a revolution that is now upon us. Although it started more than 40 years ago. Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 3 / 37
  4. 4. Why Parallel computing? To achieve performance guarantee. It generally leads to lower power consumption. Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 4 / 37
  5. 5. Parallel computing challenges Programming for serial computing is already dicult. Programming for parallel computing will only exacerbate this diculty. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 5 / 37
  6. 6. Parallel computing approaches Symmetric Multiprocessing Identical processors. A single shared main memory. A single OS instance. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 6 / 37
  7. 7. Parallel computing approaches Symmetric Multiprocessing Identical processors. A single shared main memory. A single OS instance. Víctor Manuel Jáquez Leal () Asymmetric Multiprocessing Might be dierent type of processors. Might have private memory. Might run dierent OS. GStreamer SysLink 2011-10-22 Sat 6 / 37
  8. 8. Symmetric Multiprocessing SMP It is the most common conguration. It has bandwidth issues. It lacks of fault-tolerant features. The cache coherency mechanism consumes too much power. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 7 / 37
  9. 9. Asymmetric Multiprocessing AMP A problem in one kernel/CPU will not aect the whole system. Applications must use a dierent API. Requires more hardware resource management. Wider kernel memory consumption. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 8 / 37
  10. 10. Another AMP Conguration One master processor. Multiple remote processors. slave processors are treated as devices (under /dev). One shared memory but registering private regions. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 9 / 37
  11. 11. Inter-Processors Communication IPC An IPC mechanism is key for the AMP architecture. Sharing memory regions between processors. Data interchange. Passing messages between processors. Task synchronization. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 10 / 37
  12. 12. Examples of such multi-core SoCs Texas Instruments's OMAP processors Qualcomm's SnapDragon processors Broadcom's SiByte family of processors Freescale's 8641D dual-core PowerPC-based SOC. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 11 / 37
  13. 13. What is SysLink? SysLink is the latest generation AMP/IPC driver developed by Texas Instrument. It provides mechanisms to control and communicate with remote processors. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 12 / 37
  14. 14. Objective This talk will describe the main SysLink components and how we could map those concepts into GStreamer elements. No OpenMAX No extra libraries (syslink-userspace / libmemmgr / etc.) Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 13 / 37
  15. 15. Objective This talk will describe the main SysLink components and how we could map those concepts into GStreamer elements. No OpenMAX No extra libraries (syslink-userspace / libmemmgr / etc.) Warning!!! Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 13 / 37
  16. 16. Project Gitorious repository page https://gitorious.org/vj-pandaboard/syslink/ These slides are also here https://gitorious.org/vjaquez-misc/syslink-talk/ Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 14 / 37
  17. 17. History of TI's AMP/IPC implementations (1/6) OMAP2: Reference implementation. OMAP3: DSP/Bridge OMAP2/OMAP3/DaVinci: DSPLink OMAP4: SysLink OMAP4/OMAP5: rpmsg / remoteproc Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 15 / 37
  18. 18. History of TI's AMP/IPC implementations (2/6) OMAP2: Reference implementation. dsp-gateway by Nokia. Full open source. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 16 / 37
  19. 19. History of TI's AMP/IPC implementations (3/6) OMAP3: DSP/Bridge Full featured implementation. Only supports DSP (C64x+). Linux side into stagering repository. DSP side is restricted. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 17 / 37
  20. 20. History of TI's AMP/IPC implementations (4/6) OMAP2/OMAP3/DaVinci: DSPLink A stripped implementation of DSP/Bridge. It was not submitted to upstream. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 18 / 37
  21. 21. History of TI's AMP/IPC implementations (5/6) OMAP4: SysLink Supports other types of cores (ARM Cortex-M3 and C64x) Handled in downstream (Ubuntu) Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 19 / 37
  22. 22. History of TI's AMP/IPC implementations (6/6) OMAP4/OMAP5: rpmsg / remoteproc Developed completely in upstream. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 20 / 37
  23. 23. Key features of SysLink Messaging: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  24. 24. Key features of SysLink Messaging: . . . Dynamic memory management: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  25. 25. Key features of SysLink Messaging: . . . Dynamic memory management: . . . Dynamic loading: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  26. 26. Key features of SysLink Messaging: . . . Dynamic memory management: . . . Dynamic loading: . . . Power Management: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  27. 27. Key features of SysLink Messaging: . . . Dynamic memory management: . . . Dynamic loading: . . . Power Management: . . . Zero-copy shared memory: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  28. 28. Key features of SysLink Messaging: . . . Dynamic memory management: . . . Dynamic loading: . . . Power Management: . . . Zero-copy shared memory: . . . 2D memory buers allocation: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  29. 29. Key features of SysLink Messaging: . . . Dynamic memory management: . . . Dynamic loading: . . . Power Management: . . . Zero-copy shared memory: . . . 2D memory buers allocation: . . . Remote function calls: . . . Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 21 / 37
  30. 30. SysLink v2 architecture Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 22 / 37
  31. 31. SysLink v2 components MultiProc: Identies each available processor. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  32. 32. SysLink v2 components MultiProc: Identies each available processor. SharedRegion: Handles shared memory areas. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  33. 33. SysLink v2 components MultiProc: Identies each available processor. SharedRegion: Handles shared memory areas. Gate: Protects memory regions from remote processors. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  34. 34. SysLink v2 components MultiProc: Identies each available processor. SharedRegion: Handles shared memory areas. Gate: Protects memory regions from remote processors. MessageQ: Variable length message passing across processors. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  35. 35. SysLink v2 components MultiProc: Identies each available processor. SharedRegion: Handles shared memory areas. Gate: Protects memory regions from remote processors. MessageQ: Variable length message passing across processors. Notify: Registers callbacks execute when a remote event occur. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  36. 36. SysLink v2 components MultiProc: Identies each available processor. SharedRegion: Handles shared memory areas. Gate: Protects memory regions from remote processors. MessageQ: Variable length message passing across processors. Notify: Registers callbacks execute when a remote event occur. HeapBuf : Manages xed size buers within the shared memory. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  37. 37. SysLink v2 components MultiProc: Identies each available processor. SharedRegion: Handles shared memory areas. Gate: Protects memory regions from remote processors. MessageQ: Variable length message passing across processors. Notify: Registers callbacks execute when a remote event occur. HeapBuf : Manages xed size buers within the shared memory. HeapMem: Manages variable size buers within the shared memory. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 23 / 37
  38. 38. Remote Command Messaging RCM It handles remote function calls. RCM client It manages the allocation, sending, receiving of RCM messages to/from the RCM server. RCM Server It processes inbound messages received from an RCM client. After processing a message, the server will return the message to the client. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 24 / 37
  39. 39. RCM Usage Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 25 / 37
  40. 40. OMAP4430 Pandaboard Dual-core ARM Cortex-A9 MPCore. Dual-core ARM Cortex-M3. DSP C64x (not much info). 3D graphic acceleration PowerVR SGX540. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 26 / 37
  41. 41. Ducati The two Cortex-M3 processors comprise Ducati. Sys M3: runs the Notify driver. App M3: executes the entire multimedia software. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 27 / 37
  42. 42. Distributed Codec Engine DCE libdce It is library to remotely invoke the hardware accelerated codecs on IVA-HD via SysLink/RCM. DCE AppM3 It is similar in purpose to remote CE for the OMAP4430. CE is a framework that enables applications to instantiate and work with codecs and algorithms using a common API. It is synchronous (sends a buer and waits for the output). It is open source (TI's licence). Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 28 / 37
  43. 43. Distributed OpenMAX DOMX It is a RPC shim for an IL client to use a component on Ducati DOMX AppM3 It is asynchronous. It is closed (distributed binary only). Tightly coupled with OpenMAX IL API. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 29 / 37
  44. 44. DMM/TILER It organizes video/image memory in a 2D fashion. It facilitates Also Allocating. Rotating Freeing. Mirroring Mapping. the 2D blocks or its rectangular of 2D blocks in the container(s). subsections. To be replaced by GEM (The Graphics Execution Manager) https://lwn.net/Articles/283798 Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 30 / 37
  45. 45. Row Stride The stride is the number of bytes from one row of pixels in memory to the next row of pixels in memory. If padding bytes are present, the stride is wider than the width of the image. DMM/TILER 2D buers require 4KiB row-stride GstStrideTransform (2009 thread) http://gstreamer.freedesktop.org/wiki/NewGstVideo video/x-raw-yuv-strided ??? https://gitorious.org/gstreamer-omap/gst-pluginsbase/trees/v0.10.35+ti/gst/stride Use GstMeta in 0.11 ??? Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 31 / 37
  46. 46. 2D TILER Buer Pool GstDucatiBuerPool (1D) https://gitorious.org/gstreamer-omap/gstducati/blobs/master/src/gstducatibuerpool.c But for 2D ?? Use GstBuerPool ??? Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 32 / 37
  47. 47. Sync vs Async output buers DCE is sync No problem here. DOMX is async How to handle it? GstPad tastks and queue in a thread ? Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 33 / 37
  48. 48. rpmsg / remoteproc rpmsg A virtio-based messaging bus that allows kernel drivers to communicate with remote processors available on the system. remoteproc A generic driver that maintains the state of the remote processor(s). Firmware loading. Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 34 / 37
  49. 49. Conclusion Is it possible to live without OpenMAX? How to handle GEM/TILER buers? strides? buer pool? Asynchronous output buers? Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 35 / 37
  50. 50. Image credits ASMP vs SMP http://ohlandl.ipv7.net/CPU/ASMP_SMP.html Who's Afraid of Asymmetric Multiprocessing? http://rtcmagazine.com/articles/view/100404 Ducati for dummies http://www.omappedia.org/wiki/Ducati_For_Dummies SysLink project http://omappedia.org/wiki/Syslink_Project Work in Progress (CC BY-NC 2.0) blumpy https://secure.ickr.com/photos/10537966@N00/325853852/ Pandaboard (C) Tiago Maluta https://secure.ickr.com/photos/22248939@N04/5877274222/ Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 36 / 37
  51. 51. Thanks! Questions? vjaquez@igalia.com :) Víctor Manuel Jáquez Leal () GStreamer SysLink 2011-10-22 Sat 37 / 37

×