Gstreamer plugin devpt_1


Published on

Presentation takes a look at Gstramer framwork and describes plugin development using gstreamer.

Level: Novice and above

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Gstreamer plugin devpt_1

  1. 1. Shivaramraje NimbalkarJoshi
  2. 2. <ul><li>Gstreamer Framework </li></ul><ul><li>Gstreamer Pipeline </li></ul><ul><li>Gstreamer Plugin </li></ul><ul><li>Sample Plugin </li></ul>
  3. 3. <ul><li>Gstreamer is a open source multimedia framework. </li></ul><ul><li>Gstreamer is built like a self wound clock spring around a Gstreamer Base Class, GstObject </li></ul><ul><li>GstObject itself is derived from GObject class from the Glib. </li></ul><ul><li>The basic building block of every gstreamer media processing unit is called a plugin </li></ul>
  4. 4. Gobject GstObject GstElement
  5. 5. <ul><li>Gstreamer gives the freedom of connecting one or more media processing components to attain the requisite media processing. </li></ul><ul><li>Gstreamer’s media processing unit is called as Pipeline </li></ul><ul><li>A pipeline takes care of the media processing operation intended to be performed. </li></ul><ul><li>Pipeline handles the clocking mechanism, synchronization, scheduling and control message flow between the included sub processing units. </li></ul>
  6. 6. <ul><li>Plugin - Plug-in is an media processing program that can easily be installed and used in a pipeline. </li></ul><ul><li>The intended media processing is split into signal processing units and appropriate plugins are developed to accomplish the intended media processing. </li></ul><ul><li>Gstreamer plugins can handle any type of media audio, video, images, data. </li></ul><ul><li>Gstreamer plugins are normally derived from a base class GstElement which is directly derived from GstObject. </li></ul><ul><li>GstPipeline itself is derived from GstElement. </li></ul>
  7. 7. <ul><li>Gstreamer Plugins can be broadly classified in the following categories </li></ul><ul><ul><li>Source Plugins – Media creator components eg. ALSA Source , File Source, Ximage Source </li></ul></ul><ul><ul><li>Sink Plugins – Media assimilator components eg. ALSA Sink, File Sink, Ximage Sink </li></ul></ul><ul><ul><li>Transform Plugins – Media transformation components eg. Volume Control, RGB Control, Fade In </li></ul></ul><ul><ul><li>Processing Plugins – Media manipulation components eg. Buffer plugin, Decoder Plugins </li></ul></ul>
  8. 8. <ul><li>Inter plugin communication is managed using pre-specified media stubs in the plugin. </li></ul><ul><li>These pre-specified stub is called as Pad in gstreamer terminology </li></ul><ul><li>A Pad can be visualized as a connector which connects two plugins similar to the power cable which connects the wall mounted power socket to a DVD player </li></ul>
  9. 9. <ul><li>The inter plugin communication can be classified into two sub categories </li></ul><ul><ul><li>Control Message Communication – Communication crucial for controlling media processing called as Events eg. EOS, QOS, Latency </li></ul></ul><ul><ul><li>Media Content Communication – Transfer of media content that needs to be processed called as Buffers </li></ul></ul>
  10. 10. <ul><li>There are two types of pads which can be specified in a plugin </li></ul><ul><ul><li>Sink Pad – The pad from which messages are received by the plugin. i.e: Input Pad </li></ul></ul><ul><ul><li>Source Pad – The pad from which messages are sent by the plugin. i.e: Output Pad </li></ul></ul><ul><li>Each of these pad can have a predefined set of properties called as “Capabilities” or “Caps” </li></ul>
  11. 11. <ul><li>Caps are used in validation of communication between two plugins </li></ul><ul><li>Two plugins connected to each other negotiate at run time to make sure that there is not format mis-match in the media data during data processing period. </li></ul><ul><li>A Pad can have more than one set of Caps </li></ul>
  12. 12. File Source Wavparse ALSA Sink Source pad Sink pad Clock Tick Source Pipeline WAVE Decoder Pipeline Command line: gst-launch filesrc location=“test.wav“ ! wavparse ! alsasink device=“default“
  13. 13. <ul><li>Gstreamer Core and Base packages provide an array of base classes </li></ul><ul><li>The appropriate base classes can be inherited to create a plugin that satisfies a given requirement </li></ul><ul><li>Most of the standard functional behavior is already implemented in these base classes </li></ul><ul><li>The plugin can over-ride or extend the basic functionality as per requirement </li></ul>
  14. 14. <ul><li>GstElement – The most generic base class used as base call by all other derived base classes </li></ul><ul><li>GstBaseSrc – Base Class for source plugins </li></ul><ul><li>GstBaseSink – Base Class for sink plugins </li></ul><ul><li>GstBaseTransform – Base Class for transform plugins </li></ul><ul><li>GstBin – Create a custom plugins handler similar to GstPipeline </li></ul>
  15. 15. <ul><li>Simple Plugin having 1 Sink Pad and 1 Source Pad </li></ul><ul><li>Just multiplies the input data by a number x </li></ul>
  16. 16. <ul><li>/* Definition of structure storing data for this element. */ </li></ul><ul><li>typedef struct _GstMySample { </li></ul><ul><li>GstElement element; //Base Class Data Storage </li></ul><ul><li>GstPad *sinkpad, // Sink Pad Pointer Declaration </li></ul><ul><li>GstPad *srcpad; // Source Pad Pointer Declaration </li></ul><ul><li>gboolean test_arg; // Place holder for storing value of argument </li></ul><ul><li>} GstMySample; </li></ul><ul><li>/* Standard definition defining a class for this element. */ </li></ul><ul><li>typedef struct _GstMySampleClass { </li></ul><ul><li>GstElementClass parent_class; // Base Class declaration </li></ul><ul><li>} GstMySampleClass; </li></ul>
  17. 17. <ul><li>Sample Plugins details to be stored in XML file by Gstreamer Plugin. </li></ul><ul><li>The element details are registered with the plugin during the _base_init () function, which is part of the GObject system. The _base_init () function should be set for this GObject in the function where you register the type with GLib. </li></ul><ul><li>static const GstElementDetails sample_details = { </li></ul><ul><li>&quot;A Sample Plugin&quot;, // Short Name </li></ul><ul><li>“ Audio/SampleExample&quot;, // Tree of this plugin </li></ul><ul><li>&quot;Shows the basic structure of a plugin&quot;, // Detailed description </li></ul><ul><li>&quot;your name <>“ // Contact Information </li></ul><ul><li>}; </li></ul><ul><li>static void gst_sample_base_init (gpointer klass) { </li></ul><ul><li>GstElementClass *element_class = GST_ELEMENT_CLASS (klass); </li></ul><ul><li>gst_element_class_set_details (element_class, &sample_details ); </li></ul><ul><li>} </li></ul>
  18. 18. <ul><li>GstStaticPadTemplate can be used for description of a pad that the element needs. </li></ul><ul><ul><li>A short name for the pad. </li></ul></ul><ul><ul><li>Pad direction. </li></ul></ul><ul><ul><li>Existence property. Viz. Always, Sometimes, Request </li></ul></ul><ul><ul><li>Supported Capability list. </li></ul></ul><ul><ul><li>static GstStaticPadTemplate sink_factory =GST_STATIC_PAD_TEMPLATE ( </li></ul></ul><ul><ul><li>&quot;sink&quot;, </li></ul></ul><ul><ul><li>GST_PAD_SINK, </li></ul></ul><ul><ul><li>GST_PAD_ALWAYS, </li></ul></ul><ul><ul><li>GST_STATIC_CAPS (&quot;ANY&quot;)); </li></ul></ul>
  19. 19. <ul><li>_base_init() – Function which is meant to initialize class and child class properties during each new child class creation </li></ul><ul><li>_class_init() – Function which is used to initialize the class only once, specifying what signals, arguments and virtual functions the class has and setting up global state </li></ul><ul><li>_init() – Function which is used to initialise a specific instance of this plugin </li></ul>
  20. 20. <ul><li>_set_property() – Function called when the declared arguments of the plugin are to be set </li></ul><ul><li>_get_property() – Function called when an arguments value is to be fetched </li></ul><ul><li>_setcaps() – Function is called during caps negotiation. This is the process where the linked pads decide on the stream type that will transfer between them and can respond with either “yes” (TRUE) or “no” (FALSE). If the element responds positively towards the streamtype, that type will be used on the pad </li></ul>
  21. 21. <ul><li>_change_state() – Function called when the state of the plugin needs to be change </li></ul><ul><li>A state describes whether the element instance is initialized, whether it is ready to transfer data and whether it is currently handling data. There are four states defined in GStreamer: </li></ul><ul><ul><li>GST_STATE_NULL - Default state of an element </li></ul></ul><ul><ul><li>GST_STATE_READY - Has all default resources (runtime-libraries, runtime-memory) allocated. However, it has not yet allocated or defined anything that is stream-specific </li></ul></ul><ul><ul><li>GST_STATE_PAUSED - Ready to accept and handle data </li></ul></ul><ul><ul><li>GST_STATE_PLAYING - Accept and process events and buffers with data. </li></ul></ul>
  22. 22. <ul><li>_chain() – The chain function is the function in which all data processing takes place. This function is called by gstreamer when a new data buffer becomes available on the sink pad </li></ul><ul><li>In the case of a sample plugin, _chain () functions are mostly linear functions - so for each incoming buffer, one buffer will go out, too </li></ul><ul><li>Sample plugin is designed to work in push mode of scheduling and hence has a slave mode sink pad linked to a _chain() function </li></ul>
  23. 23. <ul><li>Scheduling is a method for making sure that every element gets called once in a while to process data and prepare data for the next element. </li></ul><ul><li>A plugin can be running in master mode (task runner mode) , slave mode (push mode) and get range mode (pull mode) </li></ul><ul><li>Task Runner Mode – These plugins start a task of their own and effectively run the pipeline eg: ALSA Src </li></ul><ul><li>Pull Mode – This plugin controls data flow in the pipeline. It can provide random access to data. eg: File src </li></ul><ul><li>Push Mode – This plugin just processes the data, when it is made available at its sink pad. eg: Volume Control </li></ul><ul><li>Gstreamer informs every plugin about its scheduling mode </li></ul>
  24. 24. <ul><li>Caps negotiation is the process where elements configure themselves and each other for streaming a particular media format over their pads. </li></ul><ul><li>Since different types of elements have different requirements for the media formats they can negotiate </li></ul><ul><li>Caps negotiation can be UPSTREAM Caps Negotiation or DOWNSTREAM Caps Negotiation </li></ul><ul><li>DOWNSTREAM Caps Negotiation takes place when the plugin changes from Ready to Paused state </li></ul><ul><li>UPSTREAM Caps Negotiation takes place when the plugin at the down stream changes its configuration leading re-negotiation of all upstream plugins </li></ul>
  25. 25. <ul><li>Frameworks help to manage data processing activity </li></ul><ul><li>Plugins when designed effectively will ease management of data processing activity </li></ul><ul><li>All data processing activity can be converted to plugin to attain modularity </li></ul>
  26. 26. <ul><li>Question ? </li></ul><ul><li>Contact: </li></ul><ul><ul><li>[email_address] </li></ul></ul>