Gstreamer plugin development involves creating elements, plugins, and pads. Elements are the core components that process media streams. Plugins contain implementations of elements and are loaded on demand. Pads negotiate media flow between elements and ensure type compatibility. The chain function processes incoming buffers and passes them downstream. A simple pass-through filter would implement chain to push incoming buffers to the output pad without modification.

1. Gstreamer Plugin Development
Session VII

2. Element
• Core of Gstreamer
• Object derived from GstElement
• Source elements provides data to stream
• Filter elements acts on a data in stream
• Sink elements consumes data of stream

3. Plugin
• Loadable block of code
• Usually shared object or a dynamically linked
library
• Contains implementation of many elements or
just a single one
• Very basic functions reside in the core library
• All other are implemented as plugins
• Are only loaded when their provided elements
are requested

4. Pads
• Negotiate links and data flows between elements
• Is an port of an element – Output or Input Port
• Specific data handling capabilities
• Can restrict the data that flows through it.
• Links are allowed between two pads when data
types of two pads compatible
• Is similar to plug or jack of a physical device

5. Pads
• Source Pads
– Data flows out of one element through one or
more source pads
• Sink Pads
– Accepts incoming data flow through one or more
sink pads

6. GstMiniObject
• Stream of data are chopped up into chunks
that are passed from a source pad on one
element to a sink pad on another element
• Structure used to hold these chunks of data
• Exact type indicating what type of data
• GstMiniObject Types
– Events ( Control )
– Buffers ( Control )

7. GstBuffer
• Contains any sort of data that two linked pads
know how to handle
• Contains some sort of audio or video data that
flows from one element to another.
• Metadata describing the buffer’s contents
– Pointers to one or more GstMemory Objects
– Ref Counted objects the encapsulate a region of
memory
– Timestamp indicating the preferred display timestamp
of the content in the buffer.

8. Buffer Allocation
• Able to store chunks of memory of several
different types
• Most generic type of buffer contains memory
allocated by malloc
– Very convenient
– Not always fast, since data often needs to be
copied

9. Specialized Buffers
• Many specialized elements create buffers that
point to special memory.
– filesrc element maps a file into address space of
application ( using mmap() )
– Creates buffers that point into that address range
– Created by filesrc act like generic buffers, except
that are read only
• Buffer freeing code automatically freed by the
correct method of freeing

10. GstBufferPool
• Element might get specialized buffers is to request them
from a downstream peer through a GstBufferPool
• Downstream able to provide these objects, upstream can
use them to allocate buffers.
• Accelerated methods for copying data to hardware or
direct access to hardware
• These elements able to create a GstBufferPool for their
upstream peers.
• Example Ximagesink
– Create a buffer that contain Ximages
– Upstream peer copies the data into the buffer, it is copying
directly into the Ximage
– Enables Ximagesink draws the image directly without copy

11. GstEvent
• Information on the state of the stream flowing
between two linked pads
• Will only be sent if element explicitly support
them, else core will try to handle
automatically
• Events used to indicate ( for example )
– A media type
– End of media stream
– Cache should be flushed

12. Events
• Subtype indicating the type of contained
event
• Other contents of event depend on the
specific event type

13. Media Types & Properties
• Uses type system to ensure that the data passed
between the elements in recognized format.
• Type system is important for ensuring that the
parameters required to fully specify the format
match up correctly when linking pads between
elements.
• Each link is made between elements has a
specified type and optionally a set of properties.
• Capabilities Negotiation.

14. Basic Types
S.No Media Type Description Property Description
1 audio/* All audio types rate Sample Rate
2 channels integer Greater than 0 No of channels
3 audio/x-raw raw integer
audio
Format String
4 audio/mpeg MPEG Audio
encoding
mpegversion Integer
5 framed boolean 0 or 1 True => Frame
available,
false => no frame
6 layer Integer 1,2 or 3 Compression
layer
7 bitrate integer Greater than 0
8 audio/x-vorbis Vorbis audio
data

15. Building a plugin
• Example file element
– Single input and single output pad
– Simply pass Media Data and Event data from its
sink pad to its source pad without modification.
• Including properties
• Including signal handling
• Examples/pwg/examplefilter

16. Plugin Template
• To check out gst template,
– git clone git://anongit.freedesktop.org/gstreamer/gst-
template.git
– cd gst-template/gst-plugin/src
– ../tools/make_element MyFilter
– Creates two files
• gstmyfilter.c
• gstmyfilter.h
– autogen.sh
– make && sudo make install

17. Examing Code
• Basic code structure
• Element metadata
• GstStaticPad
• Constructor
• Plugin_init
• Specifying the pads

18. Plugin Header
#include <gst/gst.h>
/* Definition of structure storing data for this element. */
typedef struct _GstMyFilter {
GstElement element;
GstPad *sinkpad, *srcpad;
gboolean silent;
} GstMyFilter;
/* Standard definition defining a class for this element. */
typedef struct _GstMyFilterClass {
GstElementClass parent_class;
} GstMyFilterClass;
/* Standard macros for defining types for this element. */
#define GST_TYPE_MY_FILTER (gst_my_filter_get_type())
#define GST_MY_FILTER(obj) (G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_MY_FILTER,GstMyFilter))
#define GST_MY_FILTER_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_MY_FILTER,GstMyFilterClass))
#define GST_IS_MY_FILTER(obj) (G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_MY_FILTER))
#define GST_IS_MY_FILTER_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_MY_FILTER))
/* Standard function returning type information. */
GType gst_my_filter_get_type (void);

19. Setup Object
#include "filter.h"
G_DEFINE_TYPE (GstMyFilter, gst_my_filter,
GST_TYPE_ELEMENT);

20. Element Meta data
gst_element_class_set_static_metadata (klass,
"An example plugin",
"Example/FirstExample",
"Shows the basic structure of a plugin",
"your name <your.name@your.isp>");
static void gst_my_filter_class_init (GstMyFilterClass * klass)
{
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
[..]
gst_element_class_set_static_metadata (element_klass,
"An example plugin",
"Example/FirstExample",
"Shows the basic structure of a plugin",
"your name <your.name@your.isp>");
}

21. Pad Template
static GstStaticPadTemplate sink_factory =
GST_STATIC_PAD_TEMPLATE (
"sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);

22. Plugin init
static gbooleanplugin_init (GstPlugin *plugin)
{
return gst_element_register (plugin, "my_filter", GST_RANK_NONE, GST_TYPE_MY_FILTER);
}
GST_PLUGIN_DEFINE (
GST_VERSION_MAJOR,
GST_VERSION_MINOR,
my_filter,
"My filter plugin",
plugin_init,
VERSION,
"LGPL",
"GStreamer",
"http://gstreamer.net/"
)

23. Specifying Pads
static voidgst_my_filter_init (GstMyFilter *filter)
{
/* pad through which data comes in to the element */
filter->sinkpad = gst_pad_new_from_static_template (&sink_template,
"sink");
/* pads are configured here with gst_pad_set_*_function () */
gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
/* pad through which data goes out of the element */
filter->srcpad = gst_pad_new_from_static_template (&src_template,
"src");
/* pads are configured here with gst_pad_set_*_function () */
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
/* properties initial value */
filter->silent = FALSE;
}

24. Chain Function
• The function in which all data processing takes
place
• In simple filter, mostly linear functions for
each incoming buffers, one buffer will go out.

25. Simple Chain Implementation
static GstFlowReturn gst_my_filter_chain (GstPad *pad, GstObject *parent, GstBuffer *buf);
static void gst_my_filter_init (GstMyFilter * filter)
{
/* configure chain function on the pad before adding the pad to the element */
gst_pad_set_chain_function (filter->sinkpad, gst_my_filter_chain);
}
static GstFlowReturn gst_my_filter_chain (GstPad *pad, GstObject *parent, GstBuffer *buf)
{
GstMyFilter *filter = GST_MY_FILTER (parent);
if (!filter->silent)
g_print ("Have data of size %" G_GSIZE_FORMAT" bytes!n",
gst_buffer_get_size (buf));
return gst_pad_push (filter->srcpad, buf);
}