GStreamer is a multimedia framework for building applications that handle audio and video content. It provides reusable code modules called elements that can be linked together into pipelines to process multimedia. Elements include sources, filters, decoders, encoders and sinks. Pads connect elements and negotiate compatible data formats. Pipelines synchronize playback across elements and the framework supports integration with other projects and bindings to multiple programming languages.

1. GStreamer: An overview
Rodrigo Costa
Pontifical Catholic University of Rio de Janeiro
TeleM´ıdia Lab.
rodrigocosta@telemidia.puc-rio.br
April 16, 2015
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 1 / 22

2. Overview
1 GStreamer
Overview
Pipelines
Basic Concepts
Synchronization
Examples
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 2 / 22

3. What is GStreamer?
A framework for multimedia applications
Linux, Windows, OS X, . . .
10+ years old project
Similar to DirectShow, QuickTime, etc.
Designed to make it easy to write applications that handle audiovisual
contents
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 3 / 22

4. GStreamer overview
What GStreamer NOT is?
A codec implementation
An application itself
Features
Fully extensible by plugins
Flexible
Low and high level API
Often wraps other libraries (decoders, encoders, filters, etc.)
Integration with other frameworks and projects
WebKit, Firefox, Gnome, Android, iOS, . . .
Pipeline-based
Bindings to multiple languages
C/C++, Python, Java, Ruby, Pearl, etc.
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 4 / 22

5. GStreamer overview
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 5 / 22

6. Design
The fundamental design comes from the video pipeline at Oregon
Graduate Institute: Infopipes
Infopipes (2001)
High-level abstraction for building blocks that handle information
flows
Goal: simplify the task of building distributed streaming
applications by providing basic elements such as pipes, filters, buffers,
and pumps
Motivated by the lack of support of RPC-like technologies to develop
streaming and processing of distributed information
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 6 / 22

7. Infopipes
Infopipe example:
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 7 / 22

8. GStreamer pipeline
GStreamer pipeline example:
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 8 / 22

9. Basic Concepts
Elements
The most important class of objects in GStreamer
Each one has a specific function: read a file, receive data from
network, decode, multiplex, present the data, . . .
A chain of elements is created and linked together for the data to
flow through it
GStreamer ships with a large set of elements
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 9 / 22

10. Basic Concepts
Elements (cont)
Source
Filters, converters, demuxers, muxers, codecs
Sink
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 10 / 22

11. Basic Concepts
Pads
Connection points between elements
Source pads produce data
Sink pads consume data
Data always flows from src to sink pads
Can operate in pull or push mode
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 11 / 22

12. Basic Concepts
Pads (cont)
Each pad have a predefined set of properties called Capabilities or
Caps
Caps are used to validate the communication between elements (caps
restrict the type of data that flows through it)
A source pad can only be linked to a sink pad if their allowed data
types are compatible
For instance, a source pad that produces a video stream cannot be
linked to a sink pad that consumes audio streams
Elements can negotiate with each other on format to use (caps
negotiation)
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 12 / 22

13. Basic Concepts
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 13 / 22

14. Basic Concepts
Bins
Elements can be grouped into a container called bin
It can be manipulated as it were an element
It also manages the state of its children elements
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 14 / 22

15. Basic Concepts
Pipeline
A pipeline is a top-level bin
It provides a bus for communication purposes
It manages the playout synchronization
It runs in a separate thread
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 15 / 22

16. Basic Concepts
Communication:
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 16 / 22

17. Synchronization
GstClock objects provide clock time
GstClock always returns the absolute-time
The pipeline maintains a GstClock object and a base-time (time when
it goes to PLAYING state)
running-time = absolute-time - base-time
Sink elements are responsible for present the buffers in their
respective presentation time
If a buffer is delayed, the sink drops it
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 17 / 22

18. Pipeline Example
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 18 / 22

19. Example 1
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 19 / 22

20. Example 2
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 20 / 22

21. Example 3
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 21 / 22

22. Example 4
Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 22 / 22