The added value of a software framework on system design

1. The move from
a hardware centric product design
towards
a software centric product design:
Gstreamer as a driving force
Marc Leeman
@den_erpel

2. Introduction
●
The Early Days
●
First Paradigm shift
●
A new technology
●
At the start of the new millennium
●
A typical 2000 design
●
What changed?
●
A new world
●
Gstreamer, the early days
●
Gstreamer, 2nd try
●
Gstreamer, wider use
●
Gstreamer, from here on out
●
Change does not come easy
●
A Changed way of working
●
New challenges

3. The Early Days
●
Barco: Belgian American Radio Company
●
Established in 1934
●
Focus: Radios
●
Vacuum tubes
●
Long Lasting
●
In those days, buying a radio was costly
●
A status symbol

4. First Paradigm Shift
●
2nd World War
●
●
Refocus of the industry towards the War Machine
Radios were restricted
●
Control the information
●
Were hidden and penalties were severe
●
Jamming signals were (partly) aimed for detection

5. A new technology
●
The advent of television
●
●
●
Started back in the 30s, halted for 5 years
Multi standard (PAL, SECAM, NTSC)
Content driven
●
Window on the world
●
Not unlike 16:9 or HD
●
UHD?
●
Real breakthrough after exhibition of 1958
●
From the mid-60s onwards, start of to focus on industrial projects

6. At the start of a the new millennium
●
Technologies last long
●
●
●
Non disruptive changes
Improving on existing, slow move to new
Competence centre in optics
●
●
RGB projectors
Full chain development with exception of CRT
●
Components out

7. A typical 2000 design
●
Core of the design is hardware accelerated
●
High density solutions
●
FPGA at the core of the processing
●
“The rest are peripherals”
●
●
●
The rise of DSP solutions
Uc as a controller
Software as network controller
●
Management software provides logical access point
●
Hides 100s of smaller devices
●
●
Boxes each containing 10-20 PCBs
Each processing up to 4 network streams

8. What Changed?
●
Long lead times
●
Full chain development, not flexible during design
●
Maintenance is much higher once product is introduced
●
●
Custom stack; all development is integrated, very much tailored to “exceptions”
Component life time
●
Industrial range
●
Expensive
●
Underperforming wrt cutting edge
●
Industry is not driving technology, but consumer
●
Mobile... mobile... mobile...

9. Mobile?
●
Before: research was mainly on:
●
●
●
Focus on power efficiency
Focus on performance within constrained device
Android...
●
●
●
Java and AMM is not efficient
Contradicts previous focus
Solution was not to make the software better but to beef the specs of the devices
●
Same story as compression research
●
More memory, deeper memory hierarchy
●
Faster cores, more cores

10. A new world
●
Suddenly, a lot of new processor cores are flooding the
market
●
●
●
> 1GHz is the norm
> 2 cores is the norm
Sounds awfully familiar to GPP
●
●
Difference between GPP software and 'embedded' software is
decreasing faster than ever before
Gstreamer as a typical example

11. GStreamer, the early days
●
Initial attempts on MPC8349 core
●
uClibc based
●
uClibc proved very bad thread behaviour (meant for controller tasks)
●
unfair comparison: compared with MPC8245 based 'limited' protocol support
●
uC core was not as fast as advertised
●
Discontinued
●
YACS: Yet Another Custom Stack
●
●
●
Modular design was retained
2nd time to write a streaming system from scratch
2006

12. GStreamer, 2nd try
●
Something stuck
●
●
open source (experience with Linux, uclibc, busybox, ffmpeg, X11, …)
●
●
modular design
YACS just felt wrong
MGS: Media Gateway Server
●
●
●
x86-64 server based solution
removes much of the previous mentioned limitations
Small team can support a wide range of systems
●
●
cooperation
●
●
key was modularity
peer review
2008

13. GStreamer, wider use
●
Hard to not notice the advantages
●
Stability
●
Protocol support
●
Time-to-market
●
Flexibility
●
Replacement of custom stack with GStreamer
●
Integration of custom codecs in GStreamer
●
the power of an external API

14. GStreamer, from here on out...
●
Established on x86
●
Once in userland, why the focus on one architecture
●
●
NSLU2, SheevaPlug, Raspberry Pi, Odroid, ...
Platform approach
●
●
●
re-use of experience and code
encoder and platform support
For devices, added value is the same
●
Multi protocol support
●
Standard protocol support
●
Not so standard protocol support

15. Streaming,...or not?
●
RAW video handling (loss-less)
–
–
●
Low latency
High quality high bandwidth
Caps negotiation
V4l2src
V4l2sink
V4L2
V4L2
1 GHz Device handling streams +10Gbps?
–
●
GStreamer application
At first glance, it cannot be done
Not for the pure data handling
–
But for the configuration of the blocks that do the handling
–
Configuration scaler parameters
●
Caps negotiation..
V4l2src
V4L2
V4L2
Linux
Linux
FPGA
FPGA
V4l2sink
V4L2
V4L2

16. Abstraction, Obstruction
●
Raw, compressed: Oh, wibbly-wobbly, timey-wimey...
●
Keep hardware contained
–
●
As close as possible to the kernel
Standardised interfaces
–
e.g. V4L2 for RAW handling
–
Isolate frameworks behind standard interface
●
–
No new Gstreamer modules
●
●
–
e.g. OMX
Extend and improve
e.g. uri handing
e.g. Piggy back on V4L2 interface

17. Change does not come easy
●
"GStreamer is slow"
●
"GStreamer adds overhead"
●
Typically from the 'established'
●
●
Optimised code from day 1?
Prove the opposite
●
GStreamer _can_ be slow
●
BUGS, not by design
●
Fix the problems
●
●
●
Alsa
H.264 parsing
fixing a bug can never be an excuse for re-writing a complete software stack

18. A Changed Way of Working
●
The Linux kernel has been used for some time
●
●
●
Kernel is often very specific to the SoC/board
Kernel is linked to FPGA logic
In contrast: Gstreamer allows more re-use
●
Userspace
●
3rd party camera stacks can easily be transferred
●
Easy integration of codecs and codec implementations throughout
the divisions

19. Cooperation invites tooling
●
Tooling is no longer an arbitrary choice
–
●
Loads of commercial “good and less good” tools
In order to participate in a community, you need to follow
the rules of that community
–
A similar way of working makes life easier
●
●
Svn → git; deployment, ...
Commercial solution is limited to sugar coating
–
e.g. layer on top of git

20. New Challenges
●
The future is software
●
●
●
Training, training
Migrate from hardware to software
Software is cheap
●
●
Anyone can create a network system
Typical reflex
●
Protectionism
●
●
●
Do not use GPL
Do not use LGPL
Do not use Open Source Software

21. New Challenges
●
●
Luckily we past that phase before software became the
dominating factor
Cooperation is key
●
Focus on standards
●
●
Added value is in solutions, not components
●
●
●
Step away from “our use-case is special”
Is our added value really in supporting e.g. camera X support?
Keep moving...
The return of the Garage start-ups?

22. In Practice
●
Software design is now the core of product design
–
–
●
Used to be hardware driven
Board created, software was added
Now we see inter-divisional architecture meetings
–
●
Cross checking and review of design
Gstreamer provides a stable environment to develop in
–
–
Peer review (when submitted)
–
Common reference within the company
–
●
Stable APIs
Easy to get external experts for temporary bottlenecks

23. In Practice
●
Training is key
–
–
Convert to Software designers
–
●
Excess in hardware designers
Again, external training
Focus is on adding value, not re-inventing the wheel

24. Questions?
www.barco.com
www.twitter.com/Barco
www.facebook.com/Barco
www.youtube.com/BarcoTV