The WebKit WPE and GTK ports are aiming to leverage GstWebRTC as their WebRTC backend. Over the years we made progress towards this goal both in WebKit and in GStreamer. During this talk we will present the current integration status of GstWebRTC in WebKit and the plans we have for the coming months. (c) FOSDEM 2025 1 & 2 February 2025 https://fosdem.org/2025/schedule/event/fosdem-2025-4651-webrtc-support-in-webkitgtk-and-wpewebkit-with-gstreamer-status-update/

1. WebRTC support in WebKitGTK and
WebRTC support in WebKitGTK and
WPEWebKit with GStreamer
WPEWebKit with GStreamer
Philippe Normand
Philippe Normand
FOSDEM 2025
FOSDEM 2025
1
1 /
/ 30
30

2. Outline
Outline
Intro
Intro
Current status
Current status
Practical use-cases
Practical use-cases
On-going work & plans
On-going work & plans
2
2 /
/ 30
30

3. About me
About me
Multimedia engineer at Igalia since ~2009
Multimedia engineer at Igalia since ~2009
Working on GStreamer and WebKit Linux ports
Working on GStreamer and WebKit Linux ports
Was involved in the OpenWebRTC project and its integration in WebKitGTK
Was involved in the OpenWebRTC project and its integration in WebKitGTK
3
3 /
/ 30
30

4. Intro - WebKit
Intro - WebKit
FOSS Web engine, maintained by Apple, Igalia, Sony
FOSS Web engine, maintained by Apple, Igalia, Sony
WebView API, generally allowing native apps to render the web
WebView API, generally allowing native apps to render the web
Web browsers
Web browsers
Set-top-box UIs
Set-top-box UIs
News readers
News readers
Platform-specific (upstream) ports:
Platform-specific (upstream) ports:
Apple products
Apple products
Sony Playstation
Sony Playstation
Linux Desktop (GTK port)
Linux Desktop (GTK port)
Linux Embedded (WPE port)
Linux Embedded (WPE port)
Windows & JSCOnly
Windows & JSCOnly
4
4 /
/ 30
30

5. Intro - Why not use LibWebRTC? (1/2)
Intro - Why not use LibWebRTC? (1/2)
5
5 /
/ 30
30

6. Intro - Why not use LibWebRTC? (2/2)
Intro - Why not use LibWebRTC? (2/2)
WPE and WebKitGTK support LibWebRTC but:
WPE and WebKitGTK support LibWebRTC but:
BoringSSL license preventing use in GPL apps
BoringSSL license preventing use in GPL apps
Footprint in tarballs
Footprint in tarballs
Integration with GStreamer video decoders very fragile
Integration with GStreamer video decoders very fragile
Lack of hardware-accelerated video encoding
Lack of hardware-accelerated video encoding
Why do we need another Multimedia framework anyway?
Why do we need another Multimedia framework anyway?
Let’s now talk about GStreamer and GstWebRTC ;)
Let’s now talk about GStreamer and GstWebRTC ;)
6
6 /
/ 30
30

7. Intro - GStreamer (1/2)
Intro - GStreamer (1/2)
Cross-platform open-source multimedia framework
Cross-platform open-source multimedia framework
Widely deployed on embedded platforms (even in the ISS space station!)
Widely deployed on embedded platforms (even in the ISS space station!)
Custom plugins available, depending on target Set-top-box platforms
Custom plugins available, depending on target Set-top-box platforms
Checkout the
Checkout the GStreamer: State of the Union 2025
GStreamer: State of the Union 2025 talk tomorrow in the Open Media
talk tomorrow in the Open Media
devroom (K.3.401) at 3pm!
devroom (K.3.401) at 3pm!
7
7 /
/ 30
30

8. Intro - GStreamer (2/2)
Intro - GStreamer (2/2)
8
8 /
/ 30
30

9. Intro - GstWebRTC
Intro - GstWebRTC
one GStreamer element (
one GStreamer element ( webrtcbin
webrtcbin ) for integration in pipelines
) for integration in pipelines
Can be thought of as a PeerConnection object
Can be thought of as a PeerConnection object
Signals and properties to perform Offer/Answer
Signals and properties to perform Offer/Answer
one GStreamer library shipping public API (
one GStreamer library shipping public API ( libgstwebrtc.so
libgstwebrtc.so )
)
Defines classes for WebRTC-related objects (Transceivers, …)
Defines classes for WebRTC-related objects (Transceivers, …)
Pluggable ICE and Transport backend support (only libnice currently)
Pluggable ICE and Transport backend support (only libnice currently)
9
9 /
/ 30
30

10. Current status
Current status
10
10 /
/ 30
30

11. WebRTC in WebKit, using GStreamer
WebRTC in WebKit, using GStreamer
Dedicated pipeline for audio/video capture
Dedicated pipeline for audio/video capture
Another pipeline for encoding and streaming over the network
Another pipeline for encoding and streaming over the network
Seamless integration with hardware-accelerated encoders
Seamless integration with hardware-accelerated encoders
Zero-copy encoding from webcam to RTP payloader possible
Zero-copy encoding from webcam to RTP payloader possible
Incoming streams routed to separate pipelines for playback
Incoming streams routed to separate pipelines for playback
Seamless integration with hardware-accelerated decoders
Seamless integration with hardware-accelerated decoders
11
11 /
/ 30
30

12. Stream capture
Stream capture
One pipeline per capture device (Camera, Microphone, Desktop)
One pipeline per capture device (Camera, Microphone, Desktop)
Permission request handling
Permission request handling
For
For getDisplayMedia()
getDisplayMedia() :
:
Desktop portal (PipeWire) mandatory
Desktop portal (PipeWire) mandatory
Capture pipeline connects to PipeWire, gets frames as DMABufs
Capture pipeline connects to PipeWire, gets frames as DMABufs
12
12 /
/ 30
30

13. MediaStream
MediaStream handling
handling
webkitmediastreamsrc
webkitmediastreamsrc GStreamer source element acting as Observer for:
GStreamer source element acting as Observer for:
Capturer pipelines
Capturer pipelines
Canvas /
Canvas / <video>
<video> elements
elements
Incoming WebRTC
Incoming WebRTC MediaStream
MediaStream s
s
Each
Each MediaStreamTrack
MediaStreamTrack maps to a
maps to a GstPad
GstPad in
in webrtcbin
webrtcbin
13
13 /
/ 30
30

14. PeerConnection
PeerConnection handling (1/2)
handling (1/2)
RealtimeIncomingSource
RealtimeIncomingSource
webrtcbin
webrtcbin
RealtimeOutgoingVideoSource
RealtimeOutgoingVideoSource
RealtimeOutgoingAudioSource
RealtimeOutgoingAudioSource
mediastreamsrc
mediastreamsrc parsebin
parsebin appsink
appsink
Video Encoder
Video Encoder Video RTP Payloader
Video RTP Payloader
Audio Encoder
Audio Encoder Audio RTP Payloader
Audio RTP Payloader
payloaded
payloaded encoded
encoded
sink
sink sink
sink
sink
sink src
src
sink
sink
sink
sink
src
src
src
src
sink
sink src
src
sink
sink src
src
src
src
sink
sink src
src
sink
sink src
src
audio
audio
video
video
14
14 /
/ 30
30

15. PeerConnection
PeerConnection handling (2/2)
handling (2/2)
webrtcbin
webrtcbin implements the JavaScript Session Establishment Protocol (JSEP)
implements the JavaScript Session Establishment Protocol (JSEP)
Almost seamless integration with WebKit’s PeerConnection infrastructure
Almost seamless integration with WebKit’s PeerConnection infrastructure
GstPromise
GstPromise integration with WebKit’s
integration with WebKit’s Promise<T>
Promise<T>
15
15 /
/ 30
30

16. Incoming
Incoming MediaStream
MediaStream playback
playback
In JS:
In JS: videoElt.src = mediaStream
videoElt.src = mediaStream
=> In
=> In WebCore::MediaPlayerPrivateGStreamer
WebCore::MediaPlayerPrivateGStreamer :
: playbin3
playbin3
uri=mediastream://uuid
uri=mediastream://uuid
==> Internally hooks the
==> Internally hooks the MediaStream
MediaStream to
to webkitmediastreamsrc
webkitmediastreamsrc
16
16 /
/ 30
30

17. Additional WebRTC-related APIs (1/2)
Additional WebRTC-related APIs (1/2)
Statistics
Statistics
Queried to
Queried to webrtcbin
webrtcbin using a GObject action signal
using a GObject action signal
Some informations can’t be filled by
Some informations can’t be filled by webrtcbin
webrtcbin
frames-decoded
frames-decoded ,
, frames-encoded
frames-encoded ,
, bitrate
bitrate , … filled in by WebKit
, … filled in by WebKit
DataChannel,
DataChannel, RTCDataChannel
RTCDataChannel maps fairly well with
maps fairly well with GstWebRTCDataChannel
GstWebRTCDataChannel
17
17 /
/ 30
30

18. Additional WebRTC-related APIs (2/2)
Additional WebRTC-related APIs (2/2)
replaceTrack()
replaceTrack()
pc.createOffer({offerToReceiveAudio: true, offerToReceiveVideo:
pc.createOffer({offerToReceiveAudio: true, offerToReceiveVideo:
true})
true})
18
18 /
/ 30
30

19. Practical use-cases
Practical use-cases
19
19 /
/ 30
30

20. The obvious ones: Video calls
The obvious ones: Video calls
Jitsi: Very early stage bring-up
Jitsi: Very early stage bring-up
Livekit: We can connect and receive A/V. Sending not working yet
Livekit: We can connect and receive A/V. Sending not working yet
20
20 /
/ 30
30

21. Amazon Luna
Amazon Luna
Game streaming, gamepad events sent using DataChannel
Game streaming, gamepad events sent using DataChannel
Optional support for live WebCam/Mic overlay to Twitch (WIP)
Optional support for live WebCam/Mic overlay to Twitch (WIP)
Demo
Demo
21
21 /
/ 30
30

22. Zoo, Industrial modeling application
Zoo, Industrial modeling application
CAD model rendered server-side with WebRTC
CAD model rendered server-side with WebRTC
DataChannel for sending pointer events
DataChannel for sending pointer events
Demo
Demo
22
22 /
/ 30
30

23. On-going work
On-going work
23
23 /
/ 30
30

24. WebRTC Devtools in WebKit (1/3)
WebRTC Devtools in WebKit (1/3)
Currently a well-hidden WebInspector feature:
Currently a well-hidden WebInspector feature: gatherWebRTCLogs(func)
gatherWebRTCLogs(func)
24
24 /
/ 30
30

25. WebRTC Devtools in WebKit (2/3)
WebRTC Devtools in WebKit (2/3)
End goal: Events and stats gathering for live graphing and post-mortem analysis
End goal: Events and stats gathering for live graphing and post-mortem analysis
Support for LibWebRTC and GstWebRTC WebKit builds
Support for LibWebRTC and GstWebRTC WebKit builds
JSON stream emitted by WebKit’s PeerConnection backends, including timestamped
JSON stream emitted by WebKit’s PeerConnection backends, including timestamped
events, example:
events, example:
{"peer-connection":"7F1C6E013520","timestamp":1725960727633365.8,"type":"event","event":{"message":"PeerConnection creat
{"peer-connection":"7F1C6E013520","timestamp":1725960727633365.8,"type":"event","event":{"message":"PeerConnection creat
{"peer-connection":"7F1C6E013680","timestamp":1725960735855362.8,"type":"stats","event":{"type":"inbound-rtp","id":"rtp-
{"peer-connection":"7F1C6E013680","timestamp":1725960735855362.8,"type":"stats","event":{"type":"inbound-rtp","id":"rtp-
Backend enabled using an env variable:
Backend enabled using an env variable: WEBKIT_WEBRTC_JSON_EVENTS_FILE
WEBKIT_WEBRTC_JSON_EVENTS_FILE
Basic web frontend able to read such JSON file and render graphs
Basic web frontend able to read such JSON file and render graphs
25
25 /
/ 30
30

26. WebRTC Devtools in WebKit (3/3)
WebRTC Devtools in WebKit (3/3)
Graphs showing basic stats about inbound and outbound RTP streams
Graphs showing basic stats about inbound and outbound RTP streams
26
26 /
/ 30
30

27. More features
More features
Basic support for network conditions simulation (packet loss, …) ✅
Basic support for network conditions simulation (packet loss, …) ✅
Simulcast: ✅-ish
Simulcast: ✅-ish
SVC (VP8 ✅, VP9/AV1 🚧)
SVC (VP8 ✅, VP9/AV1 🚧)
Hardware-accelerated encoding 🚧(promising results on RPi/OMX and
Hardware-accelerated encoding 🚧(promising results on RPi/OMX and
desktop/VAAPI)
desktop/VAAPI)
E2EE with SFrame 🚧(pending implementation details and potential interop issues)
E2EE with SFrame 🚧(pending implementation details and potential interop issues)
27
27 /
/ 30
30

28. Access to capture devices from
Access to capture devices from
sandboxed WebKit
sandboxed WebKit
The current capture device pipeline runs in WebProcess (BAD!)
The current capture device pipeline runs in WebProcess (BAD!)
Ideally the WebProcess should be sandboxed, hence no direct access to capture
Ideally the WebProcess should be sandboxed, hence no direct access to capture
devices
devices
Currently we allow-list v4l devices in the sandbox
Currently we allow-list v4l devices in the sandbox
Plan (2025): Integration with the desktop Camera portal, giving us access to
Plan (2025): Integration with the desktop Camera portal, giving us access to
PipeWire nodes
PipeWire nodes
WPE on Embedded platforms will still need v4l devices allow-listing
WPE on Embedded platforms will still need v4l devices allow-listing
28
28 /
/ 30
30

29. UDP streaming from sandboxed WebKit
UDP streaming from sandboxed WebKit
The current streaming pipeline runs in WebProcess (BAD!)
The current streaming pipeline runs in WebProcess (BAD!)
Ideally the network usage should be restricted to the NetworkProcess
Ideally the network usage should be restricted to the NetworkProcess
libgstwebrtc
libgstwebrtc now supports custom ICE implementations
now supports custom ICE implementations
librice
librice provides a Sans-IO implementation for ICE handling
provides a Sans-IO implementation for ICE handling
Plan (2025): Implement a librice-based ICE backend in the WebProcess, handling
Plan (2025): Implement a librice-based ICE backend in the WebProcess, handling
IPC I/O with NetworkProcess
IPC I/O with NetworkProcess
29
29 /
/ 30
30

30. Thanks!
Thanks!
Any question?
Any question?
30
30 /
/ 30
30