By Philippe Normand. WPEWebKit is a WebKit flavour (also known as port) specially crafted for embedded platforms and use-cases. During this talk I would present WPEWebKit's architecture with a special emphasis on its multimedia backend based on GStreamer. I would also demonstrate various use-cases for WPE, spanning from Kiosk apps and Set-top-box user-interfaces to advanced scenarios such as Web overlays for live TV broadcasting. WPEWebKit is designed for simplicity and performance. It allows application developers to easily deploy hardware-accelerated fullscreen (or not) browsers with multimedia support, small (both in memory usage and disk space) and light as possible, and implementing the most relevant HTML specifications. Traditionally WebKit ports are associated with a specific widget toolkit library (GTK, Qt, Cocoa,...) but WPEWebKit breaks with this monolithic design and thus enables a new range of use-cases. By delegating the final web page rendering to dedicated view-backends, WPEWebKit allows flexible and tight integration in a wide range of hardware platforms. We also provide a Qt5 QML plugin that can easily replace the deprecated QtWebKit-based module. WPEWebKit leverages GStreamer for its multiple multimedia backends, ensuring your WPEWebKit-based browser supports zero-copy hardware video decoding on the most common embedded platforms such as the Raspberry Pi, i.MX6 and i.MX8M SoCS. WPEWebKit can also be used in pure GStreamer applications! Thanks to the GstWPE plugin, web-pages can be "injected" in GStreamer pipelines as audio and video streams. This new plugin thus enables use-cases such has HTML overlays. WPEWebKit is an open source project with a growing community, and it is developed within the ecosystem of the WebKit project, which powers many open source and proprietary web browsers. (c) FOSDEM 2020 Brussels, 1 & 2 February https://fosdem.org/2020/schedule/event/ema_wpe/

1. WPE, The WebKit port forWPE, The WebKit port for
Embedded platformsEmbedded platforms
FOSDEM 2020, BrusselsFOSDEM 2020, Brussels
Philippe Normand

2. Talk outlineTalk outline
Intro
WPE architecture overview
Using WPE in Qt applications
Using WPE in GStreamer Multimedia applications
Outro / Demo

3. Who am IWho am I
Fiddling with WebKit and GStreamer since 2009
WebKit committer and reviewer
GStreamer committer
Partner at Igalia:
Worker-owned coop, currently around 90 happy
Igalians around the world
Provides consulting services for various Free
So ware projects

4. WPE, Web PlatformWPE, Web Platform
for Embeddedfor Embedded

5. What is WebKitWhat is WebKit
WebEngine aimed for embedding HTML/CSS/JS in
native applications
Forked from KHTML by Apple in 2004
Powers Safari, but also dozens of applications on
various platforms
APIs provided by WebKit ports

6. Multi-process!Multi-process!

7. WPE, a decoupled WebKit portWPE, a decoupled WebKit port
Upstream in webkit.org
6 months release cycle, security updates
Not tied to any widget toolkit
Rendering and input events handling via loadable
backends

8. Rendering backendsRendering backends
Wayland-EGL extension usually required
Various implementations, most notably:
FDO backend
RDK backend (various devices supported under
the RDK/Comcast umbrella)

9. WPEBackend-FDOWPEBackend-FDO
Depends on EGL, usually provided by Mesa or binary
drivers
High level API provided for applications (browsers,
but also other apps!)
Recommended by upstream WPE community

10. Cog, the minimal WPE-basedCog, the minimal WPE-based
browserbrowser
Optional (but recommended) dependency on
WPEBackend-FDO
No window Chrome (optional GTK Chrome could be
considered)
Single web view
Can be operated through D-Bus

11. Running Cog without WaylandRunning Cog without Wayland
compositorcompositor
Through the DRM backend!
Wayland buffers (or DMABufs) imported as GBM
Buffer objects
Rendering through DRM
Input events handled with libinput
App use-cases: Kiosks, Set-Top-Box UIs, ... any
fullscreen display!

12. Cog-based app example (1/2)Cog-based app example (1/2)
DLNA MediaServer ContentDirectory client
embedded in a WebKit WebExtension
WebExtension dynamically modifies the DOM to
display MediaServer contents
Video playback done with an injected <video>
element

13. Cog-based app example (2/2)Cog-based app example (2/2)
Demo: https://www.youtube.com/watch?
v=VEQRBVKnN8o

14. Using WPE in QtUsing WPE in Qt
applicationsapplications

15. WPEQtWPEQt
Dependency on WPEBackend-FDO
Provides a QML module
API compatibility with the unmaintained QWebView
QtWebKit-based module

16. Pros & ConsPros & Cons
✔ Well maintained & security issues support
✔ Works on Wayland-EGL and EGLFS QPA
environments
❌ Linux only
❌ Dependency on Wayland-EGL

17. Enabling itEnabling it
ENABLE_WPE_QT_API CMake option in upstream
WPEWebKit
At runtime, ensure libqtwpe.so is in
$QML2_IMPORT_PATH

18. How-ToHow-To
import org.wpewebkit.qtwpe 1.0
Window {
id: main_window
WPEView {
url: qsTr("https://fosdem.org")
onTitleChanged: {
main_window.title = title;
}
}
}

19. Using WPE inUsing WPE in
GStreamerGStreamer
applicationsapplications

20. GStreamerGStreamer

21. GstWPEGstWPE
GStreamer source element producing a video
stream of a WPE WebView
Zero-copy from WPEBackend-FDO EGLImages to
GStreamer GL
Use-cases: HTML overlays, streaming/cloud
browsers

22. HTML overlaysHTML overlays

23. Ongoing workOngoing work
Audio support: one src pad per audio stream (From
<audio> and <video> HTML elements +
WebAudio + WebRTC)
Improved navigation handling (touch, mouse scroll,
keyboard)

24. OutroOutro

25. OK! Show me the codeOK! Show me the code
WPEWebKit (+WPEQt), WPEBackend-FDO, Cog:
GstWPE: upstream GStreamer
Yocto overlay:
Buildroot: recipes already upstream
https://wpewebkit.org
https://github.com/Igalia/meta-
webkit

26. Questions?