Janus Client-Side
Dr Alex Gouaillard
CoSMo Software
WARNING !
2
● This is a pineapple-free presentation. No pineapple has been used during
the making of this presentation, and no pizza was tortured nor coerced to
have an indecent relationship with a pineapple!
WARNING !
3
● This is a pineapple-free presentation. No pineapple has been used during
the making of this presentation, and no pizza was tortured nor coerced to
have an indecent relationship with a pineapple!
● This is a VoIP-free presentation! The words Kamailio, FreeSwitch, and
Asterisk will have no part in this presentation. If this hurts your feelings, or
your (religious) beliefs, …. C’est la vie!
We are CoSMo
We used to cure cancer with ML and AI (ten years ago),and then webrtc came along.
4
We are CoSMo
● Based in Singapour (HQ), Manila, France, Madrid and L.A. (streaming)
● Multiple open-source collaborators
○ Google,
○ Apple,
○ Xyrsis,
○ Meteecho,
○ ...
● Gift economy:Contributions
to standards and testing
○ W3C
○ IETF
○ IMTC
○ AOMedia
5
We Love (C++) WebRTC, C, ASM, codecs, and testing
Which makes us very popular because nobody else do.
● Contribution to Open-Source
○ KITE (With Google, Used by Apple and others)
○ No-code fuzzing
○ E2EME (PERC, SFrame)
○ H264 simulcast
● Ongoing Open-Source
○ H.265 HW accelerated (with INTEL and Apple) => working, being tested by A, prep for upstream
○ Real-time AV1 => Proposed to AOM, to be presented at board meeting sep 25th
○ Media Transport / Web Transport / Datagram on QUIC => working impl. On top of picoQuic
● Non-open source research
○ Narval
○ Real-Time Server Side Ad-Insertion
○ Real-Time Digital Forensic Watermaking of Video Media
6
We are CoSMo
● Free Testing results and white papers for the community at large
● IETF HAckathons open to all (including commercial non open-source media
servers).
7
W3C TPAC 2019 - WPT.kite
8
W3C TPAC 2019 - WPT.kite
9
We have our own Media Server ...
10
We still love Janus!
● Video Conferencing vs Streaming
● Hybrid ? (webinar, conference, courses, …..)
● IoT? => 100% Janus
● Legacy?
○ VoIP => 100% Janus
○ Streaming (RTMP, HLS, MPEG-DASH, ….)
The fact that the Janus team participate to IETF, is so knowledgeable, and
answers so fast makes the difference for us.
11
Client-side, server-side, PaaS and operation
Need to Collaborate
12
The WebRTC A-Team(s)
10 years ago, a crack student unit was sent to PhD courses by a European court
for WebRTC, a technology they didn’t write. These men promptly graduated from
maximum security universities to the Napoli and Singapore Underground.
Today, still wanted by their government-paid past teachers, they survive as
WebRTC consultants.
If you have a WebRTC problem, if no one else can help, and if you can find them,
maybe you can hire… the WebRTC A-Team.
13
The WebRTC A-Team(s)
14
15
Note:
Only real Rockstar:
Lorenzo Miniero
There is more to webrtc than p2p
There is more to webrtc than media server
0. Minimum viable SYSTEM:
● 1 Client app (web app in chrome)
● 1 server
1. Real Media Infrastructure:
● Load balancing
● Auto-scaling
● As many client types as possible
● Metrics
● ICE servers
2. Platform as a Service:
● Authentification
● Accounts, Tenants management
● Payment
● Recording, Ad, Watermarking, ...
ANNEX: Tooling:
● Unit testing
● Interop testing
● CI/CL
● Load Testing 16
There is more to webrtc than p2p
There is more to webrtc than media server
0. Minimum viable SYSTEM:
● 1 Client app (web app in chrome)
● 1 server
1. Real Media Infrastructure:
● Load balancing
● Auto-scaling
● As many client types as possible
● Metrics
● ICE servers
2. Platform as a Service:
● Authentification
● Accounts, Tenants management
● Payment
● Recording, Ad, Watermarking, ...
ANNEX: Tooling:
● Unit testing
● Interop testing
● CI/CL
● Load Testing 17
Let’s dive! Libwebrtc is easy
You want to add a video codec?
Just slide it in the encoder part of the code.
Easy right … ?
18
Let’s dive! Libwebrtc is easy
Well, wait,
You need the decoder too,
And the rtcp feedback loop …. 19
Let’s dive! Libwebrtc is easy
+ JSEP, + SRTP, +ICE, ...
20
As a Product maintainer …
I do not want to deal with fetching, building, or
validating libWebRTC!
I want to link against external / system libWebRTC
○ More Options
■ Additional Codecs, E2EE, OpenSSL instead of BoringSSL
○ pre-built
■ Architectures (x86_64, arm64), compilers
■ Release only, No source code, Artefact Factory
○ validated
■ Google Unit Tests, End-to-end testing, Dashboard
○ packaged
■ relocatable, Installable, Usable without DevEnv
○ “Importable” in other DevEnvs
■ CMake package, Toolchains (android!)
I want to rebase / update easily
○ Commit and branch exact Versioning of binaries
○ Migration guides / scripts
21
Unit Testing 1/4
22
Unit Testing 2/4
23
Unit Testing 3/4
24
Unit Testing 4/4
25
CoSMo Add-On(s) to Janus Janus
Server
SFU
Qt Desk. &
Mobile
Client
Electron
Desktop
Client
Signal
lib
Web
RTC
lib
Web
RTCFull
Custom
+
E2E
Encryp
lib
Web
RTC
OBS-Studio
Signal
lib
Web
RTC
Signal
WebRTC – Qt
Wrappers
26
27
Java/ obj-c
Wrapper
C++ API
arm
C++ API
x64
libWebRTC Build Types
custom build options
E2EE
OpenSSL 1.1g
Watermarking
Additional Codecs
G.729
VP9 SVC
Medooze
Jitsi
Janus
Web App
React Native
App
(iOS/android)
OBS-Studio
Qt App
Desktop
M
obile
&
IoT
IoT
Electron
App
Browsers
Chrome
Firefox
Safari
Edgium
Native
Mobile
Java API
C++ API
obj-c API
Swift API
Millicast
Raspberry Pi
TizenOS
Native Flutter
Mobile App
69
28
People come at it from all different angles: Web First
1. Most people Starts with a web app in Chrome
2. Extending to more browser is already
Challenge in itself even if it’s getting better
(see next slide)
3. Then people want native, desktop or mobile,
With minimum extra effort. It means reusing
JS code (signalling, UI/UX, logic) The usual suspects:
● React-native (mobile) => needs libwebrtc
● Electron (Desktop)
29
2018 ~ 2019 - WPT.fyi: many more passing tests
30
People come at it from all different angles: Native First
1. Many people are mobile only
2. Most of those use libwebrtc for the media
Stack. Compilation can be tricky.
3. Then people want native, desktop or mobile,
With minimum extra effort. It means reusing
Native code (signalling, UI/UX, logic).
No usual suspect outside of full default apps
31
We had a dream !
1. Start with web app for simplicity
2. Reuse JS signalling APi and logic in native app to reduce code duplication
3. Except for JS, No binding (java, obj-C, ….),
4. Single source code (ASM/C/C++) for mobile and native !
5. Professional cross-platform UI framework
6. Professional cross-platform Build/Test/Report tools
And someone gave us QT (+ cmake)!
32
Dreams do come true!
It just takes more sweat than you think.
1. Build system, QtCreator
2. Thread model,
3. Memory Model,
4. Garbage Collection,
5. JS reuse,
a. QML auto-binding
b. Strings / QStrings
6. Capturer (Desktop/App capture)
7. Rendering (Image Adaptation, OnFrame callback)
8. Websocket (QtWebsocket subprotocol support limitation)
33
34

Janus conf'19: janus client side

  • 1.
    Janus Client-Side Dr AlexGouaillard CoSMo Software
  • 2.
    WARNING ! 2 ● Thisis a pineapple-free presentation. No pineapple has been used during the making of this presentation, and no pizza was tortured nor coerced to have an indecent relationship with a pineapple!
  • 3.
    WARNING ! 3 ● Thisis a pineapple-free presentation. No pineapple has been used during the making of this presentation, and no pizza was tortured nor coerced to have an indecent relationship with a pineapple! ● This is a VoIP-free presentation! The words Kamailio, FreeSwitch, and Asterisk will have no part in this presentation. If this hurts your feelings, or your (religious) beliefs, …. C’est la vie!
  • 4.
    We are CoSMo Weused to cure cancer with ML and AI (ten years ago),and then webrtc came along. 4
  • 5.
    We are CoSMo ●Based in Singapour (HQ), Manila, France, Madrid and L.A. (streaming) ● Multiple open-source collaborators ○ Google, ○ Apple, ○ Xyrsis, ○ Meteecho, ○ ... ● Gift economy:Contributions to standards and testing ○ W3C ○ IETF ○ IMTC ○ AOMedia 5
  • 6.
    We Love (C++)WebRTC, C, ASM, codecs, and testing Which makes us very popular because nobody else do. ● Contribution to Open-Source ○ KITE (With Google, Used by Apple and others) ○ No-code fuzzing ○ E2EME (PERC, SFrame) ○ H264 simulcast ● Ongoing Open-Source ○ H.265 HW accelerated (with INTEL and Apple) => working, being tested by A, prep for upstream ○ Real-time AV1 => Proposed to AOM, to be presented at board meeting sep 25th ○ Media Transport / Web Transport / Datagram on QUIC => working impl. On top of picoQuic ● Non-open source research ○ Narval ○ Real-Time Server Side Ad-Insertion ○ Real-Time Digital Forensic Watermaking of Video Media 6
  • 7.
    We are CoSMo ●Free Testing results and white papers for the community at large ● IETF HAckathons open to all (including commercial non open-source media servers). 7
  • 8.
    W3C TPAC 2019- WPT.kite 8
  • 9.
    W3C TPAC 2019- WPT.kite 9
  • 10.
    We have ourown Media Server ... 10
  • 11.
    We still loveJanus! ● Video Conferencing vs Streaming ● Hybrid ? (webinar, conference, courses, …..) ● IoT? => 100% Janus ● Legacy? ○ VoIP => 100% Janus ○ Streaming (RTMP, HLS, MPEG-DASH, ….) The fact that the Janus team participate to IETF, is so knowledgeable, and answers so fast makes the difference for us. 11
  • 12.
    Client-side, server-side, PaaSand operation Need to Collaborate 12
  • 13.
    The WebRTC A-Team(s) 10years ago, a crack student unit was sent to PhD courses by a European court for WebRTC, a technology they didn’t write. These men promptly graduated from maximum security universities to the Napoli and Singapore Underground. Today, still wanted by their government-paid past teachers, they survive as WebRTC consultants. If you have a WebRTC problem, if no one else can help, and if you can find them, maybe you can hire… the WebRTC A-Team. 13
  • 14.
  • 15.
  • 16.
    There is moreto webrtc than p2p There is more to webrtc than media server 0. Minimum viable SYSTEM: ● 1 Client app (web app in chrome) ● 1 server 1. Real Media Infrastructure: ● Load balancing ● Auto-scaling ● As many client types as possible ● Metrics ● ICE servers 2. Platform as a Service: ● Authentification ● Accounts, Tenants management ● Payment ● Recording, Ad, Watermarking, ... ANNEX: Tooling: ● Unit testing ● Interop testing ● CI/CL ● Load Testing 16
  • 17.
    There is moreto webrtc than p2p There is more to webrtc than media server 0. Minimum viable SYSTEM: ● 1 Client app (web app in chrome) ● 1 server 1. Real Media Infrastructure: ● Load balancing ● Auto-scaling ● As many client types as possible ● Metrics ● ICE servers 2. Platform as a Service: ● Authentification ● Accounts, Tenants management ● Payment ● Recording, Ad, Watermarking, ... ANNEX: Tooling: ● Unit testing ● Interop testing ● CI/CL ● Load Testing 17
  • 18.
    Let’s dive! Libwebrtcis easy You want to add a video codec? Just slide it in the encoder part of the code. Easy right … ? 18
  • 19.
    Let’s dive! Libwebrtcis easy Well, wait, You need the decoder too, And the rtcp feedback loop …. 19
  • 20.
    Let’s dive! Libwebrtcis easy + JSEP, + SRTP, +ICE, ... 20
  • 21.
    As a Productmaintainer … I do not want to deal with fetching, building, or validating libWebRTC! I want to link against external / system libWebRTC ○ More Options ■ Additional Codecs, E2EE, OpenSSL instead of BoringSSL ○ pre-built ■ Architectures (x86_64, arm64), compilers ■ Release only, No source code, Artefact Factory ○ validated ■ Google Unit Tests, End-to-end testing, Dashboard ○ packaged ■ relocatable, Installable, Usable without DevEnv ○ “Importable” in other DevEnvs ■ CMake package, Toolchains (android!) I want to rebase / update easily ○ Commit and branch exact Versioning of binaries ○ Migration guides / scripts 21
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
    CoSMo Add-On(s) toJanus Janus Server SFU Qt Desk. & Mobile Client Electron Desktop Client Signal lib Web RTC lib Web RTCFull Custom + E2E Encryp lib Web RTC OBS-Studio Signal lib Web RTC Signal WebRTC – Qt Wrappers 26
  • 27.
  • 28.
    Java/ obj-c Wrapper C++ API arm C++API x64 libWebRTC Build Types custom build options E2EE OpenSSL 1.1g Watermarking Additional Codecs G.729 VP9 SVC Medooze Jitsi Janus Web App React Native App (iOS/android) OBS-Studio Qt App Desktop M obile & IoT IoT Electron App Browsers Chrome Firefox Safari Edgium Native Mobile Java API C++ API obj-c API Swift API Millicast Raspberry Pi TizenOS Native Flutter Mobile App 69 28
  • 29.
    People come atit from all different angles: Web First 1. Most people Starts with a web app in Chrome 2. Extending to more browser is already Challenge in itself even if it’s getting better (see next slide) 3. Then people want native, desktop or mobile, With minimum extra effort. It means reusing JS code (signalling, UI/UX, logic) The usual suspects: ● React-native (mobile) => needs libwebrtc ● Electron (Desktop) 29
  • 30.
    2018 ~ 2019- WPT.fyi: many more passing tests 30
  • 31.
    People come atit from all different angles: Native First 1. Many people are mobile only 2. Most of those use libwebrtc for the media Stack. Compilation can be tricky. 3. Then people want native, desktop or mobile, With minimum extra effort. It means reusing Native code (signalling, UI/UX, logic). No usual suspect outside of full default apps 31
  • 32.
    We had adream ! 1. Start with web app for simplicity 2. Reuse JS signalling APi and logic in native app to reduce code duplication 3. Except for JS, No binding (java, obj-C, ….), 4. Single source code (ASM/C/C++) for mobile and native ! 5. Professional cross-platform UI framework 6. Professional cross-platform Build/Test/Report tools And someone gave us QT (+ cmake)! 32
  • 33.
    Dreams do cometrue! It just takes more sweat than you think. 1. Build system, QtCreator 2. Thread model, 3. Memory Model, 4. Garbage Collection, 5. JS reuse, a. QML auto-binding b. Strings / QStrings 6. Capturer (Desktop/App capture) 7. Rendering (Image Adaptation, OnFrame callback) 8. Websocket (QtWebsocket subprotocol support limitation) 33
  • 34.