IIT-RTC 2017 Qt WebRTC Tutorial (Qt Janus Client)

1. Native C++ WebRTC Tutorial Dr Alex and David BUI CoSMo Software

2. Introduction and goal Half of the communication applications nowadays, are not web applications. Native apps, whether mobile, desktop or IoT represent more than 50% of the usage of communication. WebRTC is today the best technology to implement a communication app, and has the sweet~~ advantage of being on-the-wire compatible with the browsers. Facebook, Whatsapp, viber, … all use a variation of the webrtc code internally. However many think WebRTC Native is just too hard! This tutorial is here to demystify webrtc code and show how, given the right tools and the right approach, you can write your own communication app, scratch that, communication system, in 4 hours. 2

3. Introduction and goal Build a native app C++ app that can connect to janus, a webrtc media server and display a remote stream. The app and janus, together, provide an entire client / server system, that can scale down to be run entirely on IoT chips raspberry pie 3, as well as scale up to millions of users if need be. The GUI framework chosen, QT, not only is pure C++, can run unchanged on all desktop Operating systems (MacOsX, Windows, Linux, …), but is also widely used in embedded applications, and can run on any kind of ARM, or IoT chips. 3

4. Introduction and goal To make it achievable in 4 hours, we will provide you with some components: - A libwebrtc installer that will provide webrtc libraries, headers, and other variables, ready to use. - A pre-configured, hosted janus instance with the “video-room” plugin, to connect to. - A continuous stream is available from the server 4

5. Compiling and using WebRTC … Don’t do it yourself Mastering Google tools requires time and patience: Depots_tools, GN, Ninja Compilers are differents on each platform: CL, MSBUILD, GCC, CLang, LLVM… As well as linkers. Compiling is hell: time consumption, flags and definitions are hidden and the layout is changing frequently. Keep the code up-to-date through the versions: macros are changing, folders are renamed, methods deleted... and maintain a backward compatibility with older versions of libwebrtc, every 6 weeks. Customizing the code is not simple and test infrastructures are not provided by Google. 5

6. Compiling and using WebRTC … Don’t do it yourself Mastering Google tools requires time and patience: Depots_tools, GN, Ninja Compilers are differents on each platform: CL, MSBUILD, GCC, CLang, LLVM… As well as linkers. Compiling is hell: time consumption, flags and definitions are hidden and the layout is changing frequently. Keep the code up-to-date through the versions: macros are changing, folders are renamed, methods deleted... and maintain a backward compatibility with older versions of libwebrtc, every 6 weeks. Customizing the code is not simple and test infrastructures are not provided by Google. 6 WebRTC is not designed to be used as a library

7. Summary ➔ CMake : the basics ◆ First app 7 ➔ Qt5 QML module ◆ Hello World ◆ QML Signal Slot ◆ Websocket ◆ Janus API ◆ Debug w Qt Creator ➔ WebRTC ◆ JSEP Overview ◆ Offer impl. ◆ Answer impl. ◆ Trickle ICE impl. 30 mn 45 mn 45 mn 60 mn ➔ Qt5 C++ App ◆ CMake: find_package() ◆ main.cpp file ◆ Link against Qt5 library ◆ Runtime ◆ Signal Slot

8. CMAKE: Let’s get started 8 ● Make a project folder ○ This is our source dir mkdir CMakeTutorial

9. CMAKE: Let’s get started In a new file CMakeLists.txt : project( myFirstApp ) 9 ● Make a project folder ○ This is our source dir ● CMakeLists.txt is the main file ○ In Source Dir

10. CMAKE: Let’s get started Add in CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( myFirstApp ) 10 ● Make a project folder ○ This is our source dir ● CMakeLists.txt is the main file ○ In Source Dir

11. CMAKE: Let’s get started ● Make a project folder ○ This is our source dir ● CMakeLists.txt is the main file ○ In Source Dir ● Main.cpp ○ In Source Dir main.cpp: #include <iostream> void main(int argc, char** argv) { std::cout << “Hello world” << std::endl; } 11

12. CMAKE: Let’s get started 12 ● Make a project folder ○ This is our source dir ● CMakeLists.txt is the main file ○ In Source Dir ● Main.cpp ○ In Source Dir Edit CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( myFirstApp ) add_executable( myApp main.cpp )

13. CMAKE: Let’s get started 13 ● Make a project folder ○ This is our source dir ● CMakeLists.txt is the main file ○ In Source Dir ● Main.cpp ○ In Source Dir ● CMake is out-of-source build style New folder to be out-of-source ● mkdir MYBUILD ● cd MYBUILD

14. CMAKE: Let’s get started 14 ● Make a project folder ○ This is our source dir ● CMakeLists.txt is the main file ○ In Source Dir ● Main.cpp ○ In Source Dir ● CMake is out-of-source build style ○ CMake commands in Build Dir 1. Configure the project cmake .. 2. Compile with cross-platform command cmake --build . 3. Run the project and make sure everything works fine.

15. CMAKE: Wrap-Up CMake ● CMakeLists.txt ● project() ● add_executable() ● CMake configure ● CMake build ● run I don’t care …. I love it ! Windows / Mac / Linux / … MSVC / XCode / CLang / ... 15

16. Summary ➔ CMake : the basics ◆ First app 16 ➔ Qt5 QML module ◆ Hello World ◆ QML Signal Slot ◆ Websocket ◆ Janus API ◆ Debug w Qt Creator ➔ Qt5 C++ App ◆ CMake: find_package() ◆ main.cpp file ◆ Link against Qt5 library ◆ Runtime ◆ Signal Slot ➔ WebRTC ◆ JSEP Overview ◆ Offer impl. ◆ Answer impl. ◆ Trickle ICE impl. 30 mn 45 mn 45 mn 60 mn

17. Qt5 C++ app: Cmake FindPackage Create CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( QtFirstApp ) 17 ● In a new source directory Qt5App

18. Qt5 C++ app: Cmake FindPackage Add in CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( QtFirstApp ) find_package( Qt5 COMPONENTS Widgets ) 18 ● In a new source directory Qt5App ● find_package()

19. Qt5 C++ app: Cmake FindPackage: Background FindXXX.cmake ● One canonical install, ● possibly shipped with CMake, ● old school xxxConfig.cmake ● Handle multiple installs ○ Which one? Use -D …. ● Shipped with library (always in sync) ● The current way of things 19 ● In a new source directory Qt5App ● find_package() ○ FindXXX.cmake ○ XXXConfig.cmake

20. Qt5 C++ app: Cmake FindPackage CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( myFirstApp ) message( STATUS “Qt5 Dir: ${Qt5Core_DIR}.” ) find_package( Qt5 COMPONENTS Widgets ) message( STATUS “Qt5 Dir: ${Qt5Core_DIR}.” ) 20 ● In a new source directory Qt5App ● find_package() ○ FindXXX.cmake ○ XXXConfig.cmake ● cmake -DQt5_DIR=...

21. Qt5 C++ app: main main.cpp : #include <QApplication> int main(int argc, char *argv[]) { QApplication app(argc, argv); return app.exec(); } Create a new file main.cpp in the same folder as CMakeLists.txt Since we are using Qt5 GUI, the app will be running as a QApplication object type. To run the app, you simply need to call exec(). 21

22. Qt5 C++ app: main CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( myFirstApp ) message( STATUS “Qt5 Dir: ${Qt5Core_DIR}.” ) find_package( Qt5 COMPONENTS Widgets ) message( STATUS “Qt5 Dir: ${Qt5Core_DIR}.” ) add_executable( QtFirstApp main.cpp ) 22 ● In a new source directory Qt5App ● find_package() ○ FindXXX.cmake ○ XXXConfig.cmake ● cmake -DQt5_DIR=... ● Add executable

23. Qt5 C++ app: link to qt libs At this point, you should have a compilation error. The linker can not resolve the QApplication inclusion. Link the app against Qt library because we are using Qt5 Widgets module. Then compile and run the app. Add to CMakeLists.txt : cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR ) project( myFirstApp ) message( STATUS “Qt5 Dir: ${Qt5Core_DIR}.” ) find_package( Qt5 COMPONENTS Widgets ) message( STATUS “Qt5 Dir: ${Qt5Core_DIR}.” ) add_executable( QtFirstApp main.cpp ) target_link_libraries( QtFirstApp Qt5::Widgets ) 23

24. Qt5 C++ app: WIN: 64 bits and generators In a Build directory : Configure with a correct generator to have the app in 64 bits. cmake -DQt5_DIR=path_to_qt5 -G “Visual Studio 14 2015 Win64” .. Compile cmake --build . 24 ● In a new source directory Qt5App ● find_package() ○ FindXXX.cmake ○ XXXConfig.cmake ● cmake -DQt5_DIR=... ● Add executable ● cmake -G ○ “Visual Studio 14 2015 <arch>” ■ Win64 ○ “NMake Makefiles” 2 ways to use 64 bits environments: Admin CMD + x64 tools Admin CMD + vcvarsall.bat

25. Qt5 C++ app: run: WIN: the Qt DLL problem Windows users : If you thought it would be that easy to run a Qt5 app… Wrong! You will need to copy the missing .dll from Qt5 folder to the executable folder. For example, I have 3 errors like this: 25 In C:Qt5.9.2msvc2015_64bin, copy ● Qt5Widgetsd.dll ● Qt5Cored.dll ● Qt5Guid.dll and paste in the build directory

26. Qt5 C++ app: run: Signal Slot 101 Now, it would be nice to have a button which can send a message to someone else. In general, Qt objects can communicate with one another by a system called Signals and Slots. In main.cpp: #include <QApplication> #include <QPushButton> int main(int argc, char *argv[]){ QApplication app(argc, argv); QPushButton button("Quit"); button.show(); return app.exec(); } 26

27. Qt5 C++ app: run: Signal Slot 101 A Signal is emitted when a particular event occurs, like a click. A Slot is a function that is called in response to a particular signal. Let’s connect the button to the app : When we click on the button, the app will quit. In main.cpp: int main(int argc, char *argv[]){ QApplication app(argc, argv); QPushButton button("Quit"); QObject::connect( &button, SIGNAL( clicked() ), &app, SLOT( quit() ) ); button.show(); return app.exec(); } 27

28. Qt5 C++ app: run: Signal Slot 101 28 1. Compile with cross-platform command cmake --build . 2. Run the project and make sure everything works fine. The project should be already configured, we only need to recompile the source files.

29. Qt5 C++ app: Wrap-Up CMake: cmake -G cmake -D<VAR>=<VALUE> .. find_package() COMPONENTS REQUIRED message( STATUS “” ) target_link_library() Cmake Qt5: Cmake -DQt5_DIR= .. target_link_library() QT5::CORE Qt5: Signal Slot 29

31. Qt5 QML app: Hello world 31 Let’s make an app with QML. It is a module of Qt and allows you to build fluid animated user interfaces. QML is a JavaScript compatible scripting language ● In a new source directory QMLTutorial

32. Qt5 QML app: Hello world main.qml: import QtQuick 2.0 import QtQuick.Window 2.2 Window { visible : true Rectangle { text { text: “Hello World” } } } 32 ● In a new source directory QMLTutorial ● New file main.qml

33. Qt5 QML app: Hello world qml.qrc: <RCC> <qresource prefix=”/”> <file>main.qml</file> </qresource> </RCC> 33 ● In a new source directory QMLTutorial ● New file main.qml ● New file qml.qrc which lists all qml files

34. Qt5 QML app: Hello world CMakeLists.txt: cmake_minimum_required( VERSION 3.2.0 FATAL_ERROR) project( QMLApp ) find_package( Qt5 COMPONENTS Quick Widgets REQUIRED ) set( CMAKE_AUTORCC ON ) add_executable( QMLApp main.cpp qml.qrc ) target_link_libraries( QMLApp Qt5::Widgets Qt5::Quick ) 34 ● In a new source directory QMLTutorial ● New file main.qml ● New file qml.qrc which lists all qml files ● Add qml.qrc to CMakeLists.txt in “add_executable()” ● Add set( CMAKE_AUTORCC ON )

35. Qt5 QML app: QML Signal / Slot syntax Edit main.qml : import QtQuick.Controls 1.4 35 Let’s make a button that change the text “Hello World” when we are clicking on it, all in QML. ● Import QtQuick.Controls 1.4

36. Qt5 QML app: QML Signal / Slot syntax Edit main.qml : import QtQuick.Controls 1.4 … Rectangle { … } Button { id: myButton text : "emit signal" anchors.centerIn: parent } 36 ● Import QtQuick.Controls 1.4 ● Add a Button after Rectangle ○ We need an id to identify it ○ And place it in the middle of the Window with anchors.centerIn

37. Qt5 QML app: QML Signal / Slot syntax Edit main.qml : Button { id: myButton text : "emit signal" anchors.centerIn: parent signal buttonClicked(string text) } 37 ● Import QtQuick.Controls 1.4 ● Add a Button after Rectangle ○ We need an id to identify it ○ And place it in the middle of the Window with anchors.centerIn ● Add a signal taking a string as an argument

38. Qt5 QML app: QML Signal / Slot syntax Edit main.qml : Button { id: myButton text : "emit signal" anchors.centerIn: parent signal buttonClicked(string text) onClicked: { myButton.buttonClicked.connect(myText.changeText) myButton.buttonClicked("Button Clicked") } } 38 ● Import QtQuick.Controls 1.4 ● Add a Button after Rectangle ○ We need an id to identify it ○ And place it in the middle of the Window with anchors.centerIn ● Add a signal taking a string as an argument ● Add the behaviors when the button is Clicked

39. Qt5 QML app: QML Signal / Slot syntax Edit main.qml : Text { id: myText text: "Hello World" } 39 Now the Button is connected, we need to redefine a bit the Text ● Add an id to Text : myText

40. Qt5 QML app: QML Signal / Slot syntax Edit main.qml : Text { id: myText text: "Hello World" function changeText(text) { myText.text = text } } 40 Now the Button is connected, we need to redefine a bit the Text ● Add an id to Text : myText ● Add a function that change the attribute text

41. Qt5 QML app: QML Signal / Slot syntax 41 Here you go! In the build directory: 1. Compile with cross-platform command cmake --build . 2. Run the project and make sure everything works fine.

42. Qt5 QML app: Websockets ● Websocket Object ○ url ○ active ○ onTextMessageReceived ○ onStatusChanged ○ sendTextMessage WebSocket { id: janusWebSocketWrapper url: "wss://..." active: false onTextMessageReceived: {} onStatusChanged: {} } 42

43. Qt5 QML app: Websockets 43 ● Websocket Object ○ url ○ active (Set active to true to connect) ○ onTextMessageReceived ○ onStatusChanged (monitor ==>>) ○ sendTextMessage onStatusChanged: { console.log("Status Changed: "+ status) if (status === WebSocket.Error) { console.log("Websocket error: " + errorString) } else if (status === WebSocket.Open) { console.log("Websocket open.") } else if (status === WebSocket.Closed) { console.log("Websocket closed.") } else { console.log("Websocket status yet unsupported.") } }

44. Qt5 QML app: Janus Basic msg flow: random ids 44

45. Qt5 QML app: Example: Logging to Janus function sendLogin(room, login){ logMessage('Sending login: '+login+', for room: '+room) var msg = { request: "login", payload: { username: login, token: token, room: room } }; sendMessage(msg); } 45

46. Qt5 QML app: Debug QML + C++ with Qt Creator ● CMAKE_CXX_FLAGS_DEBUG Add in CMakeLists.txt: set( CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DQT_QML_DEBUG ") + Setting in QtCreator (screen captures) 46

47. Qt5 QML app: Wrap-Up Cmake Qt5: set( CMAKE_AUTORCC ON ) -DQT_QML_DEBUG Qt5 QML: Window { } Rectangle { } Qrc file Signals/Slots WebSocket Function name(args) { } 47 CMake: set (Variable Value)

49. WebRTC installer and CMake With the libwebrtc installer, you can use the library quite easily. With CMake, you only need these : ● find_package() ● target_link_libraries Let’s try the library! 49

50. WebRTC installer and CMake First, we need a source file using libwebrtc function. ● In a new source directory WebRTCSimpleApp ● Make a new file SimpleApp.cpp 50 In a new main.cpp file : int main(int argc, char** argv) { return 0; }

51. WebRTC installer and CMake First, we need a source file using libwebrtc function. ● In a new source directory WebRTCSimpleApp ● Make a new file SimpleApp.cpp ● Include a header from libwebrtc 51 In a new SimpleApp.cpp file : #include "webrtc/api/peerconnectioninterface.h" int main(int argc, char** argv) { return 0; }

52. WebRTC installer and CMake First, we need a source file using libwebrtc function. ● In a new source directory WebRTCSimpleApp ● Make a new file SimpleApp.cpp ● Include a header from libwebrtc ● Call functions from the library 52 In a new SimpleApp.cpp file : #include "webrtc/api/peerconnectioninterface.h" int main(int argc, char** argv) { rtc::scoped_refptr<webrtc::PeerConnectionFactoryInterface> peer_connection_factory = webrtc::CreatePeerConnectionFactory(); return 0; }

53. WebRTC installer and CMake Now, let’s compile the source file with CMake. ● Add the cmake_minimum_required ● Add the project 53 In a new CMakeLists.txt : cmake_minimum_required( VERSION 3.3 ) project( SimpleApp )

54. WebRTC installer and CMake Now, let’s compile the source file with CMake. ● Add the cmake_minimum_required ● Add the project ● Install libwebrtc if you haven’t yet 54 In a new CMakeLists.txt : cmake_minimum_required( VERSION 3.3 ) project( SimpleApp )

55. WebRTC installer and CMake Now, let’s compile the source file with CMake. ● Add the cmake_minimum_required ● Add the project ● Install libwebrtc if you haven’t yet ● find_package() so you have access to the library and configuration 55 In a new CMakeLists.txt : cmake_minimum_required( VERSION 3.3 ) project( SimpleApp ) find_package( libwebrtc REQUIRED )

56. WebRTC installer and CMake Now, let’s compile the source file with CMake. ● Add the cmake_minimum_required ● Add the project ● Install libwebrtc if you haven’t yet ● find_package() so you have access to the library ● ${WEBRTC_LIBRARIES} : this variable includes all you need 56 In a new CMakeLists.txt : cmake_minimum_required( VERSION 3.3 ) project( SimpleApp ) find_package( libwebrtc REQUIRED ) add_executable( SimpleApp SimpleApp.cc ) target_link_libraries( SimpleApp ${WEBRTC_LIBRARIES} )

57. WebRTC JSEP Overview ● (add one slide peerconnection exchange) 57

58. WebRTC JSEP: QML Janus offer ● Janus Subscribe Flow ○ publishers ○ Create handle ○ subscribe I got offer, let s give it to webrtc stack 58

59. WebRTC JSEP: C++ webrtc answer ● PeerconnectionFactory ● Peerconnection ● PeerconnectionObserver ○ setRemoteDescription() ○ createAnswer() ○ setLocalDescription() ● createSessionDescriptionObser ver I got an answer, let s give it back to QML 59

60. WebRTC JSEP: QML answer to janus and ack. ● Janus Subscribe Flow ○ Feed::new ○ Open ○ Answer ○ .... Main.cpp Main.qml 60

61. WebRTC trickle ICE: C++ and QML ● Peerconnection ● PeerconnectionObserver ○ onIceCandidate() ● Janus Subscribe Flow ○ Trickle Main.cpp Main.qml 61

62. WebRTC trickle ICE: C++ and QML ● Peerconnection ● PeerconnectionObserver ○ onAddStream() ● MediaStream ○ getVideoTracks() ● VideoTrackInterface ○ onFrame() ● VideoFrame ● QML WebRTC Renderer ○ provided Main.cpp Main.qml 62

IIT-RTC 2017 Qt WebRTC Tutorial (Qt Janus Client)

IIT-RTC 2017 Qt WebRTC Tutorial (Qt Janus Client)

Recommended

More Related Content

What's hot

What's hot(20)

Similar to IIT-RTC 2017 Qt WebRTC Tutorial (Qt Janus Client)

Similar to IIT-RTC 2017 Qt WebRTC Tutorial (Qt Janus Client)(20)

More from Alexandre Gouaillard

More from Alexandre Gouaillard(20)

Recently uploaded

Recently uploaded(20)

IIT-RTC 2017 Qt WebRTC Tutorial (Qt Janus Client)