Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WebRTC Browsers n Stacks Implementation differences

3,474 views

Published on

Architecture diagrams and difference between webrtc implementation in broswers and stacks.

Published in: Technology

WebRTC Browsers n Stacks Implementation differences

  1. 1. Browsers and stack implementation differences Dr. Alex Gouaillard
  2. 2. Plan • global architectures of browsers and stacks • -- chrome / firefox / webkit • -- webrtc stand alone, webrtc in chrome, webrtc in firefox • - how to trace the different browsers without recompiling
  3. 3. WebRTC Arch
  4. 4. LibWEBRTC Variation
  5. 5. Chrome Variation
  6. 6. Chrome Variation • Wait .. What? Chrome does not use libwebrtc? Why?
  7. 7. Chrome Layers
  8. 8. Chrome Layers
  9. 9. Chrome Variation • Wait .. What? Chrome does not use libwebrtc? Why? – Chrome layering – Sandbox and per tab security – Echo canceling – Special capturers (screen/window/tab) – Capturer access security – Pretty much the same applies to all browsers
  10. 10. Chrome Variations • OK, Chrome does not use libwebrtc? Why should I Care? • If you use the webrtc lib in your project, and expect the same results as chrome, be ready for big surprises! – No Screen/windows/tab sharing – Obsolete mac capturer – No HTTP PROXY support
  11. 11. LibWEBRTC src Files layout
  12. 12. Libwebrtc src file layout • base/ • common_audio/ • common_video/ • libjingle/ • modules/ • p2p/ • sound/ • system_wrappers/ • tools/ • video/ • video_engine/ • voice_engine/ • Too many to list • Basic signal processing (FFT, window, transforms, …) • Frame buffer, I420, libyuv, … • Session management • See next slide • ICE Transports: TURN, STUN, DTLS, …. • Platform specific sound system wrappers • Platform specific low level wrappers (threads, timer, …) • Poach and weaponize!! • Video encoder/decoder/rec and send stream • Capturer, Channel, Encoder, Sync, REMB, Send • Channel, DTMF, …
  13. 13. Libwebrtc src file layout • audio_coding/ • audio_conference_mixer/ • audio_device/ • audio_processing/ • bitrate_controller/ • desktop_capture/ • media_file/ • pacing/ • remote_bitrate_estimator/ • rtp_rtcp/ • utility/ • video_capture/ • video_coding/ • video_processing/ • video_render/ • ACM2, NACK, NETQ, DTMF tone gen., …. • mixer • Platform specific • Noise, typing, gain detection, NS, Echo Cancellation, … • Bandwidth estimation and bitrate controller • Desktop capture (abstract only) • Audio files reading/writing • Paced sender • Obvious • RTP, RTCP, Payloads, FEC, … • More Audio files utilities, rtp_dump, recorder, player • Generic and platform dependent capturers (OLD) • Codecs, encoders and decoders, send and rec, jitter, … • Video processing and image enhancement • Generic and platform dependent renderers (OLD)
  14. 14. Libwebrtc src file layout • Why do I care? – 1. you might need to go in the code – 2. you need to know the layout for debugging: chrome.exe --enable-logging -- vmodule=*/webrtc/*=2,*/libjingle/*=2,*=-2 --no- sandbox
  15. 15. LibWEBRTC Class Arch
  16. 16. LibWEBRTC Arch
  17. 17. LibWEBRTC DEPS • binutils • Boringssl • Colorama • Drmemory • Expat • Icu • instrumented_libraries • Jsoncpp • Libjpeg • libjpeg_turbo • Libsrtp • libudev • libvpx • Libyuv • llvm-build • Nss • Ocmock • openmax_dl • Opus • Protobuf • Sqlite • Syzygy • Usrsctp • Yasm • zlib
  18. 18. Webrtc.org
  19. 19. Chrome Specific Webrtc Bits • //src/content/renderer/media/ • //src/third_party/WebKit/public/platform/ • //src/third_party/WebKit/Source/modules/mediastream • //src/content/browser/media/ • //src/content/browser/media/capture/ • //src/media/capture/video/ • //src/content/browser/renderer_host/media/ • . • . • . • . • . • . • .
  20. 20. Chrome Specific Webrtc Bits // MediaStreamManager is used to open/enumerate media capture devices (video // supported now). Call flow: // 1. GenerateStream is called when a render process wants to use a capture // device. // 2. MediaStreamManager will ask MediaStreamUIController for permission to // use devices and for which device to use. // 3. MediaStreamManager will request the corresponding media device manager(s) // to enumerate available devices. The result will be given to // MediaStreamUIController. // 4. MediaStreamUIController will, by posting the request to UI, let the // users to select which devices to use and send callback to // MediaStreamManager with the result. // 5. MediaStreamManager will call the proper media device manager to open the // device and let the MediaStreamRequester know it has been done.
  21. 21. Chrome GUM example capturers / frames / Encoding / ….
  22. 22. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser)GUM JS API (tab/sandbox) 1. Send request
  23. 23. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser)GUM JS API (tab/sandbox) 2. Check if MST is already available
  24. 24. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser)GUM JS API (tab/sandbox) Security Manager (source, origin) 3. Check rights 2. Check if MST is already available - NO
  25. 25. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) GUM JS API (tab/sandbox) 4. Ask Corresponding capturer type to start capturing 3. Check rights - OK
  26. 26. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) 4. Ask Corresponding capturer type to create one - OK V 5. Store the MST
  27. 27. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V 6. Trigger callback Keep feeding frames
  28. 28. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V NOTE 1: second call for same device with same constraints will directly return the MST, that allows to share streams across tabs without blocking
  29. 29. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V NOTE 2: Recently, a second call for the same device but with different constraints (think simulcast) will indeed return a different resolution. Before it would return the first resolution asked.
  30. 30. Chrome Specific Webrtc Bits Media Stream Manager (singleton@browser) Audio Capturer Video Capturer Security Manager (source, origin) A GUM JS API (tab/sandbox) V NOTE 3: Not only this allow to share cams across processes, it allows for global echo cancellation (yes, including the key strokes). Before tabs could cross feed.
  31. 31. Chrome Screensharing 2 steps (1) Media Stream Manager (singleton@browser) Screen/Windows/Tab Capturer Security Manager (source, origin) Screensharing (extension) 1 2 3 4 S 5
  32. 32. Chrome Screensharing 2 steps (2) Media Stream Manager (singleton@browser) Screen/Windows/Tab Capturer S GUM JS API (tab/sandbox) With ID
  33. 33. Firefox Variation
  34. 34. Firefox Variation – Desktop 1process
  35. 35. Firefox Variation – B2G
  36. 36. FireFox Arch • PeerConnection – PeerConnection.js -- shim translation layer to let us do API adaptation to the C++ – PeerConnectionImpl -- C++ implementation of the PeerConnection interface. – SIPCC -- handles SDP and media negotiation. Provided by Cisco but not a downstream. • Media – Webrtc.org/GIPS -- handles media encoding and decoding. Downstream from Google. – MediaConduit -- Generic wrapper around Webrtc.org – MediaPipeline -- Wrapper to hold the MediaConduit, mtransport subsystem, and the SRTP contexts, as well as interface with MediaStreams. • Transport – mtransport -- generic transport subsystem with implementations for ICE, DTLS, etc. – NSS -- new DTLS stack. Mentioned because we need to land the new version of NSS – nICEr -- ICE stack; downstream from reSIProcate project – nrappkit --portable runtime, utility library; downstream from nrappkit.sourceforge.net • DataChannel – DataChannel implementation in the DOM – libsctp -- SCTP implementation; downstream from the BSD SCTP guys
  37. 37. Webkit (safari ?) Current work: - Implement datachannel for webkit (yin liu) - Implement webrtc.org as a back end for webkit (adam tiamou) - Implement windows support for OWR
  38. 38. Webkit Arch
  39. 39. Webkit Arch
  40. 40. Webkit Webrtc Arch
  41. 41. Webkit Arch – Data Channel
  42. 42. Webkit Arch – Data Channel

×