Simulcast/SVC @ IIT-RTC 2019

1.
SFU’s, Simulcast andSVC What’s new in WebRTC? Lorenzo Miniero @elminiero IIT Real-Time Communication 2019 – WebRTC Track October 15th 2019, Chicago, IL, USA

2.
A few wordsabout me Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Main author of Janus® Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero

3.
WebRTC topologies: full-mesh https://webrtchacks.com/webrtc-beyond-one-one/

4.
WebRTC topologies: MCU(Multipoint Control Unit) https://webrtchacks.com/webrtc-beyond-one-one/

5.
WebRTC topologies: SFU(Selective Forwarding Unit) https://webrtchacks.com/webrtc-beyond-one-one/

6.
Simulcast in anutshell https://webrtchacks.com/sfu-simulcast/

7.
SVC as adifferent way to encode multiple tracks https://webrtchacks.com/chrome-vp9-svc/

8.
Simulcast vs. SVC •Simulcast • Same source, same m-line • Streams of different “quality” are separate tracks • Each track is a different SSRC • Each track can be decoded indepedently from others • SVC • Same source, same m-line • Streams of different “quality” are layers of the same “thing” • All tracks share the same SSRC (since they’re layers) • Each track depends on the previous to be decoded • Less bandwidth, but more CPU intensive Fun fact – Simulcast in browsers also enables temporal scalability Allows to drop to lower framerate without sacriﬁcing quality

9.

10.

11.
Both only makesense with an SFU on the path • Browsers can’t negotiate receiving part of simulcast • ... unless you’re Philipp Hancke’s browser! • https://webrtchacks.com/a-playground-for-simulcast-without-an-sfu/ • It wouldn’t make much sense anyway! • Why receive all “versions” when you only need one? • Job for a Selective Forwarding Unit! • Pretty much all SFU’s support simulcast today • Janus (wink wink! ) • Jitsi • mediasoup • Medooze • ... • Most support some ﬂavour of SVC as well (more on that later)

12.

13.

14.

15.

16.
Tackling simulcast atthe IETF 104 hackathon https://trac.ietf.org/trac/ietf/meeting/wiki/104hackathon/webrtc

17.
Enabling simulcast theugly way (SDP munging) a=ssrc:659652645 cname:Taj3/ieCnLbsUFoH a=ssrc:659652645 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:659652645 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:659652645 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:98148385 cname:Taj3/ieCnLbsUFoH a=ssrc:98148385 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:98148385 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:98148385 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc-group:FID 659652645 98148385

18.
Enabling simulcast theugly way (SDP munging) a=ssrc:659652645 cname:Taj3/ieCnLbsUFoH a=ssrc:659652645 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:659652645 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:659652645 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:98148385 cname:Taj3/ieCnLbsUFoH a=ssrc:98148385 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:98148385 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:98148385 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc-group:FID 659652645 98148385

19.
Enabling simulcast theugly way (SDP munging) a=ssrc:659652645 cname:Taj3/ieCnLbsUFoH a=ssrc:659652645 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:659652645 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:659652645 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:98148385 cname:Taj3/ieCnLbsUFoH a=ssrc:98148385 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:98148385 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:98148385 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:1982135572 cname:Taj3/ieCnLbsUFoH a=ssrc:1982135572 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:1982135572 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:1982135572 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:2523084908 cname:Taj3/ieCnLbsUFoH a=ssrc:2523084908 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:2523084908 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:2523084908 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:3604909222 cname:Taj3/ieCnLbsUFoH a=ssrc:3604909222 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:3604909222 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:3604909222 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:1893605472 cname:Taj3/ieCnLbsUFoH a=ssrc:1893605472 msid:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk 028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc:1893605472 mslabel:i1zOaprU7rZzMDaOXFdqwkq7Q6wP6f3cgUgk a=ssrc:1893605472 label:028ab73b-cdd0-4b61-a282-ea0ed0c6a9bb a=ssrc-group:SIM 659652645 1982135572 3604909222 a=ssrc-group:FID 659652645 98148385 a=ssrc-group:FID 1982135572 2523084908 a=ssrc-group:FID 3604909222 1893605472

20.

21.

22.

23.
Hardcoded table forsimulcast bitrates https://webrtchacks.com/a-playground-for-simulcast-without-an-sfu/

24.
Testing simulcasting viaSDP munging https://www.meetecho.com/blog/simulcast-janus-ssrc/

25.
Enter “rid” (Firefox>= 46) var sender = pc.getSenders().find(s => s.track.kind == "video"); var parameters = sender.getParameters(); if(!parameters) parameters = {}; parameters.encodings = [ { rid: "h", active: true, maxBitrate: 900000 }, { rid: "m", active: true, maxBitrate: 300000, scaleResolutionDownBy: 2 }, { rid: "l", active: true, maxBitrate: 100000, scaleResolutionDownBy: 4 } ]; sender.setParameters(parameters);

26.
Encoding simulcast infoin the SDP offer a=extmap:6/sendonly urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id [..] a=rid:h send a=rid:m send a=rid:l send a=simulcast: send rid=h;m;l a=ssrc:2232294953 cname:{d31e1342-72d7-45e3-9b8d-9419cf7121b6} a=ssrc:2149721810 cname:{d31e1342-72d7-45e3-9b8d-9419cf7121b6} a=ssrc:2539596807 cname:{d31e1342-72d7-45e3-9b8d-9419cf7121b6}

27.
Encoding simulcast infoin the SDP answer a=extmap:6/recvonly urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id [..] a=rid:h recv a=rid:m recv a=rid:l recv a=simulcast: recv rid=h;m;l

28.
Testing Firefox’s ﬂavourof rid-based simulcasting https://www.meetecho.com/blog/simulcast-janus-ssrc/

29.
Conﬁguring the simulcastenvelope the “spec” way pc.addTransceiver(track, { direction: "sendrecv", streams: [stream], sendEncodings: [ { rid: "h", active: true, maxBitrate: 900000 }, { rid: "m", active: true, maxBitrate: 300000, scaleResolutionDownBy: 2 }, { rid: "l", active: true, maxBitrate: 100000, scaleResolutionDownBy: 4 } ] }); https://www.w3.org/TR/webrtc/#simulcast-functionality

30.
Hey, where’s mySSRC?! a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id [..] a=rid:h send a=rid:m send a=rid:l send a=simulcast:send h;m;l

31.
SDP answer a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=extmap:6urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id [..] a=rid:h recv a=rid:m recv a=rid:l recv a=simulcast: recv h;m;l

32.
Why remove theSSRC from the SDP? • Many implementations rely on the SSRC for demultiplexing • RTP/RTCP from multiple streams all muxed together • SSRC used to recognize one stream from another • Missing SSRCs break most of those applications • Chrome’s perspective: the problem of mapping “rid” to “ssrc” • Both are in the SDP, but how are they mapped? • Order-based just a convention, and at the time not speciﬁed anywhere • https://tools.ietf.org/html/draft-alvestrand-mmusic-simulcast-ssrc-00 Solution: parse rid RTP extension on the recipient side • SSRC in related packet allows for speciﬁc rid↔ssrc association • Once you know the SSRC, keep on multiplexing on that

33.

34.

35.
Mapping “rid” valuesto SSRC

36.
Mapping “rid” valuesto SSRC

37.
Testing rid-based simulcastingvia sendEncodings https://www.meetecho.com/blog/simulcast-janus-ssrc/

38.
What about SVC? https://webrtchacks.com/chrome-vp9-svc/

39.
Currently only availablein Chrome, and behind a ﬂag /opt/google/chrome/google-chrome --user-data-dir=/home/user/customprofile --no-first-run --force-fieldtrials= WebRTC-SupportVP9SVC/EnabledByFlag_2SL3TL/

40.
Testing VP9 SVCin Chrome https://www.meetecho.com/blog/vp9-svc-in-janus-meetecho-cosmo/

41.
AV1 is coming!(and SVC is mandated) https://aomediacodec.github.io/av1-spec/

42.
Prefeﬁned scalability modes NameSpatial Layers Resolution Ratio Temporal Layers Inter-layer dependency L1T2 1 2 L1T3 1 3 L2T1 2 2:1 1 Yes L2T2 2 2:1 2 Yes L2T3 2 2:1 3 Yes S2T1 2 2:1 1 No S2T2 2 2:1 2 No S2T3 2 2:1 3 No L2T1h 2 1.5:1 1 Yes L2T2h 2 1.5:1 2 Yes L2T3h 2 1.5:1 3 Yes S2T1h 2 1.5:1 1 No S2T2h 2 1.5:1 2 No S2T3h 2 1.5:1 3 No ... ... ... ... ... https://w3c.github.io/webrtc-svc/#scalabilitymodes*

43.
Prefeﬁned scalability modes NameSpatial Layers Resolution Ratio Temporal Layers Inter-layer dependency ... ... ... ... ... L3T1 3 2:1 1 Yes L3T2 3 2:1 2 Yes L3T3 3 2:1 3 Yes S3T1 3 2:1 1 No S3T2 3 2:1 2 No S3T3 3 2:1 3 No L3T2_KEY 3 2:1 2 Yes L3T3_KEY 3 2:1 3 Yes L4T5_KEY 4 2:1 5 Yes L4T7_KEY 4 2:1 7 Yes L3T2_KEY_SHIFT 3 2:1 2 Yes L3T3_KEY_SHIFT 3 2:1 3 Yes L4T5_KEY_SHIFT 4 2:1 5 Yes L4T7_KEY_SHIFT 4 2:1 7 Yes https://w3c.github.io/webrtc-svc/#scalabilitymodes*

44.
Programmable SVC –IETF 105 hackathon https://twitter.com/intenttoship/status/1152955435898605568

45.
WebRTC F2F duringW3C TPAC 2019 https://www.w3.org/2011/04/webrtc/wiki/September_19-20_2019

46.
What it maylook like in the future pc.addTransceiver(track, { direction: "sendrecv", streams: [stream], sendEncodings: [ { scalabilityMode: "L3T3", maxBitrate: 600000, maxFramerate: 30, spatialLayers: [ { active: true, maxBitrate: 50000, temporalLayerWeights: [0.6, 0.2, 0.2] }, { active: true, maxBitrate: 150000, temporalLayerWeights: [0.5, 0.4, 0.2] }, { active: false, temporalLayerWeights: [0.6, 0.3, 0.3] } ] }, ] }); https://github.com/w3c/webrtc-svc/issues/14

47.
What’s next? –IETF Hackathon in Singapore! https://www.ietf.org/how/runningcode/hackathons/106-hackathon/

48.
Thanks! Questions? Comments? Getin touch! • https://twitter.com/elminiero • https://twitter.com/meetecho • https://www.meetecho.com

Simulcast/SVC @ IIT-RTC 2019

More Related Content

What's hot

Similar to Simulcast/SVC @ IIT-RTC 2019

More from Lorenzo Miniero

Recently uploaded

In this document

Simulcast/SVC @ IIT-RTC 2019