SlideShare a Scribd company logo
WebRTC-to-SIP and back
It’s not all about audio and video!
Lorenzo Miniero
@lminiero@fosstodon.org
OpenSIPS Summit
May 15th 2024, Valencia
A chance to practice my broken Spanish!
Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus
Contacts and info
• lorenzo@meetecho.com
• https://fosstodon.org/@lminiero
• https://www.meetecho.com
• https://lminiero.it
Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, ACM, etc.)
• Proudly brewed in sunny Napoli, Italy
A bit of context: Janus, WebRTC and SIP
Janus
General purpose, open source WebRTC server
• https://github.com/meetecho/janus-gateway
• Demos and documentation: https://janus.conf.meetecho.com
• Community: https://janus.discourse.group/
SIP plugin in Janus
https://janus.conf.meetecho.com/docs/sip
An endpoint of behalf of WebRTC users
• Janus SIP plugin acts as a collection of SIP endpoints, not a server/trunk
• SIP stack implemented with Sofia-SIP
• WebRTC users only see the Janus API (JSON), no SIP
• No transcoding, media is only relayed
• Built-in recording (separate media legs)
• Simplifies life for web developers
• No need to worry about a SIP stack (only SIP URIs)
• Basic methods/events to handle dialogs (call, answer, hangup, message, etc.)
• Allows SIP headers injection/interception in many requests
• Support for more advanced features too (e.g., hold, transfer, etc.)
An endpoint of behalf of WebRTC users
• Janus SIP plugin acts as a collection of SIP endpoints, not a server/trunk
• SIP stack implemented with Sofia-SIP
• WebRTC users only see the Janus API (JSON), no SIP
• No transcoding, media is only relayed
• Built-in recording (separate media legs)
• Simplifies life for web developers
• No need to worry about a SIP stack (only SIP URIs)
• Basic methods/events to handle dialogs (call, answer, hangup, message, etc.)
• Allows SIP headers injection/interception in many requests
• Support for more advanced features too (e.g., hold, transfer, etc.)
Works great with OpenSIPS!
Workshop on Janus and SIP (lesson/tutorial) at OpenSIPS 2020
https://www.youtube.com/watch?v=fv9KwrguR-4&t=3544s
Audio and video are “easy”
• Both SIP and WebRTC use SDP and RTP/RTCP
• WebRTC uses SDP/RTP/RTCP on “steroids”
• Apart from this, just differences in encryption (WebRTC mandates DTLS-SRTP)
• Media is basically encoded, packaged and sent the same way
• As long as the same codec is used, they’re interoperable
• When they aren’t, transcoding helps (but Janus won’t do it for you)
• WebRTC has mandatory-to-implement codecs
• Opus and G.711 for audio, VP8 and H.264 (baseline) for video
• G.711 and H.264 should guarantee compliance with legacy equipment
Audio and video are “easy”
• Both SIP and WebRTC use SDP and RTP/RTCP
• WebRTC uses SDP/RTP/RTCP on “steroids”
• Apart from this, just differences in encryption (WebRTC mandates DTLS-SRTP)
• Media is basically encoded, packaged and sent the same way
• As long as the same codec is used, they’re interoperable
• When they aren’t, transcoding helps (but Janus won’t do it for you)
• WebRTC has mandatory-to-implement codecs
• Opus and G.711 for audio, VP8 and H.264 (baseline) for video
• G.711 and H.264 should guarantee compliance with legacy equipment
Audio and video are “easy”
• Both SIP and WebRTC use SDP and RTP/RTCP
• WebRTC uses SDP/RTP/RTCP on “steroids”
• Apart from this, just differences in encryption (WebRTC mandates DTLS-SRTP)
• Media is basically encoded, packaged and sent the same way
• As long as the same codec is used, they’re interoperable
• When they aren’t, transcoding helps (but Janus won’t do it for you)
• WebRTC has mandatory-to-implement codecs
• Opus and G.711 for audio, VP8 and H.264 (baseline) for video
• G.711 and H.264 should guarantee compliance with legacy equipment
SIP sometimes does more than that, though!
• A whole ecosystem of other protocols that could be used
• Real-Time Text (T.140 over RTP)
• Message Session Relay Protocol (MSRP)
• Binary Floor Control Protocol (BFCP)
• Fax (T.38 over RTP)
• ...
• These protocols can’t simply be gateway-ed to WebRTC
• WebRTC supports RTP, but only for audio/video, not generic data
• Custom protocols are not supported at all
• WebRTC-to-SIP gateways will in general strip them from the SDP
• We can’t rely on a WebRTC browser to simply reject unsupported media
• An unsupported m-line will cause an exception in setRemoteDescription
SIP sometimes does more than that, though!
• A whole ecosystem of other protocols that could be used
• Real-Time Text (T.140 over RTP)
• Message Session Relay Protocol (MSRP)
• Binary Floor Control Protocol (BFCP)
• Fax (T.38 over RTP)
• ...
• These protocols can’t simply be gateway-ed to WebRTC
• WebRTC supports RTP, but only for audio/video, not generic data
• Custom protocols are not supported at all
• WebRTC-to-SIP gateways will in general strip them from the SDP
• We can’t rely on a WebRTC browser to simply reject unsupported media
• An unsupported m-line will cause an exception in setRemoteDescription
SIP sometimes does more than that, though!
• A whole ecosystem of other protocols that could be used
• Real-Time Text (T.140 over RTP)
• Message Session Relay Protocol (MSRP)
• Binary Floor Control Protocol (BFCP)
• Fax (T.38 over RTP)
• ...
• These protocols can’t simply be gateway-ed to WebRTC
• WebRTC supports RTP, but only for audio/video, not generic data
• Custom protocols are not supported at all
• WebRTC-to-SIP gateways will in general strip them from the SDP
• We can’t rely on a WebRTC browser to simply reject unsupported media
• An unsupported m-line will cause an exception in setRemoteDescription
Data channels to the rescue!
Data channels to the rescue!
What are data channels?
• Arbitrary real-time connection for pretty much anything
• Bidirectional data between two WebRTC peers
• Support for multiple channels of different kinds
• Supports ordered/unordered and reliable/unreliable
• Generic data sent via SCTP and encapsulated in DTLS
• SCTP implements features, DTLS implements security
• Negotiated as an application in the SDP
• m=application 9 UDP/DTLS/SCTP webrtc-datachannel
What are data channels?
• Arbitrary real-time connection for pretty much anything
• Bidirectional data between two WebRTC peers
• Support for multiple channels of different kinds
• Supports ordered/unordered and reliable/unreliable
• Generic data sent via SCTP and encapsulated in DTLS
• SCTP implements features, DTLS implements security
• Negotiated as an application in the SDP
• m=application 9 UDP/DTLS/SCTP webrtc-datachannel
What are data channels?
• Arbitrary real-time connection for pretty much anything
• Bidirectional data between two WebRTC peers
• Support for multiple channels of different kinds
• Supports ordered/unordered and reliable/unreliable
• Generic data sent via SCTP and encapsulated in DTLS
• SCTP implements features, DTLS implements security
• Negotiated as an application in the SDP
• m=application 9 UDP/DTLS/SCTP webrtc-datachannel
That’s better, but still not enough!
• Non audio/video protocols have their own technical requirements
• RTT and T.38 will have custom RTP packetization rules
• BFCP/MSRP/others will require custom UDP or TCP transport
• All this will be reflected in the SDP negotiation
• WebRTC only supports m=application + UDP/DTLS/SCTP
• It’s the only non audio/video thing it can negotiate
• A gateway will have more work to do than with RTP audio/video
• Translate between SDP formats (e.g., custom protocol ↔ application)
• Gateway the protocol itself (e.g., RTP/T.140 ↔ data channel)
That’s better, but still not enough!
• Non audio/video protocols have their own technical requirements
• RTT and T.38 will have custom RTP packetization rules
• BFCP/MSRP/others will require custom UDP or TCP transport
• All this will be reflected in the SDP negotiation
• WebRTC only supports m=application + UDP/DTLS/SCTP
• It’s the only non audio/video thing it can negotiate
• A gateway will have more work to do than with RTP audio/video
• Translate between SDP formats (e.g., custom protocol ↔ application)
• Gateway the protocol itself (e.g., RTP/T.140 ↔ data channel)
That’s better, but still not enough!
• Non audio/video protocols have their own technical requirements
• RTT and T.38 will have custom RTP packetization rules
• BFCP/MSRP/others will require custom UDP or TCP transport
• All this will be reflected in the SDP negotiation
• WebRTC only supports m=application + UDP/DTLS/SCTP
• It’s the only non audio/video thing it can negotiate
• A gateway will have more work to do than with RTP audio/video
• Translate between SDP formats (e.g., custom protocol ↔ application)
• Gateway the protocol itself (e.g., RTP/T.140 ↔ data channel)
A practical example: Real-Time Text (T.140)
• Text transmitted instantly, as it is typed or created
• ITU-T T.140 (Protocol for multimedia application text conversation)
• Allows for real-time editing (e.g., backspace, rewriting)
• T.140 messages transported over RTP
• RFC 4103 (RTP Payload for Text Conversation)
• Redundancy implemented via RED (RFC 2198)
Specification to use T.140 over data channels
• T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865)
• https://www.rfc-editor.org/rfc/rfc8865.html
A practical example: Real-Time Text (T.140)
• Text transmitted instantly, as it is typed or created
• ITU-T T.140 (Protocol for multimedia application text conversation)
• Allows for real-time editing (e.g., backspace, rewriting)
• T.140 messages transported over RTP
• RFC 4103 (RTP Payload for Text Conversation)
• Redundancy implemented via RED (RFC 2198)
Specification to use T.140 over data channels
• T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865)
• https://www.rfc-editor.org/rfc/rfc8865.html
A practical example: Real-Time Text (T.140)
• Text transmitted instantly, as it is typed or created
• ITU-T T.140 (Protocol for multimedia application text conversation)
• Allows for real-time editing (e.g., backspace, rewriting)
• T.140 messages transported over RTP
• RFC 4103 (RTP Payload for Text Conversation)
• Redundancy implemented via RED (RFC 2198)
Specification to use T.140 over data channels
• T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865)
• https://www.rfc-editor.org/rfc/rfc8865.html
SIP plugin in Janus normally...
SIP plugin in Janus + RTT
https://github.com/meetecho/janus-gateway/pull/3231
Example: SDP offer from SIP/RTT endpoint
v=0
o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74
s=Omnitor_SDP_v1.1
c=IN IP4 192.168.1.74
t=0 0
m=text 1024 RTP/AVP 99 98
a=rtpmap:99 red/1000
a=fmtp:99 98/98/98
a=rtpmap:98 t140/1000
Example: SDP offer to WebRTC endpoint
v=0
o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74
s=Omnitor_SDP_v1.1 t=0 0
a=group:BUNDLE data
a=msid-semantic: WMS janus
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.168.1.74
a=sendrecv
a=sctp-port:5000
a=mid:0
[.. ICE/DTLS details follow ..]
Testing: TIPcon1 (SIP/RTT)
https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
Testing: Janus SIP demo (WebRTC)
https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
A more complex example: MSRP
• Instant Messaging protocol negotiated in SIP/SDP
• RFC 4975 (Message Session Relay Protocol)
• Requires a reliable transport (e.g., TCP)
• Used m=message m-line type for negotiation
• Protocol must be TCP/MSRP or TCP/TLS/MSRP
• path attribute defines endpoints
Specification to use MSRP over data channels
• Message Session Relay Protocol (MSRP) over Data Channels (RFC 8873)
• https://www.rfc-editor.org/rfc/rfc8873.html
A more complex example: MSRP
• Instant Messaging protocol negotiated in SIP/SDP
• RFC 4975 (Message Session Relay Protocol)
• Requires a reliable transport (e.g., TCP)
• Used m=message m-line type for negotiation
• Protocol must be TCP/MSRP or TCP/TLS/MSRP
• path attribute defines endpoints
Specification to use MSRP over data channels
• Message Session Relay Protocol (MSRP) over Data Channels (RFC 8873)
• https://www.rfc-editor.org/rfc/rfc8873.html
A more complex example: MSRP
• Instant Messaging protocol negotiated in SIP/SDP
• RFC 4975 (Message Session Relay Protocol)
• Requires a reliable transport (e.g., TCP)
• Used m=message m-line type for negotiation
• Protocol must be TCP/MSRP or TCP/TLS/MSRP
• path attribute defines endpoints
Specification to use MSRP over data channels
• Message Session Relay Protocol (MSRP) over Data Channels (RFC 8873)
• https://www.rfc-editor.org/rfc/rfc8873.html
Much more complex to handle than RTT
• T.140 still uses RTP (and UDP)
• Easier to integrate in existing Janus SIP plugin bridging
• Complexity mostly in SDP translation (and maybe RED)
• MSRP requires a reliable transport protocol
• With TCP, who connects to who?
• Integrating TCP with UDP poll loop can be tricky (head-of-line blocking)
• On the WebRTC side, we need to be able to provide MSRP path
• It won’t be in the SDP (attribute stripped) and dcsa unsupported by browsers
• Needs out of band (Janus SIP plugin API?) mechanism for that
Much more complex to handle than RTT
• T.140 still uses RTP (and UDP)
• Easier to integrate in existing Janus SIP plugin bridging
• Complexity mostly in SDP translation (and maybe RED)
• MSRP requires a reliable transport protocol
• With TCP, who connects to who?
• Integrating TCP with UDP poll loop can be tricky (head-of-line blocking)
• On the WebRTC side, we need to be able to provide MSRP path
• It won’t be in the SDP (attribute stripped) and dcsa unsupported by browsers
• Needs out of band (Janus SIP plugin API?) mechanism for that
Much more complex to handle than RTT
• T.140 still uses RTP (and UDP)
• Easier to integrate in existing Janus SIP plugin bridging
• Complexity mostly in SDP translation (and maybe RED)
• MSRP requires a reliable transport protocol
• With TCP, who connects to who?
• Integrating TCP with UDP poll loop can be tricky (head-of-line blocking)
• On the WebRTC side, we need to be able to provide MSRP path
• It won’t be in the SDP (attribute stripped) and dcsa unsupported by browsers
• Needs out of band (Janus SIP plugin API?) mechanism for that
SIP plugin in Janus + MSRP
https://github.com/meetecho/janus-gateway/tree/sip-msrp
Example: SDP offer from Blink (SIP/MSRP)
v=0
o=- 3922178296 3922178296 IN IP4 192.168.1.74
s=Blink 5.5.1 (Linux)
t=0 0
m=message 2855 TCP/MSRP *
c=IN IP4 192.168.1.74
a=path:msrp://192.168.1.74:2855/59a83c96684d4fc5fa41;tcp
a=accept-types:message/cpim text/* image/* 
application/im-iscomposing+xml
a=accept-wrapped-types:text/* image/* 
application/im-iscomposing+xml
a=setup:active
Example: SDP offer to WebRTC endpoint
v=0
o=- 3922178296 3922178296 IN IP4 192.168.1.74
s=Blink 5.5.1 (Linux)
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS *
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.168.1.74
a=sendrecv
a=sctp-port:5000
[.. ICE/DTLS details follow ..]
Testing: Janus SIP demo (WebRTC)
What’s next?
• Both RTT and MSRP “work” in Janus, but there’s a lot to do!
• RTT is in a better shape, but needs testing with actual endpoints
• MSRP in a more embrional stage, needs a lot of love
• Multiple protocols in same PeerConnection not supported yet either
• Data channels allow for that (single m-line, multiple labels)
• We need better mapping when dealing with SDP translation
• Maybe add support for other protocols?
• Not sure what’s really used and needed, out there
• Is this effort really worth it, though?
• Probably yes for RTT (Emergency Services mandate it)
• Other protocols may have better “solutions” already
What’s next?
• Both RTT and MSRP “work” in Janus, but there’s a lot to do!
• RTT is in a better shape, but needs testing with actual endpoints
• MSRP in a more embrional stage, needs a lot of love
• Multiple protocols in same PeerConnection not supported yet either
• Data channels allow for that (single m-line, multiple labels)
• We need better mapping when dealing with SDP translation
• Maybe add support for other protocols?
• Not sure what’s really used and needed, out there
• Is this effort really worth it, though?
• Probably yes for RTT (Emergency Services mandate it)
• Other protocols may have better “solutions” already
What’s next?
• Both RTT and MSRP “work” in Janus, but there’s a lot to do!
• RTT is in a better shape, but needs testing with actual endpoints
• MSRP in a more embrional stage, needs a lot of love
• Multiple protocols in same PeerConnection not supported yet either
• Data channels allow for that (single m-line, multiple labels)
• We need better mapping when dealing with SDP translation
• Maybe add support for other protocols?
• Not sure what’s really used and needed, out there
• Is this effort really worth it, though?
• Probably yes for RTT (Emergency Services mandate it)
• Other protocols may have better “solutions” already
What’s next?
• Both RTT and MSRP “work” in Janus, but there’s a lot to do!
• RTT is in a better shape, but needs testing with actual endpoints
• MSRP in a more embrional stage, needs a lot of love
• Multiple protocols in same PeerConnection not supported yet either
• Data channels allow for that (single m-line, multiple labels)
• We need better mapping when dealing with SDP translation
• Maybe add support for other protocols?
• Not sure what’s really used and needed, out there
• Is this effort really worth it, though?
• Probably yes for RTT (Emergency Services mandate it)
• Other protocols may have better “solutions” already
Thanks! Questions? Comments?
Contacts
• https://fosstodon.org/@lminiero
• https://twitter.com/elminiero
• https://twitter.com/meetecho
• https://www.meetecho.com/blog/

More Related Content

Similar to WebRTC and SIP not just audio and video @ OpenSIPS 2024

Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptxBridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Alberto González Trastoy
 
Janus/Asterisk @ Astricon 2017
Janus/Asterisk @ Astricon 2017Janus/Asterisk @ Astricon 2017
Janus/Asterisk @ Astricon 2017
Lorenzo Miniero
 
Fuzzing RTC @ Kamailio World 2019
Fuzzing RTC @ Kamailio World 2019Fuzzing RTC @ Kamailio World 2019
Fuzzing RTC @ Kamailio World 2019
Lorenzo Miniero
 
FOSDEM 2020: How can we make WebRTC Easier?
FOSDEM 2020: How can we make WebRTC Easier?FOSDEM 2020: How can we make WebRTC Easier?
FOSDEM 2020: How can we make WebRTC Easier?
SeanDuBois3
 
Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021
Lorenzo Miniero
 
Upperside Webinar- WebRTC from the service provider prism-final
Upperside Webinar- WebRTC from the service provider prism-finalUpperside Webinar- WebRTC from the service provider prism-final
Upperside Webinar- WebRTC from the service provider prism-final
Amir Zmora
 
Janus RTP forwarders @ FOSDEM 2020
Janus RTP forwarders @ FOSDEM 2020Janus RTP forwarders @ FOSDEM 2020
Janus RTP forwarders @ FOSDEM 2020
Lorenzo Miniero
 
WebRTC security+more @ KamailioWorld 2018
WebRTC security+more @ KamailioWorld 2018WebRTC security+more @ KamailioWorld 2018
WebRTC security+more @ KamailioWorld 2018
Lorenzo Miniero
 
Can SFUs and MCUs be friends @ IIT-RTC 2020
Can SFUs and MCUs be friends @ IIT-RTC 2020Can SFUs and MCUs be friends @ IIT-RTC 2020
Can SFUs and MCUs be friends @ IIT-RTC 2020
Lorenzo Miniero
 
WebRTC standards update - November 2014
WebRTC standards update - November 2014WebRTC standards update - November 2014
WebRTC standards update - November 2014
Victor Pascual Ávila
 
WHIP and Janus @ IIT-RTC 2021
WHIP and Janus @ IIT-RTC 2021WHIP and Janus @ IIT-RTC 2021
WHIP and Janus @ IIT-RTC 2021
Lorenzo Miniero
 
WebRTC. Yet Another Overview, for IT Technicians.
WebRTC. Yet Another Overview, for IT Technicians.WebRTC. Yet Another Overview, for IT Technicians.
WebRTC. Yet Another Overview, for IT Technicians.
Vladimir Beloborodov
 
How to video.
How to video.How to video.
How to video.
Cotap Engineering
 
Workshop oracle
Workshop oracleWorkshop oracle
Workshop oracle
Douglas Tait
 
WebRTC, Whats in it for me?
WebRTC, Whats in it for me?WebRTC, Whats in it for me?
WebRTC, Whats in it for me?
SeanDuBois3
 
BWE in Janus
BWE in JanusBWE in Janus
BWE in Janus
Lorenzo Miniero
 
WebRTC for Telco: Informa's WebRTC Global Summit Preconference
WebRTC for Telco: Informa's WebRTC Global Summit PreconferenceWebRTC for Telco: Informa's WebRTC Global Summit Preconference
WebRTC for Telco: Informa's WebRTC Global Summit Preconference
Tsahi Levent-levi
 
WebRTC Standards from Tim Panton
WebRTC Standards from Tim PantonWebRTC Standards from Tim Panton
WebRTC Standards from Tim Panton
Alan Quayle
 
Multistream in Janus @ CommCon 2019
Multistream in Janus @ CommCon 2019Multistream in Janus @ CommCon 2019
Multistream in Janus @ CommCon 2019
Lorenzo Miniero
 
Real-Time Text and WebRTC @ Kamailio World 2023
Real-Time Text and WebRTC @ Kamailio World 2023Real-Time Text and WebRTC @ Kamailio World 2023
Real-Time Text and WebRTC @ Kamailio World 2023
Lorenzo Miniero
 

Similar to WebRTC and SIP not just audio and video @ OpenSIPS 2024 (20)

Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptxBridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
Bridging_WebRTC_with_SIP_Alberto_WebRTCventures_Cluecon2023_NoVideo.pptx
 
Janus/Asterisk @ Astricon 2017
Janus/Asterisk @ Astricon 2017Janus/Asterisk @ Astricon 2017
Janus/Asterisk @ Astricon 2017
 
Fuzzing RTC @ Kamailio World 2019
Fuzzing RTC @ Kamailio World 2019Fuzzing RTC @ Kamailio World 2019
Fuzzing RTC @ Kamailio World 2019
 
FOSDEM 2020: How can we make WebRTC Easier?
FOSDEM 2020: How can we make WebRTC Easier?FOSDEM 2020: How can we make WebRTC Easier?
FOSDEM 2020: How can we make WebRTC Easier?
 
Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021
 
Upperside Webinar- WebRTC from the service provider prism-final
Upperside Webinar- WebRTC from the service provider prism-finalUpperside Webinar- WebRTC from the service provider prism-final
Upperside Webinar- WebRTC from the service provider prism-final
 
Janus RTP forwarders @ FOSDEM 2020
Janus RTP forwarders @ FOSDEM 2020Janus RTP forwarders @ FOSDEM 2020
Janus RTP forwarders @ FOSDEM 2020
 
WebRTC security+more @ KamailioWorld 2018
WebRTC security+more @ KamailioWorld 2018WebRTC security+more @ KamailioWorld 2018
WebRTC security+more @ KamailioWorld 2018
 
Can SFUs and MCUs be friends @ IIT-RTC 2020
Can SFUs and MCUs be friends @ IIT-RTC 2020Can SFUs and MCUs be friends @ IIT-RTC 2020
Can SFUs and MCUs be friends @ IIT-RTC 2020
 
WebRTC standards update - November 2014
WebRTC standards update - November 2014WebRTC standards update - November 2014
WebRTC standards update - November 2014
 
WHIP and Janus @ IIT-RTC 2021
WHIP and Janus @ IIT-RTC 2021WHIP and Janus @ IIT-RTC 2021
WHIP and Janus @ IIT-RTC 2021
 
WebRTC. Yet Another Overview, for IT Technicians.
WebRTC. Yet Another Overview, for IT Technicians.WebRTC. Yet Another Overview, for IT Technicians.
WebRTC. Yet Another Overview, for IT Technicians.
 
How to video.
How to video.How to video.
How to video.
 
Workshop oracle
Workshop oracleWorkshop oracle
Workshop oracle
 
WebRTC, Whats in it for me?
WebRTC, Whats in it for me?WebRTC, Whats in it for me?
WebRTC, Whats in it for me?
 
BWE in Janus
BWE in JanusBWE in Janus
BWE in Janus
 
WebRTC for Telco: Informa's WebRTC Global Summit Preconference
WebRTC for Telco: Informa's WebRTC Global Summit PreconferenceWebRTC for Telco: Informa's WebRTC Global Summit Preconference
WebRTC for Telco: Informa's WebRTC Global Summit Preconference
 
WebRTC Standards from Tim Panton
WebRTC Standards from Tim PantonWebRTC Standards from Tim Panton
WebRTC Standards from Tim Panton
 
Multistream in Janus @ CommCon 2019
Multistream in Janus @ CommCon 2019Multistream in Janus @ CommCon 2019
Multistream in Janus @ CommCon 2019
 
Real-Time Text and WebRTC @ Kamailio World 2023
Real-Time Text and WebRTC @ Kamailio World 2023Real-Time Text and WebRTC @ Kamailio World 2023
Real-Time Text and WebRTC @ Kamailio World 2023
 

More from Lorenzo Miniero

SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
Lorenzo Miniero
 
Getting AV1/SVC to work in the Janus WebRTC Server
Getting AV1/SVC to work in the Janus WebRTC ServerGetting AV1/SVC to work in the Janus WebRTC Server
Getting AV1/SVC to work in the Janus WebRTC Server
Lorenzo Miniero
 
The challenges of hybrid meetings @ CommCon 2023
The challenges of hybrid meetings @ CommCon 2023The challenges of hybrid meetings @ CommCon 2023
The challenges of hybrid meetings @ CommCon 2023
Lorenzo Miniero
 
Become a rockstar using FOSS!
Become a rockstar using FOSS!Become a rockstar using FOSS!
Become a rockstar using FOSS!
Lorenzo Miniero
 
Janus SFU cascading @ IIT-RTC 2022
Janus SFU cascading @ IIT-RTC 2022Janus SFU cascading @ IIT-RTC 2022
Janus SFU cascading @ IIT-RTC 2022
Lorenzo Miniero
 
SIP transfer with Janus/WebRTC @ OpenSIPS 2022
SIP transfer with Janus/WebRTC @ OpenSIPS 2022SIP transfer with Janus/WebRTC @ OpenSIPS 2022
SIP transfer with Janus/WebRTC @ OpenSIPS 2022
Lorenzo Miniero
 
WebRTC, RED and Janus @ ClueCon21
WebRTC, RED and Janus @ ClueCon21WebRTC, RED and Janus @ ClueCon21
WebRTC, RED and Janus @ ClueCon21
Lorenzo Miniero
 
JamRTC @ Wonder WebRTC unConference
JamRTC @ Wonder WebRTC unConferenceJamRTC @ Wonder WebRTC unConference
JamRTC @ Wonder WebRTC unConference
Lorenzo Miniero
 
Scaling WebRTC deployments with multicast @ IETF 110 MBONED
Scaling WebRTC deployments with multicast @ IETF 110 MBONEDScaling WebRTC deployments with multicast @ IETF 110 MBONED
Scaling WebRTC deployments with multicast @ IETF 110 MBONED
Lorenzo Miniero
 
Janus + NDI @ ClueCon 2021
Janus + NDI @ ClueCon 2021Janus + NDI @ ClueCon 2021
Janus + NDI @ ClueCon 2021
Lorenzo Miniero
 
Can WebRTC help musicians? @ FOSDEM 2021
Can WebRTC help musicians? @ FOSDEM 2021Can WebRTC help musicians? @ FOSDEM 2021
Can WebRTC help musicians? @ FOSDEM 2021
Lorenzo Miniero
 
Virtual IETF meetings with WebRTC @ IETF 109 MOPS
Virtual IETF meetings with WebRTC @ IETF 109 MOPSVirtual IETF meetings with WebRTC @ IETF 109 MOPS
Virtual IETF meetings with WebRTC @ IETF 109 MOPS
Lorenzo Miniero
 
Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020
Lorenzo Miniero
 
Turning live events to virtual with Janus
Turning live events to virtual with JanusTurning live events to virtual with Janus
Turning live events to virtual with Janus
Lorenzo Miniero
 
Janus workshop @ RTC2019 Beijing
Janus workshop @ RTC2019 BeijingJanus workshop @ RTC2019 Beijing
Janus workshop @ RTC2019 Beijing
Lorenzo Miniero
 
Simulcast/SVC @ IIT-RTC 2019
Simulcast/SVC @ IIT-RTC 2019Simulcast/SVC @ IIT-RTC 2019
Simulcast/SVC @ IIT-RTC 2019
Lorenzo Miniero
 
Welcome to JanusCon! -- Past, Present and Future of Janus
Welcome to JanusCon! -- Past, Present and Future of JanusWelcome to JanusCon! -- Past, Present and Future of Janus
Welcome to JanusCon! -- Past, Present and Future of Janus
Lorenzo Miniero
 
Janus @ ClueCon 2019
Janus @ ClueCon 2019Janus @ ClueCon 2019
Janus @ ClueCon 2019
Lorenzo Miniero
 

More from Lorenzo Miniero (18)

SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Getting AV1/SVC to work in the Janus WebRTC Server
Getting AV1/SVC to work in the Janus WebRTC ServerGetting AV1/SVC to work in the Janus WebRTC Server
Getting AV1/SVC to work in the Janus WebRTC Server
 
The challenges of hybrid meetings @ CommCon 2023
The challenges of hybrid meetings @ CommCon 2023The challenges of hybrid meetings @ CommCon 2023
The challenges of hybrid meetings @ CommCon 2023
 
Become a rockstar using FOSS!
Become a rockstar using FOSS!Become a rockstar using FOSS!
Become a rockstar using FOSS!
 
Janus SFU cascading @ IIT-RTC 2022
Janus SFU cascading @ IIT-RTC 2022Janus SFU cascading @ IIT-RTC 2022
Janus SFU cascading @ IIT-RTC 2022
 
SIP transfer with Janus/WebRTC @ OpenSIPS 2022
SIP transfer with Janus/WebRTC @ OpenSIPS 2022SIP transfer with Janus/WebRTC @ OpenSIPS 2022
SIP transfer with Janus/WebRTC @ OpenSIPS 2022
 
WebRTC, RED and Janus @ ClueCon21
WebRTC, RED and Janus @ ClueCon21WebRTC, RED and Janus @ ClueCon21
WebRTC, RED and Janus @ ClueCon21
 
JamRTC @ Wonder WebRTC unConference
JamRTC @ Wonder WebRTC unConferenceJamRTC @ Wonder WebRTC unConference
JamRTC @ Wonder WebRTC unConference
 
Scaling WebRTC deployments with multicast @ IETF 110 MBONED
Scaling WebRTC deployments with multicast @ IETF 110 MBONEDScaling WebRTC deployments with multicast @ IETF 110 MBONED
Scaling WebRTC deployments with multicast @ IETF 110 MBONED
 
Janus + NDI @ ClueCon 2021
Janus + NDI @ ClueCon 2021Janus + NDI @ ClueCon 2021
Janus + NDI @ ClueCon 2021
 
Can WebRTC help musicians? @ FOSDEM 2021
Can WebRTC help musicians? @ FOSDEM 2021Can WebRTC help musicians? @ FOSDEM 2021
Can WebRTC help musicians? @ FOSDEM 2021
 
Virtual IETF meetings with WebRTC @ IETF 109 MOPS
Virtual IETF meetings with WebRTC @ IETF 109 MOPSVirtual IETF meetings with WebRTC @ IETF 109 MOPS
Virtual IETF meetings with WebRTC @ IETF 109 MOPS
 
Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020Janus Workshop @ ClueCon 2020
Janus Workshop @ ClueCon 2020
 
Turning live events to virtual with Janus
Turning live events to virtual with JanusTurning live events to virtual with Janus
Turning live events to virtual with Janus
 
Janus workshop @ RTC2019 Beijing
Janus workshop @ RTC2019 BeijingJanus workshop @ RTC2019 Beijing
Janus workshop @ RTC2019 Beijing
 
Simulcast/SVC @ IIT-RTC 2019
Simulcast/SVC @ IIT-RTC 2019Simulcast/SVC @ IIT-RTC 2019
Simulcast/SVC @ IIT-RTC 2019
 
Welcome to JanusCon! -- Past, Present and Future of Janus
Welcome to JanusCon! -- Past, Present and Future of JanusWelcome to JanusCon! -- Past, Present and Future of Janus
Welcome to JanusCon! -- Past, Present and Future of Janus
 
Janus @ ClueCon 2019
Janus @ ClueCon 2019Janus @ ClueCon 2019
Janus @ ClueCon 2019
 

Recently uploaded

How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 

Recently uploaded (20)

How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 

WebRTC and SIP not just audio and video @ OpenSIPS 2024

  • 1. WebRTC-to-SIP and back It’s not all about audio and video! Lorenzo Miniero @lminiero@fosstodon.org OpenSIPS Summit May 15th 2024, Valencia
  • 2. A chance to practice my broken Spanish!
  • 3. Who am I? Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Main author of Janus Contacts and info • lorenzo@meetecho.com • https://fosstodon.org/@lminiero • https://www.meetecho.com • https://lminiero.it
  • 4. Just a few words on Meetecho • Co-founded in 2009 as an academic spin-off • University research efforts brought to the market • Completely independent from the University • Focus on real-time multimedia applications • Strong perspective on standardization and open source • Several activities • Consulting services • Commercial support and Janus licenses • Streaming of live events (IETF, ACM, etc.) • Proudly brewed in sunny Napoli, Italy
  • 5. A bit of context: Janus, WebRTC and SIP Janus General purpose, open source WebRTC server • https://github.com/meetecho/janus-gateway • Demos and documentation: https://janus.conf.meetecho.com • Community: https://janus.discourse.group/
  • 6. SIP plugin in Janus https://janus.conf.meetecho.com/docs/sip
  • 7. An endpoint of behalf of WebRTC users • Janus SIP plugin acts as a collection of SIP endpoints, not a server/trunk • SIP stack implemented with Sofia-SIP • WebRTC users only see the Janus API (JSON), no SIP • No transcoding, media is only relayed • Built-in recording (separate media legs) • Simplifies life for web developers • No need to worry about a SIP stack (only SIP URIs) • Basic methods/events to handle dialogs (call, answer, hangup, message, etc.) • Allows SIP headers injection/interception in many requests • Support for more advanced features too (e.g., hold, transfer, etc.)
  • 8. An endpoint of behalf of WebRTC users • Janus SIP plugin acts as a collection of SIP endpoints, not a server/trunk • SIP stack implemented with Sofia-SIP • WebRTC users only see the Janus API (JSON), no SIP • No transcoding, media is only relayed • Built-in recording (separate media legs) • Simplifies life for web developers • No need to worry about a SIP stack (only SIP URIs) • Basic methods/events to handle dialogs (call, answer, hangup, message, etc.) • Allows SIP headers injection/interception in many requests • Support for more advanced features too (e.g., hold, transfer, etc.)
  • 9. Works great with OpenSIPS! Workshop on Janus and SIP (lesson/tutorial) at OpenSIPS 2020 https://www.youtube.com/watch?v=fv9KwrguR-4&t=3544s
  • 10. Audio and video are “easy” • Both SIP and WebRTC use SDP and RTP/RTCP • WebRTC uses SDP/RTP/RTCP on “steroids” • Apart from this, just differences in encryption (WebRTC mandates DTLS-SRTP) • Media is basically encoded, packaged and sent the same way • As long as the same codec is used, they’re interoperable • When they aren’t, transcoding helps (but Janus won’t do it for you) • WebRTC has mandatory-to-implement codecs • Opus and G.711 for audio, VP8 and H.264 (baseline) for video • G.711 and H.264 should guarantee compliance with legacy equipment
  • 11. Audio and video are “easy” • Both SIP and WebRTC use SDP and RTP/RTCP • WebRTC uses SDP/RTP/RTCP on “steroids” • Apart from this, just differences in encryption (WebRTC mandates DTLS-SRTP) • Media is basically encoded, packaged and sent the same way • As long as the same codec is used, they’re interoperable • When they aren’t, transcoding helps (but Janus won’t do it for you) • WebRTC has mandatory-to-implement codecs • Opus and G.711 for audio, VP8 and H.264 (baseline) for video • G.711 and H.264 should guarantee compliance with legacy equipment
  • 12. Audio and video are “easy” • Both SIP and WebRTC use SDP and RTP/RTCP • WebRTC uses SDP/RTP/RTCP on “steroids” • Apart from this, just differences in encryption (WebRTC mandates DTLS-SRTP) • Media is basically encoded, packaged and sent the same way • As long as the same codec is used, they’re interoperable • When they aren’t, transcoding helps (but Janus won’t do it for you) • WebRTC has mandatory-to-implement codecs • Opus and G.711 for audio, VP8 and H.264 (baseline) for video • G.711 and H.264 should guarantee compliance with legacy equipment
  • 13. SIP sometimes does more than that, though! • A whole ecosystem of other protocols that could be used • Real-Time Text (T.140 over RTP) • Message Session Relay Protocol (MSRP) • Binary Floor Control Protocol (BFCP) • Fax (T.38 over RTP) • ... • These protocols can’t simply be gateway-ed to WebRTC • WebRTC supports RTP, but only for audio/video, not generic data • Custom protocols are not supported at all • WebRTC-to-SIP gateways will in general strip them from the SDP • We can’t rely on a WebRTC browser to simply reject unsupported media • An unsupported m-line will cause an exception in setRemoteDescription
  • 14. SIP sometimes does more than that, though! • A whole ecosystem of other protocols that could be used • Real-Time Text (T.140 over RTP) • Message Session Relay Protocol (MSRP) • Binary Floor Control Protocol (BFCP) • Fax (T.38 over RTP) • ... • These protocols can’t simply be gateway-ed to WebRTC • WebRTC supports RTP, but only for audio/video, not generic data • Custom protocols are not supported at all • WebRTC-to-SIP gateways will in general strip them from the SDP • We can’t rely on a WebRTC browser to simply reject unsupported media • An unsupported m-line will cause an exception in setRemoteDescription
  • 15. SIP sometimes does more than that, though! • A whole ecosystem of other protocols that could be used • Real-Time Text (T.140 over RTP) • Message Session Relay Protocol (MSRP) • Binary Floor Control Protocol (BFCP) • Fax (T.38 over RTP) • ... • These protocols can’t simply be gateway-ed to WebRTC • WebRTC supports RTP, but only for audio/video, not generic data • Custom protocols are not supported at all • WebRTC-to-SIP gateways will in general strip them from the SDP • We can’t rely on a WebRTC browser to simply reject unsupported media • An unsupported m-line will cause an exception in setRemoteDescription
  • 16. Data channels to the rescue!
  • 17. Data channels to the rescue!
  • 18. What are data channels? • Arbitrary real-time connection for pretty much anything • Bidirectional data between two WebRTC peers • Support for multiple channels of different kinds • Supports ordered/unordered and reliable/unreliable • Generic data sent via SCTP and encapsulated in DTLS • SCTP implements features, DTLS implements security • Negotiated as an application in the SDP • m=application 9 UDP/DTLS/SCTP webrtc-datachannel
  • 19. What are data channels? • Arbitrary real-time connection for pretty much anything • Bidirectional data between two WebRTC peers • Support for multiple channels of different kinds • Supports ordered/unordered and reliable/unreliable • Generic data sent via SCTP and encapsulated in DTLS • SCTP implements features, DTLS implements security • Negotiated as an application in the SDP • m=application 9 UDP/DTLS/SCTP webrtc-datachannel
  • 20. What are data channels? • Arbitrary real-time connection for pretty much anything • Bidirectional data between two WebRTC peers • Support for multiple channels of different kinds • Supports ordered/unordered and reliable/unreliable • Generic data sent via SCTP and encapsulated in DTLS • SCTP implements features, DTLS implements security • Negotiated as an application in the SDP • m=application 9 UDP/DTLS/SCTP webrtc-datachannel
  • 21. That’s better, but still not enough! • Non audio/video protocols have their own technical requirements • RTT and T.38 will have custom RTP packetization rules • BFCP/MSRP/others will require custom UDP or TCP transport • All this will be reflected in the SDP negotiation • WebRTC only supports m=application + UDP/DTLS/SCTP • It’s the only non audio/video thing it can negotiate • A gateway will have more work to do than with RTP audio/video • Translate between SDP formats (e.g., custom protocol ↔ application) • Gateway the protocol itself (e.g., RTP/T.140 ↔ data channel)
  • 22. That’s better, but still not enough! • Non audio/video protocols have their own technical requirements • RTT and T.38 will have custom RTP packetization rules • BFCP/MSRP/others will require custom UDP or TCP transport • All this will be reflected in the SDP negotiation • WebRTC only supports m=application + UDP/DTLS/SCTP • It’s the only non audio/video thing it can negotiate • A gateway will have more work to do than with RTP audio/video • Translate between SDP formats (e.g., custom protocol ↔ application) • Gateway the protocol itself (e.g., RTP/T.140 ↔ data channel)
  • 23. That’s better, but still not enough! • Non audio/video protocols have their own technical requirements • RTT and T.38 will have custom RTP packetization rules • BFCP/MSRP/others will require custom UDP or TCP transport • All this will be reflected in the SDP negotiation • WebRTC only supports m=application + UDP/DTLS/SCTP • It’s the only non audio/video thing it can negotiate • A gateway will have more work to do than with RTP audio/video • Translate between SDP formats (e.g., custom protocol ↔ application) • Gateway the protocol itself (e.g., RTP/T.140 ↔ data channel)
  • 24. A practical example: Real-Time Text (T.140) • Text transmitted instantly, as it is typed or created • ITU-T T.140 (Protocol for multimedia application text conversation) • Allows for real-time editing (e.g., backspace, rewriting) • T.140 messages transported over RTP • RFC 4103 (RTP Payload for Text Conversation) • Redundancy implemented via RED (RFC 2198) Specification to use T.140 over data channels • T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865) • https://www.rfc-editor.org/rfc/rfc8865.html
  • 25. A practical example: Real-Time Text (T.140) • Text transmitted instantly, as it is typed or created • ITU-T T.140 (Protocol for multimedia application text conversation) • Allows for real-time editing (e.g., backspace, rewriting) • T.140 messages transported over RTP • RFC 4103 (RTP Payload for Text Conversation) • Redundancy implemented via RED (RFC 2198) Specification to use T.140 over data channels • T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865) • https://www.rfc-editor.org/rfc/rfc8865.html
  • 26. A practical example: Real-Time Text (T.140) • Text transmitted instantly, as it is typed or created • ITU-T T.140 (Protocol for multimedia application text conversation) • Allows for real-time editing (e.g., backspace, rewriting) • T.140 messages transported over RTP • RFC 4103 (RTP Payload for Text Conversation) • Redundancy implemented via RED (RFC 2198) Specification to use T.140 over data channels • T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865) • https://www.rfc-editor.org/rfc/rfc8865.html
  • 27. SIP plugin in Janus normally...
  • 28. SIP plugin in Janus + RTT https://github.com/meetecho/janus-gateway/pull/3231
  • 29. Example: SDP offer from SIP/RTT endpoint v=0 o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74 s=Omnitor_SDP_v1.1 c=IN IP4 192.168.1.74 t=0 0 m=text 1024 RTP/AVP 99 98 a=rtpmap:99 red/1000 a=fmtp:99 98/98/98 a=rtpmap:98 t140/1000
  • 30. Example: SDP offer to WebRTC endpoint v=0 o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74 s=Omnitor_SDP_v1.1 t=0 0 a=group:BUNDLE data a=msid-semantic: WMS janus m=application 9 UDP/DTLS/SCTP webrtc-datachannel c=IN IP4 192.168.1.74 a=sendrecv a=sctp-port:5000 a=mid:0 [.. ICE/DTLS details follow ..]
  • 32. Testing: Janus SIP demo (WebRTC) https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
  • 33. A more complex example: MSRP • Instant Messaging protocol negotiated in SIP/SDP • RFC 4975 (Message Session Relay Protocol) • Requires a reliable transport (e.g., TCP) • Used m=message m-line type for negotiation • Protocol must be TCP/MSRP or TCP/TLS/MSRP • path attribute defines endpoints Specification to use MSRP over data channels • Message Session Relay Protocol (MSRP) over Data Channels (RFC 8873) • https://www.rfc-editor.org/rfc/rfc8873.html
  • 34. A more complex example: MSRP • Instant Messaging protocol negotiated in SIP/SDP • RFC 4975 (Message Session Relay Protocol) • Requires a reliable transport (e.g., TCP) • Used m=message m-line type for negotiation • Protocol must be TCP/MSRP or TCP/TLS/MSRP • path attribute defines endpoints Specification to use MSRP over data channels • Message Session Relay Protocol (MSRP) over Data Channels (RFC 8873) • https://www.rfc-editor.org/rfc/rfc8873.html
  • 35. A more complex example: MSRP • Instant Messaging protocol negotiated in SIP/SDP • RFC 4975 (Message Session Relay Protocol) • Requires a reliable transport (e.g., TCP) • Used m=message m-line type for negotiation • Protocol must be TCP/MSRP or TCP/TLS/MSRP • path attribute defines endpoints Specification to use MSRP over data channels • Message Session Relay Protocol (MSRP) over Data Channels (RFC 8873) • https://www.rfc-editor.org/rfc/rfc8873.html
  • 36. Much more complex to handle than RTT • T.140 still uses RTP (and UDP) • Easier to integrate in existing Janus SIP plugin bridging • Complexity mostly in SDP translation (and maybe RED) • MSRP requires a reliable transport protocol • With TCP, who connects to who? • Integrating TCP with UDP poll loop can be tricky (head-of-line blocking) • On the WebRTC side, we need to be able to provide MSRP path • It won’t be in the SDP (attribute stripped) and dcsa unsupported by browsers • Needs out of band (Janus SIP plugin API?) mechanism for that
  • 37. Much more complex to handle than RTT • T.140 still uses RTP (and UDP) • Easier to integrate in existing Janus SIP plugin bridging • Complexity mostly in SDP translation (and maybe RED) • MSRP requires a reliable transport protocol • With TCP, who connects to who? • Integrating TCP with UDP poll loop can be tricky (head-of-line blocking) • On the WebRTC side, we need to be able to provide MSRP path • It won’t be in the SDP (attribute stripped) and dcsa unsupported by browsers • Needs out of band (Janus SIP plugin API?) mechanism for that
  • 38. Much more complex to handle than RTT • T.140 still uses RTP (and UDP) • Easier to integrate in existing Janus SIP plugin bridging • Complexity mostly in SDP translation (and maybe RED) • MSRP requires a reliable transport protocol • With TCP, who connects to who? • Integrating TCP with UDP poll loop can be tricky (head-of-line blocking) • On the WebRTC side, we need to be able to provide MSRP path • It won’t be in the SDP (attribute stripped) and dcsa unsupported by browsers • Needs out of band (Janus SIP plugin API?) mechanism for that
  • 39. SIP plugin in Janus + MSRP https://github.com/meetecho/janus-gateway/tree/sip-msrp
  • 40. Example: SDP offer from Blink (SIP/MSRP) v=0 o=- 3922178296 3922178296 IN IP4 192.168.1.74 s=Blink 5.5.1 (Linux) t=0 0 m=message 2855 TCP/MSRP * c=IN IP4 192.168.1.74 a=path:msrp://192.168.1.74:2855/59a83c96684d4fc5fa41;tcp a=accept-types:message/cpim text/* image/* application/im-iscomposing+xml a=accept-wrapped-types:text/* image/* application/im-iscomposing+xml a=setup:active
  • 41. Example: SDP offer to WebRTC endpoint v=0 o=- 3922178296 3922178296 IN IP4 192.168.1.74 s=Blink 5.5.1 (Linux) t=0 0 a=group:BUNDLE 0 a=extmap-allow-mixed a=msid-semantic: WMS * m=application 9 UDP/DTLS/SCTP webrtc-datachannel c=IN IP4 192.168.1.74 a=sendrecv a=sctp-port:5000 [.. ICE/DTLS details follow ..]
  • 42. Testing: Janus SIP demo (WebRTC)
  • 43. What’s next? • Both RTT and MSRP “work” in Janus, but there’s a lot to do! • RTT is in a better shape, but needs testing with actual endpoints • MSRP in a more embrional stage, needs a lot of love • Multiple protocols in same PeerConnection not supported yet either • Data channels allow for that (single m-line, multiple labels) • We need better mapping when dealing with SDP translation • Maybe add support for other protocols? • Not sure what’s really used and needed, out there • Is this effort really worth it, though? • Probably yes for RTT (Emergency Services mandate it) • Other protocols may have better “solutions” already
  • 44. What’s next? • Both RTT and MSRP “work” in Janus, but there’s a lot to do! • RTT is in a better shape, but needs testing with actual endpoints • MSRP in a more embrional stage, needs a lot of love • Multiple protocols in same PeerConnection not supported yet either • Data channels allow for that (single m-line, multiple labels) • We need better mapping when dealing with SDP translation • Maybe add support for other protocols? • Not sure what’s really used and needed, out there • Is this effort really worth it, though? • Probably yes for RTT (Emergency Services mandate it) • Other protocols may have better “solutions” already
  • 45. What’s next? • Both RTT and MSRP “work” in Janus, but there’s a lot to do! • RTT is in a better shape, but needs testing with actual endpoints • MSRP in a more embrional stage, needs a lot of love • Multiple protocols in same PeerConnection not supported yet either • Data channels allow for that (single m-line, multiple labels) • We need better mapping when dealing with SDP translation • Maybe add support for other protocols? • Not sure what’s really used and needed, out there • Is this effort really worth it, though? • Probably yes for RTT (Emergency Services mandate it) • Other protocols may have better “solutions” already
  • 46. What’s next? • Both RTT and MSRP “work” in Janus, but there’s a lot to do! • RTT is in a better shape, but needs testing with actual endpoints • MSRP in a more embrional stage, needs a lot of love • Multiple protocols in same PeerConnection not supported yet either • Data channels allow for that (single m-line, multiple labels) • We need better mapping when dealing with SDP translation • Maybe add support for other protocols? • Not sure what’s really used and needed, out there • Is this effort really worth it, though? • Probably yes for RTT (Emergency Services mandate it) • Other protocols may have better “solutions” already
  • 47. Thanks! Questions? Comments? Contacts • https://fosstodon.org/@lminiero • https://twitter.com/elminiero • https://twitter.com/meetecho • https://www.meetecho.com/blog/