Greg Kawere, profile picture
Uploaded byGreg Kawere
122 views

Scaling server side web rtc applications the janus challenge by lorenzo miniero

This document discusses strategies for scaling server-side WebRTC applications using the Janus gateway. It begins with an overview of Janus' modular architecture and plugins. It then examines approaches to scaling beyond a single Janus instance, including plain load balancing, media resource brokering, and sharing the load across multiple Janus servers to serve the same context. Load balancing works well for independent streams but has limitations for plugins that require related streams to be on the same server. Media resource brokering allocates contexts to servers to improve scalability but may not help for very large contexts.

Embed presentation

Download to read offline
Scaling server-side WebRTC applications: the Janus challenge Lorenzo Miniero @elminiero CommCon 2018 June 27th 2018, Wotton House, Surrey (UK)
Sorry Dan, pineapples on pizza = BIG NO
A picture of me as a kid for no reason Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Open source believer Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero
1. No, I don’t know what happened...
2. Yes, Police is still looking for who stole my cheeks
Ok, let’s stop clowning around... Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Open source believer Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero
What is Janus? Janus General purpose, open source WebRTC server • https://github.com/meetecho/janus-gateway • Demos and documentation: https://janus.conf.meetecho.com • Community: https://groups.google.com/forum/#!forum/meetecho-janus
Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
Extensible Architecture and API
Extensible Architecture and API
A quick look at plugins: SIP gateway
A quick look at plugins: RTP rebroadcaster
A quick look at plugins: Audio MCU
A quick look at plugins: Video SFU
The first step towards scaling: benchmarking • Measuring Janus performance is not trivial... • Different plugins have different behaviours • Different technologies have different impact • Need to focus on individual features • SIP Gateway (SIP plugin) • Multimedia streaming (Streaming plugin) • Audio mixing (AudioBridge plugin) • Video routing (VideoRoom plugin) • ... What about the results?1 • When just relaying, bandwidth is typically the real bottleneck • Linear vs. exponential resource increase depends on plugin and topologies 1 https://janus.conf.meetecho.com/citeus
The first step towards scaling: benchmarking • Measuring Janus performance is not trivial... • Different plugins have different behaviours • Different technologies have different impact • Need to focus on individual features • SIP Gateway (SIP plugin) • Multimedia streaming (Streaming plugin) • Audio mixing (AudioBridge plugin) • Video routing (VideoRoom plugin) • ... What about the results?1 • When just relaying, bandwidth is typically the real bottleneck • Linear vs. exponential resource increase depends on plugin and topologies 1 https://janus.conf.meetecho.com/citeus
The first step towards scaling: benchmarking • Measuring Janus performance is not trivial... • Different plugins have different behaviours • Different technologies have different impact • Need to focus on individual features • SIP Gateway (SIP plugin) • Multimedia streaming (Streaming plugin) • Audio mixing (AudioBridge plugin) • Video routing (VideoRoom plugin) • ... What about the results?1 • When just relaying, bandwidth is typically the real bottleneck • Linear vs. exponential resource increase depends on plugin and topologies 1 https://janus.conf.meetecho.com/citeus
Beyond a single Janus instance: different strategies • Once performance is evaluated, other aspects come into play • Does a user stream need to stay within the plugin? • Can it be sent to/received from somewhere else? • Will this affect other connections? • How much is the context important? • What level of orchestration is needed? • All those questions can lead to different answers/solutions, e.g. 1 Plain load balancing 2 Media resource brokering (MRB) 3 Sharing the “burden” 4 Cascaded delivery 5 ...
Beyond a single Janus instance: different strategies • Once performance is evaluated, other aspects come into play • Does a user stream need to stay within the plugin? • Can it be sent to/received from somewhere else? • Will this affect other connections? • How much is the context important? • What level of orchestration is needed? • All those questions can lead to different answers/solutions, e.g. 1 Plain load balancing 2 Media resource brokering (MRB) 3 Sharing the “burden” 4 Cascaded delivery 5 ...
Plain load balancing • The simplest strategy of them all • You have N Janus instances available • You allocate users on one according to some logic • No orchestration needed, just knowledge of Janus addresses • User has list vs. HAProxy vs. nginx vs. ... • Logic can differ depending on the application • Pick a random server? • Use round robin? • Pick the one closest to the user? • Pick the least loaded? • Something else and/or any combination of the above?
Plain load balancing • The simplest strategy of them all • You have N Janus instances available • You allocate users on one according to some logic • No orchestration needed, just knowledge of Janus addresses • User has list vs. HAProxy vs. nginx vs. ... • Logic can differ depending on the application • Pick a random server? • Use round robin? • Pick the one closest to the user? • Pick the least loaded? • Something else and/or any combination of the above?
Plain load balancing • The simplest strategy of them all • You have N Janus instances available • You allocate users on one according to some logic • No orchestration needed, just knowledge of Janus addresses • User has list vs. HAProxy vs. nginx vs. ... • Logic can differ depending on the application • Pick a random server? • Use round robin? • Pick the one closest to the user? • Pick the least loaded? • Something else and/or any combination of the above?
Plain load balancing examples
Plain load balancing examples
Plain load balancing examples
Plain load balancing limitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
Plain load balancing limitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
Plain load balancing limitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
Plain load balancing limitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
Plain load balancing limitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
Media resource brokering • A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
Media resource brokering • A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
Media resource brokering • A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
Media resource brokering • A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
Media resource brokering use case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
Media resource brokering use case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
Media resource brokering use case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
Media resource brokering use case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
Media resource brokering examples
Media resource brokering examples
Media resource brokering examples
Media resource brokering examples
Sharing the “burden” • A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
Sharing the “burden” • A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
Sharing the “burden” • A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
Sharing the “burden” • A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
Sharing the “burden” use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
Sharing the “burden” use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
Sharing the “burden” use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
Sharing the “burden” use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
Sharing the “burden” example
Sharing the “burden” example
Sharing the “burden” example
Sharing the “burden” example
Sharing the “burden” example
RTP forwarding • All strategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
RTP forwarding • All strategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
RTP forwarding • All strategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
RTP forwarding • All strategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
RTP forwarding example
RTP forwarding example
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
While “plain” RTP, and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
A specific scenario: broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
A specific scenario: broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
A specific scenario: broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
A specific scenario: broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
SOLEIL architecture: a simple overview
SOLEIL architecture: a simple overview
Another scenario: cascaded conferencing • Multiple VideoRoom instances “feeding” each other • Publisher on Janus X → relayed on other Janus instances too • Users can subscribe even when not on Janus X • Can be used for scaling purposes • Potential audience of publisher goes beyond capabilities of single Janus instance • Broadcasting seen before applied to conferencing, basically • Very interesting for geographic access to rooms too, though • Users always contact Janus closest to them • Behind the scenes, the Janus instances exchange media for all publishers • Typically, intra- and inter-datacenter network speed > Users network speed
Another scenario: cascaded conferencing • Multiple VideoRoom instances “feeding” each other • Publisher on Janus X → relayed on other Janus instances too • Users can subscribe even when not on Janus X • Can be used for scaling purposes • Potential audience of publisher goes beyond capabilities of single Janus instance • Broadcasting seen before applied to conferencing, basically • Very interesting for geographic access to rooms too, though • Users always contact Janus closest to them • Behind the scenes, the Janus instances exchange media for all publishers • Typically, intra- and inter-datacenter network speed > Users network speed
Another scenario: cascaded conferencing • Multiple VideoRoom instances “feeding” each other • Publisher on Janus X → relayed on other Janus instances too • Users can subscribe even when not on Janus X • Can be used for scaling purposes • Potential audience of publisher goes beyond capabilities of single Janus instance • Broadcasting seen before applied to conferencing, basically • Very interesting for geographic access to rooms too, though • Users always contact Janus closest to them • Behind the scenes, the Janus instances exchange media for all publishers • Typically, intra- and inter-datacenter network speed > Users network speed
Another scenario: cascaded conferencing
Another scenario: cascaded conferencing
A few words on nuts and bolts • We use Docker A LOT in our deployments • Even though Janus can deal with NATs, we do prefer it to have a public IP • Host networking makes it tricky to place more instances on the same machine • Pipework: assign a dedicated IP address to a container https://github.com/jpetazzo/pipework • Docker native macvlan network driver also allows you to assign custom IPs to containers. We’ll move to it sooner or later... • Several people (including us) using Janus in clustered environments (Docker Swarm, Kubernetes, ...) • Auto-scaling by spawning new replicas as needed • Live monitoring of machines’ resources (e.g., CPU, # connections, bandwidth) drives the deployment process
A few words on nuts and bolts • We use Docker A LOT in our deployments • Even though Janus can deal with NATs, we do prefer it to have a public IP • Host networking makes it tricky to place more instances on the same machine • Pipework: assign a dedicated IP address to a container https://github.com/jpetazzo/pipework • Docker native macvlan network driver also allows you to assign custom IPs to containers. We’ll move to it sooner or later... • Several people (including us) using Janus in clustered environments (Docker Swarm, Kubernetes, ...) • Auto-scaling by spawning new replicas as needed • Live monitoring of machines’ resources (e.g., CPU, # connections, bandwidth) drives the deployment process
A few words on nuts and bolts • We use Docker A LOT in our deployments • Even though Janus can deal with NATs, we do prefer it to have a public IP • Host networking makes it tricky to place more instances on the same machine • Pipework: assign a dedicated IP address to a container https://github.com/jpetazzo/pipework • Docker native macvlan network driver also allows you to assign custom IPs to containers. We’ll move to it sooner or later... • Several people (including us) using Janus in clustered environments (Docker Swarm, Kubernetes, ...) • Auto-scaling by spawning new replicas as needed • Live monitoring of machines’ resources (e.g., CPU, # connections, bandwidth) drives the deployment process
A real-world example
A real-world example
Thanks! Questions? Comments? Contacts • https://twitter.com/elminiero • https://twitter.com/meetecho • http://www.meetecho.com

More Related Content

PDF
Scaling WebRTC applications with Janus
byLorenzo Miniero
 
PDF
Can SFUs and MCUs be friends @ IIT-RTC 2020
byLorenzo Miniero
 
PDF
Janus SFU cascading @ IIT-RTC 2022
byLorenzo Miniero
 
PPTX
Architecting your WebRTC application for scalability, Arin Sime
byAlan Quayle
 
PDF
Janus RTP forwarders @ FOSDEM 2020
byLorenzo Miniero
 
PDF
An SFU/MCU integration for heterogeneous environments
byGiacomo Vacca
 
PPTX
Deep Dive into Apache Kafka
byconfluent
 
PPTX
OVN - Basics and deep dive
byTrinath Somanchi
 
Scaling WebRTC applications with Janus
byLorenzo Miniero
 
Can SFUs and MCUs be friends @ IIT-RTC 2020
byLorenzo Miniero
 
Janus SFU cascading @ IIT-RTC 2022
byLorenzo Miniero
 
Architecting your WebRTC application for scalability, Arin Sime
byAlan Quayle
 
Janus RTP forwarders @ FOSDEM 2020
byLorenzo Miniero
 
An SFU/MCU integration for heterogeneous environments
byGiacomo Vacca
 
Deep Dive into Apache Kafka
byconfluent
 
OVN - Basics and deep dive
byTrinath Somanchi
 

What's hot

PDF
SIP trunking in Janus @ Kamailio World 2024
byLorenzo Miniero
 
PDF
High-Performance Networking Using eBPF, XDP, and io_uring
byScyllaDB
 
PDF
Janus & docker: friends or foe
byAlessandro Amirante
 
PDF
FreeSWITCH as a Microservice
byEvan McGee
 
PDF
Overview of kubernetes network functions
byHungWei Chiu
 
PDF
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
byThomas Graf
 
PPTX
Linux and Java - Understanding and Troubleshooting
byJérôme Kehrli
 
PDF
Alexei vladishev - Open Source Monitoring With Zabbix
byAndré Déo
 
PPTX
Introduction to Apache Kafka
byJeff Holoman
 
PDF
Open vSwitch 패킷 처리 구조
bySeung-Hoon Baek
 
PDF
Networking in Java with NIO and Netty
byConstantine Slisenka
 
PDF
NFS(Network File System)
byudamale
 
PPTX
CCNA 2 Routing and Switching v5.0 Chapter 11
byNil Menon
 
PPTX
Apache kafka
byKumar Shivam
 
PPTX
CCNA 2 Routing and Switching v5.0 Chapter 8
byNil Menon
 
PDF
Kafka Overview
byiamtodor
 
PDF
Unified stateful big data processing in Apache Beam (incubating)
byAljoscha Krettek
 
PPTX
Group-7-DHCPv4.pptx
byIvanTabanag1
 
PDF
An Introduction to Apache Kafka
byAmir Sedighi
 
PPTX
TechWiseTV Workshop: Cisco Catalyst 9800 Series Wireless Controller
byRobb Boyd
 
SIP trunking in Janus @ Kamailio World 2024
byLorenzo Miniero
 
High-Performance Networking Using eBPF, XDP, and io_uring
byScyllaDB
 
Janus & docker: friends or foe
byAlessandro Amirante
 
FreeSWITCH as a Microservice
byEvan McGee
 
Overview of kubernetes network functions
byHungWei Chiu
 
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
byThomas Graf
 
Linux and Java - Understanding and Troubleshooting
byJérôme Kehrli
 
Alexei vladishev - Open Source Monitoring With Zabbix
byAndré Déo
 
Introduction to Apache Kafka
byJeff Holoman
 
Open vSwitch 패킷 처리 구조
bySeung-Hoon Baek
 
Networking in Java with NIO and Netty
byConstantine Slisenka
 
NFS(Network File System)
byudamale
 
CCNA 2 Routing and Switching v5.0 Chapter 11
byNil Menon
 
Apache kafka
byKumar Shivam
 
CCNA 2 Routing and Switching v5.0 Chapter 8
byNil Menon
 
Kafka Overview
byiamtodor
 
Unified stateful big data processing in Apache Beam (incubating)
byAljoscha Krettek
 
Group-7-DHCPv4.pptx
byIvanTabanag1
 
An Introduction to Apache Kafka
byAmir Sedighi
 
TechWiseTV Workshop: Cisco Catalyst 9800 Series Wireless Controller
byRobb Boyd
 

Similar to Scaling server side web rtc applications the janus challenge by lorenzo miniero

PDF
Multistream in Janus @ CommCon 2019
byLorenzo Miniero
 
PDF
Janus + Audio @ Open Source World
byLorenzo Miniero
 
PDF
Janus/SIP @ OpenSIPS 2017
byLorenzo Miniero
 
PDF
Janus workshop @ RTC2019 Beijing
byLorenzo Miniero
 
PDF
BWE in Janus
byLorenzo Miniero
 
PDF
Janus/SIP @ OpenSIPS 2019
byLorenzo Miniero
 
PDF
Janus/Asterisk @ Astricon 2017
byLorenzo Miniero
 
PDF
Janus: an open source and general purpose WebRTC (gateway) server
byDevDay
 
PDF
Multistream in SIP and NoSIP @ OpenSIPS Summit 2025
byLorenzo Miniero
 
PDF
Janus Workshop pt.2 @ ClueCon 2021
byLorenzo Miniero
 
PDF
WebRTC Infrastructure Design
byNeeraj Chandra
 
PDF
Janus conf'19: janus client side
byAlexandre Gouaillard
 
PDF
Write a SocialTV app @ OpenSIPS 2021
byLorenzo Miniero
 
PDF
Janus Workshop @ ClueCon 2020
byLorenzo Miniero
 
PDF
Fuzzing Janus @ IPTComm 2019
byLorenzo Miniero
 
PDF
Janus @ ClueCon 2019
byLorenzo Miniero
 
PDF
WebRTC Rockstars Asian Tour 2017
byLorenzo Miniero
 
PDF
Janus @ RTC2017 Beijing
byLorenzo Miniero
 
PDF
Janus @ WebRTC Meetup Stockholm
byLorenzo Miniero
 
PDF
Janus @ DevDay Napoli
byLorenzo Miniero
 
Multistream in Janus @ CommCon 2019
byLorenzo Miniero
 
Janus + Audio @ Open Source World
byLorenzo Miniero
 
Janus/SIP @ OpenSIPS 2017
byLorenzo Miniero
 
Janus workshop @ RTC2019 Beijing
byLorenzo Miniero
 
BWE in Janus
byLorenzo Miniero
 
Janus/SIP @ OpenSIPS 2019
byLorenzo Miniero
 
Janus/Asterisk @ Astricon 2017
byLorenzo Miniero
 
Janus: an open source and general purpose WebRTC (gateway) server
byDevDay
 
Multistream in SIP and NoSIP @ OpenSIPS Summit 2025
byLorenzo Miniero
 
Janus Workshop pt.2 @ ClueCon 2021
byLorenzo Miniero
 
WebRTC Infrastructure Design
byNeeraj Chandra
 
Janus conf'19: janus client side
byAlexandre Gouaillard
 
Write a SocialTV app @ OpenSIPS 2021
byLorenzo Miniero
 
Janus Workshop @ ClueCon 2020
byLorenzo Miniero
 
Fuzzing Janus @ IPTComm 2019
byLorenzo Miniero
 
Janus @ ClueCon 2019
byLorenzo Miniero
 
WebRTC Rockstars Asian Tour 2017
byLorenzo Miniero
 
Janus @ RTC2017 Beijing
byLorenzo Miniero
 
Janus @ WebRTC Meetup Stockholm
byLorenzo Miniero
 
Janus @ DevDay Napoli
byLorenzo Miniero
 

Recently uploaded

PDF
One Agent to Rule Them All - The Fellowship of AI Agents
byIvo Andreev
 
PPTX
SNG460-CNG489_Week6_3-7Nov25_Chp6-Part1.pptx
byberayseray382
 
PPTX
Privacy Preserving Detection of Sensitive Data Exposure (1).pptx
byNikitaMandhan4
 
PDF
Manual vs Automated Accessibility Testing – What to Choose in 2025.pdf
bykalichargn70th171
 
PDF
What Is A Woman (WIAW) Token – Smart Contract Security Audit Report by EtherA...
byEtherAuthority
 
PPTX
Deep Dive into Durable Functions, presented at Cloudbrew 2025
byJoonas Westlin
 
PDF
Blueprint to build quality before the code exists - StackConnect Milan 2025
byLudovicoBesana1
 
PPTX
application security presentation 2 by harman
byharmanideapad
 
PPTX
Why Your Business Needs Snowflake Consulting_ From Data Silos to AI-Ready Cloud
byChetu
 
PDF
How Modern Custom Software is Revolutionizing Mortgage Lending Processes -Ma...
byMatellio
 
PDF
Navigating SEC Regulations for Crypto Exchanges Preparing for a Compliant Fut...
byzak jasper
 
PDF
Revolutionising-SQL-Data-Modelling-with-SqlDBM.pdf
bySQL DBM
 
PDF
How NetSuite Cloud ERP Helps Businesses Overcome Legacy System Downtime.
byTechWize
 
PPTX
Lecture 3 - Scheduling - Operating System
bynurulalomador
 
PPT
Oracle Integration Cloud – Complete Step-by-Step Integration Workflow Guide
byjayasuryanexinfo
 
PDF
Jeremy Millul - An NYU Computer Science Graduate
byJeremy Millul
 
PPTX
Modern P&C Insurance Software for Smarter, Faster Operations.pptx
byInsurance Tech Services
 
PDF
Imed Eddine Bouchoucha | computer engineer | software Architect
byImed Bouchoucha
 
PPTX
List on Python Programming Language.pptx
byRICHARDALONSAGAY1
 
PDF
Database Management Systems(DBMS):UNIT-II Relational Data Model BCA SEP SEM ...
byKuvempu University
 
One Agent to Rule Them All - The Fellowship of AI Agents
byIvo Andreev
 
SNG460-CNG489_Week6_3-7Nov25_Chp6-Part1.pptx
byberayseray382
 
Privacy Preserving Detection of Sensitive Data Exposure (1).pptx
byNikitaMandhan4
 
Manual vs Automated Accessibility Testing – What to Choose in 2025.pdf
bykalichargn70th171
 
What Is A Woman (WIAW) Token – Smart Contract Security Audit Report by EtherA...
byEtherAuthority
 
Deep Dive into Durable Functions, presented at Cloudbrew 2025
byJoonas Westlin
 
Blueprint to build quality before the code exists - StackConnect Milan 2025
byLudovicoBesana1
 
application security presentation 2 by harman
byharmanideapad
 
Why Your Business Needs Snowflake Consulting_ From Data Silos to AI-Ready Cloud
byChetu
 
How Modern Custom Software is Revolutionizing Mortgage Lending Processes -Ma...
byMatellio
 
Navigating SEC Regulations for Crypto Exchanges Preparing for a Compliant Fut...
byzak jasper
 
Revolutionising-SQL-Data-Modelling-with-SqlDBM.pdf
bySQL DBM
 
How NetSuite Cloud ERP Helps Businesses Overcome Legacy System Downtime.
byTechWize
 
Lecture 3 - Scheduling - Operating System
bynurulalomador
 
Oracle Integration Cloud – Complete Step-by-Step Integration Workflow Guide
byjayasuryanexinfo
 
Jeremy Millul - An NYU Computer Science Graduate
byJeremy Millul
 
Modern P&C Insurance Software for Smarter, Faster Operations.pptx
byInsurance Tech Services
 
Imed Eddine Bouchoucha | computer engineer | software Architect
byImed Bouchoucha
 
List on Python Programming Language.pptx
byRICHARDALONSAGAY1
 
Database Management Systems(DBMS):UNIT-II Relational Data Model BCA SEP SEM ...
byKuvempu University
 

Scaling server side web rtc applications the janus challenge by lorenzo miniero

  • 1.
    Scaling server-side WebRTCapplications: the Janus challenge Lorenzo Miniero @elminiero CommCon 2018 June 27th 2018, Wotton House, Surrey (UK)
  • 2.
    Sorry Dan, pineappleson pizza = BIG NO
  • 3.
    A picture ofme as a kid for no reason Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Open source believer Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero
  • 4.
    1. No, Idon’t know what happened...
  • 5.
    2. Yes, Policeis still looking for who stole my cheeks
  • 6.
    Ok, let’s stopclowning around... Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Open source believer Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero
  • 7.
    What is Janus? Janus Generalpurpose, open source WebRTC server • https://github.com/meetecho/janus-gateway • Demos and documentation: https://janus.conf.meetecho.com • Community: https://groups.google.com/forum/#!forum/meetecho-janus
  • 8.
    Modular architecture • Thecore only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
  • 9.
    Modular architecture • Thecore only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
  • 10.
    Modular architecture • Thecore only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
  • 11.
    Modular architecture • Thecore only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
  • 12.
  • 13.
  • 14.
    A quick lookat plugins: SIP gateway
  • 15.
    A quick lookat plugins: RTP rebroadcaster
  • 16.
    A quick lookat plugins: Audio MCU
  • 17.
    A quick lookat plugins: Video SFU
  • 18.
    The first steptowards scaling: benchmarking • Measuring Janus performance is not trivial... • Different plugins have different behaviours • Different technologies have different impact • Need to focus on individual features • SIP Gateway (SIP plugin) • Multimedia streaming (Streaming plugin) • Audio mixing (AudioBridge plugin) • Video routing (VideoRoom plugin) • ... What about the results?1 • When just relaying, bandwidth is typically the real bottleneck • Linear vs. exponential resource increase depends on plugin and topologies 1 https://janus.conf.meetecho.com/citeus
  • 19.
    The first steptowards scaling: benchmarking • Measuring Janus performance is not trivial... • Different plugins have different behaviours • Different technologies have different impact • Need to focus on individual features • SIP Gateway (SIP plugin) • Multimedia streaming (Streaming plugin) • Audio mixing (AudioBridge plugin) • Video routing (VideoRoom plugin) • ... What about the results?1 • When just relaying, bandwidth is typically the real bottleneck • Linear vs. exponential resource increase depends on plugin and topologies 1 https://janus.conf.meetecho.com/citeus
  • 20.
    The first steptowards scaling: benchmarking • Measuring Janus performance is not trivial... • Different plugins have different behaviours • Different technologies have different impact • Need to focus on individual features • SIP Gateway (SIP plugin) • Multimedia streaming (Streaming plugin) • Audio mixing (AudioBridge plugin) • Video routing (VideoRoom plugin) • ... What about the results?1 • When just relaying, bandwidth is typically the real bottleneck • Linear vs. exponential resource increase depends on plugin and topologies 1 https://janus.conf.meetecho.com/citeus
  • 21.
    Beyond a singleJanus instance: different strategies • Once performance is evaluated, other aspects come into play • Does a user stream need to stay within the plugin? • Can it be sent to/received from somewhere else? • Will this affect other connections? • How much is the context important? • What level of orchestration is needed? • All those questions can lead to different answers/solutions, e.g. 1 Plain load balancing 2 Media resource brokering (MRB) 3 Sharing the “burden” 4 Cascaded delivery 5 ...
  • 22.
    Beyond a singleJanus instance: different strategies • Once performance is evaluated, other aspects come into play • Does a user stream need to stay within the plugin? • Can it be sent to/received from somewhere else? • Will this affect other connections? • How much is the context important? • What level of orchestration is needed? • All those questions can lead to different answers/solutions, e.g. 1 Plain load balancing 2 Media resource brokering (MRB) 3 Sharing the “burden” 4 Cascaded delivery 5 ...
  • 23.
    Plain load balancing •The simplest strategy of them all • You have N Janus instances available • You allocate users on one according to some logic • No orchestration needed, just knowledge of Janus addresses • User has list vs. HAProxy vs. nginx vs. ... • Logic can differ depending on the application • Pick a random server? • Use round robin? • Pick the one closest to the user? • Pick the least loaded? • Something else and/or any combination of the above?
  • 24.
    Plain load balancing •The simplest strategy of them all • You have N Janus instances available • You allocate users on one according to some logic • No orchestration needed, just knowledge of Janus addresses • User has list vs. HAProxy vs. nginx vs. ... • Logic can differ depending on the application • Pick a random server? • Use round robin? • Pick the one closest to the user? • Pick the least loaded? • Something else and/or any combination of the above?
  • 25.
    Plain load balancing •The simplest strategy of them all • You have N Janus instances available • You allocate users on one according to some logic • No orchestration needed, just knowledge of Janus addresses • User has list vs. HAProxy vs. nginx vs. ... • Logic can differ depending on the application • Pick a random server? • Use round robin? • Pick the one closest to the user? • Pick the least loaded? • Something else and/or any combination of the above?
  • 26.
  • 27.
  • 28.
  • 29.
    Plain load balancinglimitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
  • 30.
    Plain load balancinglimitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
  • 31.
    Plain load balancinglimitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
  • 32.
    Plain load balancinglimitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
  • 33.
    Plain load balancinglimitations • The only safe option is the SIP plugin • All streams are independent of each other • Even those that are related, are related via the SIP infrastructure • It might work with the Streaming plugin... • ... as long as you make sure the same mountpoints exist on all servers... • ... and they’re all fed with the same data! • With other plugins (e.g., AudioBridge or VideoRoom), it’s not as simple • Users meant for the same room may end up on different servers • ... unless the balancing component is aware of the context (“room”) • What about hosting the same room on different servers? • May work, but is also messy • Different contexts, multiple signalling channels, events overhead
  • 34.
    Media resource brokering •A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
  • 35.
    Media resource brokering •A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
  • 36.
    Media resource brokering •A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
  • 37.
    Media resource brokering •A step above Plain load balancing • You have N Janus instances available • You allocate users based on logic AND context • Orchestration needed, here • You need to know which Janus instances are available... • ... and which one is doing what • Logic will differ depending on the context • Before allocating users, you allocate the context itself (e.g., a conference room) • Users are then sent wherever their context belongs to • Unlike Plain load balancing, can be used for everything • You just need to implement context awareness somehow, and track resources
  • 38.
    Media resource brokeringuse case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
  • 39.
    Media resource brokeringuse case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
  • 40.
    Media resource brokeringuse case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
  • 41.
    Media resource brokeringuse case • Will work nicely with all plugins, if you manage the context • Context may give more info to SIP balancing logic • Conference rooms and Streaming mountpoints can be associated to instances • Broadcasting with Streaming plugin... • ... if MRB knows Janus X hosts mountpoint Y, it can send users there • Conference rooms with AudioBridge and/or VideoRoom... • New rooms can be “deployed” on a specific instance • Attempts to join room Y can be sent to Janus instance responsible for it • Won’t help much when the same resource needs to be scaled, though • e.g., many small rooms vs. one very large room • For that, some different approach is needed
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
    Sharing the “burden” •A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
  • 47.
    Sharing the “burden” •A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
  • 48.
    Sharing the “burden” •A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
  • 49.
    Sharing the “burden” •A bit different from Brokering described before • You have N Janus instances available • You use one or more of them to serve the same context • Again, orchestration needed here too • You need to know which Janus instances are available... • ... and which one is serving which “subset” of the resource • Logic will differ depending on the resource • You identify how you can “split” the resource (unrelated parts) • Users are then sent where the resource “subset” is allocated • Less flexible than Balancing and Brokering, and more specific • Can greatly help optimize delivery scenarios, though, depending on the context
  • 50.
    Sharing the “burden”use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
  • 51.
    Sharing the “burden”use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
  • 52.
    Sharing the “burden”use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
  • 53.
    Sharing the “burden”use case • A typical use case is larger conferences • Many active participants, with many viewers • The first step is identifying the resources, and how they’re related • Publishers will send media: they don’t “depend” on each other • Subscribers only receive media: they depend on a specific publisher • Publishers can be the “subset”, here • A large conference room becomes N 1-to-many smaller broadcasts • Each publisher→subscribers distribution is allocated separately • Controlling application makes sure that viewer picks streams from the right place • Bob from Janus1, Alice from Janus2, Tom from Janus3, Jim from Janus2, ...
  • 54.
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
    RTP forwarding • Allstrategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
  • 60.
    RTP forwarding • Allstrategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
  • 61.
    RTP forwarding • Allstrategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
  • 62.
    RTP forwarding • Allstrategies so far assume related streams stay together • In smaller rooms, publishers and subscribers are all on the same server • In larger rooms, only the publisher and its subscribers are on the same server • No scenario where this relationship is broken • Apart from SIP and Streaming, but those are different (legacy interop) • How can streams from different servers interact? • Janus plugins don’t talk to each other... • ... and neither do different Janus instances! RTP forwarding A way to relay RTP behind the curtains, without WebRTC • e.g., relay VideoRoom publisher’s media to a Streaming mountpoint
  • 63.
  • 64.
  • 65.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 66.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 67.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 68.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 69.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 70.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 71.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 72.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 73.
    While “plain” RTP,and not WebRTC? • Why not use WebRTC to “move” media around? • WebRTC uses RTP too, after all, and has a lot of useful stuff • Orchestrated properly, you can have one Janus see another Janus as a WebRTC user • Many reasons why we went for “plain” RTP, actually 1 Recipient may not be WebRTC compliant (e.g., FFmpeg script) 2 A lot of existing tools support RTP (and other things) natively 3 You may not need an encrypted stream (although RTP forwarders can do SRTP) 4 You may need to “feed” many recipients at the same time 5 Plain RTP is simpler to distribute in different topologies 6 You can take advantage of multicast networks easily 7 More in general, WebRTC has too much overhead for that
  • 74.
    A specific scenario:broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
  • 75.
    A specific scenario:broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
  • 76.
    A specific scenario:broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
  • 77.
    A specific scenario:broadcasting • Tree topology with several levels • Broadcaster is at the root • Intermediate levels are relayers (RTP trunks) • Attendees attached at edge nodes (WebRTC) • Waterfalls and Valves • Only one flow between related nodes • Different encodings can be different flows • Number of relay levels can vary • No levels → Broadcaster feeding edge nodes directly • 1-N levels → distribution network (unicast/simulcast) SOLEIL Streaming Of Large Scale Events over Internet cLouds
  • 78.
    SOLEIL architecture: asimple overview
  • 79.
    SOLEIL architecture: asimple overview
  • 80.
    Another scenario: cascadedconferencing • Multiple VideoRoom instances “feeding” each other • Publisher on Janus X → relayed on other Janus instances too • Users can subscribe even when not on Janus X • Can be used for scaling purposes • Potential audience of publisher goes beyond capabilities of single Janus instance • Broadcasting seen before applied to conferencing, basically • Very interesting for geographic access to rooms too, though • Users always contact Janus closest to them • Behind the scenes, the Janus instances exchange media for all publishers • Typically, intra- and inter-datacenter network speed > Users network speed
  • 81.
    Another scenario: cascadedconferencing • Multiple VideoRoom instances “feeding” each other • Publisher on Janus X → relayed on other Janus instances too • Users can subscribe even when not on Janus X • Can be used for scaling purposes • Potential audience of publisher goes beyond capabilities of single Janus instance • Broadcasting seen before applied to conferencing, basically • Very interesting for geographic access to rooms too, though • Users always contact Janus closest to them • Behind the scenes, the Janus instances exchange media for all publishers • Typically, intra- and inter-datacenter network speed > Users network speed
  • 82.
    Another scenario: cascadedconferencing • Multiple VideoRoom instances “feeding” each other • Publisher on Janus X → relayed on other Janus instances too • Users can subscribe even when not on Janus X • Can be used for scaling purposes • Potential audience of publisher goes beyond capabilities of single Janus instance • Broadcasting seen before applied to conferencing, basically • Very interesting for geographic access to rooms too, though • Users always contact Janus closest to them • Behind the scenes, the Janus instances exchange media for all publishers • Typically, intra- and inter-datacenter network speed > Users network speed
  • 83.
  • 84.
  • 85.
    A few wordson nuts and bolts • We use Docker A LOT in our deployments • Even though Janus can deal with NATs, we do prefer it to have a public IP • Host networking makes it tricky to place more instances on the same machine • Pipework: assign a dedicated IP address to a container https://github.com/jpetazzo/pipework • Docker native macvlan network driver also allows you to assign custom IPs to containers. We’ll move to it sooner or later... • Several people (including us) using Janus in clustered environments (Docker Swarm, Kubernetes, ...) • Auto-scaling by spawning new replicas as needed • Live monitoring of machines’ resources (e.g., CPU, # connections, bandwidth) drives the deployment process
  • 86.
    A few wordson nuts and bolts • We use Docker A LOT in our deployments • Even though Janus can deal with NATs, we do prefer it to have a public IP • Host networking makes it tricky to place more instances on the same machine • Pipework: assign a dedicated IP address to a container https://github.com/jpetazzo/pipework • Docker native macvlan network driver also allows you to assign custom IPs to containers. We’ll move to it sooner or later... • Several people (including us) using Janus in clustered environments (Docker Swarm, Kubernetes, ...) • Auto-scaling by spawning new replicas as needed • Live monitoring of machines’ resources (e.g., CPU, # connections, bandwidth) drives the deployment process
  • 87.
    A few wordson nuts and bolts • We use Docker A LOT in our deployments • Even though Janus can deal with NATs, we do prefer it to have a public IP • Host networking makes it tricky to place more instances on the same machine • Pipework: assign a dedicated IP address to a container https://github.com/jpetazzo/pipework • Docker native macvlan network driver also allows you to assign custom IPs to containers. We’ll move to it sooner or later... • Several people (including us) using Janus in clustered environments (Docker Swarm, Kubernetes, ...) • Auto-scaling by spawning new replicas as needed • Live monitoring of machines’ resources (e.g., CPU, # connections, bandwidth) drives the deployment process
  • 88.
  • 89.
  • 90.
    Thanks! Questions? Comments? Contacts •https://twitter.com/elminiero • https://twitter.com/meetecho • http://www.meetecho.com