Alberto Gonzalez Trastoy, was among the speakers at Agora’s Real-Time Engagement 2020 Conference. His presentation was about what makes building a live video application more complicated than a regular web app. Isn’t WebRTC supposed to handle everything for you? Alberto describes some of the unexpected nuances and challenges a web developer may encounter building real-time engagement and communications applications. This includes networking, interoperability, scalability and security. He also discusses other complexities in building WebRTC applications and offers tools and alternatives to solve them.
Building WebRTC based interesting features and services . WebRTC to stream from remote machine in IOT.
Details of Ramudroid a bot meant for cleaning outdoors uses webrtc stream for remote navigation .
A video annotation service, to upload and/or record videos and make time synced playback of annotations. The video content is automatically tagged, clasified and described using congitive service.
Slides for the talk I made at IIT-RTC 2021 about WHIP (WebRTC-HTTP ingestion protocol) and how it can help foster adoption of WebRTC in traditional broadcasting tools. The slides also cover my open source implementations of WHIP server (based on Janus) and WHIP client (based on GStreamer), and interoperability tests with other implementations.
Talk@JanusCon2019: Janus, WebRTC and ML - Fantastic technologies and how to m...Paolo Saviano
The presentation will show how to integrate Artificial Intelligence world with WebRTC using the Janus WebRTC Server. Taking advantage of a modular architecture, a Janus instance was configured to provide the handled media to a *Stream Elaborator Network* as a low-latency RTP stream, and to receive back information on the processed media through a data channel, everything in a seamless way.
The presentation will explore how we built a tool that can be specialized to execute different Computer Vision tasks, from the architecture till the code.
This is an introduction on Janus and its WebRTC features to the ClueCon audience, with a few words on how it can be used to complement FreeSwitch in some interesting scenarios.
My (quite boring) slides on what we needed to do in Janus to support multiple streams of the same type (e.g., 3 video streams) on the same PeerConnection.
Scaling WebRTC deployments with multicast @ IETF 110 MBONEDLorenzo Miniero
An overview of how multicast can be used to scale WebRTC deployments, with focus on the Virtual Event Platform used to provide remote participation support to IETF meetings, given during the MBONED WG session at IETF 110.
Building WebRTC based interesting features and services . WebRTC to stream from remote machine in IOT.
Details of Ramudroid a bot meant for cleaning outdoors uses webrtc stream for remote navigation .
A video annotation service, to upload and/or record videos and make time synced playback of annotations. The video content is automatically tagged, clasified and described using congitive service.
Slides for the talk I made at IIT-RTC 2021 about WHIP (WebRTC-HTTP ingestion protocol) and how it can help foster adoption of WebRTC in traditional broadcasting tools. The slides also cover my open source implementations of WHIP server (based on Janus) and WHIP client (based on GStreamer), and interoperability tests with other implementations.
Talk@JanusCon2019: Janus, WebRTC and ML - Fantastic technologies and how to m...Paolo Saviano
The presentation will show how to integrate Artificial Intelligence world with WebRTC using the Janus WebRTC Server. Taking advantage of a modular architecture, a Janus instance was configured to provide the handled media to a *Stream Elaborator Network* as a low-latency RTP stream, and to receive back information on the processed media through a data channel, everything in a seamless way.
The presentation will explore how we built a tool that can be specialized to execute different Computer Vision tasks, from the architecture till the code.
This is an introduction on Janus and its WebRTC features to the ClueCon audience, with a few words on how it can be used to complement FreeSwitch in some interesting scenarios.
My (quite boring) slides on what we needed to do in Janus to support multiple streams of the same type (e.g., 3 video streams) on the same PeerConnection.
Scaling WebRTC deployments with multicast @ IETF 110 MBONEDLorenzo Miniero
An overview of how multicast can be used to scale WebRTC deployments, with focus on the Virtual Event Platform used to provide remote participation support to IETF meetings, given during the MBONED WG session at IETF 110.
Slides for the presentation on how you can get SFUs and MCUs to actually be friends, which I presented at the virtual edition of IIT-RTC 2020. The slides cover some of the pros and cons of both, and some use cases where you may actually want to use both. At the end, a few words are spent on how to use browsers as an MCU instead, which might make them being used with SFUs even easier.
Slides for the presentation I did remotely at Open Source World, to talk about audio-only WebRTC applications, and what we've done in Janus to improve and cover the requirements so far.
These slides cover a workshop called "Having fun with Janus and WebRTC" at the virtual edition of OpenSIPS 2021. The workshop guided viewers to how they could use different features in Janus to build a WebRTC Social TV application, including how to write a new plugin in JavaScript to build a virtual remote.
Slides for the talk I made at the virtual edition of FOSDEM 2022, on how to use WHIP for WebRTC broadcasting ingestion, and how the distribution process could be done via WebRTC as well, e.g., via Janus (and the SOLEIL architecture).
Slides for the presentation I made at ClueCon 21 on the experimental RED support in WebRTC, and how we've started tinkering with it in Janus. The presentation also addresses a more generic overview on audio features in WebRTC.
Slides for the presentation on Insertable Streams and E2EE in WebRTC I presented at the virtual edition of ClueCon 2020. After an introduction on the past and recent E2EE efforts, the slides also present some efforts to integrate the technology in the Janus WebRTC server as well.
This is the slide deck I presented at the first CommCon event in the UK: it goes through some of the possible strategies for scaling WebRTC applications, mostly if you're using Janus but not only.
Virtual IETF meetings with WebRTC @ IETF 109 MOPSLorenzo Miniero
An overview of how the Janus WebRTC Server was used to serve virtual IETF meetings at scale, with focus on how audio and video streams were handled in different ways, given during the MOPS WG session at IETF 109. Some considerations on some specific enhancements made between IETF 108 and 109 are provided as well.
Janus: an open source and general purpose WebRTC (gateway) serverDevDay
Janus è un server WebRTC open source e modulare, concepito per essere a tutti gli effetti uno strumento "general purpose" per la realizzazione di complessi scenari multimediali. In quanto tale, si presta a supportare applicazioni di vario tipo, a partire da scenari più tradizionali quali conferencing, e-learning e streaming multimediale, per arrivare ad applicazioni più innovative che coinvolgano dispositivi eterogenei. La presentazione partirà da una breve panoramica su WebRTC, per coprire poi l'architettura di Janus e le possibili topologie di utilizzo, fino a presentare alcuni esempi reali di utilizzo e casi d'uso di successo.
Lorenzo Miniero
Lorenzo Miniero è uno dei fondatori di Meetecho, azienda specializzata in applicazioni multimediali e comunicazioni in real-time. Lorenzo ha conseguito laurea e dottorato di ricerca presso l'Università di Napoli Federico II, università della quale la stessa Meetecho è spin-off accademico. È un attivo "contributor" alle attività di standardizzazione della Internet Engineering Task Force (IETF), ed è noto principalmente come l'autore del server WebRTC open source Janus.
Slides for the 60 minutes "part 2" Janus workshop I presented at the virtual edition of ClueCon 2021. This time the slides covered Janus ability to bridge WebRTC and non-WebRTC applications to do interesting things, especially with the help of plain RTP and RTP forwarders. Check the conference recordings to see the actual demos in action.
Workshop on SIP/WebRTC load testing, as presented at Kamailio World 2017. May make less sense without the live demos to support it, but can still contain info some might find useful.
Slides on the new Janus Lua plugin, as presented at the Real-time Communications devroom at FOSDEM2018. Describes in detail what the challenges were, and how we addressed them, with a few real examples at the end (including a demo written specifically for this session).
The slides for a talk I presented at the WebRTC track of IIT-RTC in Chicago, titled "SFU's, Simulcast and SVC: what's new in WebRTC?". Mostly a high-level introduction to how simulcast and SVC work (or not) today in browsers, how they came to be and where they might be headed from a standards perspective.
Slides for the presentation on how you can get SFUs and MCUs to actually be friends, which I presented at the virtual edition of IIT-RTC 2020. The slides cover some of the pros and cons of both, and some use cases where you may actually want to use both. At the end, a few words are spent on how to use browsers as an MCU instead, which might make them being used with SFUs even easier.
Slides for the presentation I did remotely at Open Source World, to talk about audio-only WebRTC applications, and what we've done in Janus to improve and cover the requirements so far.
These slides cover a workshop called "Having fun with Janus and WebRTC" at the virtual edition of OpenSIPS 2021. The workshop guided viewers to how they could use different features in Janus to build a WebRTC Social TV application, including how to write a new plugin in JavaScript to build a virtual remote.
Slides for the talk I made at the virtual edition of FOSDEM 2022, on how to use WHIP for WebRTC broadcasting ingestion, and how the distribution process could be done via WebRTC as well, e.g., via Janus (and the SOLEIL architecture).
Slides for the presentation I made at ClueCon 21 on the experimental RED support in WebRTC, and how we've started tinkering with it in Janus. The presentation also addresses a more generic overview on audio features in WebRTC.
Slides for the presentation on Insertable Streams and E2EE in WebRTC I presented at the virtual edition of ClueCon 2020. After an introduction on the past and recent E2EE efforts, the slides also present some efforts to integrate the technology in the Janus WebRTC server as well.
This is the slide deck I presented at the first CommCon event in the UK: it goes through some of the possible strategies for scaling WebRTC applications, mostly if you're using Janus but not only.
Virtual IETF meetings with WebRTC @ IETF 109 MOPSLorenzo Miniero
An overview of how the Janus WebRTC Server was used to serve virtual IETF meetings at scale, with focus on how audio and video streams were handled in different ways, given during the MOPS WG session at IETF 109. Some considerations on some specific enhancements made between IETF 108 and 109 are provided as well.
Janus: an open source and general purpose WebRTC (gateway) serverDevDay
Janus è un server WebRTC open source e modulare, concepito per essere a tutti gli effetti uno strumento "general purpose" per la realizzazione di complessi scenari multimediali. In quanto tale, si presta a supportare applicazioni di vario tipo, a partire da scenari più tradizionali quali conferencing, e-learning e streaming multimediale, per arrivare ad applicazioni più innovative che coinvolgano dispositivi eterogenei. La presentazione partirà da una breve panoramica su WebRTC, per coprire poi l'architettura di Janus e le possibili topologie di utilizzo, fino a presentare alcuni esempi reali di utilizzo e casi d'uso di successo.
Lorenzo Miniero
Lorenzo Miniero è uno dei fondatori di Meetecho, azienda specializzata in applicazioni multimediali e comunicazioni in real-time. Lorenzo ha conseguito laurea e dottorato di ricerca presso l'Università di Napoli Federico II, università della quale la stessa Meetecho è spin-off accademico. È un attivo "contributor" alle attività di standardizzazione della Internet Engineering Task Force (IETF), ed è noto principalmente come l'autore del server WebRTC open source Janus.
Slides for the 60 minutes "part 2" Janus workshop I presented at the virtual edition of ClueCon 2021. This time the slides covered Janus ability to bridge WebRTC and non-WebRTC applications to do interesting things, especially with the help of plain RTP and RTP forwarders. Check the conference recordings to see the actual demos in action.
Workshop on SIP/WebRTC load testing, as presented at Kamailio World 2017. May make less sense without the live demos to support it, but can still contain info some might find useful.
Slides on the new Janus Lua plugin, as presented at the Real-time Communications devroom at FOSDEM2018. Describes in detail what the challenges were, and how we addressed them, with a few real examples at the end (including a demo written specifically for this session).
The slides for a talk I presented at the WebRTC track of IIT-RTC in Chicago, titled "SFU's, Simulcast and SVC: what's new in WebRTC?". Mostly a high-level introduction to how simulcast and SVC work (or not) today in browsers, how they came to be and where they might be headed from a standards perspective.
Wading through the mobile WebRTC developer gauntlet (IIT RTC Conference sessi...Brian Pulito
Today real-time communications in mobile devices typically means general purpose, non-contextual audio and video calling. WebRTC promises more by embedding real-time with context. For mobile this means apps enabled with WebRTC capabilities. While mobile implementations of WebRTC are now available for iOS and Android, there are still many decisions that need to be made to take advantage of it. Things like what frameworks to use, native vs. hybrid, how to handle security and much more. This session will discuss going from concept to delivered application and what steps and decisions need to be made along the way.
This week, the WebRTC WG is having its first "testing focused" meeting to review the status of testing of WebRTC 1.0. Our coming session will be a summary of this meeting.
Event sponsored by WebRTC.Ventures
Status of WebRTC across Asia by Alan Quayle +++Alan Quayle
Status of WebRTC across Asia by Alan Quayle, and a group of leading experts contributing to the reality, not the hype, of WebRTC.
It’s 2020, WebRTC (Web Real Time Communications) became known in 2011 when Google open sourced intellectual property it had bought in previous years. Gossip about those acquisitions began in 2009. The IETF (Internet Engineering Task Force) was already laying the groundwork with Opus (voice codec) officially in 2010, and back in 2009 the discussion process started that became WebRTC. It’s been roughly one decade. Did WebRTC change everything? Is WebRTC everywhere?
WebRTC myths and misconceptions. Understanding the two components of WebRTC, the open source project, and the standards track.
Reviewing the achievements of WebRTC across Asia.
Understanding why ‘WebRTC’ companies such as Vidyo and Tokbox did not achieve big exits.
What is the current status of WebRTC, where are the standards, where is the innovation edge?
What is happening across Asia on WebRTC? Understanding the difference service providers adoption of WebRTC. Across telcos, CPaaS, UCaaS. CCaaS, in-app communication platforms, and enterprises.
Case studies on WebRTC implementation across Asia.
Recommendations for WebRTC in Asia.
8 pre launch steps to go with the web rtc based application developmentMoonTechnolabsPvtLtd
Experiencing and interacting with people via live video has become quite popular in numerous applications across various industries. With real-time engagement through live video, hundreds and thousands of businesses managed to transform the way they operate. Today, business enterprises, telemedicine platforms, online education platforms, entertainment & sports platforms, virtual event platforms, e-commerce, and everything else in between, use the power of WebRTC technology to establish communications in real-time via mobile applications.
https://www.moontechnolabs.com/blog/pre-launch-steps-with-webrtc-based-application-development/
WebRTC Live Q&A Session #4 - WebRTC in WebKit and the story around Apple and ...Amir Zmora
A live Q&A session about WebRTC in general and an update about Apple's work on WebRTC. Session included internal information from a meeting between Apple engineers and the people behind the WebRTC-in-WebKit initiative.
Session by Alex Gouailard, Dan Burnett and Amir Zmora
WebRTC for Telco: Informa's WebRTC Global Summit PreconferenceTsahi Levent-levi
The preconference workshop I did at Informa's WebRTC Global Summit in London, 31st of March 2014
It is targeted at bringing people up to speed with what WebRTC is, how people and vendors are using it today and placing it also in the context of the telecom world (which is the focus of this specific conference).
One of the common challenges in the digital space is improving the speed of releases without compromising the of quality of your app. The root of the problem is the market - customer expectations are on the rise, the app market is crowded, and app development is difficult. The solution is test automation.
Watch Perfecto and Infostretch demonstrate Quantum, an established open-source test framework, to run robust, repeatable, and continuous test scenarios.
In this technical webinar, the audience will learn how to use the test framework to
-Create robust and maintainable test automation scripts
-Extend open-source with advanced automation capabilities
-Execute cross-platform mobile and web tests in parallel
-Plug the newly created tests easily to the CI (Continuous Integration) workflow
-Drive fast developer feedback with an advanced reporting library
WebSphere Liberty Rtcomm: WebRTC Middleware for the EnterpriseBrian Pulito
In order to provide the type of services their customers crave, your clients need to be able to provide blazing fast communication capabilities and access important information in the blink of an eye. WebRTC (Web Real-Time Communications) allows for the creation of next-generation communication applications without the need for browser plugins. WebSphere Application Server Liberty Profile is changing the way people communicate by making it easy to provide web page context as part of real-time conversations. This webinar will cover all of the real-time communications features recently released in WebSphere Liberty, including the new Rtcomm feature for rapid development of WebRTC based applications, and the open-source Rtcomm client-side libraries. (link to webinar replay: http://www.websphereusergroup.org/khatch/go/gallery/item/1543395?type=video)
WebRTC Webinar and Q&A - IP Address Privacy and Microsoft Edge InteroperabilityAmir Zmora
WebRTC webinar explaining what was all the hype around IP address privacy in WebRTC, what are the risks and how WebRTC is handling them. Webinar also talks about WebRTC browser interoperability and specifically interoperability with Microsoft Edge.
Webinar is part of the monthly WebRTC live Q&A sessions by Alex Gouailard, Dan Burnett and Amir Zmora
A modified version of my Desert Code Camp 2011.2 presentation on SignalR from November 5th, 2011.
It's modified since I'm more of a talker and rarely utilize bullet points and much text in my slides.
Similar to Things I wished I knew before building my first WebRTC app - RTE2020 (20)
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
3. Open source project
Real Time Communications framework
Secure
Updated frequently
Used in many major platforms and applications
Available on all modern browsers and native clients
WebRTC Basics
4. “This app doesn’t work on iPhone”
“Hi, hello! Can you hear me? I can hear you, but you can’t hear
me”
“Yes I can see you, but video looks blurry”
“My microphone is not working, wait a second, I will restart my
computer”
“I can’t connect, I think it is because I have very slow internet”
But mistakes can cause user responses like…
9. Some open Source Alternatives:Some commercial alternatives:
Scalability
10. Networking issues: Restrictive
Review your checklist, monitor and
NAT Traversal
Proxy and firewall rules
- Proxy authentication from clients required?
- Proxy blocking access to IP addresses
- Firewall rules
- NAT
Solution
11. Networking issues: Congested
Network congestion
- Too many hosts in a local network
- Low bandwidth
- Interference from outside sources or
faulty cabling
WebRTC has error resilience mechanisms
but there is a limit. In that case, optimize,
monitor and keep track of the logs
Solution
12. Video Room Type Minimal Available Bandwidth
required (at the client side)
8 participants video room with Lo-res
Video (240x180) + HD Audio
~2 Mbps
8 participants video room with SD Video
(640x480) + HD Audio
~8 Mbps
8 participants video room with HD Video
(1280x720) + HD Audio
~22 Mbps
WebRTC Video Bandwidth Requirements
13. How To Overcome Those Limits?
• Minimize the number of videos the client subscribes to
• Use VP8 Simulcast for large conferences or broadcasting*
• Minimize video resolution and frame-rate
• Optimize based on device type
• Keep audio as first-class citizen
*With codecs we always need to compromise. If most users use Safari and have good internet connection, H264 codec
might be the way to go
14. App Optimization Example
Layout example where the main speaker appears in the pink/red
square and the other participants appear at the right
17. WebRTC E2EME Solution
Using Insertable Streams API to use secure frames mechanism
Demo available here: https://webrtc.github.io/samples/src/content/peerconnection/endtoend-encryption/ (experimental)
18. Web Testing vs WebRTC Testing
Both need functional testing
Compatibility testing ≠ Interoperability testing
N x Performance testing
19. WebRTC Testing and Debugging Tools
And other testing and debugging proprietary applications…
Testing Debugging
Chrome WebRTC Internals
20. “This app doesn’t work on iPhone”, said someone using chrome on iPhone
“Hi, hello! Can you hear me? I can hear you, but you can’t hear me”, said someone failing to accept
or blocking access to the microphone
“Yes I can see you, but video looks blurry”, said someone using a video app that doesn’t use SVC
or simulcast
“My microphone is not working, wait a second, I will restart my computer”, said someone with faulty
headphones
“I can’t connect, I think it is because I have very slow internet”, said someone about an app that
doesn’t prioritize audio and optimize available bandwidth
Back To The User Issues
I wanted to introduce this talk with an accurate representation working on RTC apps:
From a naïve version of me 5 years ago to a more experience one.
During that process I discovered how WebRTC is not just another browser API, it has its own community of experts
Today after dozens of apps built for different use cases, and more now to help interacting in a global pandemic.
So why we use WebRTC, well it is the open source standard to go for low latency streaming!
It does sound familiar. And I left out the “I can’t hear you. You are muted” because it is more of a UI/UX thing
But those are user problems that can happen due to mistakes in the implementation. An implementation that doesn’t lack challenges…
WebRTC is build to be easy to use but it is also different from using any other browser API.
This is because it converges hardware. telephony and software. What makes building a live video application more complicated than a regular web app?
Interoperability, which just simply refers to “How good some devices interact with each other”
Interoperability, which just simply refers to “How good some devices interact with each other” is a common challenge.
Basic one to one communication using WebRTC works in most desktop and mobile scenarios.
But some more advanced features like screensharing or managing multiple peer connections are also supported by most. But….Not all
In some situations hardware and OS system also plays a role limiting some functionalities.
Cameras and microphones are not equal in each device and are a common source of user problems.
Handling those errors properly will be key for a good user experience
(Debugging some of this types of issues might require of debugging tools like WebRTC internals or wireshark.)
Browser or OS! The main functions are working on all major browsers. Being safari and edge the latest to support WebRTC
But there are still some general interoperability issues:
Different codec preferences for each browser
Older browser versions with specific bugs. Major browser upgrades WebRTC in edge . Edge was rebuilt on top of Chromium makes it easier now
Others: Safari
-Screenshare
-Safari WebRTC on mac* using h264
-1-1 audio/video call, the integration with this major browsers is quite easy, the problems start to appear in more complex scenarios….
iOS implementation has some bugs/restrictions
Forget about using other browsers other than Safari
Some restrictions on autoplay rules (Guide to Safari webrtc in WebRTC Hacks has some very useful info)
Safari iOS is not ready for WebRTC screen sharing
In a recent project for many to many video proctoring and additional one to one calls we encountered all those issues. For example, if you want to send more than one media stream then the previous video/audio is muted
Since Edge was rebuilt on top of Chromium, having MS Edge using working consistently using WebRTC is not a struggle anymore, example using a multiparty WebRTC app
Firefox, is also in sync with the WebRTC implementation and there aren’t any major differences between Chromium and Firefox that I am aware of today
Scalability doesn’t lack it’s challenges. Mesh video call doesn’t work well beyond 4-5 participants… (CPU/BW). We need media servers for:
1) Scalability, Multiple Participants in a Video Call (helps reduce the number of streams a client needs to send,usually to one)
2) Integration with Other Communication Technologies (PSTN via SIP trunking or streaming through RTMP to services)
3) Processing of Media Streams (processing of video and audio streams at a very low level, like being able to run computer vision models)
Server can handle hundreds of media streams, but limit to vertical scaling. Horizontal scaling with geolocation is a common approach for production rtc apps. To scale media servers horizontally one common approach is to build a dispatcher distributing requests from participants to different media servers. Slightly more advanced, doing geographical cascading, which can reduce latency between participants in different regions by letting each participant send and receive video from closest media server
New codecs and standards like SVC (Scalable Video Coding) are helping to scale from the client side to send better quality at lower bitrates and the right quality for each participant
But of course there are some limitations if we compare with VOD…
OSS:
Jitsi SFU and implements ints own signaling using Jingle(XMPP)
Janus general purpose WebRTC server that can be setup as an SFU. Plugin architecture: SIP Gateway, VP9-SVC Video Room, live streaming…
Kurento can also be configured to function as SFU or MCU, or both, in a single instance. OpenVidu, a new platform to facilitate the use of Kurento functionalities from a higher-level client in your web or mobile applications
We have worked with all of them for production projects or, at least, demos.
Also, there are other popular platforms that weren’t originally developed to be WebRTC media servers but have WebRTC media server capabilities:
Asterisk, FreeSwitch: Mostly used in telephony applications it also supports WebRTC and it is frequently used in conjunction with JsSIP or SIP.js
Pion: New stack for Web Real-Time Communications. Pion is built on Go and allows developers to use the WebRTC stack as small pieces of lego. Can be used to build a SFU
CPaaS
Will scale probably to millions of connections without you having to handle the distribution between servers/maintenance or geolocation. You just need to use their SDK and you are good to just focus on the client solution
Checklist of proxy and firewall rules:
-TCP Ports like 443 should be allowed
-UDP ports used for RTP connection 1025-65535 should be open too. If not at least UDP 3478 for TURN
-Persistent WSS should be allowed for the signaling
-NAT essentially hides a home or office's internal network from the public internet
(Tech note) NAT Traversal:
Nat traversal (reaching client IP address hidden by NAT) is achieved using the WebRTC build in ICE gathering (protocols are STUN and, as a last resort, TURN (less than 1/3 of calls need that but chances are that you will need it if you are in a restrictive network))
But still you will need a TURN server to skip network limitations. You can deploy it yourself using coturn or use some 3rd party provider (CPaaS will handle this for you)
Monitoring is usually built in some CPaaS but there are also some 3rd party platforms like callstats that handle it.
(Or you can build it yourself strong webrtc errors in a logging database)
----
More on NAT traversal:
Clients are typically situated on networks designed to protect them from public requests and may not have a public IP address => this often introduce complicated hurdles.
Connecting to a simple web server is as easy as making an HTTP request VS WebRTC needs to use ICE, which provides a multitude of connection types, each of which may be tried in order to establish a successful connection.
It in your network can be caused by faulty cabling, interference from outside sources or as the result of a collision.
Also, too many hosts in a domain or not enough bandwidth (internet pipe size) can generate congestion and overload the network
Network congestion => High error rate/packet loss & Might cause low quality media
How to optimize?
First, you’d need to know your use case and architecture. Is it a webinar, video chat, panel? And from there measure a minimal available bandwidth required
As an example here I calculated an 8 party video room using video bitrate estimations based on resolution (at 24fps). Assuming a SFU media server (one video uplink and 7 downlink)
And although it will change depending on the codec used (VP9 better, AV1 even better) that’s the idea…
Since not everyone has 22Mbps available how can we handle HD quality? How do we overcome these limits?
Collaboration/presentation use cases might not need to display all the participants in a grid. We can show the dominant speaker and the rest as thumbnails
Collaboration/presentation use cases might not need to display all the participants in a grid. We can show the dominant speaker and the rest as thumbnails
Also mobile phones have less CPU so if you want to keep the best experience for mobile keep the number of displayed videos small
Encrypted end-to-end.
Core protocols defined by the IETF for providing WebRTC security: SRTP for media traffic and DTLS-SRTP for key negotiation
This is an ideal scenario that gets more complicated if we need to support multiparty with media servers in between
we have an intermediate participant, the media server, which would decrypt and re-encrypt the media. Obviously, that’s not great if you don’t trust the media server
Media streams are temporarily decrypted within the cloud servers and then immediately re-encrypted before being sent through the internet to the subscribing client. This decryption is necessary for managing group calls, other types of media exchange, intelligent quality control, and session recording
E2EE with insertable streams demo from webrtc-samples where Middlebox represents what the media server would see. Insertable Streams is not supported by default in Chrome yet, so you might need to enable that in chrome://flags in Canary.
Kudos to cosmoSoftware, google and the rest of the open source community building this encryption mechanism. Called Secure Frames
Using Insertable Streams API
Compatibility testing on a basic web app would be mainly focused on display in different devices and resolutions. Different operating systems display certain app elements differently.
VS
Interoperability testing needs, IN ADDITION to test for compatibility between different browsers and OS. How the RTC communication behaves, codecs used, etc.
N times what a basic web app would need…
For performance, while basic web apps will focus on the single page load, CPU usage of the server and so on. For RTC, and in specific, WebRTC, you will need to test the bandwidth limitations when sending and receiving media with different number of participants.
Also stress testing is easy to do for basic web apps, just open new tabs. But for WebRTC, you can reach the client bandwidth and CPU limit quick, so you will need multiple devices or VMs to properly stress test, a device isn’t enough.
For testing and debugging network or interoperability challenges mentioned before using KITE webrtc specific selenium based framework will help identify problems in your app
Some proprietary apps that we have used are BrowserStack or testRTC for testing or callstats for monitoring/debugging
For debugging WebRTC internals it is a quick way to identify WebRTC problems: it can be used to debug the flow of WebRTC sessions to determine issues during development
Wireshark will be a more advanced alternative to get more granularity, down to seeing the packets one by one
KITE for interoperability testing which uses selenium to launch browsers to check if video is sent or receives, and also goes into other details such as if the ICE gathering was successful.
In this image we are testing with 4 browsers, for testing with Safari you need to have a Safari device or VM.
WebRTC internals: it can be used to debug the flow of WebRTC sessions to determine issues during development
For example we can see here the outbound video and audio streams. And Video stopped being sent after a few seconds (could be user or pli packets stopped due to hardware?)
Back to the user issues, now, based on what I explained we could guess what could have been the problem for each user…
Because applications today have a high standard and things are supposed to work, always. I hope you learned and won’t make the same mistakes I did in the past.