Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Freifunk Munich - How to scale Jitsi

How we built an open video conferencing service to help people stay connected during corona

You can watch the Youtube Recording here (german):

  • Login to see the comments

Freifunk Munich - How to scale Jitsi

  1. 1. ffmuc: How to scale Jitsi
  2. 2. awlnx ● Senior Network Engineer @AS51324 ● Twitter @awlnx Krombel ● IT Consultant ● Twitter @kr0mbel Who are we?
  3. 3. Jitsi to close the social gap during corona • An upcoming Freifunk Meeting was about to get canceled because of Corona • In the past we used NextCloud talk but we had issues (max 6. users per conference) ■ We needed something different for at least 20 - 30 people ■ jitsi seemed easy and straightforward to install ● “apt install jitsi-meet” => Done ¯_(ツ)_/¯
  4. 4. Debian package wins over Docker • We had some issues with docker-proxy in the past • docker image didn’t look up to date • With scaling in mind ■ Planned using dedicated hardware for Jitsi anyway ■ Debian packages are available
  5. 5. First test - looks promising • testing with some people of Freifunk Munich • Worked like a charm, with a handful of people • The idea was born: ■ Maybe other people have the same problem? ■ Why not open it for the public? ■ Maybe teachers, healthcare workers, etc need that, too?
  6. 6. We need insights! Let’s monitor! • How many users are on the platform? • What’s the impact on the Hardware/VMs? • Do we have bottlenecks? If so: Where?
  7. 7. We have to scale! • One server cannot take the load => Bottleneck CPU on Videobridge • Scale horizontally → Videobridges need dedicated hardware and we may need many! → Looking for sponsors
  8. 8. How we scale!
  9. 9. The project gains attention • Interviews • BR, DigitalCourage, Focus Online, Stimme, SZ, … • diverse Listen
  10. 10. Problems start - Prosody • Change network_backend to “epoll” - No more 1024 connections as a limit
  11. 11. Don’t tune only one part of the stack - nginx • 502 all over the place • Raise the number of nginx workers • Raise max open files limit
  12. 12. More problems - JVB Version 1 • CPU Usage … • Upgrade to JVB Version 2 → Bottlenecks moved to context switches and PPS
  13. 13. Firefox breaks user experience • Users experience problems especially in large conferences (25-40 people) • Add warning for Firefox users … it’s mostly ignored … so problems still exists • Deactivate Firefox support and release FreifunkMeet app →
  14. 14. Upgrades sometimes break stuff • apt update && apt upgrade • Configs get wiped … good to have a testing environment • Once again this time … ar x … scp and git diff ...
  15. 15. Lessons learned • Dedicated Hardware… non-shared resources (NOT the 2€/month VM ) • Know your audience and react to behaviour • Know your technical limitations • Check, test, fix, tune and repeat • Monitoring, Monitoring, Monitoring
  16. 16. Thank you very much! A special thanks to all who helped writing/translating the FAQs. Also thank you to all the users and of course our supporters: