Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Overcoming the Challenges in Testing WebRTC Services

2,811 views

Published on

My session for WebRTC Summit, Nov 2015, Santa Clara

WebRTC sits at the intersection between VoIP and the Web. As such, it poses some interesting challenges for those developing services on top of it, but also for those who need to test and monitor these services.

In this session, Tsahi Levent-Levi will review the various challenges posed by WebRTC when it comes to testing and monitoring and on ways to overcome them.

Published in: Technology
  • Be the first to comment

Overcoming the Challenges in Testing WebRTC Services

  1. 1. Overcoming the Challenges in Testing WebRTC Services Tsahi Levent-Levi November 2015
  2. 2. Let me tell you a little secret People don’t usually test WebRTC implementations…
  3. 3. Why is that? • WebRTC is still niche/experimental to their business • They are a startup with limited resources • Adding features takes priority over stability • Is this VoIP or Web?
  4. 4. At an intersection of worlds 4 Our Internet Web Traditional VoIP
  5. 5. The world of VoIP testing • Focused on DUTs (Devices Under Test) • Rely heavily on slow moving standards and implementations • Vast majority of testing solutions are on premise • Assume limited geographies 5
  6. 6. The world of Web testing • Focused on different browsers • Rely heavily on the GUI • Responsive design causes GUI changes • GUI and API testing treated as separate • Geography doesn’t matter for functionality 6
  7. 7. 5 Major Challenges When trying to test your WebRTC service
  8. 8. Browser version changes Do you know if your service will not break next week? #1
  9. 9. Browsers update cycle in 2015 This translates to a regression test needed on a monthly basis – not taking into account changes you make to the service 40 41 42 43 44 45 46 35 36 37 38 39 40 Dec-14 Feb-15 Apr-15 May-15 Jul-15 Sep-15 Oct-15 Dec-15 Chrome Firefox
  10. 10. Stable, Beta, Dev and Canary Stable What your customers are using Beta What you should expect in the next release Dev Where you can complain about breakage – and expect fixes in time Canary A nightly build of whatever is available at that moment in time Testing only against the Stable version of browsers means you find out issues only after the service breaks for your customers.
  11. 11. BrowserYour app A Word about Mobile At what pace do you update you app’s WebRTC lib? WebRTC WebRTC 6-8 weeks update cycle
  12. 12. NAT traversal How does your service fair in varying network conditions? #2
  13. 13. WebRTC isn’t always P2P Anywhere between 5-50% of your calls get relayed via TURN servers client client Web server TURN server
  14. 14. Have you setup your relays properly? • Support for UDP • Support for TCP
  15. 15. Location, Location, Location When a call in Paris gets relayed through a TURN server in the US – you lose a customer…
  16. 16. Server scale Have you tested your service with 10 browsers? 100? 1,000? More? #3
  17. 17. What do you test at scale? • Signaling? • Media relays? • Backend media processing? • Single session multiparty scale? • End-to-end service scale? • Do you add variance to your scale testing?
  18. 18. “ A typical conversation… Oh, it should work for 100 people in the same conference. We did test it with 12 people – the number of machines we had available – and it worked perfectly fine.
  19. 19. Service uptime Do you know if your service is really running? #4
  20. 20. What people monitor today Memory Storage CPU Network Ping
  21. 21. End-to-end • There are many moving parts in your service • WebRTC requires 3 or more servers to operate • You need to make sure it runs end-to-end
  22. 22. Orchestration How do you automate the mechanics of communications in your service? #5
  23. 23. Web testing is simpler (most of the time) 1. Write your script for a browser 2. Run it once 3. Run it in parallel on 10,000 browsers at once 4. You’re done
  24. 24. WebRTC is about coordinating people 1. A user and an agent 2. A user and a PSTN/mobile call 3. N users joining in on a single call 4. Automatic/dynamic/manual pairing of the users 5. …
  25. 25. Don’t take WebRTC testing lightly
  26. 26. About testRTC Come check us out at testRTC.com
  27. 27. Thank You! Tsahi Levent-Levi tsahil@testRTC.com @tsahil

×