Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Media processing with serverless architecture

332 views

Published on

Practical study of how to make AI/ML media processing leveraging WebRTC technology. In this slide, I explained how to bind WebRTC GW and media processing with serverless container architecture. I also explained Pros & Cons of this architecture for applying media.

This slide was for RTC conference Korea 2018 https://rtckorea.com/ .

Published in: Technology
  • Be the first to comment

Media processing with serverless architecture

  1. 1. Copyright © NTT Communications Corporation. All rights reserved. RTC Conference Korea 2018 on November 1st, 2018 Kensaku Komatsu NTT Communications Media Processing with Serverless Architecture
  2. 2. Copyright © NTT Communications Corporation. All rights reserved. Kensaku Komatsu Technical manager of ECL SkyWay Web application evangelist at NTT Communications in Silicon valley office @komasshu kensakukomatsu
  3. 3. Copyright © NTT Communications Corporation. All rights reserved. Main Idea of the tech talk today 3 A practice of Serverless Architecture for Media Processing using WebRTC.
  4. 4. Copyright © NTT Communications Corporation. All rights reserved. AI / IoT
  5. 5. Copyright © NTT Communications Corporation. All rights reserved. Agenda of the talk 5 • Current business in WebRTC • WebRTC GW – enabler for AI/IoT • Media processing with Serverless architecture
  6. 6. Copyright © NTT Communications Corporation. All rights reserved. Agenda of the talk 6 • Current business in WebRTC • WebRTC GW – enabler for AI/IoT • Media processing with Serverless architecture
  7. 7. Copyright © NTT Communications Corporation. All rights reserved. ECL SkyWay in nutshell 7 https://webrtc.ecl.ntt.com/en/
  8. 8. Copyright © NTT Communications Corporation. All rights reserved. Service model 8 P2P / SFU connectivity Client SDK
  9. 9. Copyright © NTT Communications Corporation. All rights reserved. Customer journey : rarejob ( online education ) 9https://japan.cnet.com/article/35116616/
  10. 10. Copyright © NTT Communications Corporation. All rights reserved. Use cases 10 • Online Education • Online healthcare • Field engineer support • …
  11. 11. Copyright © NTT Communications Corporation. All rights reserved. Wish from our customer 11 • Recording • Voice recognition, translation • Video detection, recognition • Emotion detection • Low latency live streaming • VR / AR / Avatar • …
  12. 12. Copyright © NTT Communications Corporation. All rights reserved. • Recording • Voice recognition, translation • Video detection, recognition • Emotion detection • Avatar • … Question 12 How do we support these needs from our customer?
  13. 13. Copyright © NTT Communications Corporation. All rights reserved. Agenda of the talk 13 • Current business in WebRTC • WebRTC GW – enabler for AI/IoT • Media processing with Serverless architecture
  14. 14. Copyright © NTT Communications Corporation. All rights reserved. Our answer 14 Use WebRTC GW for server side media processing
  15. 15. Copyright © NTT Communications Corporation. All rights reserved. What’s SkyWay WebRTC GW 15 • Protocol gateway of WebRTC • Enabler for server side media processing • Can get RTP media stream from client • Controllable with REST API • Easy to connect from skyway client • Supported platform : Linux and Windows WebRTC GW Processing engineRTP
  16. 16. Copyright © NTT Communications Corporation. All rights reserved. 16 https://github.com/skyway/skyway-webrtc-gateway
  17. 17. Copyright © NTT Communications Corporation. All rights reserved. Use case 1 : server side recording 17 • WebRTC GW … stream out media data as RTP streaming • Bind decoder / encoder when needed • POST media data to cloud storage or else WebRTC GW Decoder / EncoderRTP Storage POST
  18. 18. Copyright © NTT Communications Corporation. All rights reserved. Use case 2 : voice recognition 18 • WebRTC GW … stream out media data as RTP streaming • Bind decoder to get raw PCM data stream • POST voice data to Voice recognition PaaS • POST recognition result to cloud database. WebRTC GW Decoder RTP DB POST Voice recognition
  19. 19. Copyright © NTT Communications Corporation. All rights reserved. • WebRTC GW … stream out media data as RTP streaming • Bind decoder to get raw PCM data stream • POST voice data to Voice recognition PaaS • POST recognition result to cloud database. Question 19 How do we manage these service for each scenario?
  20. 20. Copyright © NTT Communications Corporation. All rights reserved. Agenda of the talk 20 • Current business in WebRTC • WebRTC GW – enabler for AI/IoT • Media processing with Serverless architecture
  21. 21. Copyright © NTT Communications Corporation. All rights reserved. Base idea to manage server side processing 21 • Apply micro-service model for each media processing scenario. • Agility, Flexibility and scalability 21 WebRTC GW Micro Service #1 RTP Micro Service #2 Micro Service #3
  22. 22. Copyright © NTT Communications Corporation. All rights reserved. Issues, we faced with 22 • We found that… • Sometime, media processing engine makes state issue. • Scalability issue for each components • Especially media encoder and recognition process requires high resource usage. • Have to allocate much resources for above micro servers.
  23. 23. Copyright © NTT Communications Corporation. All rights reserved. Serverless architecture 23 • AKA ”Function as a service” • Event base execution model • Each function will be executed and killed in each session cycle • Easy to get scalability • No-need to care about long-term state issue 23 spawn
  24. 24. Copyright © NTT Communications Corporation. All rights reserved. Our challenge 24 • Apply serverless architecture into server side media processing scenario. • Build serverless system from scratch on top of docker container • Current FaaS is focused to only RPC GW Decoder recognition GW Decoder recognition API GW orchestrator spawn
  25. 25. Copyright © NTT Communications Corporation. All rights reserved. Demo 25
  26. 26. Copyright © NTT Communications Corporation. All rights reserved. Pros and Cons 26 • Pros • Flexibility, Scalability and Agility • Do not need to care about execution time than short lived functions • Easy to re-use each components • Each scenario is well isolated, e.g. security perspective • Cons • Need to have consolidate monitoring • Controlling each components while set upping orchestration is difficult • Several timing issue will happen, especially video streaming.
  27. 27. Copyright © NTT Communications Corporation. All rights reserved. Summary 27 • Current business in WebRTC • It is used for bunch of use cases • Media processing is needed • WebRTC GW – enabler for AI/IoT • Protocol GW for WebRTC • Can be orchestrated with media processing engines • Media processing with Serverless architecture • Flexibility, scalability and agility • Well isolated and security • Need to have more practice and management system.
  28. 28. Copyright © NTT Communications Corporation. All rights reserved. Thanks!! @komasshu kensakukomatsu

×