The document presents a tech talk by Kensaku Komatsu on serverless architecture for media processing using WebRTC, highlighting its applications in online education and healthcare. It discusses the use of WebRTC gateway (gw) as an enabler for AI/IOT and server-side media processing, and describes various use cases including voice recognition and live streaming. Challenges and advantages of implementing serverless architecture in media processing are also outlined, emphasizing the need for effective management systems.

1. Copyright © NTT Communications Corporation. All rights reserved.
RTC Conference Korea 2018 on November 1st, 2018
Kensaku Komatsu
NTT Communications
Media Processing with
Serverless Architecture

2. Copyright © NTT Communications Corporation. All rights reserved.
Kensaku Komatsu
Technical manager of ECL SkyWay
Web application evangelist
at NTT Communications
in Silicon valley office
@komasshu
kensakukomatsu

3. Copyright © NTT Communications Corporation. All rights reserved.
Main Idea of the tech talk today
3
A practice of Serverless Architecture
for Media Processing using WebRTC.

4. Copyright © NTT Communications Corporation. All rights reserved.
AI / IoT

5. Copyright © NTT Communications Corporation. All rights reserved.
Agenda of the talk
5
• Current business in WebRTC
• WebRTC GW – enabler for AI/IoT
• Media processing with Serverless architecture

6. Copyright © NTT Communications Corporation. All rights reserved.
Agenda of the talk
6
• Current business in WebRTC
• WebRTC GW – enabler for AI/IoT
• Media processing with Serverless architecture

7. Copyright © NTT Communications Corporation. All rights reserved.
ECL SkyWay in nutshell
7
https://webrtc.ecl.ntt.com/en/

8. Copyright © NTT Communications Corporation. All rights reserved.
Service model
8
P2P / SFU
connectivity
Client SDK

9. Copyright © NTT Communications Corporation. All rights reserved.
Customer journey : rarejob ( online education )
9https://japan.cnet.com/article/35116616/

10. Copyright © NTT Communications Corporation. All rights reserved.
Use cases
10
• Online Education
• Online healthcare
• Field engineer support
• …

11. Copyright © NTT Communications Corporation. All rights reserved.
Wish from our customer
11
• Recording
• Voice recognition, translation
• Video detection, recognition
• Emotion detection
• Low latency live streaming
• VR / AR / Avatar
• …

12. Copyright © NTT Communications Corporation. All rights reserved.
• Recording
• Voice recognition, translation
• Video detection, recognition
• Emotion detection
• Avatar
• …
Question
12
How do we support these needs
from our customer?

13. Copyright © NTT Communications Corporation. All rights reserved.
Agenda of the talk
13
• Current business in WebRTC
• WebRTC GW – enabler for AI/IoT
• Media processing with Serverless architecture

14. Copyright © NTT Communications Corporation. All rights reserved.
Our answer
14
Use WebRTC GW
for server side media processing

15. Copyright © NTT Communications Corporation. All rights reserved.
What’s SkyWay WebRTC GW
15
• Protocol gateway of WebRTC
• Enabler for server side media processing
• Can get RTP media stream from client
• Controllable with REST API
• Easy to connect from skyway client
• Supported platform : Linux and Windows
WebRTC
GW
Processing
engineRTP

16. Copyright © NTT Communications Corporation. All rights reserved. 16
https://github.com/skyway/skyway-webrtc-gateway

17. Copyright © NTT Communications Corporation. All rights reserved.
Use case 1 : server side recording
17
• WebRTC GW … stream out media data as RTP
streaming
• Bind decoder / encoder when needed
• POST media data to cloud storage or else
WebRTC
GW
Decoder
/ EncoderRTP
Storage
POST

18. Copyright © NTT Communications Corporation. All rights reserved.
Use case 2 : voice recognition
18
• WebRTC GW … stream out media data as RTP
streaming
• Bind decoder to get raw PCM data stream
• POST voice data to Voice recognition PaaS
• POST recognition result to cloud database.
WebRTC
GW
Decoder
RTP
DB
POST
Voice
recognition

19. Copyright © NTT Communications Corporation. All rights reserved.
• WebRTC GW … stream out media data as RTP
streaming
• Bind decoder to get raw PCM data stream
• POST voice data to Voice recognition PaaS
• POST recognition result to cloud database.
Question
19
How do we manage these service
for each scenario?

20. Copyright © NTT Communications Corporation. All rights reserved.
Agenda of the talk
20
• Current business in WebRTC
• WebRTC GW – enabler for AI/IoT
• Media processing with Serverless architecture

21. Copyright © NTT Communications Corporation. All rights reserved.
Base idea to manage server side processing
21
• Apply micro-service model for each media
processing scenario.
• Agility, Flexibility and scalability
21
WebRTC
GW
Micro
Service
#1
RTP
Micro
Service #2
Micro
Service #3

22. Copyright © NTT Communications Corporation. All rights reserved.
Issues, we faced with
22
• We found that…
• Sometime, media processing engine makes
state issue.
• Scalability issue for each components
• Especially media encoder and recognition
process requires high resource usage.
• Have to allocate much resources for above
micro servers.

23. Copyright © NTT Communications Corporation. All rights reserved.
Serverless architecture
23
• AKA ”Function as a service”
• Event base execution model
• Each function will be executed and killed in
each session cycle
• Easy to get scalability
• No-need to care about long-term state issue
23
spawn

24. Copyright © NTT Communications Corporation. All rights reserved.
Our challenge
24
• Apply serverless architecture into server side
media processing scenario.
• Build serverless system from scratch on top of
docker container
• Current FaaS is focused to only RPC
GW Decoder recognition
GW Decoder recognition
API GW orchestrator
spawn

25. Copyright © NTT Communications Corporation. All rights reserved.
Demo
25

26. Copyright © NTT Communications Corporation. All rights reserved.
Pros and Cons
26
• Pros
• Flexibility, Scalability and Agility
• Do not need to care about execution time
than short lived functions
• Easy to re-use each components
• Each scenario is well isolated, e.g. security
perspective
• Cons
• Need to have consolidate monitoring
• Controlling each components while set
upping orchestration is difficult
• Several timing issue will happen, especially
video streaming.

27. Copyright © NTT Communications Corporation. All rights reserved.
Summary
27
• Current business in WebRTC
• It is used for bunch of use cases
• Media processing is needed
• WebRTC GW – enabler for AI/IoT
• Protocol GW for WebRTC
• Can be orchestrated with media processing
engines
• Media processing with Serverless architecture
• Flexibility, scalability and agility
• Well isolated and security
• Need to have more practice and
management system.

28. Copyright © NTT Communications Corporation. All rights reserved.
Thanks!!
@komasshu
kensakukomatsu