1.
"Extreme" Tes+ng of WebRTC Applica+ons
2015/11/4(Wed) @WebRTC Meetup Japan
iwashi86(Yoshimasa Iwase)
atakigawa(Akinori Takigawa)

2.
■Name
Yoshimasa Iwase / @iwashi86
■Work
NTT Communications,
DevOps Engineer of SkyWay
■Name
Akinori Takigawa / @kenkenkenken8
■Work
CTO of Nefrock Inc.

3.
Copyright © NTT Communica+ons Corpora+on. All rights reserved.
3
SkyWay = WebRTC plaUorm (CPaaS)

4.
TL; DR
• Tes+ng WebRTC plaUorm/app is super hard
WebRTCプラットフォーム/アプリのテストは超大変
• Let’s automate it! and we did it!
そこで自動化したく、やってみた！

5.
TL; DR
• Tes+ng WebRTC plaUorm/app is super hard
WebRTCプラットフォーム/アプリのテストは超大変
• Let’s automate it! and we did it!
そこで自動化したく、やってみた！

6.
Why so tough?
なぜ大変？

7.
Reason 1 : Manual opera+on (手動)
Video 1 Video2
① ② ③

8.
Reason 1+α : Tes+ng signaling is not enough
Video 1 Video2
Signaling Succeeded
Video s+ll black
シグナリング単体じゃ不十分

9.
Reason 2 : Various condi+ons (Clients)
+ iOS, Android, Na+ve …
条件多すぎ（クライアントだけで)

10.
Reason 2+α : Aggressive update of clients
like, gUM on HTTP just doesn’t work
アップデート早すぎ(クライアントだけで)
gUMはHTTPで動かなくなったり
6-8 weeks update!

11.
Reason3 : Various condi+ons (Network)
NAT-X
① Endpoint Independent Filtering X Endpoint Independent Mapping
… over 80 cases (NATタイプだけで発着を考えると80種類超)
条件多すぎ（ネットワークの種類も)

12.
Reason3 : Various condi+ons (Network)
NAT-X
NAT-Y
UDP blocked!
① Endpoint Independent Filtering X Endpoint Independent Mapping
… over 80 cases (NATタイプだけで発着を考えると80種類超)
条件多すぎ（ネットワークの種類も)

13.
Reason3 : Various condi+ons (Network)
NAT-X
NAT-Y
UDP blocked!
① Endpoint Independent Filtering X Endpoint Independent Mapping
… over 80 cases (NATタイプだけで発着を考えると80種類超)
条件多すぎ（ネットワークの種類も)
Just tes+ng in the same machine isn’t enough

14.
TL; DR
• Tes+ng WebRTC plaUorm/app is super hard
WebRTCプラットフォーム/アプリのテストは超大変
• Let’s automate it! and we did it!
そこで自動化したく、やってみた！

15.
Ubuntu with Xvn
Architecture Overview
Ubuntu with Xvn
NAT
instance
NAT
instance
Intranet1 Intranet2
Web Server
(hosts test
app page)
Virtual Internet
VPC
Network
Segment
TURN
server

16.
Browser1
DataChannel Tes+ng
Browser2

17.
MediaChannel Tes+ng
Browser1 Browser2
video
(gUM)
video
(gUM)

18.
Browser x Browser x NAT type x NAT type
・Browsers
・ Chrome Stable, Chrome Beta, Chrome Unstable
・ Firefox Stable (, Firefox Unstable)
・NAT types
・Theore+cally 9 types (RFC4787)
・some types not configurable with iptables
-> wrote it from scratch
・4 * 4 * 9 * 9 = 1296
uh…

19.
NAT types
hup://www.netmanias.com/en/?m=view&id=techdocs&no=6065

20.
Ubuntu with Xvn
Architecture Overview
Ubuntu with Xvn
NAT
instance
NAT
instance
Intranet1 Intranet2
Web Server
(hosts test
app page)
Virtual Internet
VPC
Network
Segment
TURN
server

21.
Ubuntu
with Xvn
Architecture Overview (Detailed ver.)
Intranet1 Intranet2
Web Server
(hosts test
app page)
Virtual Internet
VPC
Network
Segment
Ubuntu
with Xvn
Ubuntu
with Xvn
Ubuntu
with Xvn
Ubuntu
with Xvn
16 clients
NAT
Type
9
NAT
Type
8
NAT
Type
7
NAT
Type
6
NAT
Type
5
NAT
Type
4
NAT
Type
3
NAT
Type
2
NAT
Type
1
Ubuntu
with Xvn
Ubuntu
with Xvn
Ubuntu
with Xvn
Ubuntu
with Xvn
Ubuntu
with Xvn
16 clients
NAT
Type
9
NAT
Type
8
NAT
Type
7
NAT
Type
6
NAT
Type
5
NAT
Type
4
NAT
Type
3
NAT
Type
2
NAT
Type
1
TURN server

22.
Test result and Packet visualiza+on
see -> hup://status.skyway.io

23.
Thanks!!