The document presents the full matrix auto test framework for WebRTC and web applications, introduced by Kensaku Komatsu from NTT Communications. It outlines the framework's four phases (bootstrap, configure, orchestrate/server_ctl, and orchestrate/job_ctl) for automated testing in a cloud environment, and addresses the issues faced during implementation. The framework is planned to be open-sourced in fall or winter after currently being in a private repository.

1. FULL MATRIX AUTO
TEST FRAMEWORK
FOR
S F W E B R T C M E E T U P , J U N E 1 5 T H , 2 0 1 6
K E N S A K U K O M A T S U @ N T T C O M M U N I C A T I O N S

2. INTRODUCE TO MYSELF
• Who am I ?
– Kensaku Komatsu (call me Ken)
– Working at NTT Communications in Palo Alto office.
• Position
– Technical manager of SkyWay
– Web Application Evangelist of NTT communications
• Social
– Google Developer Expert (HTML5)
– Chairman of WebRTC conference in Japan

3. ACTIVITIES
https://skyway.io/ https://box.skyway.io/

4. MAIN IDEA FOR TODAY
Testing!!

5. GENERIC TESTING FOR WEB APP
Order of N

6. WEBRTC TESTING …..

7. WEBRTC TESTING …..
NAT type
combination
?
PROXY ?
NW
condition?
Devices?

8. n
https://www.flickr.com/photos/58782395@N0

9. Need
Framework !!
https://www.flickr.com/photos/janti

10. LAST YEAR: BUILT TESTING ENV.

11. LAST YEAR: BUILT TESTING ENV.

12. LAST YEAR : DASH BOARD
http://status.skyway.io/

13. LAST YEAR : IMPLEMENT EACH NAT
FROM SCRATCH

14. ISSUES
• Specific for our system.
– On top of our cloud system : Cloudn
– Use our specific servers : SkyWay only
• Every components were deeply bounded together

15. Make it
Open source !!

16. AUTO TEST FRAMEWORK
• Framework has 4 phases
–Bootstrap
–Configure
–Orchestrate / server_ctl
–Orchestrate / job_ctl

17. AUTO TEST FRAMEWORK
:: BOOTSTRAP
Built VMs
on top of AWS-VPC
via CloudFormation

18. AUTO TEST FRAMEWORK
:: CONFIGURE
Install each components
Using Ansible
• Reference signalling srv
• Coturn
• Sample Web apps
• NAT
• Chrome & FF
• …

19. AUTO TEST FRAMEWORK
:: ORCHESTRATE/SERVER_CTL
Execute server programs.

20. AUTO TEST FRAMEWORK
:: ORCHESTRATE/JOB_CTL
Run Selenium for all
patterns.

21. HOW TO ADOPT TO EACH PROJECT?
• Framework has 4 phases
–Bootstrap
–Configure
–Orchestrate / server_ctl
–Orchestrate / job_ctl
Update each
script and
configuration
files.

22. CURRENT STATUS
http://52.196.173.127/
Caused by ….
Web Driver mismatch?
Due to resource limitation?
(t2.micro)

23. RELEASE PLAN
Will be open-sourced in fall or winter.
Now private
repo.

24. THANKS!
@komasshu
/kensaku.komatsu
/in/kensaku-komatsu-6528333b
Slideshare: /KensakuKOMATSU
/eastandwest