The slide for SF WebRTC Meetup on June 15th, 2016. Brief introduction about our auto test framework for exponential order of test pattern for WebRTC testing.

1. FULL MATRIX AUTO
TEST FRAMEWORK
FOR
S F W E B R T C M E E T U P , J U N E 1 5 T H , 2 0 1 6
K E N S A K U K O M A T S U @ N T T C O M M U N I C A T I O N S

2. INTRODUCE TO MYSELF
• Who am I ?
– Kensaku Komatsu (call me Ken)
– Working at NTT Communications in Palo Alto office.
• Position
– Technical manager of SkyWay
– Web Application Evangelist of NTT communications
• Social
– Google Developer Expert (HTML5)
– Chairman of WebRTC conference in Japan

3. ACTIVITIES
https://skyway.io/ https://box.skyway.io/

4. MAIN IDEA FOR TODAY
Testing!!

5. GENERIC TESTING FOR WEB APP
Order of N

6. WEBRTC TESTING …..

7. WEBRTC TESTING …..
NAT type
combination
?
PROXY ?
NW
condition?
Devices?

8. n
https://www.flickr.com/photos/58782395@N0

9. Need
Framework !!
https://www.flickr.com/photos/janti

10. LAST YEAR: BUILT TESTING ENV.

11. LAST YEAR: BUILT TESTING ENV.

12. LAST YEAR : DASH BOARD
http://status.skyway.io/

13. LAST YEAR : IMPLEMENT EACH NAT
FROM SCRATCH

14. ISSUES
• Specific for our system.
– On top of our cloud system : Cloudn
– Use our specific servers : SkyWay only
• Every components were deeply bounded together

15. Make it
Open source !!

16. AUTO TEST FRAMEWORK
• Framework has 4 phases
–Bootstrap
–Configure
–Orchestrate / server_ctl
–Orchestrate / job_ctl

17. AUTO TEST FRAMEWORK
:: BOOTSTRAP
Built VMs
on top of AWS-VPC
via CloudFormation

18. AUTO TEST FRAMEWORK
:: CONFIGURE
Install each components
Using Ansible
• Reference signalling srv
• Coturn
• Sample Web apps
• NAT
• Chrome & FF
• …

19. AUTO TEST FRAMEWORK
:: ORCHESTRATE/SERVER_CTL
Execute server programs.

20. AUTO TEST FRAMEWORK
:: ORCHESTRATE/JOB_CTL
Run Selenium for all
patterns.

21. HOW TO ADOPT TO EACH PROJECT?
• Framework has 4 phases
–Bootstrap
–Configure
–Orchestrate / server_ctl
–Orchestrate / job_ctl
Update each
script and
configuration
files.

22. CURRENT STATUS
http://52.196.173.127/
Caused by ….
Web Driver mismatch?
Due to resource limitation?
(t2.micro)

23. RELEASE PLAN
Will be open-sourced in fall or winter.
Now private
repo.

24. THANKS!
@komasshu
/kensaku.komatsu
/in/kensaku-komatsu-6528333b
Slideshare: /KensakuKOMATSU
/eastandwest