LT 2020.5.16(Sat)

1. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Test Automation Journey
Parallel Execution of E2E Test
May 16, 2020
第５回Seleniumユーザーコミュニティ勉強会
@woosyume

2. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
• Software Engineer
• Test automation TF lead
• Scrum Master
• At Raku*nRanking Frontend team
• Organizer
• Japan Selenium UC Kansai branch
• Korea Selenium UC (co-founder)
• Speaker
• SeleniumConf 2019
• Rakuten Technology Conference 2019
• Regional Scrum Gathering Tokyo 2020
• DevRel Earth 2020 (TBU)
Profile
@woosyume

3. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Test Automation Journey
Feel blue
with
manual test
Investigate
recursive
tests
Consider
which part
can be
automated
Build a
Strategy
Automate
tests
Migrate to
Shared
Environment
Optimizing
•Execution time
•Load
•Understandable
Reporting
Fully-
Automated
with high-
performance

4. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Problem
It takes a long time…
2Hours

5. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Current Workflow
DatabaseAPI
Test Automation
Application
Front Page
Create expected value dynamically
& Store it as in-memory cache
As post action
Commit
Set destination
And access

6. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
3 Ways to reduce execution time
• Distributing test environments to nodes
Edge
on Windows
Chrome
in iOS
Firefox
on Ubuntu
Safari
on MacOS
Hub and Nodes

7. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
3 Ways to reduce execution time
• Distributing test environments to multiple pods
Selenium
Code Image
Selenium
Hub Image
Browser
Image
Containers
Selenium Code
Container
Selenium Hub
Container
Browser
ContainerBrowser
Container

8. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
3 Ways to reduce execution time
• Executing tests with multiple threads
Chrome
in iOS
thread-1
thread-2
thread-3
thread-4

9. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Concept
• Run multiple threads maximizing performance of cores

10. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Prototype with plugin
• maven-surfire-plugin
d Before
After
440s
231s
50%↗

11. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Prototype with plugin
• The more I executes test cases, the more error occurred
Before
After
4520s
1809s

12. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Root Cause
• WebDriver is NOT ‘Thread-safe’
Cannot make sure that each driver instances works as an instance.
Launch a driver

13. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
How did I resolve the issue
• Initialize local variable for each thread
• ThreadLocal supports to limit a scope of variable

14. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
But I found another answer
• TestNG framework officially supports Selenium
• https://testng.org/doc/selenium.html
• JUnit just uses webdriver set from maven
• It seems more stable to use ‘multiple-thread function’ of TestNG than
plugin with forcible modification of the existing code
Easily migrate with Eclipse plugin
pom.xml
testng.xml

15. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Test Automation Journey
Feel blue
with
manual test
Investigate
recursive
tests
Consider
which part
can be
automated
Build a
Strategy
Automate
tests
Migrate to
Shared
Environment
Optimizing
•Execution time
•Load
•Understandable
Reporting
Fully-
Automated
with high-
performance

16. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Next Station of this journey
DatabaseAPI
Test Automation
Application
Front Page
Create expected value dynamically
& Store it as in-memory cache
As post action
Commit
Set destination
And access

17. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
もしこの旅行を始めたいけど、
テスト自動化の価値が分かってもらってなく
始められない方におすすめします。
https://www.slideshare.net/mavenant/selenium-effective-roi-for-test-automation

18. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Thankyoufor your listening