Technology supporting the release of HOME'S

Technology supporting
the release of HOME'S
Naoto Kishino (Software Engineer in Test at LIFULL)
April 19th, 2019 at Selenium Conf Tokyo

© LIFULL Co.,Ltd. 本書の無断転載、複製を固く禁じます。2
Introduction
Naoto Kishino
(木住野奈夫人)
Software Engineer in Test Group
(3 members)
Test Automation/CI/CD
Group vision :
自動化と高速化を進め、プロとして最高のDXを実現する
Create the best DX (Developer Experience) in the world.
@naotospace
NA070

Overview
We accelerated release frequency
from two times a week to four times a
week.
We achieved this goal by solving some
problems in the automated system testing
which is executed before release.

Agenda
1.About LIFULL HOME’S
2.About development process
3.Problems in our automated system testing
4.Solutions
5.Conclusion

About LIFULL HOME’S
Do you know LIFULL HOME’S?✋

● Information service about Real Estate
in Japan.
● Registration number’s No.1 in Japan

Development process

About development process
Team A Team B Team C
Multiple teams create
many new features at the same time.

Team B Team C
These new features become
one release package.
Team A

Test
Team B Team CTeam A
Release!
Teams do some tests for their function.
After passing all the tests, package will be release.

Release!
Test
Hundreds of new features release per month.
It was able to release on Tuesday and Thursday.
Monday Tuesday Wednesday Thursday Friday
◯ ◯

About development problem
Large-scale features can only released on Tuesday.
X (Delay😥) → → →
→ Release
Release!
Too Slow….

About development Team
Change the number of releasable day in a week
◯ ◯
◯ ◯ ◯ ◯

Problems in our
automated system testing

Problems in our automated system testing
Release!
• E2E(End-to-End) test using Selenium.
• Executed before release.
• To make sure no incident is increasing by degrades.
System testing

High cost!
We almost spent 4 person-day on per test.
Bottleneck!!!😥

Why bottleneck?
1. Too many test cases
2. Too flaky
3. Maintainability was too low

1. Too many test cases
● No test plan
● What we want to assure is not clear

2. Too flaky
● Test code were implemented
in complex processing
● Some cases include unsuitable step
in automated testing

● Complex test code
● Low readability

Solution 1
Recreate test plan

Solutions
1. Too many test cases -> Test plan
2. Too flaky
Solution 1 Recreate test plan

Define the scopes about
manual testing and automated testing
• Manual：Specification based
– New features and regression test
• Automated：Risk based
– Comprehensive regression test

All functions in LIFULL HOME’S
Manual
Testing
Automated
Testing
New
Feature

Manual
Testing
Automated
Testing
New
Feature

Prioritize test scopes
• Page views and page’s values on Google Analytics
– Business critical
• Incidents of the production

● Decreased to suitable number
○ 1000 cases -> 600 cases
● Defined the rule of adding test cases
○ Unsuitable test case prevented.

Solution 2
New testing framework “Bucky”

Solutions.
2. Too flaky -> Bucky
3. Maintainability was too low -> Bucky
Solution 2 New testing framework “Bucky”

$ bucky run -case case_1
Data
base
Save
test result
Execute test
TestC
ode
TestC
ode
Test
Case
Page
object
Element
LocaterElement
Location
Page
object
Make
test report
Bucky-core Bucky-management

Solutions.
2. Too flaky -> Retest

Improve retest function
$ bucky run -r 3 $ bucky rerun --job_id 1 -r 3

Solutions.
3. Maintainability was too low-> Test code in YAML

Test code in YAML

What’s YAML?
https://yaml.org/spec/1.2/spec.html
# YAML example
american:
- Boston Red Sox
- Detroit Tigers
- New York Yankees
national:
- New York Mets
- Chicago Cubs
- Atlanta Braves

- case_name: favorite_1
func: Save to favorite.
desc: Save Real Estate to favorite from detail page
procs:
- proc: Open Real Estate’s detail page.
exec:
operate: go
url: www.homes.co.jp/chintai/b-0000000000000/
- proc: Save to favorite.
exec:
operate: click
page: bukken_detail
part: favorite
- proc: Check whether target Real Estate is saved as
favorite.
exec:
verify: assert_exist_part
page: bukken_detail
part: complete_favorite

Test code in YAML
● Very simple
○ No programming skills dependence
○ No if and loop
○ Reduce complexity
-> Improved maintainability
-> Stabilization test execution

# Open Real Estate’s detail page.
self.driver.get('www.homes.co.jp/chintai/b-0000000000000/’)
# Save to favorite.
page = bukkenluist.Page(self.driver)
page.favorite.click
# Check whether target Real Estate is saved as favorite.
self.verify.is_true(self.is_exist_part(page, 'complete_favorite’))
def is_exist_part(page, part_name):
try:
getattr(page, part_name)
return True
except NoSuchElementException:
return False
except Exception as e:
raise Exception(e.message)
- case_name: favorite_1
func: Save to favorite.
desc: Save Real Estate to favorite from detail page
procs:
- proc: Open Real Estate’s detail page.
exec:
operate: go
url: www.homes.co.jp/chintai/b-0000000000000/
- proc: Save to favorite.
exec:
operate: click
page: bukken_detail
part: favorite
- proc: Check whether target Real Estate is saved
as favorite.
exec:
verify: assert_exist_part
page: bukken_detail
part: complete_favorite
Bucky-coreOld（Python)

Conclusion

Solutions.
3. Maintainability was too low-> Test code in YAML
Conclusion

Testing cost per day decreased.
Conclusion
0
5
10
15
20
25
30
35
2 days 4 days
32h (4 people * 8h)
6h (3 people * 2h)

The number of releasable days increase!!
Conclusion
◯ ◯
◯ ◯ ◯ ◯

Bucky have many useful functions!
We published as OSS.
Please take a look and have a try of Bucky!
https://github.com/lifull-dev/bucky-core
https://github.com/lifull-dev/bucky-management

Thank you for listening.
Have a good Selenium life.

Technology supporting the release of HOME'S

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Technology supporting the release of HOME'S

Similar to Technology supporting the release of HOME'S (20)

Recently uploaded

Recently uploaded (20)

Technology supporting the release of HOME'S