This document discusses using machine learning to help automate UI tests. It describes collecting input forms from the internet and labeling them to train a machine learning model. The model can then infer the topic of new input forms based on attributes like IDs and classes. This allows the model to automatically fill in test forms, handling most of the implementation details of UI tests. A prototype was created to demonstrate labeling sample forms and having the model pre-fill responses. The document suggests trying this approach for UI tests as it is challenging and fun.

1. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Introduce Machine
Learning Into UI Tests
2 0
20
0 0 21
SWET Group
@toshiya_komoda

2. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Bio
! LOF U 5LJL
⁃ 0
⁃ 7 0
! 1C / AL % IP %
⁃ / JL IC JC ALJ KU

3. Copyright © DeNA Co.,Ltd. All Rights Reserved.
5 years
700 UI tests
60000 API tests

4. Copyright © DeNA Co.,Ltd. All Rights Reserved.
ObservaBon
! /9 COPO NC C OU PL I K J KP K
! COPO NC KLP

5. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Can machine learning make
UI Tests easier

6. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Input Topic IdenBficaBon with Machine Learning
! LA O LK CPCAP K K II K K PCTP DLNJO
⁃ 8W AL NOC A KKLP F K IC II NL ICJO P LKAC%
! 9NC L O LN
⁃ O K :CJ KP A : J I N PU DLN K P L A CKP A P LK K
0N SI K OC C / I A P LK COP K
4 K C 6 K NK K
FPP O. NT %LN O ( ,% )-

7. Copyright © DeNA Co.,Ltd. All Rights Reserved.
How it works !

8. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Training Machine Learning Model
Training Data
Internet
Model
Collect input forms
and label them
Learning algorithm

9. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Text Forms
Text Form A
Text Form B

10. Copyright © DeNA Co.,Ltd. All Rights Reserved.
HTML of text forms
Text Form A
Text Form B
<input class="input-text email emailaddress required" type="text”
id="dwfrm_profile_customer_email” name="dwfrm_profile_customer_email"
value=”” minlength="6” maxlength="255" autocomplete="off”
ariarequired="true">
<input class="input-text firstname validaddress required" type="text"
id="dwfrm_profile_customer_firstname”
name="dwfrm_profile_customer_firstname" value="" minlength="0"
maxlength="255" aria-required="true">

11. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Form topics Inferred by the machine learning model
Model
Text Form A
Text Form B
<input class="input-text email emailaddress required" type="text”
id="dwfrm_profile_customer_email” name="dwfrm_profile_customer_email"
value=”” minlength="6” maxlength="255" autocomplete="off”
ariarequired="true">
<input class="input-text firstname validaddress required" type="text"
id="dwfrm_profile_customer_firstname”
name="dwfrm_profile_customer_firstname" value="" minlength="0"
maxlength="255" aria-required="true">
EMAIL
User First Name
Inference of form topics

12. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Filling in text forms according to the inferred topics
Text Form A
Text Form B
All the details of test implementa[on are
handled by machine learning model

13. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Prototype
! PF
FPP O. PF %ALJ PLOF U OCJ KP A OCICAPLN
! CT CN JCKP I OP C
⁃ KLP KPC N PC S PF KU PCOP DN JCSLN
⁃ J AF KC IC NK K JL CI K 9UPFLK /9 OCN CN
! KDCNCKAC AA N AU
⁃ ,
• LN OCN NC OPN P LK C 4 KCOC C : PC
⁃ NLLJ DLN J NL CJCKP

14. Copyright © DeNA Co.,Ltd. All Rights Reserved.
Why don’t you try
machine learning in UI tests ?
It’s challenging and fun!