UI testing a procedure to check application’s user interface. Graphical User Interface (GUI) is the front-end of the software where the user interacts with software through it. This presentation consists of several methods used for user interface testing with their reviews.

1. User interface Testing
Presented By:
Priyanka Chauhan
M.tech (15303013)

2. Contents:
 Introduction
 Approaches for UI testing
 Literature Survey
◦ Methods used
◦ Approaches overview
 References

3. Introduction
 a procedure to check application’s user interface
 to identify whether application has a proper functional
 involves conducting a set of tasks and comparing the result of
same with the anticipated output and ability to repeat equal
set of tasks many times with various data inputs and equal
level of exactness
 Should be started at early stage as it accelerates
development, improves quality and diminishes risks towards
the end of the cycle
 infinite number of different input instances as there are many
users
 Graphical User Interface (GUI) is the front-end of the software
where the user interacts with software through it.

4. Approaches for UI testing
 Manual based
◦ Based on the domain and application
knowledge of the tester
 Automatic testing
 Capture and Replay
◦ Based on capture and replay of user
sessions
 Model-based testing
◦ Based on the execution of user
sessions selected from a model of the
GUI

5. Literature Survey
1. Test Automation for Graphical User Interfaces: A Review
 Model Based Testing
◦ Modeling GUIs: Finite State Machine
◦ Extracting a GUI Structure Automatically: GUI Ripping tool is used.
 Combinatorial Interaction Testing
◦ Selenium: Used to test web applications, which uses scripts to run tests on the
browser.
◦ Web Portal Test: an automation process which tests a web portal for all the missing
links, file type mismatches, unreachable files etc.
2. Combining Model-Based and Capture-Replay Testing
Techniques of Graphical User Interfaces
 Model-based and capture and replay techniques
◦ advantages of our approach are automated derivation, low maintainability, no code
needed
 Three phases for approach:
◦ Definition
◦ Test case creation and execution
◦ Analysis

6. • Approach Overview

7. 3. Rule-Based Exploratory Testing of Graphical User Interfaces
◦ An approach to GUI testing that combines aspects of manual exploratory testing with
rule-based test automation.
◦ uses short, automated rules to increase the bug detection capability of recorded
exploratory test sessions
 Rule based testing
◦ a form of automated testing which can simplify the verification and validation logic of a
test as well as reduce the chances that a test will break when a GUI is changed
 Exploratory testing
◦ an approach to testing GUIs manually that leverages a human tester’s ingenuity, but is
expensive to perform repeatedly
 Approach: Rule-Based Exploratory Testing (R-BET)
◦ LEET is a tool used to perform a pilot evaluation to determine whether R-BET can be
applied to a set of sample testing situations.
 Weakness of evaluations
◦ self-evaluations
◦ narrow number of test applications used
◦ low number of rules overall

8. Fig. Structure of R-BET as implemented by
LEET

9. 4. Pattern Based GUI testing for Mobile Applications
◦ assess the feasibility of using the Pattern Based GUI Testing approach, PBGT, to test
mobile applications
◦ model based testing approach
◦ increase systematization, reusability and diminish the effort in modeling and testing
◦ based on the concept of User Interface Test Patterns (UITP)
 Model-based testing
◦ multiple MBT approaches such as Spec#, VAN4GUIM, Event Flow Graphs (EFGs),
Labeled State Transition Systems (LSTSs) and Finite State Machines (FSMs)
◦ Many approaches do not generate test scripts, Most of the scripts have only one path and
the models do not allow cycles resulting in bigger models
◦ Many models used do not describe the testing goals in a higher level of abstraction
 PBGT approach
◦ aims to solve these issues by providing UITP (User Interface Test Patterns)
 UITP
◦ A UI Test Pattern describes a generic test strategy, formally defined by a set of test goals,
denoted as
< Goal; V;A;C;P >
still need of improvement as the gestures are different in both mobile and
web applications

10. 5. Functional and User Interface Model for Generating Test Cases
◦ Test cases generated from use cases models are focused on core functionality of
the system
◦ test cases generated from task models are used to user interface testing which
concerned with details of user interactions.
◦ Individually these models capture partial system behavior so we combined both of
them
 Approach
◦ Use case models and task models are transformed to FSM respectively.
◦ Approach formally integrates the model for use case models and task models.
◦ resulting integrated model is then used to generate test cases which capture
more complete and detailed user interactions and secondary system
interactions.
Fig. FSM for Buy Products Tasks

11. 6. Automatic Testing of Natural User Interfaces
 Natural user interfaces
◦ do not require the use of such artificial devices such as keyboard and
computer mouse
◦ Examples are touch screens, speech recognition, or gesture recognition
 Approach:
(a) Training pathway
(b) Testing pathway
Fig. Schematic overview of the proposed approach, showing the pathways for
training the motion synthesis system (top) and generating novel test data
(bottom).

12. 7. Graphical User Interface Testing Using Evolutionary Algorithms
 optimization theory and evolutionary algorithms (EA) concepts were
applied for GUI testing
 aim of the proposed methodology is to find the test case sequence(s)
which produce(s) the maximum amount of changes inside the GUI while
preserving all other constraints
 Approach
◦ method starts by randomly generating the initial set of test cases,
which is then improved through measuring and evaluating specific
fitness functions
◦ By using optimization theory, very good results were obtained for
accelerating the automatic test of large interfaces
◦ Resources and time saving by replacing manual testing with
automation testing
 The software application that we designed for testing user interfaces is
called EvoGuiTest
 implemented in JavaScript

13. 8. A Model Based Testing Approach for Mobile Device
◦ the interaction between man and machine have become more
complex
◦ an automated testing approach will be the best solution
 Approach
◦ using OCR for generating GUI model for mobile device
software regardless the target OS or programming languages,
which makes the GUI widgets recognition easier by
constructing windows transferring graph automatically.
◦ the realized GUI modeling algorithm (OCR ripper) is the initial
step of the entire modeling process, which needs additional
events information to build a relatively complete mode
◦ GUI ripper is feasible
◦ OCR algorithm is usually vulnerable and greatly impacted by
the GUI windows’ location, text fonts, background colors and
etc.
◦ we have to add new algorithm for image processing in order
to improve the accuracy of widgets recognition

14. Fig. GUI traversing and recognition via OCR

15. References
1. Abdulrahman Ahmed: “Test Automation for Graphical User Interfaces: A Review”,
IEEE, 2014
2. Vladimir Entin, Mathias Winder, Bo Zhang, Stephan Christmann: “Combining
Model-Based and Capture-Replay Testing Techniques of Graphical User
Interfaces”, IEEE Computer Society, Pg. 572-577, 2011
3. Theodore D. Hellmann and Frank Maurer: “Rule-Based Exploratory Testing of
Graphical User Interfaces”, IEEE Computer Society, Agile Conference, Pg. 107-
116, 2011
4. Pedro Costa, Miguel Nabuco, Ana C. R. Paiva: “Pattern Based GUI testing for
Mobile Applications”, IEEE Computer Society, 9th International Conference on
the Quality of Information and Communications Technology, Pg. 66-74, 2014
5. Zhu Bin, Wang Anbao: “Functional and User Interface Model for Generating Test
Cases”, IEEE Computer Society, ACIS 11th International Conference on
Computer and Information Science, Pg. 605-610, 2012
6. Chris J. Hunt · Guy J. Brown · Gordon Fraser: “Automatic Testing of Natural
User Interfaces”, IEEE International Conference on Software Testing,
Verification, and Validation, Pg. 123-132, 2014
7. Genţiana Ioana Laţiu, Octavian Creţ, Lucia Văcariu: “Graphical User Interface
Testing Using Evolutionary Algorithms”, IEEE, 2011
8. Yumei Wu, Zhifang Liu: “A Model Based Testing Approach for Mobile Device”,
IEEE, 2012 International Conference on Industrial Control and Electronics
Engineering, 2012

16.  [9] Samer Al-Zain , Derar Eleyan , Joy Garfield: “Automated User Interface Testing
for Web
 Applications and TestComplete”, CUBE '12 Proceedings of the CUBE International
Information Technology Conference, ACM, 2012
 [10] Young-Min Baek, Doo-Hwan Bae: “Automated Model-Based Android GUI
Testing using Multi-level GUI Comparison Criteria”, Proceedings of the 31st
IEEE/ACM International Conference on Automated Software Engineering, ACM,
2016
 [11] Domenico Amalfitano, Nicola Amatucci, Anna Rita Fasolino, Porfirio
Tramontana: “A Conceptual Framework for the Comparison of Fully Automated
GUI Testing Techniques”, 30th IEEE/ACM International Conference on Automated
Software Engineering Workshop, 2015
 [12] Eman M. Saleh , Omar Al Sheik Salem: “A Model-Driven Engineering
Transition-Based GUI Testing Technique”, International Conference on
Computational Science and Computational Intelligence, IEEE, 2015
 [13] Haowen Zhu, Xiaojun Ye, Xiaojun Zhang, Ke Shen: “A Context-aware
Approach for Dynamic GUI Testing of Android Applications”, IEEE 39th Annual
International Computers, Software & Applications Conference, 2015
 [14] Hsiang-Lin Wen, Chia-Hui Lin, Tzong-Han Hsieh, and Cheng-Zen Yang:
“PATS: A Parallel GUI Testing Framework for Android Applications”, 39th Annual
International Computers, Software & Applications Conference, IEEE, 2015
 [15] Pekka Aho, Matias Suarez, Atif Memon, Teemu Kanstrén: “Making GUI
Testing Practical: Bridging the Gaps”, 12th International Conference on
Information Technology - New Generations, IEEE, 2015

17.  [16] Ali Darvish, Carl K. Chang: “Black-box Test Data Generation for GUI
Testing”, 14th International Conference on Quality Software, IEEE, 2014
 [17] Abdul Rauf, Mohammad N. Alanazi: “Using Artificial Intelligence to
Automatically Test GUI”, 9th International Conference on Computer Science &
Education (ICCSE 2014), IEEE, 2014
 [18] Rodrigo M. L. M. Moreira, Ana C. R. Paiva, Atif Memon: “A Pattern-Based
Approach for GUI Modeling and Testing”, 24th International Symposium on
Software Reliability Engineering (ISSRE), IEEE, 2013
 [19] G. Appasami, V. Mohan Raj, Md.Zubair: “Automated Testing of Silverlight
and Moonlight Applications”, Proceedings of the National Conference on
Innovations in Emerging Technology, IEEE, 2011
 [20] Sebastian Bauersfeld: “GUIdiff – A Regression Testing Tool for Graphical
User Interfaces”, Sixth International Conference on Software Testing, Verification
and Validation, IEEE, 2013