Gui path oriented test generation algorithms paper
GUI Path Oriented Test Generation Algorithms Izzat Alsmadi Department of computer science North Dakota state university firstname.lastname@example.org (performance testing), how does theABSTRACT system react if its environment does notTesting software manually is a labor behave as expected (robustness testing),intensive process. Efficient automated and how long can we rely on the correcttesting can significantly reduce the functioning of the system (reliabilityoverall cost of software development testing).and maintenance. Graphical User User interfaces have steadilyInterfaces (GUI’s) code has some grown more rich, more user interactivecharacteristics that distinguish it from and more sophisticated over time. Inthe rest of the project code. Generating many applications one of the majortest cases from the GUI code requires improvements that are suggested withdifferent algorithms from those usually the new releases is to improve the userapplied in test case generation. We interface.developed several GUI test generation Generating test cases can happenautomated algorithms that do not need from requirements, design or the actualany user involvement and that ensure GUI implementation. Although it istest adequacy in the generated test cases. expected that those three should beThe test cases are generated from an consistent and related, yet they haveXML GUI model or tree that represents different levels of abstraction.the GUI structure. This work contributed Requirements and design are usually of ato the goal of developing fully GUI test high level of abstraction to generateautomated framework. from them the test cases. On the other hand the task of generating the test casesGeneral Terms from the GUI implementation modelUser interface, Automatic test case will be delayed until we implement thegeneration. GUI, which is usually occurred in theKeywords late implementation. We should not have any problems in delaying GUI testingTest Automation, GUI Testing, Test CaseGeneration. giving the fact that a tool will automate the generation and executing process.1. INTRODUCTION We designed a tool in C# that uses Testing tries to answer the reflection to serialize the GUI controlfollowing questions(3): Does the system components or widgets. Certain controldo what it should do, or does its properties are selected to be serialized.behavior comply with its functional These properties are relevant to the userspecification (conformance testing), how interface. The application then uses thefast can the system perform its tasks XML file that is produced to build the GUI tree or the event flow graph and
generate the test cases. Generating the appropriate test cases for these paths.test cases takes into consideration the These techniques can further betree structure. Test cases are selected classified as static and dynamic. Staticwith the consideration of the techniques are often based on symboliceffectiveness of the selected test suit. We execution, whereas dynamic techniqueswill study the fault detection obtain the necessary data by executingeffectiveness of our test case selections. the program under test. Goal-oriented The algorithms developed to techniques identify test cases covering agenerate test cases from the GUI are selected goal such as a statement ornovels. The two factors that affect the branch, irrespective of the path taken.suggested algorithms were first Intelligent techniques of automated testgenerating a valid test scenario in which case generation rely on complexeach edge is a legal edge in the actual GUI computations to identify test cases. Themodel. The second factor is ensuring a real challenge to test generation is in thecertain level of effectiveness on the generation of test cases that are capablegenerated test scenarios. of detecting faults in the IUT. We will The next section introduces the list some of the works related to thisrelated work. Section 3 lists the goals of thisresearch and describes the work done toward paper. Goga(2) introduce an algorithmthose goals. Section 4 introduces in bases on probabilistic approach. Itsummary the developed GUI Auto tool. suggests combining the test generationSection 5 presents the conclusion and future and the test execution in one phase.work. Tretmans(3) studied test case generation algorithms for implementations that2. RELATED WORK communicate via inputs and outputs, Software testing is about based on specifications using Labelledchecking the correctness of the system Transition Systems (LTS). In MulSawand confirming that the implementation project (4), the team use 2conforms to the specifications. complementary frameworks, TestEraConformance testing checks whether a and Korat for specification based testblack box Implementation Under Test automation. To test a method, TestEra(IUT) behaves correctly with respect to and Korat automatically generate allits specification. The work in this paper non-isomorphic test cases from theis related to test case generation methods pre-condition and check itsalgorithms, automatic test case correctness using its post-condition as ageneration and GUI test case generation test oracle. There are several papersin software testing. Several approaches related to this project. We have a similarhave been proposed for test case approach that focus on GUI testing. Asgeneration, mainly random, path- explained earlier, one of the goals of ouroriented, goal-oriented and intelligent automatic generation of test scenarios isapproaches (5) and domain testing to produce non-isomorphic test(which includes equivalence scenarios. We also check the results ofpartitioning, boundary-value testing, and the tests through comparing the outputthe category-partition method) (7). Path- results with event tables generated fromoriented techniques generally use control the specification. Those event tables areflow information to identify a set of similar to the pre post condition eventpaths to be covered and generate the tables. Clay (6) presented an overview
for model based software testing using for another tool that is capable ofUML. Prior to test case generation, we producing small adequate test-sets thatdevelop an XML model tree that can successfully verify that anrepresents the actual GUI that is implementation of the specificationserialized from the implementation. Test produced is correct.cases are then generated from the XMLmodel. Turner and Robson  have In the specific area of GUI test casesuggested a new technique for the generation, Memon (14) has severalvalidation of OOPS which emphasizes papers about automatically generatingthe interaction between the features and test cases from the GUI using an AIthe object’s state. Each feature is planner, the process is not totallyconsidered as a mapping from its starting automatic and requires the user decisionor input states to its resultant or output to set current and goal states. The AIstates affected by any stimuli. Tse, Chan, planner will find the best way to reachand Chen (9) and (11) introduce normal the goal states giving the current state.forms for an axiom based test case Another issue with respect to thisselection strategy for Object oriented research is that it does not address theprograms and equivalent sequences of problem of the huge number of statesoperations as an integration approach for that a GUI in even small application canobject oriented test case generation. Orso have and hence may generate too manyand Silva (10) introduce some of the test cases. The idea of defining the GUIchallenges that Object Oriented state as the collection state of eachtechnologies added to the process of control and that the change of a singlesoftware testing. Rajanna (12) studies property in one control will lead to athe impact and usefulness of automated new state is valid but is the reason forsoftware testing tools during the producing the huge amount of possiblemaintenance phase of a software product GUI states. We considered in ourby citing the pragmatic experiences research another alternative definition ofgained from the maintenance of a a GUI state. By generate an XML treecritical, active, and very large that represent the GUI structure, we cancommercial software product as a case define the GUI state as embedded in thisstudy. It demonstrated that most of the tree. This means that if the tree structureerror patterns reported during the is changed, which is something that canmaintenance are due to the inadequate be automatically checked, then wetest coverage, which is often the consider this as a GUI state change.outcome of manual testing, by relating Although we generate this treethe error patterns and the capability of dynamically at run time and then anyvarious test data generators at detecting change in the GUI will be reflected inthem. Stanford paper (13) is an example the current tree, yet this definition can beof using formal methods in defining the helpful in certain cases where we wantspecifications through object to trigger some events ( like regressionspecification tool that check for some testing ) if the GUI state is changed.properties like correctness. It is hoped Mikkolainen (15) discusses some issuesthat the application produced by this related to GUI test automationproject should form the groundwork challenges. Alexander (16) and Haward present the concept of critical path
testing for GUI test case generation. 7. Beizer, Boris. Software Testing Techniques.They define the critical paths as those Second Edition. New York, Van Nostrand Reinhold, 1990.paths that have “repeated” edges or 8. Turner, C.D. and D.J. Robson. The State-event in many test cases. The approach based Testing of Object-Oriented Programs.utilizes earlier manually created test Proceedings of the 1993 IEEE Conference oncases through a captureplay back tool. Software Maintenance (CSM- 93), Montreal,Although this is expected to be an Quebec, Canada, Sep. 1993. 9. T.H. Tse, F.T. Chan, H.Y. Chen. An Axiom-effective way of defining critical paths, Based Test Case Selection Strategy for Object-yet it is not automatically calculated. As Oriented Programs. University of Hong Kong,an alternative to the need of defining Hong Kong. 94.critical paths from run time, we define in 10. Orso, Alessandro, and Sergio Silva. Openone algorithm static critical paths issues and research directions in Object Oriented testing. Italy. AQUIS98.through the use of metric weights. The 11. T.H. Tse, F.T. Chan, H.Y. Chen. In Blackmetric weight is calculated by counting and White: An Integrated Approach to Object-all the children- or grand children for a Oriented Program Testing. University of Hongcontrol. Other ways of defining critical Kong, Hong Kong. 96.paths is by measuring delay time during 12. Rajanna V. Automated Software Testing Tools and Their Impact on Softwareexecution, or by manually locating Maintenance- An Experience. Tata Consultancycritical paths from specification. From Services.the specification a critical path can be a 13. Stanford, Matthew. Object specification toolpath that is calling an external API, using VTL. Master dissertation. University ofsaving to or calling an external file. Sheffield. 2002. 14. Memon, Atef. Hierarchical GUI Test Case Generation Using Automated Planning. IEEE3. GOALS AND APPROACHES transactions on software engineering. 2001. vol 27. 15. Mikkolainen, Markus. Automated4.CONCLUSION AND FUTURE WORK Graphical User Interface Testing. 2006. www.cs.helsinki.fi/u/paakki/mikkolainen.pdf. 16. Alexander K, Ames and Haward Jie.5. REFERENCES Critical Paths for GUI Regression Testing.1. Pettichord, Bret. Homebrew test automation. www.cse.ucsc.edu/~sasha/proj/gui_testing.ThoughtWorks. Sep. 2004. pdfwww.io.com/~wazmo/ papers/homebrew_test_automation_200409.pdf.2. Goga, N. A probabilistic coverage for on-the-ytest generation algorithms. Jan. 2003.fmt.cs.utwente.nl/publications/files/398_covprob.ps.gz.3. Jan Tretmans: Test Generation with Inputs,Outputs, and Quiescence. TACAS 1996: 127-146.4. Software Design Group. MIT. ComputerScience and Artificial Intelligence Laboratory.2006. http://sdg.csail.mit.edu/index.html.5. Prasanna, M, S.N. Sivanandam R.Venkatesan.and R.Sundarrajan. A survey on automatic testcase generation. Academic Open InternetJournal. Vol. 15. 2005.6. Williams, Clay. Software testing and theUML. ISSRE99. 99. http://www.chillarege.com/fastabstracts/issre99/.