Techniques and Tools for Rich Internet Applications Testing

870 views

Published on

The User Interfaces of Rich Internet Applications (RIAs) present a richer functionality and enhanced usability than the ones of traditional Web applications which are obtained by means of a successful combination of heterogeneous technologies, frameworks, and communication models. Due to its increased complexity, dynamicity, and responsiveness, testing the user interfaces of an RIA is more complex than testing the user interfaces of a traditional Web application and requires that effective and efficient testing techniques are proposed and validated. In this paper we analyse the most critical open issues in RIA testing automation and propose a classification framework that characterizes existing RIA testing techniques from four different perspectives. Driven by this classification, we present a set of testing techniques that can be used for automatically and semi-automatically generating test cases, for executing them and evaluating their results. Some examples of applying the proposed techniques for testing real Ajax applications will also be shown in the paper.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
870
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Techniques and Tools for Rich Internet Applications Testing

  1. 1. Techniques and Tools for RichTechniques and Tools for RichInternet Applications TestingInternet Applications TestingDomenicoDomenico AmalfitanoAmalfitanoAnna RitaAnna Rita FasolinoFasolinoPorfirio TramontanaPorfirio TramontanaDipartimentoDipartimento di Informatica edi Informatica e SistemisticaSistemisticaUniversity of Naples Federico II, ItalyUniversity of Naples Federico II, Italy
  2. 2. 22Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010RIA interfaceRIA interface testingtesting RIAs represent the new generation of Web applications, providingricher, more interactive, dynamic and usable user interfaces thantraditional ones. AJAX (Asynchronous JavaScript and XML) is a set of technologies(JavaScript, XML, XMLHttpRequest objects) providing one of the mostdiffused approach for implementing RIAs. But the enhancements provided by RIAs have a price in terms ofinterface complexity and testability Finding processes and techniques for an efficientand effective testing of RIA interfaces is a verychallenging task
  3. 3. 33Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010From hypertexts toFrom hypertexts to RIAsRIAs Hypertexts consist of a set of static HTML pagesHypertexts consist of a set of static HTML pages Web 1.0 Applications (in particular Thin ClientWeb 1.0 Applications (in particular Thin ClientWeb Applications) are clientWeb Applications) are client--server applicationsserver applicationsin which the server side manages all thein which the server side manages all theapplication logicapplication logic InIn RIAsRIAs, a large portion of the application logic is, a large portion of the application logic ismanaged on the client side and an importantmanaged on the client side and an importantamount of code is devoted to the user interfaceamount of code is devoted to the user interfacemanagementmanagement
  4. 4. 44Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010RIA User InterfaceRIA User Interface It is implemented by Web pages composed by individualcomponents, which can be updated, deleted or added at run timeindependently. The manipulation of the page components is performed by an Ajaxengine written in JavaScript Its status changes due to Javascript event handlers elaborations. Event Handlers are triggered by user events or other external events(such as timeout events or asynchronous responses from the server). The RIA User Interface can be considered like adynamically changing event-driven softwaresystem (EDS).
  5. 5. 55Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010Some RIA Testing Open issuesSome RIA Testing Open issues How the artifacts to be tested can be modelled?How the artifacts to be tested can be modelled? Artifacts can be dynamically implemented and modified, so theArtifacts can be dynamically implemented and modified, so themodel cannot be statically definedmodel cannot be statically defined How test cases can be automatically generated?How test cases can be automatically generated? How oracles can be defined?How oracles can be defined? How test cases can be automatically replayed?How test cases can be automatically replayed? How oracles can be evaluated?How oracles can be evaluated? What kind of tools can supporting testing processes?What kind of tools can supporting testing processes? What kind of defects can be automatically tested?What kind of defects can be automatically tested? A set of models, techniques, tools and processes will beA set of models, techniques, tools and processes will bepresented and discussed in order to categorize possiblepresented and discussed in order to categorize possiblesolutions to this open issuessolutions to this open issues
  6. 6. 66Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010RIA InterfaceRIA Interface Event based testing techniques of GUI ofdesktop applications are based on models suchas: Event Flow Graphs. Event Interaction Graphs. Finite State Machines. Due to the similarities between RIAs and GUIs,all these models may be used to model thebehaviour of an RIA.
  7. 7. 77Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010• FSM represents all the elaboration states where the RIA receives any inputsolicitation by its user.• Each state of the RIA is described by the client Interface shown to the userat that interaction time.• Each client interface is characterized only by the sub-set of its activewidgets that are ‘clickable’ or, more in general, that have a registeredevent listener and a corresponding event handler.• Transitions are associated with user interactions that trigger the RIAmigration towards the new state.FSMState WidgetClient InterfaceTransitionEvent+Event Handler10..1*11 *+prev+next11*1Finite StateFinite State MachineMachine ModelModel• In a first approximation,different Client Interfacescorrespond to different States• The number of possiblestates may be unlimited !• A more useful model willconsider ‘equivalent’ ClientInterfaces as the same State
  8. 8. 88Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010FSM Reverse Engineering TechniqueFSM Reverse Engineering Technique In the past, we have proposed a reverse engineeringapproach for obtaining the FSM that: Is based on the analysis of the execution traces of theRIA; Solves the problem of FSM states and transitionsexplosion by heuristic criteria that aim at clusteringequivalent states and transitions of the FSM. In example, two client interfaces may be considered equivalentif they contain the same set of containers with the same IDattribute value and containing the same type of widgets1. D. Amalfitano, A. R. Fasolino, P. Tramontana, Reverse Engineering Finite State Machines from Rich Internet Applications,(WCRE 2008).2. D. Amalfitano, A. R. Fasolino, P. Tramontana, Experimenting a Reverse Engineering Technique for Modelling the Behaviour ofRich Internet Applications, (ICSM 2009).3. D. Amalfitano, A. R. Fasolino, P. Tramontana, An Iterative Approach for the Reverse Engineering of Rich Internet Applications,(IARIA ICIW 2010).
  9. 9. 99Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010Test case generation techniquesTest case generation techniques A possible approach for test cases generationA possible approach for test cases generationbased on dynamic analysis consists in thebased on dynamic analysis consists in thecollection of execution traces and in theircollection of execution traces and in theirtransformation to test casestransformation to test cases Execution traces may be collected:Execution traces may be collected: By a manual approach based on the analysis of theBy a manual approach based on the analysis of theinteractions with an RIA of real users or testers;interactions with an RIA of real users or testers; By an automatic approach based on CrawlingBy an automatic approach based on Crawlingtechniques;techniques; By mixing manual and automatic approachesBy mixing manual and automatic approaches
  10. 10. 1010Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010ManualManual vsvs Automatic Test Case GenerationAutomatic Test Case Generation Manual approaches can produce test cases that actually reproduceManual approaches can produce test cases that actually reproducessthe real user behaviours from execution tracesthe real user behaviours from execution traces The automatic approaches can be executed in a completelyThe automatic approaches can be executed in a completelyautomated wayautomated way Both approaches produces very huge test suitesBoth approaches produces very huge test suites But they should be the only practicable approaches in absence ofBut they should be the only practicable approaches in absence ofdocumentation and/or the server side source codedocumentation and/or the server side source code Techniques can be proposed for stopping the execution traces colTechniques can be proposed for stopping the execution traces collectionlectionand for reducing the test suite size on the basis of model/codeand for reducing the test suite size on the basis of model/code coveragecoveragecriteriacriteria State coverage, Transition coverage, Path coverageState coverage, Transition coverage, Path coverage LOC coverage, Function coverage, Script coverageLOC coverage, Function coverage, Script coverageD. Amalfitano, A. R. Fasolino and P. Tramontana, "Rich Internet Application Testing Using Execution Trace Data," ThirdInternational Conference on Software Testing, Verification, and Validation Workshops, 2010 (TESTBEDS 2010), IEEE CSPress, pp.274-283
  11. 11. 1111Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010Testing OracleTesting Oracle The automatic definition of oracles is possible only in some casThe automatic definition of oracles is possible only in some cases:es: Invariant TestingInvariant Testing The oracle can be expressed by means of queries on the interfaceThe oracle can be expressed by means of queries on the interface Crash TestingCrash Testing The oracle consists of the absence of raised exceptionsThe oracle consists of the absence of raised exceptions Regression TestingRegression Testing The oracle is the equivalence with the interface obtained by theThe oracle is the equivalence with the interface obtained by the originaloriginalversion of the mutated applicationversion of the mutated application Otherwise, oracles must be manually defined for each test caseOtherwise, oracles must be manually defined for each test case Objective oracles can be automatically evaluatedObjective oracles can be automatically evaluated E.g. oracles defined in terms ofE.g. oracles defined in terms of XPathXPath queriesqueries Subjective oracles must be manually evaluatedSubjective oracles must be manually evaluated E.g.: some usability or accessibility issues are usually manuallE.g.: some usability or accessibility issues are usually manuallyyevaluatedevaluated
  12. 12. 1212Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010Testing Automation Tools CategoriesTesting Automation Tools Categories CrawlerCrawler That automatically navigates RIA interfacesThat automatically navigates RIA interfaces CapturerCapturer That collects execution traces by observing the user navigationThat collects execution traces by observing the user navigation Test Case GeneratorTest Case Generator That extracts and encodes test cases from execution tracesThat extracts and encodes test cases from execution traces Test Suite ReducerTest Suite Reducer That reduces the size of a test suite trying to not reduce itsThat reduces the size of a test suite trying to not reduce itseffectivenesseffectiveness Test CaseTest Case ReplayerReplayer That automatically replays encoded test casesThat automatically replays encoded test cases Assertion GeneratorAssertion Generator That automatically generates oraclesThat automatically generates oracles Assertion VerifierAssertion Verifier That automatically verifies oracle assertionsThat automatically verifies oracle assertions
  13. 13. 1313Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010CReRIACReRIA Supports theSupports theuser duringuser duringthe executionthe executiontracestracescollectioncollection Produces theProduces theFSM on theFSM on thebasis of thebasis of thecollectedcollectedexecutionexecutiontraces and oftraces and ofthe chosenthe chosenclusteringclusteringheuristicsheuristicsAvailable from http://wpage.unina.it/ptramont/downloads.htm
  14. 14. 1414Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010CrawlRIACrawlRIA Explores the states of the interface of a RIA by triggeringExplores the states of the interface of a RIA by triggeringthe events related to the widgets of the interface, with athe events related to the widgets of the interface, with adepth first or a breadth first strategydepth first or a breadth first strategy Tree branches are cut when an interface equivalent to anotherTree branches are cut when an interface equivalent to anotheralready visited one is encounteredalready visited one is encountered Heuristic clustering criteria are used to evaluate the equivalenHeuristic clustering criteria are used to evaluate the equivalencecebetween the interfacesbetween the interfaces Produces a set of execution traces, all starting from theProduces a set of execution traces, all starting from thesame starting statesame starting stateCrawlRIACrawlRIA is currently under restructuring and it will be online soon!is currently under restructuring and it will be online soon!
  15. 15. 1515Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010TestRIATestRIA Transforms traces in executableTransforms traces in executabletest cases (as Seleniumtest cases (as Seleniumexecutable tests)executable tests) Reduces test suites accordingReduces test suites accordingto some FSM model coverageto some FSM model coveragecriteriacriteria Supports assertion generationSupports assertion generation Replays encoded test casesReplays encoded test casesD. Amalfitano, A. R. Fasolino and P. Tramontana, "Rich Internet Application Testing UsingExecution Trace Data," Third International Conference on Software Testing, Verification, andValidation Workshops, 2010 (TESTBEDS 2010), IEEE CS Press, pp.274-283Available from http://wpage.unina.it/ptramont/downloads.htm
  16. 16. 1616Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010DynaRIADynaRIA Dynamically capturesDynamically capturesinformation about theinformation about theexecution sessions of aexecution sessions of aRIARIA Produces metrics, staticProduces metrics, staticand dynamic views of theand dynamic views of theRIA interfacesRIA interfaces Monitors JS crashes,Monitors JS crashes,network exception andnetwork exception andother fault causesother fault causes Replays executionReplays executionsessionssessionsDomenico Amalfitano, Anna Rita Fasolino, Armando Polcaro, PorfirioTramontana, DynaRIA: a Tool for Ajax Web Application Comprehension, Tool demoat the 18th IEEE International Conference on Program Comprehension, ICPC 2010Available from http://wpage.unina.it/ptramont/downloads.htm
  17. 17. 1717Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010PuttingPutting allall togethertogether::CrashCrash TestingTesting ProcessProcessfunction initMenu() {var uls = document.getElementsByTagName("ul");for (i = 0; i < uls.length+1; i++) {if (uls[i].className == "menuList") {decorateMenu(uls[i]);}}}function renderTableListId(listId) {hideTodosLayers();document.forms.todoForms.listId.value = listId;todos.forceRenderTodos(listId, replyRenderTable);tracker(/ajax/renderTableListId);}function showAddTodoList() {hideTodosLayers();$("addNewListDiv").style.display="inline";document.forms.addNewListForm.name.focus();alert(hiAll);} ExamplesExamples ofof faultsfaults thatthat cancan bebe discovereddiscovered:: JSJS unreferencedunreferenced objectsobjects ArrayArray out ofout of boundbound
  18. 18. 1818Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010PuttingPutting allall togethertogether::UserUser VisibleVisible FaultFault TestingTesting ProcessProcess ExamplesExamples:: InvariantInvariant TestingTesting AccessibilityAccessibility testingtesting Error page detectionError page detection Application Invariant TestingApplication Invariant Testing OracleOracle testingtesting ……
  19. 19. 1919Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010PuttingPutting allall togethertogether::RegressionRegression TestingTesting ProcessProcess TheThe oracleoracle isis automaticallyautomatically defineddefined inin termsterms ofofequivalenceequivalence withwith the interfacethe interface obtainedobtained byby executingexecuting thethesamesame test case on thetest case on the originaloriginal RIARIA versionversion
  20. 20. 2020Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010ConclusionsConclusions The RIAThe RIA testingtesting isis aa relativelyrelatively newnew butbut terriblyterriblychallengingchallenging fieldfield WeWe havehave proposedproposed somesome processesprocesses supportingsupportingthethe testingtesting automationautomation in ain a clientclient sideside reversereverseengineeringengineering scenarioscenario withoutwithout no accessno access toto serverserverside code andside code and documentationdocumentation TheThe processesprocesses areare supportedsupported byby a set ofa set of toolstools thatthat weweareare developingdeveloping andand thatthat areare freelyfreely availableavailable onon mymywebsitewebsite at http://at http://wpage.unina.itwpage.unina.it//ptramontptramont
  21. 21. 2121Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010FutureFuture worksworks Execution of larger case studies and/or empirical studiesExecution of larger case studies and/or empirical studiesin order to evaluate effectiveness and efficiency ofin order to evaluate effectiveness and efficiency oftesting processes by varying:testing processes by varying: Number and type of usersNumber and type of users ApplicationsApplications Reduction techniquesReduction techniques The type of defects (by using a suitable fault model and faultThe type of defects (by using a suitable fault model and faultinjection techniques)injection techniques) Comparison with other emerging approaches/tools, suchComparison with other emerging approaches/tools, suchas the ones of the SERG group in Delft and of IRST inas the ones of the SERG group in Delft and of IRST inTrentoTrento
  22. 22. 2222Porfirio TramontanaPorfirio Tramontana –– WSE 2010WSE 2010 –– TimisoaraTimisoara –– 18/09/201018/09/2010

×