GUI Tetsing

919 views

Published on

  • Be the first to comment

GUI Tetsing

  1. 1. Csci 565 Spring 2009 H.Reza
  2. 2. <ul><li>Event-based systems </li></ul><ul><li>GUI Testing </li></ul><ul><ul><li>Concepts, strategies, etc </li></ul></ul><ul><li>The Currency Conversion Program </li></ul><ul><ul><li>To show unit testing for GUI </li></ul></ul><ul><ul><li>To show integration testing for GUI </li></ul></ul><ul><ul><li>To show system testing for GUI </li></ul></ul>
  3. 3. <ul><li>What is GUI? </li></ul><ul><ul><li>Universal client alternative to traditional forms/commands based user interfaces that accepts as input user-generated and/or system-generated events from a set of finite events and produces predicable graphical outputs </li></ul></ul>
  4. 4. <ul><li>GUIs are organized in hierarchical fashion and Composed of objects: </li></ul><ul><ul><li>Objects has a set of fixed properties having discrete values </li></ul></ul><ul><ul><ul><li>Menu </li></ul></ul></ul><ul><ul><ul><ul><li>provide almost continuous availability of the various features of the systems; </li></ul></ul></ul></ul><ul><ul><ul><li>Buttons /keyboard shortcuts </li></ul></ul></ul><ul><ul><ul><ul><li>enable the user to navigate and access the various functions of their application. </li></ul></ul></ul></ul><ul><ul><ul><li>User interacts and manipulate with objects using events (or sequence of events) </li></ul></ul></ul><ul><ul><ul><li>Windows allow multiple windows to be displayed at the same time </li></ul></ul></ul><ul><ul><ul><ul><li>Windows provides functionality using fields in which text or numeric data can be entered </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Windows operate in two mode: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Modal (restrict the user's foucs and interactions to one window) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Modalless ( does not restrict monopolozes the GUI interactions) </li></ul></ul></ul></ul></ul>
  5. 5. <ul><li>The GUI events can be classified as </li></ul><ul><ul><li>Restricted-focus events: open modal windows </li></ul></ul><ul><ul><ul><li>E.g. Set Language </li></ul></ul></ul><ul><ul><li>Unrestricted-focus events open modeless windows </li></ul></ul><ul><ul><ul><li>E.g., Replace in MS World </li></ul></ul></ul><ul><ul><li>Termination Events : close modal windows </li></ul></ul><ul><ul><ul><li>E.g., OK and Cancel </li></ul></ul></ul><ul><ul><li>System- interaction events: interact with underlying software to perform some action </li></ul></ul><ul><ul><ul><li>E.g., Copy event used for coping objects to the clipboard </li></ul></ul></ul>
  6. 6. <ul><li>GUI testing issues </li></ul><ul><ul><li>The event-driven nature of GUIs </li></ul></ul><ul><ul><ul><li>presents the first serious testing difficulty because users can click on any pixel on the screen and there so many of them </li></ul></ul></ul><ul><ul><li>Unsolicited events </li></ul></ul><ul><ul><ul><li>E.g., when the printer is out of papers </li></ul></ul></ul><ul><ul><li>The infinite sequence of events (or permutations of events) </li></ul></ul><ul><ul><li>GUI can be represented by Object oriented paradigm </li></ul></ul><ul><ul><li>Hidden synchronization and dependencies (objects in the same windows and different windows) </li></ul></ul><ul><ul><ul><li>E.g., if a check box is set to true, a text box intended to accept a numeric value elsewhere in the window may be made inactive or invisible. </li></ul></ul></ul><ul><ul><li>Many events in/Many events out </li></ul></ul><ul><ul><li>Testability (difficulty to specify, to execute and to analyze test cases) </li></ul></ul><ul><ul><ul><li>event-based system +hidden interactions </li></ul></ul></ul><ul><ul><ul><li>Coverage criteria??? (memon et al uses events, and events sequences) </li></ul></ul></ul><ul><ul><li>Suffers from some of problems related to COTs (or precompiled components) </li></ul></ul>
  7. 7. <ul><li>There are serious legal issues related to both design and testing of GUI </li></ul><ul><ul><li>Legal issues </li></ul></ul><ul><ul><ul><li>Privacy/security </li></ul></ul></ul><ul><ul><ul><ul><li>E.g., access to the privileged/stored data </li></ul></ul></ul></ul><ul><ul><ul><li>Safety/ reliability </li></ul></ul></ul><ul><ul><ul><ul><li>E.g., GUI for Safety critical systems can affect life-or-death decisions (functionality and ease of use) </li></ul></ul></ul></ul><ul><ul><ul><li>Accessibility </li></ul></ul></ul><ul><ul><ul><ul><li>Equal access for disabled users </li></ul></ul></ul></ul>
  8. 8. <ul><li>Focus on errors to reduce the scope of tests </li></ul><ul><li>Separation of concerns (divide and conquer) </li></ul><ul><li>Use Test design techniques when it is applicable (e.g., blackbox) </li></ul>
  9. 9. <ul><li>GUI errors include </li></ul><ul><ul><li>Data validation </li></ul></ul><ul><ul><li>Incorrect field defaults </li></ul></ul><ul><ul><li>Mishandling of server process failures </li></ul></ul><ul><ul><li>Mandatory fields, not mandatory </li></ul></ul><ul><ul><li>Wrong fields retrieved by queries </li></ul></ul><ul><ul><li>Incorrect search criteria </li></ul></ul><ul><ul><li>Field order </li></ul></ul><ul><ul><li>Multiple database rows returned, single row expected </li></ul></ul><ul><ul><li>Currency of data on screens </li></ul></ul><ul><ul><li>Window object/DB field correspondence </li></ul></ul><ul><ul><li>Correct window modality? </li></ul></ul><ul><ul><li>Window system commands not available/don’t work </li></ul></ul><ul><ul><li>Control state alignment with state of data in window? </li></ul></ul><ul><ul><li>Focus on objects needing it? </li></ul></ul><ul><ul><li>Menu options align with state of data or application mode? </li></ul></ul><ul><ul><li>Action of menu commands aligns with state of data in window </li></ul></ul><ul><ul><li>Synchronization of window object content </li></ul></ul><ul><ul><li>State of controls aligns with state of data in window? </li></ul></ul>
  10. 10. <ul><li>Testing Strategies include </li></ul><ul><li>Functionality </li></ul><ul><ul><li>Unit testing (Buttons) </li></ul></ul><ul><ul><li>Integrations Testing (synchronizations) </li></ul></ul><ul><ul><li>System Testing (functional behaviors according to requirements or functional testing) </li></ul></ul><ul><li>Non-Functionality(Quality) </li></ul><ul><ul><li>Usability testing </li></ul></ul><ul><ul><li>Security testing </li></ul></ul><ul><ul><li>… </li></ul></ul>
  11. 11. <ul><li>Model-Based Approach to test GUI requires: </li></ul><ul><ul><li>Identify components and Events </li></ul></ul><ul><ul><li>Select a hierarchal formal model (e.g., EFSM) to represent GUI elements (components) </li></ul></ul><ul><ul><li>Intra vs. inter components </li></ul></ul><ul><ul><li>Use model to generate threats or event sequences of various lengths corresponding to visible & observable system outputs </li></ul></ul><ul><ul><ul><li>Total number of events sequences increases when the length increase </li></ul></ul></ul><ul><ul><ul><li>It might impractical to test for ALL possible event sequences </li></ul></ul></ul><ul><ul><ul><ul><li>be selective? </li></ul></ul></ul></ul>
  12. 12. <ul><li>Unit and Integration testing minimally needed </li></ul><ul><li>Main focus is on System Testing </li></ul><ul><ul><li>Model? </li></ul></ul><ul><ul><ul><li>Event-driven Petri nets (EDPNs) </li></ul></ul></ul><ul><ul><ul><li>Testing threads </li></ul></ul></ul>
  13. 13. <ul><li>Event driven </li></ul><ul><li>Unit “ button level ” testing </li></ul><ul><li>Not much integration testing </li></ul><ul><li>System level testing: Models </li></ul><ul><ul><li>UML Behavioral models </li></ul></ul><ul><ul><li>StateCharts </li></ul></ul><ul><ul><li>Finite State Machines </li></ul></ul>
  14. 15. <ul><li>Starts testing (functional and structural) of user-supplied code for buttons </li></ul><ul><ul><li>Compute </li></ul></ul><ul><ul><ul><li>Valid and invalid data (e.g., non-numerical, etc) </li></ul></ul></ul><ul><ul><li>Clear </li></ul></ul><ul><ul><li>Quit </li></ul></ul><ul><li>Two approaches </li></ul><ul><ul><li>Using Drivers (good) </li></ul></ul><ul><ul><li>Using GUI (system level for input/output events) </li></ul></ul>
  15. 16. <ul><li>Not too much of Integration testing </li></ul><ul><li>Three possible options: </li></ul><ul><ul><li>Focus on all logic in one place and use the value of the option buttons as conditions in IF tests </li></ul></ul><ul><ul><li>Use OO approach using methods invocation to exchange the rate value </li></ul></ul><ul><ul><li>Use visual Basic </li></ul></ul><ul><ul><ul><li>Event procedures for each option button uses constant value for specific exchange rate </li></ul></ul></ul><ul><ul><ul><li>(e.g. Private Sub OptEU_click () exchange =1.35 End sub) </li></ul></ul></ul>
  16. 17. <ul><li>Heavily usage of System testing </li></ul><ul><li>Need event-driven model </li></ul><ul><ul><li>Event-Driven PN (EDPN) to specify and to generate scenarios/threads </li></ul></ul><ul><ul><li>Tables 19.1 and Table 19.2 </li></ul></ul>
  17. 18. <ul><li>The process to generate threads (positive/negative scenarios) include </li></ul><ul><ul><li>Map the elements of CCP to EDPN </li></ul></ul><ul><ul><ul><li>Map input/out events to input /output events of EDPN </li></ul></ul></ul><ul><ul><ul><li>Map Data to EDPN data places </li></ul></ul></ul><ul><ul><ul><li>Map each Action to the transition in EDPN </li></ul></ul></ul><ul><ul><li>Create EDPN for each ASF (tasks) </li></ul></ul><ul><ul><li>Compose the EDPNs </li></ul></ul><ul><ul><ul><li>Use ASF graph </li></ul></ul></ul><ul><ul><li>Generate a subset (or suitable? ) of threads </li></ul></ul>
  18. 19. <ul><li>How many threads are needed? </li></ul><ul><ul><li>Coverage criteria? </li></ul></ul><ul><li>Reasonable coverage includes </li></ul><ul><ul><li>Test ALL port inputs </li></ul></ul><ul><ul><li>Test All port outputs </li></ul></ul><ul><ul><li>Test All ASF (tasks or requirements) </li></ul></ul><ul><ul><ul><li>Difficult to test each ASF </li></ul></ul></ul><ul><ul><li>Test Threats </li></ul></ul><ul><ul><ul><li>There are too many </li></ul></ul></ul><ul><ul><ul><li>Need to be selective </li></ul></ul></ul>
  19. 20. <ul><li>Definition: EDPN is tripartite-directed graph (P, D, S, In, Out) </li></ul><ul><ul><li>P: a set of port events </li></ul></ul><ul><ul><li>D: a set of data places </li></ul></ul><ul><ul><li>S: a set of transitions </li></ul></ul><ul><ul><li>In: a set of ordered Paris from (P  D)  S </li></ul></ul><ul><ul><li>Out: a set of ordered pairs from S  (P  D) </li></ul></ul><ul><li>P, D, S correspond to </li></ul><ul><ul><li>Data, Input/output Events, Action, </li></ul></ul>
  20. 22. Represent the snap shot of the simulating EDPN Tuple (p4, d2, p12, p17,p21) description m1 (1,0,0,0,0) S4 enabled; s4 fired m2 (0,1,1,1,1) Nothing enabled m3 .. mn
  21. 23. <ul><li>Port Input Events </li></ul><ul><ul><li>P1: Enter US Dollar Amount </li></ul></ul><ul><ul><li>P2: Click on Brazil </li></ul></ul><ul><ul><li>P3: Click on Canada </li></ul></ul><ul><ul><li>P4: Click on European Union </li></ul></ul><ul><ul><li>P5: Click on Japan </li></ul></ul><ul><ul><li>P6: Click on Compute Button </li></ul></ul><ul><ul><li>P7: Click on Clear Button </li></ul></ul><ul><ul><li>P8: Click on Quit Button </li></ul></ul>
  22. 24. <ul><li>Output Events </li></ul><ul><ul><li>P9: Display U.S. dollar amount </li></ul></ul><ul><ul><li>P10: Display Brazilian Reals </li></ul></ul><ul><ul><li>P11: Display Canadian dollars </li></ul></ul><ul><ul><li>P12 :Display European Community Euros </li></ul></ul><ul><ul><li>P13: Display Japanese Yen </li></ul></ul><ul><ul><li>P14: Display ellipsis </li></ul></ul><ul><ul><li>P15: Indicate Brazil </li></ul></ul><ul><ul><li>P16: Indicate Canada </li></ul></ul><ul><ul><li>P17: Indicate European Community </li></ul></ul><ul><ul><li>P18: Indicate Japan </li></ul></ul>
  23. 25. <ul><li>Output Event </li></ul><ul><ul><li>P19: Reset selected country </li></ul></ul><ul><ul><li>P20: Reset Brazil </li></ul></ul><ul><ul><li>P21: Reset Canada </li></ul></ul><ul><ul><li>P22: Reset European Community </li></ul></ul><ul><ul><li>P23: Reset Japan </li></ul></ul><ul><ul><li>P24: Reset equivalent currency amount </li></ul></ul><ul><ul><li>P25: End application </li></ul></ul>
  24. 26. <ul><li>Atomic System Functions (or actions) maps to transition </li></ul><ul><ul><li>S1: Store US Dollar amount </li></ul></ul><ul><ul><li>S2: Sense Click on Brazil </li></ul></ul><ul><ul><li>S3: Sense click on Canada </li></ul></ul><ul><ul><li>S4: Sense click on EU </li></ul></ul><ul><ul><li>S5: Sense click on Japan </li></ul></ul><ul><ul><li>S6: Sense click on clear button </li></ul></ul><ul><ul><li>S7: sense click on the Clear button </li></ul></ul><ul><ul><li>S8: Sense click on Quit button </li></ul></ul>
  25. 27. <ul><li>D1: US dollar amount entered </li></ul><ul><li>D2: country selected </li></ul>
  26. 33. <ul><li>The composition is not defined </li></ul><ul><ul><li>Place fusions? </li></ul></ul><ul><ul><ul><li>Looks like it </li></ul></ul></ul><ul><ul><li>Transition fusions? </li></ul></ul>
  27. 35. <ul><li>Consider a set of threads T = {T1, T2, T3, T4} </li></ul><ul><ul><li>Scenario 1: 1) enter US amount, 2) select EU, 3) press compute button, 4) press quit </li></ul></ul><ul><ul><ul><li>T1 = {s1, s4, s6, s8} </li></ul></ul></ul><ul><ul><li>Scenerio2: 1) User enters US$, 2) Click on Brazil, 3) Clicks on Compute, Clicks on Clear button) </li></ul></ul><ul><ul><ul><li>T2 = {s1, s2, s6, s7} </li></ul></ul></ul><ul><ul><li>Scenario 3: </li></ul></ul><ul><ul><ul><li>T3 = {s3, s1, s6, s7} </li></ul></ul></ul><ul><ul><li>Scenario 4: </li></ul></ul><ul><ul><ul><li>T4 = {s5, s1, s7, s8} </li></ul></ul></ul><ul><li>In general the set of T covers (minim set of usages for CCP) </li></ul><ul><ul><li>Every action (or S) </li></ul></ul><ul><ul><li>Every port input (every input data and input event) </li></ul></ul><ul><ul><li>Every port output (every output data and output event) </li></ul></ul>
  28. 36. <ul><li>Next-level user behavior </li></ul><ul><li>Common usage scenario </li></ul><ul><ul><li>Converts amount to all 4 currencies, clears and quits </li></ul></ul><ul><ul><li>T5 = [s1, s2, s6, s3, s6, s4, s6, s5, s6, s7, s8] </li></ul></ul>
  29. 37. <ul><li>Unexpected user behavior ( negative scenario ) </li></ul><ul><ul><li>Neg-Sencario1: User changes his/her mind about currency </li></ul></ul><ul><ul><li>T6 = [s1, s2, s3, s4, s5, s6, s7, s8] </li></ul></ul><ul><li>unexpected behavior ( negative scenario ) </li></ul><ul><ul><li>Neg-Scenario2:User toggles between 2 countries, then quits </li></ul></ul><ul><ul><li>T7 = [s1, s2, s3, s2, s3, s2, s3, s8] </li></ul></ul><ul><li>… </li></ul>
  30. 38. <ul><li>Unit testing & integration testing not important </li></ul><ul><li>System testing most complicated test. </li></ul><ul><li>ASFs and EDPNs can help identify and validate test cases </li></ul><ul><ul><li>Valid usages (positive scenarios) </li></ul></ul><ul><ul><li>Invalid usages (negative scenarios) </li></ul></ul>
  31. 39. <ul><li>How to use MBT to test GUI? </li></ul><ul><ul><li>How to measure the benefit and cost of using MBT in GUI? </li></ul></ul><ul><li>Usability testing/evaluation: requires specified requirements or formal models of GUI </li></ul><ul><ul><li>Expert reviews </li></ul></ul><ul><ul><ul><li>Heuristic evaluation </li></ul></ul></ul><ul><ul><ul><li>Guidelines review </li></ul></ul></ul><ul><ul><ul><li>Consistency inspection </li></ul></ul></ul><ul><ul><ul><li>Cognitive walkthrough </li></ul></ul></ul><ul><ul><ul><li>Formal usability inspection </li></ul></ul></ul><ul><ul><li>Usability Testing and Laboratories </li></ul></ul><ul><ul><li>Surveys Instruments </li></ul></ul><ul><ul><li>Acceptance tests </li></ul></ul><ul><ul><li>Evaluation during Active Use </li></ul></ul>
  32. 40. <ul><li>Software Testing by P. Jorgensen </li></ul><ul><li>Model-Based Testing by Utting </li></ul>

×