Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Unit 09: Web Application Testing


Published on

Published in: Technology
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ ◀ ◀ ◀ ◀
    Are you sure you want to  Yes  No
    Your message goes here

Unit 09: Web Application Testing

  1. 1. Unit 9: Web Application Testing Testing is the activity conducted to evaluate the quality of a product and to improve it by finding errors. Testingdsbw 2011/2012 q1 1
  2. 2. Testing Terminology An error is “the difference between a computed, observed, or measured value or condition and the true, specified, or theoretically correct value or condition” (IEEE standard 610.12- 1990). This “true, specified, or theoretically correct value or condition” comes from  A well-defined requirements model, if available and complete  An incomplete set of fuzzy and contradictory goals, concerns, and expectations of the stakeholders A test is a set of test cases for a specific object under test: the whole Web application, components of a Web application, a system that runs a Web application, etc. A single test case describes a set of inputs, execution conditions, and expected results, which are used to test a specific aspect of the object under testdsbw 2011/2012 q1 2
  3. 3. Testing [and] Quality Testing should address compliance not only to functional requirements but also to quality requirements, i.e., the kinds of quality characteristics expected by stakeholders. ISO/IEC 9126-1 [Software] Quality Model:dsbw 2011/2012 q1 3
  4. 4. Goals of Testing The main goal of testing is to find errors but not to prove their absence A test run is successful if errors are detected. Otherwise, it is unsuccessful and “a waste of time”. Testing should adopt a risk-based approach:  Test first and with the greatest effort those critical parts of an application where the most dangerous errors are still undetected  A further aim of testing is to bring risks to light, not simply to demonstrate conformance to stated requirements. Test as early as possible at the beginning of a project: errors happened in early development phases are harder to localize and more expensive to fix in later phases.dsbw 2011/2012 q1 4
  5. 5. Test Levels (1/2) Unit tests  Test the smallest testable units (classes, Web pages, etc.) independently of one another.  Performed by the developer during implementation. Integration tests  Evaluate the interaction between distinct and separately tested units once they have been integrated.  Performed by a tester, a developer, or both jointly. System tests  Test the complete, integrated system.  Typically performed by a specialized test team.dsbw 2011/2012 q1 5
  6. 6. Test Levels (2/2) Acceptance tests  Evaluate the system with the client in an “realistic” environment, i.e. with real conditions and real data. Beta tests  Let friendly users work with early versions of a product to get early feedback.  Beta tests are unsystematic tests which rely on the number and “malevolence” of potential users.dsbw 2011/2012 q1 6
  7. 7. Fitting Testing in the Development Process Planning: Defines the quality goals, the general testing strategy, the test plans for all test levels, the metrics and measuring methods, and the test environment. Preparing: Involves selecting the testing techniques and tools and specifying the test cases (including the test data). Performing: Prepares the test infrastructure, runs the test cases, and then documents and evaluates the results. Reporting: Summarizes the test results and produces the test reports.dsbw 2011/2012 q1 7
  8. 8. Web Testing: A Road Map Content Interface Testing Testing Usability Testing user Navigation Testing Component Testing Configuration Testing Performance Security technology Testing Testingdsbw 2011/2012 q1 8
  9. 9. Usability Usability is a quality attribute that assesses how easy user interfaces are to use. Also refers to methods for improving ease-of-use during the design process. Usability is defined by five quality components:  Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design?  Efficiency: Once users have learned the design, how quickly can they perform tasks?  Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency?  Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?  Satisfaction: How pleasant is it to use the design?dsbw 2011/2012 q1 9
  10. 10. Why Usability matters* 62% of web shoppers gave up looking for an item. (Zona study) 50% of web sales are lost because visitors can’t easily find content. (Gartner Group) 40% of repeat visitors do not return due to a negative experience. (Zona study) 85% of visitors abandon a new site due to poor design. (cPulse) Only 51% of sites complied with simple web usability principles. (Forrester study of 20 major sites)(*) data from 2011/2012 q1 10
  11. 11. Why people fail Search Findability (IA, Category names, Navigation, Links) Page design (Readability, Layout, Graphics, Amateur, Scrolling) Information (Content, Product info, Corporate info, Prices) Task support (Workflow, Privacy, Forms, Comparison, Inflexible) Fancy design (Multimedia, Back button, PDF/Printing, New window, Sound) Other (Bugs, Presence on Web, Ads, New site,Usability problems weighted by how frequently they Metaphors) caused users to fail a task [NL06]dsbw 2011/2012 q1 11
  12. 12. Top Ten (Usability) Mistakes in Web Design1. Bad search2. Pdf files for online reading3. Not changing the color of visited links4. Non-scannable text5. Fixed font size6. Page titles with low search engine visibility7. Anything that looks like an advertisement8. Violating design conventions9. Opening new browser windows10. Not answering users questionsdsbw 2011/2012 q1 12
  13. 13. Assessing Usability Two major types of assessing methods:  Usability evaluations:  Evaluators and no users  Techniques: surveys/questionnaires, observational evaluations, guideline based reviews, cognitive walkthroughs, expert reviews, heuristic evaluations  Usability tests: focus on users working with the product Usability testing is the only way to know if the Web site actually has problems that keep people from having a successful and satisfying experience.dsbw 2011/2012 q1 13
  14. 14. Usability Testing Usability testing is a methodology that employs potential users to evaluate the degree to which a website/software meets predefined usability criteria. Basic Process: 1. Watch Customers 2. They Perform Tasks 3. Note Their Problems 4. Make Recommendations 5. Iteratedsbw 2011/2012 q1 14
  15. 15. Measures of Usability Effectiveness (Ability to successfully accomplish tasks)  Percentage of goals/tasks achieved (success rate)  Number of errors Efficiency (Ability to accomplish tasks with speed and ease)  Time to complete a task  Frequency of requests for help  Number of times facilitator provides assistance  Number of times user gives updsbw 2011/2012 q1 15
  16. 16. Measures of Usability Satisfaction (Pleasing to users)  Positive and negative ratings on a satisfaction scale  Percent of favorable comments to unfavorable comments  Number of good vs. bad features recalled after test  Number of users who would use the system again  Number of times users express dissatisfaction or frustration Learnability (Ability to learn how to use site and remember it)  Ratio of successes to failures  Number of features that can be recalled after the testdsbw 2011/2012 q1 16
  17. 17. Usability Testing Roles Facilitator:  Oversees the entire test process  Plan, test, and report. Participant:  Actual or potential customer.  Representative users (marketing, designers) avoided. Observer (optional):  Records events as they occur.  Limits interaction with the customer.  Does contribute to the report.dsbw 2011/2012 q1 17
  18. 18. Usability Testing ProcessStep 1: Planning The Usability Test  Define what to test  Define which customers should be tested  Define what tasks should be tested  Write usability scenarios and tasks  Select participantsStep 2: Conducting The Usability Test  Conduct a test  Collect dataStep 3: Analyzing and Reporting The Usability Test  Compile results  Make recommendationsdsbw 2011/2012 q1 18
  19. 19. People – Context – ActivitiesStep 1: Planning The Usability Test  Define what to test  → Activities (Use Cases)  Define which customers (user profiles) to be tested  → People (Actors)  Provide a background for the activities to test  → Contextdsbw 2011/2012 q1 19
  20. 20. Usability Scenarios and Tasks Provide the participant with motivation and context to make the situation more realistic Include several tasks:  Make the first task simple  Give a goal, without describing steps Set some success criteria, examples:  N% of test participants will be able to complete x% of tasks in the time allotted.  Participants will be able to complete x% of tasks with no more than one error per task.  N% of test participants will rate the system as highly usable on a scale of x to x.dsbw 2011/2012 q1 20
  21. 21. Example of Scenario with Tasks Context:  You want to book a sailing on Royal Caribbean International for next June with your church group. The group is called “Saint Francis Summer 2010”. The group is selling out fast, so you want to book a cabin, which is close to an elevator because your leg hurts from a recent injury. Tasks to perform: 1. Open your browser 2. Click the link labeled “Royal Caribbean” 3. Tell me the available cabins in the “Saint Francis Summer 2010” group 4. Tell me a cabin number closest to an elevator 5. Book the cabin the best suits your needsdsbw 2011/2012 q1 21
  22. 22. Selecting Participants Recruit participants  In-house  recruitment firms, databases, conferences Match participants with user profiles Numbers: of participants, floaters Schedule test sessions Incentives:  Gift checks ($100 per session)  Food or gift cardsdsbw 2011/2012 q1 22
  23. 23. How Many Test Participants Are Required? The number of usability problems found in a usability test with n participants is: N(1-(1-L)n)  N : total number of usability problems in the design  L : the proportion of usability problems discovered while testing a single participant. For L = 31%dsbw 2011/2012 q1 23
  24. 24. How Many Test Participants Are Required? It seems that you need to test with at least 15 participants to discover all the usability problems However, is better to perform 3 tests with 5 participants than to perform one with 15 participants:  After the first test with 5 participants has found 85% of the usability problems, you will want to fix them in a redesign.  After creating the new design, you need to test again.  The second test with 5 users will discover most of the remaining 15% of the original usability problems that were not found in the first test (and some new one).  The new test will be able to uncover structural usability problems that were obscured in initial studies as users were stumped by surface-level usability problems.  Fix the new problems, and test …dsbw 2011/2012 q1 24
  25. 25. Usability Labs … Not NecessaryThe testing room contains office The observer side contains afurniture, video tape equipment, a powerful computer to collect themicrophone and a computer with usability data and analyze it. A one-appropriate software. way mirror separates the rooms.dsbw 2011/2012 q1 25
  26. 26. Test Side-by-Sidedsbw 2011/2012 q1 26
  27. 27. Conducting Tests: Facilitator’s Role Start with an easy task to build confidence Sit beside the person not behind the glass Use “think-out-loud” protocol Give participants time to think it through Offer appropriate encouragement Lead participants, don’t answer questions (being an enabler) Don’t act knowledgeable (treat them as the experts) Don’t get too involved in data collection Don’t jump to conclusions Don’t solve their problems immediatelydsbw 2011/2012 q1 27
  28. 28. Collecting Data Performance  Objective (what actually happened)  Usually Quantitative  Time to complete a task  Time to recover from an error  Number of errors  Percentage of tasks completed successfully  Number of clicks  Pathway information Preference  Subjective (what participants say/thought)  Usually Qualitative  Preference of versions  Suggestions and comments  Ratings or rankings (can be quantitative)dsbw 2011/2012 q1 28
  29. 29. Report findings and recommendations Make report usable for your users Include quantitative data (success rates, times, etc.) Avoid words like “few, many, several”. Include counts Use quotes Use screenshots Mention positive findings Do not use participant names, use P1, P2, P3, etc. Include recommendations Make it shortdsbw 2011/2012 q1 29
  30. 30. Component Testing Focuses on a set of tests that attempt to uncover errors in WebApp functions Conventional black-box and white-box test case design methods can be used at each architectural layer (presentation, domain, data access) Form data can be exploited systematically to find errors:  Missing/incomplete data  Type conversion problems  Value boundary violations  Fake data  Etc. Database testing is often an integral part of the component- testing regimedsbw 2011/2012 q1 30
  31. 31. Configuration Testing: Server-Side Issues Is the WebApp fully compatible with the server OS? Are system files, directories, and related system data created correctly when the WebApp is operational? Do system security measures (e.g., firewalls or encryption) allow the WebApp to execute and service users without interference or performance degradation? Has the WebApp been tested with the distributed server configuration (if one exists) that has been chosen? Is the WebApp properly integrated with database software? Is the WebApp sensitive to different versions of database software? Do server-side WebApp scripts execute properly? Have system administrator errors been examined for their affect on WebApp operations? If proxy servers are used, have differences in their configuration been addressed with on-site testing?dsbw 2011/2012 q1 31
  32. 32. Configuration Testing: Client-Side Issues Hardware—CPU, memory, storage and printing devices Operating systems—Linux, Macintosh OS, Microsoft Windows, a mobile-based OS Browser software—Internet Explorer, Mozilla/Netscape, Opera, Safari, and others User interface components—Active X, Java applets and others Plug-ins—QuickTime, RealPlayer, and many others Connectivity—cable, DSL, regular modem, T1dsbw 2011/2012 q1 32
  33. 33. Security Testing Designed to probe vulnerabilities  of the client-side environment,  the network communications that occur as data are passed from client to server and back again,  and the server-side environment On the client-side, vulnerabilities can often be traced to pre- existing bugs in browsers, e-mail programs, or communication software. On the network infrastructure On the server-side, Review the DSBW Unit on  At host level WebApp Security  At WebApp leveldsbw 2011/2012 q1 33
  34. 34. Performance Testing: Main Questions Does the server response time degrade to a point where it is noticeable and unacceptable? At what point (in terms of users, transactions or data loading) does performance become unacceptable? What system components are responsible for performance degradation? What is the average response time for users under a variety of loading conditions? Does performance degradation have an impact on system security? Is WebApp reliability or accuracy affected as the load on the system grows? What happens when loads that are greater than maximum server capacity are applied?dsbw 2011/2012 q1 34
  35. 35. Performance Testing: Load Tests A load test verifies whether or not the system meets the required response times and the required throughput. Steps: 1. Determine load profiles (what access types, how many visits per day, at what peak times, how many visits per session, how many transactions per session, etc.) and the transaction mix (which functions shall be executed with which percentage). 2. Determine the target values for response times and throughput (in normal operation and at peak times, for simple or complex accesses, with minimum, maximum, and average values). 3. Run the tests, generating the workload with the transaction mix defined in the load profile, and measure the response times and the throughput. 4. The results are evaluated, and potential bottlenecks are identified.dsbw 2011/2012 q1 35
  36. 36. Performance Testing: Stress Tests A stress test verifies whether or not the system reacts in a controlled way in “stress situations”, which are simulated by applying extreme conditions, such as unrealistic overload, or heavily fluctuating load. The test is aimed at answering the questions:  Does the server degrade ‘gently’ or does it shut down as capacity is exceeded?  Does server software generate “server not available” messages? More generally, are users aware that they cannot reach the server?  Are transactions lost as capacity is exceeded?  Is data integrity affected as capacity is exceeded?dsbw 2011/2012 q1 36
  37. 37. Performance Testing: Stress Tests (cont.)  Under what load conditions the server environment fails? How does failure manifest itself? Are automated notifications sent to technical support staff at the server site?  If the system does fail, how long will it take to come back on- line?  Are certain WebApp functions (e.g., compute intensive functionality, data streaming capabilities) discontinued as capacity reaches the 80 or 90% level?dsbw 2011/2012 q1 37
  38. 38. Performance Testing: Interpreting Graphics  Load: the number of requests that arrive at the system per time unit  Throughput: the number of requests served per time unit.  SLA: Service Level Agreementdsbw 2011/2012 q1 38
  39. 39. Test Automation Automation can significantly increase the efficiency of testing and enables new types of tests that also increase the scope (e.g. different test objects and quality characteristics) and depth of testing (e.g. large amounts and combinations of input data). Test automation brings the following benefits:  Running automated regression tests on new versions of a WebApp allows to detect defects caused by side-effects to unchanged functionality.  Various test methods and techniques would be difficult or impossible to perform manually. For example, load and stress testing requires to simulate a large number of concurrent users.  Automation allows to run more tests in less time and, thus, to run the tests more often leading to greater confidence in the system under test. Web Site Test Tools: 2011/2012 q1 39
  40. 40. References R. G. Pressman, D. Lowe: Web Engineering. A Practitioner’s Approach. McGraw Hill, 2008. Chapter 15. KAPPEL, Gerti et al. Web Engineering, John Wiley & Sons, 2006. Chapter 7. [NH06] NIELSEN, J. and LORANGER, H. 2006 Prioritizing Web Usability. New Riders Publishing. (Jakob Nielsen) www.usability.govdsbw 2011/2012 q1 40