• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Automated Exploratory Tests
 

Automated Exploratory Tests

on

  • 1,685 views

automated exploratory test tools for web application. Make your test agile. Do not overpay for your automation. Test all aspects (performance, layout, analytic) not only concentrate on one specific ...

automated exploratory test tools for web application. Make your test agile. Do not overpay for your automation. Test all aspects (performance, layout, analytic) not only concentrate on one specific thing.

Statistics

Views

Total Views
1,685
Views on SlideShare
1,685
Embed Views
0

Actions

Likes
0
Downloads
8
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Automated Exploratory Tests Automated Exploratory Tests Presentation Transcript

    • Automated ExploratoryTests for Web ApplicationZbyszek Moćkun © 2010 Cognifide Limited. In commercial confidence only.
    • ObjectiveHow Cognifide has developed automatedtests and where we are going © 2010 Cognifide Limited. In commercial confidence only.
    • IntroductionIn short about process and environment © 2010 Cognifide Limited. In commercial confidence only.
    • Methodologies, projects … Methodologies Projects • Web application• Agile: SCRUM, Kanban • Migration• Frequently releases • Content Management• No time for manual Systems (CMS) regression • Short or medium term• Lot of work at the end of projects sprint • Automation investment, no• CI Environment time for money return © 2010 Cognifide Limited. In commercial confidence only.
    • Common standards forautomationFunctional tests © 2010 Cognifide Limited. In commercial confidence only.
    • Test Automation Lifecycle Write TC Write automated script Run Manage Update © 2010 Cognifide Limited. In commercial confidence only.
    • Advantages and disadvantages • Time consuming • Cost • Coverage by automated tests • Invest and wait for return • No duplications during • Test dependency regression test • Only coded functionality is • Functionality is tested tested © 2010 Cognifide Limited. In commercial confidence only.
    • Can you believe your tests?• Incorrect Xpath (locator) used in tests• Not all data were verified (unable to write verifications for all data)• CSS/images or other files wasn’t load• Frequently design changes require xpath (locator) update• Incorrect CSS fixes (designs are not reflected by page layout)• Analytics doesn’t work• Performance decrease• Few thousands of pages (components variation – can’t test all of them © 2010 Cognifide Limited. In commercial confidence only.
    • Migration (upgrade) projects• No existing automated test for regression test• Few thousands of pages• Quick response needed• Iterative development © 2010 Cognifide Limited. In commercial confidence only.
    • Automated ExploratoryTestsNew approach, theory part © 2010 Cognifide Limited. In commercial confidence only.
    • We need automated testsbecause ... Source: http://www.testfacebook.com © 2010 Cognifide Limited. In commercial confidence only.
    • What should we test? Two perspectives: User Client © 2010 Cognifide Limited. In commercial confidence only.
    • AspectsCI Environment Client User Analytics Layout Functionality Performance © 2010 Cognifide Limited. In commercial confidence only.
    • Looking for a symptom„Symptom is a departure from normal function or feelingwhich is noticed by a patient, indicating the presence of disease or abnormality. A symptom is subjective, observed by the patient, and not measured.” http://wikipedia.org © 2010 Cognifide Limited. In commercial confidence only.
    • Symptoms Layout Performance Content URL strategy HTML code Bug JavaScript Link checker errors Server Validators response (W3C) (404) Analytics © 2010 Cognifide Limited. In commercial confidence only.
    • Oracle HeuristicsIt is used by comparing the output(s) of the system under test,for a given test case input, to the outputs that the oracledetermines that product should have.− other products (second program that uses a different algorithm to evaluate the same mathematical expression as the product under test)− a consistency oracle that compares the results of one test execution to another for similarity,− a model-based oracle that uses the same model to generate and verify system behavior,− or a human beings judgment (i.e. does the program "seem" to the user to do the correct thing?) © 2010 Cognifide Limited. In commercial confidence only.
    • Oracle Heuristic for GUI © 2010 Cognifide Limited. In commercial confidence only.
    • Reference symptoms• Previous version of application• Reference data saved on disc• Other version of the same application (different servers)• The same script takes pattern and compare• One tool for different projects Performance Layout Content HTML code monitoring © 2010 Cognifide Limited. In commercial confidence only.
    • Robots • Scripts that are looking for symptoms • The same script for all projects • Open Source toolsJavaScript Server response Validators Analytics Link checker URL Strategy errors (404) (HTLM, CSS) © 2010 Cognifide Limited. In commercial confidence only.
    • Exploratory testing„Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design, test execution and test result interpretation” Cem Kaner © 2010 Cognifide Limited. In commercial confidence only.
    • What are Automated ExploratoryTests?Meny different definition:• Record and play manual exploratory test sessions• Combined manual and automated tests• Test steps automation• Blackbox tests with input/output data generator• Not at all possible due to human factor © 2010 Cognifide Limited. In commercial confidence only.
    • What are Automated ExploratoryTests?„Tests supporting exploratory testing, not necessarily based on the test scenario, searching areas where there may be errors or a change in the application, allowing the examination of applications faster and more deeply or not to test sites available to man.” Zbyszek Moćkun„Automated Tests do not provide complete information. They provide partial information that might be useful” Cem Kaner © 2010 Cognifide Limited. In commercial confidence only.
    • Automated Exploratory Tests -advantages• One tool − Gather all symptoms − Use in all projects − Quick to introduce to project (half a day) − Resistant to appliciaton changes• Do not require deep knowledge about application, support the learning process• Integrated with Continuous Intergration env• Strong solution if combined with automated functional tests• Migration or upgrade tests• Iterative development © 2010 Cognifide Limited. In commercial confidence only.
    • Track the propagation ofchanges in the code- See what pages/components are infected by change- Positive – false (code change affect all necessary places?) © 2010 Cognifide Limited. In commercial confidence only.
    • Automated ExploratoryTestsPractical part © 2010 Cognifide Limited. In commercial confidence only.
    • Report – example of use © 2010 Cognifide Limited. In commercial confidence only.
    • URL management• Sitemap• Link crawler• CSV file with url list• Symptoms as parameter (on, off)• Comments (bug id, ...)• Hide element• Other actions• Keep in SVN © 2010 Cognifide Limited. In commercial confidence only.
    • Pattern management• Easy to use (overwrite pattern)• Checkbox (check all)• Commit to svn © 2010 Cognifide Limited. In commercial confidence only.
    • Dynamic content• Reference symptoms – need processing • Use regexp • Hide element • Write generic function• Robots – do not need any additional actions• Dynamic components (counters, date, random numbers)• Flash/video files• Embedded content (twitter, facebook,...)• Require functional test• Static components – about 75% © 2010 Cognifide Limited. In commercial confidence only.
    • Mobile version• Required (not nice to have), can’t avoid testing• Mobile pages are more static than desktop version• Do not use flash• Not all symptoms are easy to check © 2010 Cognifide Limited. In commercial confidence only.
    • Narzędzia• Selenium II/WebDriver• Komparatory (ImageMagick, Pretty Diff)• Walidatory (W3C)• Performance (JMeter, Proxy)• Serwery Proxy (BrowserMob)• Firefox/Firebug (NetExport, ConsoleExport, JS Console output redirector)• HAR Viewer (http://www.softwareishard.com/har/viewer/) © 2010 Cognifide Limited. In commercial confidence only.
    • Q&A © 2010 Cognifide Limited. In commercial confidence only.