Automated Testing Of Web Applications Using XML

  • 8,942 views
Uploaded on

css 2002 presentation

css 2002 presentation

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
8,942
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
256
Comments
1
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Automated Testing of Web Applications using XML dIon Gillard, Multitask Consulting Pty Ltd
  • 2. Agenda
    • What is a Web Application?
    • Testing Web Apps
      • Code-based testing
      • Data driven testing
    • Tools
    • Automating test case creation
    • Automating tests
    • Comparison to Commercial Tools
  • 3. What is a Web Application
    • We’ll be concentrating on J2EE Web Applications
    • A lot of the tools and concepts can be applied to other technologies
    • Web Applications consist of:
      • Server side resources
        • JSPs, Servlets, Java code, configuration
      • Client side resources
        • HTML, images, JavaScript
  • 4. Testing Web Applications – ‘Field Experience’
    • Developers test localised changes have had the desired effect
    • Without automated testing a complete regression test is impractical
    • Application yo-yos from good to bad, working to broken frequently
    • Users have little confidence in the application and wait for a ‘release’
    • Large effort goes into QA for a release
  • 5. Testing Web Applications – Why?
    • There are lots of ways to test, we aren’t going to cover all of them
    • Testing usually has a purpose
      • Does the code work
      • Does the application perform well
      • Does it respect security requirements
      • Can it handle load
      • Does it match business requirements
      • … .
  • 6. Testing web applications - How
    • We’re going to be focussing on ‘functional testing’, a.k.a. Black-box testing
    • “ Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution condition”
    • IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries
  • 7. Testing Web Applications - Implications
    • Looking at web applications from a user’s perspective - Does it do what I want, how I want it done?
    • What the code does is a side-effect for the user
    • Users expect a web application to be an ‘application’, regardless of how the data is displayed or the functionality accessed
    • User interface is still very important, however, so layout, along with look and feel must still be tested
  • 8. Testing Web Applications – What to test
    • Interactions between the user and the application
    • Expected Inputs
      • input fields, hidden fields etc, query string or URL additions
    • Unexpected inputs
      • no password, field lengths, numerical domains etc
    • Error conditions
      • Messages to the user, code ‘failure’ Error 500
    • Expected processing
      • Processing succeeds or fails
    • Expected output
      • Results of next page, database updates, messages queued etc
  • 9. Code based Testing
    • Tools such as JUnit rely on developers creating ‘test cases’ in code to ensure correctness
    • This has all the advantages and disadvantages of developers writing code
    • Several tools have been released that allow this development to be more productive, less tedious and the code more reusable
    • Writing low-level code makes it hard to focus on what the testing is for
  • 10. HttpUnit
    • Freeware ‘Browser emulator’ for use in JUnit test cases (and other places)
    • Written in Java, provided as a jar developers would use
    • Takes care of authentication, cookies, page redirection
    • Provides resulting pages as a DOM via NekoHTML
    • HTML pages don’t have to have valid XML syntax to be parsed
  • 11. HttpUnit - Sample WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret"); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); response = conversation.getResponse( request ); assertTrue( "Login not rejected", response.getText().indexOf("Login failed") != -1 );
  • 12. HttpUnit – API
    • Classes are available for creating requests in various ways (WebRequest)
    • Working with the HTTP response (WebResponse) return code, text, headers, cookies etc
    • Working with HTML artifacts
      • getDOM
      • getForms(), getFormWithName(name), getTables() getLinks(), getLinkWith(text)
    • Forms can be ‘submitted’ after retrieval from the response
  • 13. HttpUnit – Pros and Cons
    • Code can be repetitive and tedious
    • Working at a low level, you can lose sight of what is being done
    • Each WebRequest is created in isolation in a test
    • Testing a web application is typically a single host and URI, code is needed to unify tests
    • Based on JUnit, is easy to automate
    • Simple clear API is easy to understand
  • 14. HtmlUnit
    • Similar in scope to HttpUnit, but with a focus on HTML documents rather than the HTTP protocol details like requests, responses etc
    • Better JavaScript support
    • API is easier to relate to user interaction, e.g. clicking links and pressing buttons
    • Wrapper objects for every html element
    • Browser specific features can be used to fake the browser version and vendor
  • 15. HtmlUnit - continued
    • Can track frames, iframes, pop up windows
    • Listener to handle new windows being opened as part of interaction or JavaScript
    • Can easily access the other windows
    • Tab-order can be retrieved and checked
    • Assertion helper methods for unique ids, accessibility
    • Full access to the actual DOM i.e. editable
  • 16. HtmlUnit – Sample form submit WebClient client = new WebClient(); URL url = new URL("http://htmlunit.sf.net/"); HtmlPage page = (HtmlPage) client.getPage(url); // find form by id HtmlForm form = page.getFormById(“mainForm”); // ‘type’ text into the userId field HtmlInput input = form.getInput(“userId”); input.setValue(“dion”); // submit – note existing values provided in the page // are ‘automatically’ filled in form.submit();
  • 17. ServletUnit
    • Companion tool shipped with HttpUnit
    • For when you need to do less black box and more white box testing
    • Simulates a servlet container by providing Mock Objects
    • Provides API for accessing the ‘Invocation Context’
    • Other tools such as Cactus provide a fully-featured in-container environment
  • 18. ServletUnit – Sample code ServletRunner sr = new ServletRunner(); sr.registerServlet("myServlet", StatefulServlet.class.getName()); // create a client and request ServletUnitClient sc = sr.newClient(); WebRequest request = new PostMethodWebRequest( "http://test.meterware.com/myServlet" ); request.setParameter( "color", "red" ); InvocationContext ic = sc.newInvocation( request ); // call service ic.getServlet().service(request, ic.getResponse()); // test response, session, servlet instance variables etc assertNotNull("Session was not created", ic.getRequest().getSession( false ) );
  • 19. StrutsTestCase – What is it?
    • JUnit extension for testing code based on the Apache Struts web app framework
    • Provides Mock Objects for simulating the servlet container
    • Also provides an in-container Cactus extension (CactusStrutsTestCase)
    • Switching approaches means changing parent class to/from StrutsTestCase
  • 20. StrutsTestCase – How does it work?
    • Tests are based around Struts Actions
    • There are methods available for hooking into the Struts request lifecycle
    • Extra assert-like methods for checking the Struts environment, like forwards and action errors
  • 21. StrutsTestCase - Sample … public void testFailedLogin() { // setup parameters to pass to the action addRequestParameter("username","deryl"); addRequestParameter("password","express"); // set up the path to identify the action setRequestPathInfo("/login"); // perform action processing actionPerform(); // check that the user has been forwarded to “login” verifyForward("login"); // make sure the error for key ‘error.password.mismatch’ // is present in the request verifyActionErrors(new String[] {"error.password.mismatch"}); // make sure the session state is correct assertNull((String) getSession().getAttribute( "authentication")); } …
  • 22. Apache’s Cactus
    • A framework for unit testing server side Java code
    • JUnit extension to make it easier to test Servlets, EJBs, Tag Libraries, Servlet Filters etc
    • Comes with ant tasks for automation
    • Uses a ‘proxy’ mechanism to redirect test case requests to a live servlet container
  • 23. Apache’s Cactus - API
    • Provides 3 Test classes to subclass from
      • ServletTestCase, JspTestCase and FilterTestCase
    • Code testXXX methods as per usual, except you need to instantiate the object to be tested (e.g. servlet)
    • beginXXX and endXXX methods can be coded to setup or post-process HTTP related parameters such as headers, return code, the response etc
  • 24. Apache’s Cactus - Features
    • Provides IDE integration for JBuilder, VAJava
    • EJB unit testing using Servlet ‘proxy’
    • As it runs in-container, there is good documentation on setting up the proxy
    • Can integrate with HttpUnit and StrutsTestCase
  • 25. XML-based Tools
    • These tools rely on tests written in an XML format rather than java code
    • Output varies – can be HTML
    • They can be more easily generated
    • They are more ‘user’ friendly
    • Take away the reliance on technical skills to build tests
    • Work at a higher level and allow more focus on what is being tested and how
  • 26. Apache’s Latka
    • Functional testing tool that uses XML to define a series of HTTP(S) requests
    • Validations for the responses are defined as child elements of the requests
    • Has a command prompt interface
    • Can run tests via JUnit
    • Can run tests interactively via a web-app
    • Latka is moving to use Jelly as it’s XML processor
  • 27. Apache’s Latka – Document Definition
    • Suite
      • top level element for a set of tests
      • Provides default host, port and a label
    • Session
      • Wrapper for a set of requests that share HTTP state
    • Request
      • A HTTP request to be executed
      • Path, method (“get”), host, port, label
    • Validate
      • Holds all validations for a specific request
  • 28. Apache’s Latka - Validations
    • Bytelength – length of the response
    • Cookie – presence of a cookie
    • maxRequestTime
    • Regexp – tests the response matches a regular expression
    • statusCode – e.g. 200, 404
    • XPath – test the response matches an XPath expression
  • 29. Apache’s Latka - Sample
    • <suite defaultHost=&quot;jakarta.apache.org&quot; label=&quot;Taglibs&quot;>
    • <session>
          • <request path=&quot;/taglibs/&quot; label=&quot;home page&quot;>
          • <validate>
          • <statusCode />
          • <regexp pattern=&quot;Welcome to Jakarta Taglibs!&quot;/>
          • </validate>
          • </request>
          • <request path=“/logon”>
          • <param><paramName>user</paramName>
          • <paramValue>dion</paramValue></param>
          • <param><paramName>password</paramName>
          • <paramValue>dionspwd</paramValue></param>
          • <validate><statusCode />
          • </request>
    • </session>
    • </suite>
  • 30. Apache’s Latka – Running Tests
    • Command prompt
      • Latka.bat file:./TestSite.xml
    • JUnit
      • public static Test suite() {
      • TestSuite suite = new TestSuite();
      • String fileName = &quot;tests/samples/TestCommonsWebsite.xml&quot;; suite.addTest(
      • JUnitTestAdapter.createTestFromFile(fileName));
      • return suite;
      • }
  • 31. Apache’s Latka
    • Also provides validators as tags for Jelly
    • Automate via JUnit Ant task
    • Listeners can be attached to pick up test events as they happen
    • XML and Simple (console) reporter provided
    • XSL Style sheet provided to produce text from report
  • 32. Apache’s Jelly
    • Java and XML based scripting and processing engine
    • Jelly ‘scripts’ are XML documents which get parsed
    • Scripts are then ‘executed’ to produce XML, HTML, text etc
    • XML elements can be bound to java code similar to JSP custom tags and Ant tasks
  • 33. Apache’s Jelly - Taglibs
    • Core: catch, choose, expr, forEach, if, file, import, include, jelly, new, otherwise, set, thread, when, whitespace
    • Xml: attribute, copy, copyOf, element, expr, forEach, if, parse, set
    • Define: attribute, bean, invoke, invokeBody, jellyBean, script, tag, taglib,
    • Sql: dateParam, driver, param, query, setDataSource, transaction, update
  • 34. Apache’s Jelly – Taglibs (Continued)
    • JSL: applyTemplates, style, stylesheet, template
    • Ant: ant, filescanner
    • Werkz: attain, attainGoal, goal, postAction, postGoal, preAction, preGoal, project
    • Jeez: target, tagDef
    • Log: debug, error, fatal, info, trace, warn
    • Ojb: broker, store
  • 35. Apache’s Jelly – Taglibs (Continued)
    • JMS: connection, destination, mapEntry, mapMessage, message, objectMessage, property, receive, send, textMessage
    • Validate: verifier, validate, assertValid
    • Http: delete, get, header, post, put
    • Interaction: ask
    • Antlr: antlr, grammar
    • Util: tokenize
    • Html: parse
  • 36. Apache’s Jelly – Taglibs (Continued)
    • JUnit: assert, assertEquals, case, fail, run, suite
    • Swing: action, component, windowListener
    • Quartz: cronTrigger, job, waitForScheduler
    • Betwixt: introspector, parse
  • 37. Apache’s Jelly - Testing
    • Provides
      • a JUnit tag library
      • a HTTP tag library
      • a ‘cron’ like facility
      • an expression language
    • Integration with JMS, SQL etc allows page results to be checked against the database or queue
  • 38. Apache’s Jelly - Sample
    • <?xml version=&quot;1.0&quot;?>
    • <j:jelly
    • xmlns:j=&quot;jelly:core&quot;
    • xmlns:log=&quot;jelly:log&quot;
    • xmlns:v=&quot;jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary&quot;
    • xmlns=&quot;jelly:org.apache.commons.latka.jelly.HttpTagLibrary&quot;
    • trim=&quot;false&quot;>
      • <session>
      • <get var=&quot;mtc&quot; uri=&quot;http://www.multitask.com.au/&quot;/>
      • <j:if test='${mtc.statusCode == 200}'>
      • request ok
      • </j:if>
      • Results for mtc url are:
      • http return code = ${mtc.statusCode}
      • http status text = '${mtc.statusText}'
      • size of result = ${mtc.responseBodyAsString.length()}
      • response time = ${mtc.responseTime}
      • </session>
    • </jelly>
  • 39. Apache’s Jelly - Sample
    • <?xml version=&quot;1.0&quot;?>
    • <j:jelly xmlns:j=&quot;jelly:core“ xmlns:log=&quot;jelly:log&quot;
    • xmlns:v=&quot;jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary&quot;
    • xmlns=&quot;jelly:org.apache.commons.latka.jelly.HttpTagLibrary“>
      • <session>
      • <post var=&quot;jdc“
      • uri=&quot;http://developer.java.sun.com/servlet/SessionServlet&quot;>
      • <parameter name=&quot;action&quot; value=&quot;login&quot; />
      • <parameter name=&quot;url&quot; value=&quot;/developer/index.jshtml&quot; />
      • <parameter name=&quot;UserId&quot; value=&quot;XXXX&quot; />
      • <parameter name=&quot;Password&quot; value=&quot;XXXX&quot; />
      • </post>
      • jdc login result = ${jdc.statusCode}
      • good login = ${jdc.responseBodyAsString.indexOf(&quot;Sorry!&quot;) == &quot;-1&quot;}
      • <v:regexp pattern=&quot;Sox.y!&quot; var=&quot;jdc&quot;>
      • <request-failed var=&quot;jdc&quot;>bad pattern 1</request-failed>
      • </v:regexp>
      • </session>
    • </jelly>
  • 40. XMLUnit
    • JUnit extension providing comparison between XML docs
    • XML can come from lots of places: Strings, DOM, Reader
    • Accepts ‘browser-friendly’ HTML as input
    • Can validate against a DTD
  • 41. JXWeb
    • XML Script ‘language’ and extension to JXUnit
    • Uses HttpUnit under the covers
    • No JUnit integration
    • Not as easily extendable as Jelly
  • 42. JXWeb – Scripting Language
    • jxw – top level element
    • httpGet, postXml – to retrieve data
    • setRequestParameter, setCookie, setHeader
    • getLink, getTableValue, getFormParameter, getForm, getTable
    • ifEqual, isNull, loop, doLoop, exception, ifNull, save, subst, ifEqualLength, isEqual
    • Echo, set, loadProperties
  • 43. JXWeb - Sample <jxw> <set name=&quot;req&quot; value=&quot;www.testsite.com&quot;/> <httpGet response=&quot;response1&quot;/> <save name=&quot;respText&quot; file=&quot;list.html&quot;/> <getTable name=&quot;myTable&quot; tableIndex=&quot;0&quot;/> <getTableValue name=&quot;tableValue&quot; table=&quot;myTable“ row=&quot;0&quot; column=&quot;0&quot;/> <isEqual name=“tableValue&quot; value=&quot;Every Good Beer Deserves Froth“ message=“page is missing froth&quot;/> </jxw>
  • 44. AntEater – Ant Functional Testing
    • Testing framework designed around Ant
    • Supplied as a set of Ant tasks and types
    • Integrates easily into an existing build or IDE
    • Comes with a servlet container built-in to allow for ‘call backs’
  • 45. Ant Eater - Tasks
    • Action Tasks
      • httpRequest
      • soapRequest
      • Listener
    • Match – groups multiple tests
    • Tests
      • Listening: method, parameter, sendResponse
      • Header, contentEquals, regexp, responseCode, xpath, relaxng
  • 46. Ant Eater – Other tags
    • Session
    • Logger – minimal, plain, colour, xml
    • forEach
    • Group – a way of grouping variables, sessions and loggers
    • servletContainer – start internal tomcat
    • Deploy – deploy web apps
  • 47. Ant Eater - Sample … <target name=&quot;checkonline&quot;> <httpRequest path=&quot;/&quot;> <match assign=&quot;online&quot;><responseCode value=&quot;200&quot;/> </match> </httpRequest> </target> <target name=&quot;hitserver&quot; if=&quot;online&quot;> <echo>We're online!</echo> <httpRequest path=&quot;/&quot;> <match assign=&quot;apache&quot;> <header name=&quot;Server“ value=&quot;Apache/1.3.24 (Unix) Debian GNU/Linux&quot;/> </match> <match assign=&quot;unknownserver“/> </httpRequest> </target>
  • 48. Limitations
    • XML as a programming language isn’t
      • Not as expressive
      • Tedious syntax for non-techies
      • Usually not as extensible
    • Limits the interaction with external data sources
      • Other than Jelly, most don’t have a mechanism for dealing with java objects outside their framework
  • 49. Automating Test Case Creation
    • One of the main reasons developers give for limited testing is limited time
    • Developers are actually happy to test if it isn’t a significant drain on achievements
    • Testing pages or servlets helps developers think of error conditions that could easily crop up
  • 50. Automated Creation - JMeter
    • JMeter is often thought of as a load testing tool only
    • It also has a ‘proxy server’ mode where you can record browser activity
    • It can filter out requests for images or other resources using a regexp
    • It can also be used to ‘assert’ text is present on a request
  • 51. Automated Creation – ‘Homer’
    • Microsoft freebie tool in IIS 5.0 resource kit
    • Under windows acts as a proxy and records requests coming in
    • Usually used for load testing
    • Stores data in an Access .mdb file
    • Simply queries can transform this into XML and then into Latka or Jelly etc
    • Merged into VS.NET toolset
  • 52. Automated Testing
    • Once the tests are created the next thing is to get them run on a regular basis
    • This means application and developer confidence grows
    • Depending on the tool this can be achieved using the operating system and a build tool
  • 53. Apache’s Ant
    • Java based build tool
    • Has JUnit tasks as part of the ‘optional’ tasks
    • Can be scheduled using operating system constructs
    • Can provide email about failures
  • 54. CruiseControl
    • Open source tool from ThoughtWorks
    • Provides a way to kick off Ant builds based on source repository changes
    • This allows the automation of the tests to be done when the source code changes
    • Ensures that the application passes all tests
    • Flags immediately when the application breaks
  • 55. Commercial Tools
    • LoadRunner from Mercury Interactive
      • Provides access to XML data from the scripting language
      • More geared toward load testing
    • Astra QuickTest from Mercury Interactive
      • Web testing, built-in script creation via browser, cross browser tests from same test case, extract details from browser objects
      • Supports Flash, Real etc
  • 56. Commercial Tools
    • Rational RobotJ
      • Eclipse based tool
      • Uses proprietary scripting language
      • Does auto-record, regular expressions
      • Approx 1.3GB space
    • WebLoad
      • Similar features to higher cost tools
      • Multiple server support, higher load focus
      • Supports JavaScript
      • GUI and Command line driven
  • 57. Commercial Tools - continued
    • High end Java IDEs such as WebSphere Studio Application Developer 5 now include testing tools
      • Some can not be automated
      • Compete with other vendors in same tool space
  • 58. Summary
    • There are simple tools available now that can alleviate the drudgery of functional testing
    • Users can help write the functional tests
    • Automation and repetition are key
    • Choose how far you want to take testing
    • Be consistent
    • Allow for learning curve on new tools and technologies
    • Mike Bowler’s Session on Test Driven Design
  • 59. Resources
    • Ant
      • http://jakarta.apache.org/ant/
    • HttpUnit
      • http://httpunit.sourceforge.net/
    • JMeter
      • http://jakarta.apache.org/jmeter/
    • JUnit
      • http://www.junit.org
    • Latka
      • http://jakarta.apache.org/commons/latka
  • 60. Resources (Continued)
    • StrutsTestCase
      • http://strutstestcase.sourceforge.net/
    • JXWeb
      • http://qare.sourceforge.net/web/2001-12/products/jxweb/
    • Cactus
      • http://jakarta.apache.org/cactus/
    • XMLUnit
      • http://xmlunit.sourceforge.net
  • 61. Resources (Continued)
    • Homer http://webtool.rte.microsoft.com
    • HtmlUnit http://htmlunit.sourceforge.net
    • WebLoad http://www.webload.dk
    • Jelly http://jakarta.apache.org/commons/sandbox/jelly/
  • 62. Contact
    • Here all week
    • Q ‘n’ A sessions
    • [email_address]