Automated Testing of Web Applications using XML dIon Gillard, Multitask Consulting Pty Ltd

Agenda What is a Web Application? Testing Web Apps Code-based testing Data driven testing Tools Automating test case creation Automating tests Comparison to Commercial Tools

What is a Web Application?

Testing Web Apps

Code-based testing

Data driven testing

Tools

Automating test case creation

Automating tests

Comparison to Commercial Tools

What is a Web Application We’ll be concentrating on J2EE Web Applications A lot of the tools and concepts can be applied to other technologies Web Applications consist of: Server side resources JSPs, Servlets, Java code, configuration Client side resources HTML, images, JavaScript

We’ll be concentrating on J2EE Web Applications

A lot of the tools and concepts can be applied to other technologies

Web Applications consist of:

Server side resources

JSPs, Servlets, Java code, configuration

Client side resources

HTML, images, JavaScript

Testing Web Applications – ‘Field Experience’ Developers test localised changes have had the desired effect Without automated testing a complete regression test is impractical Application yo-yos from good to bad, working to broken frequently Users have little confidence in the application and wait for a ‘release’ Large effort goes into QA for a release

Developers test localised changes have had the desired effect

Without automated testing a complete regression test is impractical

Application yo-yos from good to bad, working to broken frequently

Users have little confidence in the application and wait for a ‘release’

Large effort goes into QA for a release

Testing Web Applications – Why? There are lots of ways to test, we aren’t going to cover all of them Testing usually has a purpose Does the code work Does the application perform well Does it respect security requirements Can it handle load Does it match business requirements … .

There are lots of ways to test, we aren’t going to cover all of them

Testing usually has a purpose

Does the code work

Does the application perform well

Does it respect security requirements

Can it handle load

Does it match business requirements

… .

Testing web applications - How We’re going to be focussing on ‘functional testing’, a.k.a. Black-box testing “ Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution condition” IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries

We’re going to be focussing on ‘functional testing’, a.k.a. Black-box testing

“ Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution condition”

IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries

Testing Web Applications - Implications Looking at web applications from a user’s perspective - Does it do what I want, how I want it done? What the code does is a side-effect for the user Users expect a web application to be an ‘application’, regardless of how the data is displayed or the functionality accessed User interface is still very important, however, so layout, along with look and feel must still be tested

Looking at web applications from a user’s perspective - Does it do what I want, how I want it done?

What the code does is a side-effect for the user

Users expect a web application to be an ‘application’, regardless of how the data is displayed or the functionality accessed

User interface is still very important, however, so layout, along with look and feel must still be tested

Testing Web Applications – What to test Interactions between the user and the application Expected Inputs input fields, hidden fields etc, query string or URL additions Unexpected inputs no password, field lengths, numerical domains etc Error conditions Messages to the user, code ‘failure’ Error 500 Expected processing Processing succeeds or fails Expected output Results of next page, database updates, messages queued etc

Interactions between the user and the application

Expected Inputs

input fields, hidden fields etc, query string or URL additions

Unexpected inputs

no password, field lengths, numerical domains etc

Error conditions

Messages to the user, code ‘failure’ Error 500

Expected processing

Processing succeeds or fails

Expected output

Results of next page, database updates, messages queued etc

Code based Testing Tools such as JUnit rely on developers creating ‘test cases’ in code to ensure correctness This has all the advantages and disadvantages of developers writing code Several tools have been released that allow this development to be more productive, less tedious and the code more reusable Writing low-level code makes it hard to focus on what the testing is for

Tools such as JUnit rely on developers creating ‘test cases’ in code to ensure correctness

This has all the advantages and disadvantages of developers writing code

Several tools have been released that allow this development to be more productive, less tedious and the code more reusable

Writing low-level code makes it hard to focus on what the testing is for

HttpUnit Freeware ‘Browser emulator’ for use in JUnit test cases (and other places) Written in Java, provided as a jar developers would use Takes care of authentication, cookies, page redirection Provides resulting pages as a DOM via NekoHTML HTML pages don’t have to have valid XML syntax to be parsed

Freeware ‘Browser emulator’ for use in JUnit test cases (and other places)

Written in Java, provided as a jar developers would use

Takes care of authentication, cookies, page redirection

Provides resulting pages as a DOM via NekoHTML

HTML pages don’t have to have valid XML syntax to be parsed

HttpUnit - Sample WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret"); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); response = conversation.getResponse( request ); assertTrue( "Login not rejected", response.getText().indexOf("Login failed") != -1 );

HttpUnit – API Classes are available for creating requests in various ways (WebRequest) Working with the HTTP response (WebResponse) return code, text, headers, cookies etc Working with HTML artifacts getDOM getForms(), getFormWithName(name), getTables() getLinks(), getLinkWith(text) Forms can be ‘submitted’ after retrieval from the response

Classes are available for creating requests in various ways (WebRequest)

Working with the HTTP response (WebResponse) return code, text, headers, cookies etc

Working with HTML artifacts

getDOM

getForms(), getFormWithName(name), getTables() getLinks(), getLinkWith(text)

Forms can be ‘submitted’ after retrieval from the response

HttpUnit – Pros and Cons Code can be repetitive and tedious Working at a low level, you can lose sight of what is being done Each WebRequest is created in isolation in a test Testing a web application is typically a single host and URI, code is needed to unify tests Based on JUnit, is easy to automate Simple clear API is easy to understand

Code can be repetitive and tedious

Working at a low level, you can lose sight of what is being done

Each WebRequest is created in isolation in a test

Testing a web application is typically a single host and URI, code is needed to unify tests

Based on JUnit, is easy to automate

Simple clear API is easy to understand

HtmlUnit Similar in scope to HttpUnit, but with a focus on HTML documents rather than the HTTP protocol details like requests, responses etc Better JavaScript support API is easier to relate to user interaction, e.g. clicking links and pressing buttons Wrapper objects for every html element Browser specific features can be used to fake the browser version and vendor

Similar in scope to HttpUnit, but with a focus on HTML documents rather than the HTTP protocol details like requests, responses etc

Better JavaScript support

API is easier to relate to user interaction, e.g. clicking links and pressing buttons

Wrapper objects for every html element

Browser specific features can be used to fake the browser version and vendor

HtmlUnit - continued Can track frames, iframes, pop up windows Listener to handle new windows being opened as part of interaction or JavaScript Can easily access the other windows Tab-order can be retrieved and checked Assertion helper methods for unique ids, accessibility Full access to the actual DOM i.e. editable

Can track frames, iframes, pop up windows

Listener to handle new windows being opened as part of interaction or JavaScript

Can easily access the other windows

Tab-order can be retrieved and checked

Assertion helper methods for unique ids, accessibility

Full access to the actual DOM i.e. editable

HtmlUnit – Sample form submit WebClient client = new WebClient(); URL url = new URL("http://htmlunit.sf.net/"); HtmlPage page = (HtmlPage) client.getPage(url); // find form by id HtmlForm form = page.getFormById(“mainForm”); // ‘type’ text into the userId field HtmlInput input = form.getInput(“userId”); input.setValue(“dion”); // submit – note existing values provided in the page // are ‘automatically’ filled in form.submit();

ServletUnit Companion tool shipped with HttpUnit For when you need to do less black box and more white box testing Simulates a servlet container by providing Mock Objects Provides API for accessing the ‘Invocation Context’ Other tools such as Cactus provide a fully-featured in-container environment

Companion tool shipped with HttpUnit

For when you need to do less black box and more white box testing

Simulates a servlet container by providing Mock Objects

Provides API for accessing the ‘Invocation Context’

Other tools such as Cactus provide a fully-featured in-container environment

ServletUnit – Sample code ServletRunner sr = new ServletRunner(); sr.registerServlet("myServlet", StatefulServlet.class.getName()); // create a client and request ServletUnitClient sc = sr.newClient(); WebRequest request = new PostMethodWebRequest( "http://test.meterware.com/myServlet" ); request.setParameter( "color", "red" ); InvocationContext ic = sc.newInvocation( request ); // call service ic.getServlet().service(request, ic.getResponse()); // test response, session, servlet instance variables etc assertNotNull("Session was not created", ic.getRequest().getSession( false ) );

StrutsTestCase – What is it? JUnit extension for testing code based on the Apache Struts web app framework Provides Mock Objects for simulating the servlet container Also provides an in-container Cactus extension (CactusStrutsTestCase) Switching approaches means changing parent class to/from StrutsTestCase

JUnit extension for testing code based on the Apache Struts web app framework

Provides Mock Objects for simulating the servlet container

Also provides an in-container Cactus extension (CactusStrutsTestCase)

Switching approaches means changing parent class to/from StrutsTestCase

StrutsTestCase – How does it work? Tests are based around Struts Actions There are methods available for hooking into the Struts request lifecycle Extra assert-like methods for checking the Struts environment, like forwards and action errors

Tests are based around Struts Actions

There are methods available for hooking into the Struts request lifecycle

Extra assert-like methods for checking the Struts environment, like forwards and action errors

StrutsTestCase - Sample … public void testFailedLogin() { // setup parameters to pass to the action addRequestParameter("username","deryl"); addRequestParameter("password","express"); // set up the path to identify the action setRequestPathInfo("/login"); // perform action processing actionPerform(); // check that the user has been forwarded to “login” verifyForward("login"); // make sure the error for key ‘error.password.mismatch’ // is present in the request verifyActionErrors(new String[] {"error.password.mismatch"}); // make sure the session state is correct assertNull((String) getSession().getAttribute( "authentication")); } …

Apache’s Cactus A framework for unit testing server side Java code JUnit extension to make it easier to test Servlets, EJBs, Tag Libraries, Servlet Filters etc Comes with ant tasks for automation Uses a ‘proxy’ mechanism to redirect test case requests to a live servlet container

A framework for unit testing server side Java code

JUnit extension to make it easier to test Servlets, EJBs, Tag Libraries, Servlet Filters etc

Comes with ant tasks for automation

Uses a ‘proxy’ mechanism to redirect test case requests to a live servlet container

Apache’s Cactus - API Provides 3 Test classes to subclass from ServletTestCase, JspTestCase and FilterTestCase Code testXXX methods as per usual, except you need to instantiate the object to be tested (e.g. servlet) beginXXX and endXXX methods can be coded to setup or post-process HTTP related parameters such as headers, return code, the response etc

Provides 3 Test classes to subclass from

ServletTestCase, JspTestCase and FilterTestCase

Code testXXX methods as per usual, except you need to instantiate the object to be tested (e.g. servlet)

beginXXX and endXXX methods can be coded to setup or post-process HTTP related parameters such as headers, return code, the response etc

Apache’s Cactus - Features Provides IDE integration for JBuilder, VAJava EJB unit testing using Servlet ‘proxy’ As it runs in-container, there is good documentation on setting up the proxy Can integrate with HttpUnit and StrutsTestCase

Provides IDE integration for JBuilder, VAJava

EJB unit testing using Servlet ‘proxy’

As it runs in-container, there is good documentation on setting up the proxy

Can integrate with HttpUnit and StrutsTestCase

XML-based Tools These tools rely on tests written in an XML format rather than java code Output varies – can be HTML They can be more easily generated They are more ‘user’ friendly Take away the reliance on technical skills to build tests Work at a higher level and allow more focus on what is being tested and how

These tools rely on tests written in an XML format rather than java code

Output varies – can be HTML

They can be more easily generated

They are more ‘user’ friendly

Take away the reliance on technical skills to build tests

Work at a higher level and allow more focus on what is being tested and how

Apache’s Latka Functional testing tool that uses XML to define a series of HTTP(S) requests Validations for the responses are defined as child elements of the requests Has a command prompt interface Can run tests via JUnit Can run tests interactively via a web-app Latka is moving to use Jelly as it’s XML processor

Functional testing tool that uses XML to define a series of HTTP(S) requests

Validations for the responses are defined as child elements of the requests

Has a command prompt interface

Can run tests via JUnit

Can run tests interactively via a web-app

Latka is moving to use Jelly as it’s XML processor

Apache’s Latka – Document Definition Suite top level element for a set of tests Provides default host, port and a label Session Wrapper for a set of requests that share HTTP state Request A HTTP request to be executed Path, method (“get”), host, port, label Validate Holds all validations for a specific request

Suite

top level element for a set of tests

Provides default host, port and a label

Session

Wrapper for a set of requests that share HTTP state

Request

A HTTP request to be executed

Path, method (“get”), host, port, label

Validate

Holds all validations for a specific request

Apache’s Latka - Validations Bytelength – length of the response Cookie – presence of a cookie maxRequestTime Regexp – tests the response matches a regular expression statusCode – e.g. 200, 404 XPath – test the response matches an XPath expression

Bytelength – length of the response

Cookie – presence of a cookie

maxRequestTime

Regexp – tests the response matches a regular expression

statusCode – e.g. 200, 404

XPath – test the response matches an XPath expression

Apache’s Latka - Sample <suite defaultHost=&quot;jakarta.apache.org&quot; label=&quot;Taglibs&quot;> <session> <request path=&quot;/taglibs/&quot; label=&quot;home page&quot;> <validate> <statusCode /> <regexp pattern=&quot;Welcome to Jakarta Taglibs!&quot;/> </validate> </request> <request path=“/logon”> <param><paramName>user</paramName> <paramValue>dion</paramValue></param> <param><paramName>password</paramName> <paramValue>dionspwd</paramValue></param> <validate><statusCode /> </request> </session> </suite>

<suite defaultHost=&quot;jakarta.apache.org&quot; label=&quot;Taglibs&quot;>

<session>

<request path=&quot;/taglibs/&quot; label=&quot;home page&quot;>

<validate>

<statusCode />

<regexp pattern=&quot;Welcome to Jakarta Taglibs!&quot;/>

</validate>

</request>

<request path=“/logon”>

<param><paramName>user</paramName>

<paramValue>dion</paramValue></param>

<param><paramName>password</paramName>

<paramValue>dionspwd</paramValue></param>

<validate><statusCode />

</request>

</session>

</suite>

Apache’s Latka – Running Tests Command prompt Latka.bat file:./TestSite.xml JUnit public static Test suite() { TestSuite suite = new TestSuite(); String fileName = &quot;tests/samples/TestCommonsWebsite.xml&quot;; suite.addTest( JUnitTestAdapter.createTestFromFile(fileName)); return suite; }

Command prompt

Latka.bat file:./TestSite.xml

JUnit

public static Test suite() {

TestSuite suite = new TestSuite();

String fileName = &quot;tests/samples/TestCommonsWebsite.xml&quot;; suite.addTest(

JUnitTestAdapter.createTestFromFile(fileName));

return suite;

}

Apache’s Latka Also provides validators as tags for Jelly Automate via JUnit Ant task Listeners can be attached to pick up test events as they happen XML and Simple (console) reporter provided XSL Style sheet provided to produce text from report

Also provides validators as tags for Jelly

Automate via JUnit Ant task

Listeners can be attached to pick up test events as they happen

XML and Simple (console) reporter provided

XSL Style sheet provided to produce text from report

Apache’s Jelly Java and XML based scripting and processing engine Jelly ‘scripts’ are XML documents which get parsed Scripts are then ‘executed’ to produce XML, HTML, text etc XML elements can be bound to java code similar to JSP custom tags and Ant tasks

Java and XML based scripting and processing engine

Jelly ‘scripts’ are XML documents which get parsed

Scripts are then ‘executed’ to produce XML, HTML, text etc

XML elements can be bound to java code similar to JSP custom tags and Ant tasks

Apache’s Jelly - Taglibs Core: catch, choose, expr, forEach, if, file, import, include, jelly, new, otherwise, set, thread, when, whitespace Xml: attribute, copy, copyOf, element, expr, forEach, if, parse, set Define: attribute, bean, invoke, invokeBody, jellyBean, script, tag, taglib, Sql: dateParam, driver, param, query, setDataSource, transaction, update

Core: catch, choose, expr, forEach, if, file, import, include, jelly, new, otherwise, set, thread, when, whitespace

Xml: attribute, copy, copyOf, element, expr, forEach, if, parse, set

Define: attribute, bean, invoke, invokeBody, jellyBean, script, tag, taglib,

Sql: dateParam, driver, param, query, setDataSource, transaction, update

Apache’s Jelly – Taglibs (Continued) JSL: applyTemplates, style, stylesheet, template Ant: ant, filescanner Werkz: attain, attainGoal, goal, postAction, postGoal, preAction, preGoal, project Jeez: target, tagDef Log: debug, error, fatal, info, trace, warn Ojb: broker, store

JSL: applyTemplates, style, stylesheet, template

Ant: ant, filescanner

Werkz: attain, attainGoal, goal, postAction, postGoal, preAction, preGoal, project

Jeez: target, tagDef

Log: debug, error, fatal, info, trace, warn

Ojb: broker, store

Apache’s Jelly – Taglibs (Continued) JMS: connection, destination, mapEntry, mapMessage, message, objectMessage, property, receive, send, textMessage Validate: verifier, validate, assertValid Http: delete, get, header, post, put Interaction: ask Antlr: antlr, grammar Util: tokenize Html: parse

JMS: connection, destination, mapEntry, mapMessage, message, objectMessage, property, receive, send, textMessage

Validate: verifier, validate, assertValid

Http: delete, get, header, post, put

Interaction: ask

Antlr: antlr, grammar

Util: tokenize

Html: parse

Apache’s Jelly – Taglibs (Continued) JUnit: assert, assertEquals, case, fail, run, suite Swing: action, component, windowListener Quartz: cronTrigger, job, waitForScheduler Betwixt: introspector, parse

JUnit: assert, assertEquals, case, fail, run, suite

Swing: action, component, windowListener

Quartz: cronTrigger, job, waitForScheduler

Betwixt: introspector, parse

Apache’s Jelly - Testing Provides a JUnit tag library a HTTP tag library a ‘cron’ like facility an expression language Integration with JMS, SQL etc allows page results to be checked against the database or queue

Provides

a JUnit tag library

a HTTP tag library

a ‘cron’ like facility

an expression language

Integration with JMS, SQL etc allows page results to be checked against the database or queue

Apache’s Jelly - Sample <?xml version=&quot;1.0&quot;?> <j:jelly xmlns:j=&quot;jelly:core&quot; xmlns:log=&quot;jelly:log&quot; xmlns:v=&quot;jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary&quot; xmlns=&quot;jelly:org.apache.commons.latka.jelly.HttpTagLibrary&quot; trim=&quot;false&quot;> <session> <get var=&quot;mtc&quot; uri=&quot;http://www.multitask.com.au/&quot;/> <j:if test='${mtc.statusCode == 200}'> request ok </j:if> Results for mtc url are: http return code = ${mtc.statusCode} http status text = '${mtc.statusText}' size of result = ${mtc.responseBodyAsString.length()} response time = ${mtc.responseTime} </session> </jelly>

<?xml version=&quot;1.0&quot;?>

<j:jelly

xmlns:j=&quot;jelly:core&quot;

xmlns:log=&quot;jelly:log&quot;

xmlns:v=&quot;jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary&quot;

xmlns=&quot;jelly:org.apache.commons.latka.jelly.HttpTagLibrary&quot;

trim=&quot;false&quot;>

<session>

<get var=&quot;mtc&quot; uri=&quot;http://www.multitask.com.au/&quot;/>

<j:if test='${mtc.statusCode == 200}'>

request ok

</j:if>

Results for mtc url are:

http return code = ${mtc.statusCode}

http status text = '${mtc.statusText}'

size of result = ${mtc.responseBodyAsString.length()}

response time = ${mtc.responseTime}

</session>

</jelly>

Apache’s Jelly - Sample <?xml version=&quot;1.0&quot;?> <j:jelly xmlns:j=&quot;jelly:core“ xmlns:log=&quot;jelly:log&quot; xmlns:v=&quot;jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary&quot; xmlns=&quot;jelly:org.apache.commons.latka.jelly.HttpTagLibrary“> <session> <post var=&quot;jdc“ uri=&quot;http://developer.java.sun.com/servlet/SessionServlet&quot;> <parameter name=&quot;action&quot; value=&quot;login&quot; /> <parameter name=&quot;url&quot; value=&quot;/developer/index.jshtml&quot; /> <parameter name=&quot;UserId&quot; value=&quot;XXXX&quot; /> <parameter name=&quot;Password&quot; value=&quot;XXXX&quot; /> </post> jdc login result = ${jdc.statusCode} good login = ${jdc.responseBodyAsString.indexOf(&quot;Sorry!&quot;) == &quot;-1&quot;} <v:regexp pattern=&quot;Sox.y!&quot; var=&quot;jdc&quot;> <request-failed var=&quot;jdc&quot;>bad pattern 1</request-failed> </v:regexp> </session> </jelly>

<?xml version=&quot;1.0&quot;?>

<j:jelly xmlns:j=&quot;jelly:core“ xmlns:log=&quot;jelly:log&quot;

xmlns:v=&quot;jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary&quot;

xmlns=&quot;jelly:org.apache.commons.latka.jelly.HttpTagLibrary“>

<session>

<post var=&quot;jdc“

uri=&quot;http://developer.java.sun.com/servlet/SessionServlet&quot;>

<parameter name=&quot;action&quot; value=&quot;login&quot; />

<parameter name=&quot;url&quot; value=&quot;/developer/index.jshtml&quot; />

<parameter name=&quot;UserId&quot; value=&quot;XXXX&quot; />

<parameter name=&quot;Password&quot; value=&quot;XXXX&quot; />

</post>

jdc login result = ${jdc.statusCode}

good login = ${jdc.responseBodyAsString.indexOf(&quot;Sorry!&quot;) == &quot;-1&quot;}

<v:regexp pattern=&quot;Sox.y!&quot; var=&quot;jdc&quot;>

<request-failed var=&quot;jdc&quot;>bad pattern 1</request-failed>

</v:regexp>

</session>

</jelly>

XMLUnit JUnit extension providing comparison between XML docs XML can come from lots of places: Strings, DOM, Reader Accepts ‘browser-friendly’ HTML as input Can validate against a DTD

JUnit extension providing comparison between XML docs

XML can come from lots of places: Strings, DOM, Reader

Accepts ‘browser-friendly’ HTML as input

Can validate against a DTD

JXWeb XML Script ‘language’ and extension to JXUnit Uses HttpUnit under the covers No JUnit integration Not as easily extendable as Jelly

XML Script ‘language’ and extension to JXUnit

Uses HttpUnit under the covers

No JUnit integration

Not as easily extendable as Jelly

JXWeb – Scripting Language jxw – top level element httpGet, postXml – to retrieve data setRequestParameter, setCookie, setHeader getLink, getTableValue, getFormParameter, getForm, getTable ifEqual, isNull, loop, doLoop, exception, ifNull, save, subst, ifEqualLength, isEqual Echo, set, loadProperties

jxw – top level element

httpGet, postXml – to retrieve data

setRequestParameter, setCookie, setHeader

getLink, getTableValue, getFormParameter, getForm, getTable

ifEqual, isNull, loop, doLoop, exception, ifNull, save, subst, ifEqualLength, isEqual

Echo, set, loadProperties

JXWeb - Sample <jxw> <set name=&quot;req&quot; value=&quot;www.testsite.com&quot;/> <httpGet response=&quot;response1&quot;/> <save name=&quot;respText&quot; file=&quot;list.html&quot;/> <getTable name=&quot;myTable&quot; tableIndex=&quot;0&quot;/> <getTableValue name=&quot;tableValue&quot; table=&quot;myTable“ row=&quot;0&quot; column=&quot;0&quot;/> <isEqual name=“tableValue&quot; value=&quot;Every Good Beer Deserves Froth“ message=“page is missing froth&quot;/> </jxw>

AntEater – Ant Functional Testing Testing framework designed around Ant Supplied as a set of Ant tasks and types Integrates easily into an existing build or IDE Comes with a servlet container built-in to allow for ‘call backs’

Testing framework designed around Ant

Supplied as a set of Ant tasks and types

Integrates easily into an existing build or IDE

Comes with a servlet container built-in to allow for ‘call backs’

Ant Eater - Tasks Action Tasks httpRequest soapRequest Listener Match – groups multiple tests Tests Listening: method, parameter, sendResponse Header, contentEquals, regexp, responseCode, xpath, relaxng

Action Tasks

httpRequest

soapRequest

Listener

Match – groups multiple tests

Tests

Listening: method, parameter, sendResponse

Header, contentEquals, regexp, responseCode, xpath, relaxng

Ant Eater – Other tags Session Logger – minimal, plain, colour, xml forEach Group – a way of grouping variables, sessions and loggers servletContainer – start internal tomcat Deploy – deploy web apps

Session

Logger – minimal, plain, colour, xml

forEach

Group – a way of grouping variables, sessions and loggers

servletContainer – start internal tomcat

Deploy – deploy web apps

Ant Eater - Sample … <target name=&quot;checkonline&quot;> <httpRequest path=&quot;/&quot;> <match assign=&quot;online&quot;><responseCode value=&quot;200&quot;/> </match> </httpRequest> </target> <target name=&quot;hitserver&quot; if=&quot;online&quot;> <echo>We're online!</echo> <httpRequest path=&quot;/&quot;> <match assign=&quot;apache&quot;> <header name=&quot;Server“ value=&quot;Apache/1.3.24 (Unix) Debian GNU/Linux&quot;/> </match> <match assign=&quot;unknownserver“/> </httpRequest> </target>

Limitations XML as a programming language isn’t Not as expressive Tedious syntax for non-techies Usually not as extensible Limits the interaction with external data sources Other than Jelly, most don’t have a mechanism for dealing with java objects outside their framework

XML as a programming language isn’t

Not as expressive

Tedious syntax for non-techies

Usually not as extensible

Limits the interaction with external data sources

Other than Jelly, most don’t have a mechanism for dealing with java objects outside their framework

Automating Test Case Creation One of the main reasons developers give for limited testing is limited time Developers are actually happy to test if it isn’t a significant drain on achievements Testing pages or servlets helps developers think of error conditions that could easily crop up

One of the main reasons developers give for limited testing is limited time

Developers are actually happy to test if it isn’t a significant drain on achievements

Testing pages or servlets helps developers think of error conditions that could easily crop up

Automated Creation - JMeter JMeter is often thought of as a load testing tool only It also has a ‘proxy server’ mode where you can record browser activity It can filter out requests for images or other resources using a regexp It can also be used to ‘assert’ text is present on a request

JMeter is often thought of as a load testing tool only

It also has a ‘proxy server’ mode where you can record browser activity

It can filter out requests for images or other resources using a regexp

It can also be used to ‘assert’ text is present on a request

Automated Creation – ‘Homer’ Microsoft freebie tool in IIS 5.0 resource kit Under windows acts as a proxy and records requests coming in Usually used for load testing Stores data in an Access .mdb file Simply queries can transform this into XML and then into Latka or Jelly etc Merged into VS.NET toolset

Microsoft freebie tool in IIS 5.0 resource kit

Under windows acts as a proxy and records requests coming in

Usually used for load testing

Stores data in an Access .mdb file

Simply queries can transform this into XML and then into Latka or Jelly etc

Merged into VS.NET toolset

Automated Testing Once the tests are created the next thing is to get them run on a regular basis This means application and developer confidence grows Depending on the tool this can be achieved using the operating system and a build tool

Once the tests are created the next thing is to get them run on a regular basis

This means application and developer confidence grows

Depending on the tool this can be achieved using the operating system and a build tool

Apache’s Ant Java based build tool Has JUnit tasks as part of the ‘optional’ tasks Can be scheduled using operating system constructs Can provide email about failures

Java based build tool

Has JUnit tasks as part of the ‘optional’ tasks

Can be scheduled using operating system constructs

Can provide email about failures

CruiseControl Open source tool from ThoughtWorks Provides a way to kick off Ant builds based on source repository changes This allows the automation of the tests to be done when the source code changes Ensures that the application passes all tests Flags immediately when the application breaks

Open source tool from ThoughtWorks

Provides a way to kick off Ant builds based on source repository changes

This allows the automation of the tests to be done when the source code changes

Ensures that the application passes all tests

Flags immediately when the application breaks

Commercial Tools LoadRunner from Mercury Interactive Provides access to XML data from the scripting language More geared toward load testing Astra QuickTest from Mercury Interactive Web testing, built-in script creation via browser, cross browser tests from same test case, extract details from browser objects Supports Flash, Real etc

LoadRunner from Mercury Interactive

Provides access to XML data from the scripting language

More geared toward load testing

Astra QuickTest from Mercury Interactive

Web testing, built-in script creation via browser, cross browser tests from same test case, extract details from browser objects

Supports Flash, Real etc

Commercial Tools Rational RobotJ Eclipse based tool Uses proprietary scripting language Does auto-record, regular expressions Approx 1.3GB space WebLoad Similar features to higher cost tools Multiple server support, higher load focus Supports JavaScript GUI and Command line driven

Rational RobotJ

Eclipse based tool

Uses proprietary scripting language

Does auto-record, regular expressions

Approx 1.3GB space

WebLoad

Similar features to higher cost tools

Multiple server support, higher load focus

Supports JavaScript

GUI and Command line driven

Commercial Tools - continued High end Java IDEs such as WebSphere Studio Application Developer 5 now include testing tools Some can not be automated Compete with other vendors in same tool space

High end Java IDEs such as WebSphere Studio Application Developer 5 now include testing tools

Some can not be automated

Compete with other vendors in same tool space

Summary There are simple tools available now that can alleviate the drudgery of functional testing Users can help write the functional tests Automation and repetition are key Choose how far you want to take testing Be consistent Allow for learning curve on new tools and technologies Mike Bowler’s Session on Test Driven Design

There are simple tools available now that can alleviate the drudgery of functional testing

Users can help write the functional tests

Automation and repetition are key

Choose how far you want to take testing

Be consistent

Allow for learning curve on new tools and technologies

Mike Bowler’s Session on Test Driven Design

Resources Ant http://jakarta.apache.org/ant/ HttpUnit http://httpunit.sourceforge.net/ JMeter http://jakarta.apache.org/jmeter/ JUnit http://www.junit.org Latka http://jakarta.apache.org/commons/latka

Ant

http://jakarta.apache.org/ant/

HttpUnit

http://httpunit.sourceforge.net/

JMeter

http://jakarta.apache.org/jmeter/

JUnit

http://www.junit.org

Latka

http://jakarta.apache.org/commons/latka

Resources (Continued) StrutsTestCase http://strutstestcase.sourceforge.net/ JXWeb http://qare.sourceforge.net/web/2001-12/products/jxweb/ Cactus http://jakarta.apache.org/cactus/ XMLUnit http://xmlunit.sourceforge.net

StrutsTestCase

http://strutstestcase.sourceforge.net/

JXWeb

http://qare.sourceforge.net/web/2001-12/products/jxweb/

Cactus

http://jakarta.apache.org/cactus/

XMLUnit

http://xmlunit.sourceforge.net

Resources (Continued) Homer http://webtool.rte.microsoft.com HtmlUnit http://htmlunit.sourceforge.net WebLoad http://www.webload.dk Jelly http://jakarta.apache.org/commons/sandbox/jelly/

Homer http://webtool.rte.microsoft.com

HtmlUnit http://htmlunit.sourceforge.net

WebLoad http://www.webload.dk

Jelly http://jakarta.apache.org/commons/sandbox/jelly/

Contact Here all week Q ‘n’ A sessions [email_address]

Here all week

Q ‘n’ A sessions

[email_address]