Automated Testing Of Web Applications Using XML

Automated Testing of Web Applications using XML dIon Gillard, Multitask Consulting Pty Ltd

Agenda
What is a Web Application?
Testing Web Apps
Code-based testing
Data driven testing
Tools
Automating test case creation
Automating tests
Comparison to Commercial Tools

What is a Web Application
We’ll be concentrating on J2EE Web Applications
A lot of the tools and concepts can be applied to other technologies
Web Applications consist of:
Server side resources
JSPs, Servlets, Java code, configuration
Client side resources
HTML, images, JavaScript

Testing Web Applications – ‘Field Experience’
Developers test localised changes have had the desired effect
Without automated testing a complete regression test is impractical
Application yo-yos from good to bad, working to broken frequently
Users have little confidence in the application and wait for a ‘release’
Large effort goes into QA for a release

Testing Web Applications – Why?
There are lots of ways to test, we aren’t going to cover all of them
Testing usually has a purpose
Does the code work
Does the application perform well
Does it respect security requirements
Can it handle load
Does it match business requirements
… .

Testing web applications - How
We’re going to be focussing on ‘functional testing’, a.k.a. Black-box testing
“ Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution condition”
IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries

Testing Web Applications - Implications
Looking at web applications from a user’s perspective - Does it do what I want, how I want it done?
What the code does is a side-effect for the user
Users expect a web application to be an ‘application’, regardless of how the data is displayed or the functionality accessed
User interface is still very important, however, so layout, along with look and feel must still be tested

Testing Web Applications – What to test
Interactions between the user and the application
Expected Inputs
input fields, hidden fields etc, query string or URL additions
Unexpected inputs
no password, field lengths, numerical domains etc
Error conditions
Messages to the user, code ‘failure’ Error 500
Expected processing
Processing succeeds or fails
Expected output
Results of next page, database updates, messages queued etc

Code based Testing
Tools such as JUnit rely on developers creating ‘test cases’ in code to ensure correctness
This has all the advantages and disadvantages of developers writing code
Several tools have been released that allow this development to be more productive, less tedious and the code more reusable
Writing low-level code makes it hard to focus on what the testing is for

HttpUnit
Freeware ‘Browser emulator’ for use in JUnit test cases (and other places)
Written in Java, provided as a jar developers would use
Takes care of authentication, cookies, page redirection
Provides resulting pages as a DOM via NekoHTML
HTML pages don’t have to have valid XML syntax to be parsed

HttpUnit - Sample WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( "http://www.meterware.com/servlet/TopSecret"); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); response = conversation.getResponse( request ); assertTrue( "Login not rejected", response.getText().indexOf("Login failed") != -1 );

HttpUnit – API
Classes are available for creating requests in various ways (WebRequest)
Working with the HTTP response (WebResponse) return code, text, headers, cookies etc
Working with HTML artifacts
getDOM
getForms(), getFormWithName(name), getTables() getLinks(), getLinkWith(text)
Forms can be ‘submitted’ after retrieval from the response

HttpUnit – Pros and Cons
Code can be repetitive and tedious
Working at a low level, you can lose sight of what is being done
Each WebRequest is created in isolation in a test
Testing a web application is typically a single host and URI, code is needed to unify tests
Based on JUnit, is easy to automate
Simple clear API is easy to understand

HtmlUnit
Similar in scope to HttpUnit, but with a focus on HTML documents rather than the HTTP protocol details like requests, responses etc
Better JavaScript support
API is easier to relate to user interaction, e.g. clicking links and pressing buttons
Wrapper objects for every html element
Browser specific features can be used to fake the browser version and vendor

HtmlUnit - continued
Can track frames, iframes, pop up windows
Listener to handle new windows being opened as part of interaction or JavaScript
Can easily access the other windows
Tab-order can be retrieved and checked
Assertion helper methods for unique ids, accessibility
Full access to the actual DOM i.e. editable

HtmlUnit – Sample form submit WebClient client = new WebClient(); URL url = new URL("http://htmlunit.sf.net/"); HtmlPage page = (HtmlPage) client.getPage(url); // find form by id HtmlForm form = page.getFormById(“mainForm”); // ‘type’ text into the userId field HtmlInput input = form.getInput(“userId”); input.setValue(“dion”); // submit – note existing values provided in the page // are ‘automatically’ filled in form.submit();

ServletUnit
Companion tool shipped with HttpUnit
For when you need to do less black box and more white box testing
Simulates a servlet container by providing Mock Objects
Provides API for accessing the ‘Invocation Context’
Other tools such as Cactus provide a fully-featured in-container environment

ServletUnit – Sample code ServletRunner sr = new ServletRunner(); sr.registerServlet("myServlet", StatefulServlet.class.getName()); // create a client and request ServletUnitClient sc = sr.newClient(); WebRequest request = new PostMethodWebRequest( "http://test.meterware.com/myServlet" ); request.setParameter( "color", "red" ); InvocationContext ic = sc.newInvocation( request ); // call service ic.getServlet().service(request, ic.getResponse()); // test response, session, servlet instance variables etc assertNotNull("Session was not created", ic.getRequest().getSession( false ) );

StrutsTestCase – What is it?
JUnit extension for testing code based on the Apache Struts web app framework
Provides Mock Objects for simulating the servlet container
Also provides an in-container Cactus extension (CactusStrutsTestCase)
Switching approaches means changing parent class to/from StrutsTestCase

StrutsTestCase – How does it work?
Tests are based around Struts Actions
There are methods available for hooking into the Struts request lifecycle
Extra assert-like methods for checking the Struts environment, like forwards and action errors

StrutsTestCase - Sample … public void testFailedLogin() { // setup parameters to pass to the action addRequestParameter("username","deryl"); addRequestParameter("password","express"); // set up the path to identify the action setRequestPathInfo("/login"); // perform action processing actionPerform(); // check that the user has been forwarded to “login” verifyForward("login"); // make sure the error for key ‘error.password.mismatch’ // is present in the request verifyActionErrors(new String[] {"error.password.mismatch"}); // make sure the session state is correct assertNull((String) getSession().getAttribute( "authentication")); } …

Apache’s Cactus
A framework for unit testing server side Java code
JUnit extension to make it easier to test Servlets, EJBs, Tag Libraries, Servlet Filters etc
Comes with ant tasks for automation
Uses a ‘proxy’ mechanism to redirect test case requests to a live servlet container

Apache’s Cactus - API
Provides 3 Test classes to subclass from
ServletTestCase, JspTestCase and FilterTestCase
Code testXXX methods as per usual, except you need to instantiate the object to be tested (e.g. servlet)
beginXXX and endXXX methods can be coded to setup or post-process HTTP related parameters such as headers, return code, the response etc

Apache’s Cactus - Features
Provides IDE integration for JBuilder, VAJava
EJB unit testing using Servlet ‘proxy’
As it runs in-container, there is good documentation on setting up the proxy
Can integrate with HttpUnit and StrutsTestCase

XML-based Tools
These tools rely on tests written in an XML format rather than java code
Output varies – can be HTML
They can be more easily generated
They are more ‘user’ friendly
Take away the reliance on technical skills to build tests
Work at a higher level and allow more focus on what is being tested and how

Apache’s Latka
Functional testing tool that uses XML to define a series of HTTP(S) requests
Validations for the responses are defined as child elements of the requests
Has a command prompt interface
Can run tests via JUnit
Can run tests interactively via a web-app
Latka is moving to use Jelly as it’s XML processor

Apache’s Latka – Document Definition
Suite
top level element for a set of tests
Provides default host, port and a label
Session
Wrapper for a set of requests that share HTTP state
Request
A HTTP request to be executed
Path, method (“get”), host, port, label
Validate
Holds all validations for a specific request

Apache’s Latka - Validations
Bytelength – length of the response
Cookie – presence of a cookie
maxRequestTime
Regexp – tests the response matches a regular expression
statusCode – e.g. 200, 404
XPath – test the response matches an XPath expression

Apache’s Latka - Sample
<suite defaultHost="jakarta.apache.org" label="Taglibs">
<session>
<request path="/taglibs/" label="home page">
<validate>
<statusCode />
<regexp pattern="Welcome to Jakarta Taglibs!"/>
</validate>
</request>
<request path=“/logon”>
<param><paramName>user</paramName>
<paramValue>dion</paramValue></param>
<param><paramName>password</paramName>
<paramValue>dionspwd</paramValue></param>
<validate><statusCode />
</request>
</session>
</suite>

Apache’s Latka – Running Tests
Command prompt
Latka.bat file:./TestSite.xml
JUnit
public static Test suite() {
TestSuite suite = new TestSuite();
String fileName = "tests/samples/TestCommonsWebsite.xml"; suite.addTest(
JUnitTestAdapter.createTestFromFile(fileName));
return suite;
}

Apache’s Latka
Also provides validators as tags for Jelly
Automate via JUnit Ant task
Listeners can be attached to pick up test events as they happen
XML and Simple (console) reporter provided
XSL Style sheet provided to produce text from report

Apache’s Jelly
Java and XML based scripting and processing engine
Jelly ‘scripts’ are XML documents which get parsed
Scripts are then ‘executed’ to produce XML, HTML, text etc
XML elements can be bound to java code similar to JSP custom tags and Ant tasks

Apache’s Jelly - Taglibs
Core: catch, choose, expr, forEach, if, file, import, include, jelly, new, otherwise, set, thread, when, whitespace
Xml: attribute, copy, copyOf, element, expr, forEach, if, parse, set
Define: attribute, bean, invoke, invokeBody, jellyBean, script, tag, taglib,
Sql: dateParam, driver, param, query, setDataSource, transaction, update

Apache’s Jelly – Taglibs (Continued)
JSL: applyTemplates, style, stylesheet, template
Ant: ant, filescanner
Werkz: attain, attainGoal, goal, postAction, postGoal, preAction, preGoal, project
Jeez: target, tagDef
Log: debug, error, fatal, info, trace, warn
Ojb: broker, store

JMS: connection, destination, mapEntry, mapMessage, message, objectMessage, property, receive, send, textMessage
Validate: verifier, validate, assertValid
Http: delete, get, header, post, put
Interaction: ask
Antlr: antlr, grammar
Util: tokenize
Html: parse

JUnit: assert, assertEquals, case, fail, run, suite
Swing: action, component, windowListener
Quartz: cronTrigger, job, waitForScheduler
Betwixt: introspector, parse

Apache’s Jelly - Testing
Provides
a JUnit tag library
a HTTP tag library
a ‘cron’ like facility
an expression language
Integration with JMS, SQL etc allows page results to be checked against the database or queue

Apache’s Jelly - Sample
<?xml version="1.0"?>
<j:jelly
xmlns:j="jelly:core"
xmlns:log="jelly:log"
xmlns:v="jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary"
xmlns="jelly:org.apache.commons.latka.jelly.HttpTagLibrary"
trim="false">
<session>
<get var="mtc" uri="http://www.multitask.com.au/"/>
<j:if test='${mtc.statusCode == 200}'>
request ok
</j:if>
Results for mtc url are:
http return code = ${mtc.statusCode}
http status text = '${mtc.statusText}'
size of result = ${mtc.responseBodyAsString.length()}
response time = ${mtc.responseTime}
</session>
</jelly>

Apache’s Jelly - Sample
<?xml version="1.0"?>
<j:jelly xmlns:j="jelly:core“ xmlns:log="jelly:log"
xmlns:v="jelly:org.apache.commons.latka.jelly.validators.HttpValidatorTagLibrary"
xmlns="jelly:org.apache.commons.latka.jelly.HttpTagLibrary“>
<session>
<post var="jdc“
uri="http://developer.java.sun.com/servlet/SessionServlet">
<parameter name="action" value="login" />
<parameter name="url" value="/developer/index.jshtml" />
<parameter name="UserId" value="XXXX" />
<parameter name="Password" value="XXXX" />
</post>
jdc login result = ${jdc.statusCode}
good login = ${jdc.responseBodyAsString.indexOf("Sorry!") == "-1"}
<v:regexp pattern="Sox.y!" var="jdc">
<request-failed var="jdc">bad pattern 1</request-failed>
</v:regexp>
</session>
</jelly>

XMLUnit
JUnit extension providing comparison between XML docs
XML can come from lots of places: Strings, DOM, Reader
Accepts ‘browser-friendly’ HTML as input
Can validate against a DTD

JXWeb
XML Script ‘language’ and extension to JXUnit
Uses HttpUnit under the covers
No JUnit integration
Not as easily extendable as Jelly

JXWeb – Scripting Language
jxw – top level element
httpGet, postXml – to retrieve data
setRequestParameter, setCookie, setHeader
getLink, getTableValue, getFormParameter, getForm, getTable
ifEqual, isNull, loop, doLoop, exception, ifNull, save, subst, ifEqualLength, isEqual
Echo, set, loadProperties

JXWeb - Sample <jxw> <set name="req" value="www.testsite.com"/> <httpGet response="response1"/> <save name="respText" file="list.html"/> <getTable name="myTable" tableIndex="0"/> <getTableValue name="tableValue" table="myTable“ row="0" column="0"/> <isEqual name=“tableValue" value="Every Good Beer Deserves Froth“ message=“page is missing froth"/> </jxw>

AntEater – Ant Functional Testing
Testing framework designed around Ant
Supplied as a set of Ant tasks and types
Integrates easily into an existing build or IDE
Comes with a servlet container built-in to allow for ‘call backs’

Ant Eater - Tasks
Action Tasks
httpRequest
soapRequest
Listener
Match – groups multiple tests
Tests
Listening: method, parameter, sendResponse
Header, contentEquals, regexp, responseCode, xpath, relaxng

Ant Eater – Other tags
Session
Logger – minimal, plain, colour, xml
forEach
Group – a way of grouping variables, sessions and loggers
servletContainer – start internal tomcat
Deploy – deploy web apps

Ant Eater - Sample … <target name="checkonline"> <httpRequest path="/"> <match assign="online"><responseCode value="200"/> </match> </httpRequest> </target> <target name="hitserver" if="online"> <echo>We're online!</echo> <httpRequest path="/"> <match assign="apache"> <header name="Server“ value="Apache/1.3.24 (Unix) Debian GNU/Linux"/> </match> <match assign="unknownserver“/> </httpRequest> </target>

Limitations
XML as a programming language isn’t
Not as expressive
Tedious syntax for non-techies
Usually not as extensible
Limits the interaction with external data sources
Other than Jelly, most don’t have a mechanism for dealing with java objects outside their framework

Automating Test Case Creation
One of the main reasons developers give for limited testing is limited time
Developers are actually happy to test if it isn’t a significant drain on achievements
Testing pages or servlets helps developers think of error conditions that could easily crop up

Automated Creation - JMeter
JMeter is often thought of as a load testing tool only
It also has a ‘proxy server’ mode where you can record browser activity
It can filter out requests for images or other resources using a regexp
It can also be used to ‘assert’ text is present on a request

Automated Creation – ‘Homer’
Microsoft freebie tool in IIS 5.0 resource kit
Under windows acts as a proxy and records requests coming in
Usually used for load testing
Stores data in an Access .mdb file
Simply queries can transform this into XML and then into Latka or Jelly etc
Merged into VS.NET toolset

Automated Testing
Once the tests are created the next thing is to get them run on a regular basis
This means application and developer confidence grows
Depending on the tool this can be achieved using the operating system and a build tool

Apache’s Ant
Java based build tool
Has JUnit tasks as part of the ‘optional’ tasks
Can be scheduled using operating system constructs
Can provide email about failures

CruiseControl
Open source tool from ThoughtWorks
Provides a way to kick off Ant builds based on source repository changes
This allows the automation of the tests to be done when the source code changes
Ensures that the application passes all tests
Flags immediately when the application breaks

Commercial Tools
LoadRunner from Mercury Interactive
Provides access to XML data from the scripting language
More geared toward load testing
Astra QuickTest from Mercury Interactive
Web testing, built-in script creation via browser, cross browser tests from same test case, extract details from browser objects
Supports Flash, Real etc

Commercial Tools
Rational RobotJ
Eclipse based tool
Uses proprietary scripting language
Does auto-record, regular expressions
Approx 1.3GB space
WebLoad
Similar features to higher cost tools
Multiple server support, higher load focus
Supports JavaScript
GUI and Command line driven

Commercial Tools - continued
High end Java IDEs such as WebSphere Studio Application Developer 5 now include testing tools
Some can not be automated
Compete with other vendors in same tool space

Summary
There are simple tools available now that can alleviate the drudgery of functional testing
Users can help write the functional tests
Automation and repetition are key
Choose how far you want to take testing
Be consistent
Allow for learning curve on new tools and technologies
Mike Bowler’s Session on Test Driven Design

Resources
Ant
http://jakarta.apache.org/ant/
HttpUnit
http://httpunit.sourceforge.net/
JMeter
http://jakarta.apache.org/jmeter/
JUnit
http://www.junit.org
Latka
http://jakarta.apache.org/commons/latka

Resources (Continued)
StrutsTestCase
http://strutstestcase.sourceforge.net/
JXWeb
http://qare.sourceforge.net/web/2001-12/products/jxweb/
Cactus
http://jakarta.apache.org/cactus/
XMLUnit
http://xmlunit.sourceforge.net

Resources (Continued)
Homer http://webtool.rte.microsoft.com
HtmlUnit http://htmlunit.sourceforge.net
WebLoad http://www.webload.dk
Jelly http://jakarta.apache.org/commons/sandbox/jelly/

Contact
Here all week
Q ‘n’ A sessions
[email_address]

http://at4qa.blogspot.com	92
http://automated-testing.info	44
http://www.slideshare.net	37
http://blog.bugpub.com	15
http://www.automated-testing.info	9
http://www.at4qa.blogspot.com	3
http://www.blogger.com	2
http://webcache.googleusercontent.com	1

Automated Testing Of Web Applications Using XML

by diongillard on Oct 18, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

8 Embeds 203

Statistics

Automated Testing Of Web Applications Using XML — Presentation Transcript