Agile Java Testing With
Open Source Frameworks
Presented by Mr. Viraf Karai
**********
on Wed. Feb. 25, 2009
1

Summary of this talk
Why test Why test frameworks
 
Why automate tests Unit test frameworks
 
Kinds of tests Integ test frameworks
 
When do you test Load/perf test
 
frameworks
Unit testing

Other tools (CI)

Integration testing

Resources

Acceptance testing

2

Why test ?
To ensure that the product satisfies all use­

cases/user­stories
To gain confidence in your product's abilities

To learn about the behavior of your product

To deliver a quality product to your customer

To fulfil a binding contract with the govt/military

3

A quote on testing
Perhaps one day, writing code without tests will be

considered as professionally irresponsible as
constructing a bridge without performing a
structural analysis.
Brian Goetz (JVM and concurrency guru)
author of Java Concurrency In Practice.
4

Why automate tests ?
Humans are error­prone by nature

Automated tests

Save time
●
Consume minimal resources on beefy hardware
●
Remove the drudgery from repetitive tasks
●
Can be run frequently
●
Are highly effective with continuous integration
●
5

Kinds of tests
Typically the following tests are done

Unit testing – development
●
Integration testing – development
●
Functional testing (includes load/perf testing) – QA
●
Acceptance testing – customers / product manager
●
6

When do you test ?
Agile shops are increasingly embracing TDD

Testing is a 1st class activity and should never be

done as an afterthought
Unit testing ­ while developing code

Integration testing – while and after developing code

Functional testing – by QA after code complete

Load/perf testing – by QA after code compelete

Acceptance testing – by customers after QA is done

7

Unit testing
Always done by developers

In a TDD environment, tests are written before

writing code
TDD forces you to make your code testable and can

result in very clean design if done right
Code shouldn't be checked in without passing tests.

Run by developers before code check­in and by CI

system periodically (Daily / after code check­ins)
8

Integration testing
Ideally done by both dvlp and QA (separately)

Involves more big­picture tests

Focus is on collaboration of a number of objects

Fixtures to set up are more complex than unit tests

Typically touches aspects such as network (http, ftp,

smtp) and database interactions
9

Acceptance testing
Tests used to help define when a user story is done

Done every iteration and derived from user stories

A story isn't complete until all acceptance tests pass

Big bang acceptance testing occurs after code­

complete and test­complete
Onus is on customer/product manager to verify

correctness of acceptance tests
Automation of acceptance tests is strongly desired

10

Why use test frameworks ?
Absolutely mainstream in agile environments

Used by developers, QA and customers

Used to test all tiers of an enterprise app

Usually involves a learning curve – nothing is free

ROI can be substantial once mastered

Excellent way to achieve automation

Tests almost always experience a faster turnaround

11

Unit test frameworks
Junit 4.x* – unit test Java classes and interfaces

TestNG* – unit test Java classes and interfaces

EasyMock ­ provides mock objects for interfaces in

JUnit/TestNG tests by generating objects on the fly
Both* heavily leverage Java5 annotations

Both* are supported by Java IDEs: IntelliJ, Eclipse

Both* are supported by build tools: Maven, Ant

Both* generate reports about failed tests & coverage

12

Integration test frameworks
DbUnit – used to test your persistence tier

XmlUnit – used to compare 2 XML documents

HttpUnit – used to interact with your webapp and

test forms, tables, links, etc.
Junit/TestNG with MockHttpServletRequest and

MockHttpServletResponse (Spring Framework) to
test your servlets
13

Integration test frameworks (cont'd)
Cactus – tests server components e.g. Servlets,

EJBs, servlet filters, JSP taglibs, etc.
SoapUI ­ inspect, invoke and test web services over

HTTP (supported cleanly in Eclipse and IntelliJ)
14

Functional test frameworks
Abbot, JFCUnit, Marathon – Swing func test tools

Selenium – advanced tool to test your web app –

record, playback and run tests on a large grid.
Supports Java, Ruby, Python, Perl, PHP
JWebUnit – works with HtmlUnit and Selenium to

navigate your web app and verify correctness
Canoo WebTest – similar to JWebUnit – write rules

(tests) as Ant tasks or Groovy unit tests
15

Load / perf test frameworks
LoadRunner ($) ­ generates load for back­end

servers by emulating user requests
The Grinder – load test HTTP web servers, SOAP

and REST web services. Uses Jython for test scripts.
JUnitPerf – load and perf testing with Junit

TestNG – has annotations to give you control over #

concurrent requests and # threads e.g.
@Test(invocationCount = 100, threadPoolSize = 10
16

Acceptance test frameworks
Fit, FitNesse

Concordion

Exactor

17

Other tools (continuous integration)
Consider using a separate build and continuous

integration (CI) server
Will quickly generate failed test reports with each

run. Developers should respond quickly
Hook up CI server with CM system (CVS / SVN)

and set up build times (daily / after each checkin)
Popular CI tools are Hudson, Continuum, Luntbuild,

CruiseControl, TeamCity ($)
18

Resources (web)
http://www.ambysoft.com

http://www.martinfowler.com

http://www.opensourcetesting.org/unit_java.php

http://www.onjava.com

http://www.ibm.com/developerworks

http://www.javaworld.com

http://www.parleys.com (MM)

http://developers.sun.com/learning/javaoneonline (MM)

19

Resources (books)
JUnit Recipes by J B Rainsberger

Test Driven (Practical TDD.....) by Lasse Koskela

Next Generation Java Testing by Cedric Beust

xUnit Test Patterns by Gerard Meszaros

The Art Of Agile Development by James Shore et al

20