Java Test Runner
http://jtrunner.sourceforge.net
Francesco Russo

Testing matters...
What is an error?
\"Error is a difference between the desired and
actual behavior or performance of a system or
object\"

But testing has
different flavors

Behavioral

Integration

System & Stress

User acceptance

... and more

JTR is...
... a dynamic testing tool suitable for:
• integration / system testing
• behavioral testing

\"How can JTR
help me?\"

JTR lets you (stress-)test
Standard Java™
classes

JTR lets you (stress-)test
Standard Java™
classes
J2EE 1.4 Session Beans

JTR lets you (stress-)test
J2EE 1.4 Session
Beans
Standard Java™
classes
JEE 5 Session Beans

JTR lets you (stress-)test
J2EE 1.4 Session
Beans
Standard Java™
classes
Web Services
JEE 5 Session Beans

JTR lets you (stress-)test
J2EE 1.4 Session
Beans
Standard Java™
classes
Web Services
JMS applications JEE 5 Session Beans

Parameters matter

JTR adopts Inversion of
Control
Push... ... not pull!

Parameterization
happens with...
Fixed values

Parameterization
happens with...
Statistically
generated values
Fixed values

Parameterization
happens with...
Statistically
generated values
Fixed values
Scripted values

A simple example

A really simple JTR runner...
package jtr.test.fosdem;
import jtr.runners.AbstractRunner;
import junit.framework.Assert;
public class FosdemRunner extends AbstractRunner {
@Override
public void test() throws Throwable {
Assert.assertEquals(1978, hardcodedParameter);
addMessage(\"Hardcoded parameter is: \"+hardcodedParameter);
addMessage(\"\\nGaussian parameter is: \"+gaussianParameter);
Assert.assertEquals(\"Hello FOSDEM\", scriptedParameter);
addMessage(\"\\nScripted parameter is: \"+scriptedParameter);
}
private int hardcodedParameter;
private double gaussianParameter;
private String scriptedParameter;
}

A really simple JTR runner...
package jtr.test.fosdem;
Inherit useful
behaviors
import jtr.runners.AbstractRunner;
import junit.framework.Assert;
public class FosdemRunner extends AbstractRunner {
@Override
public void test() throws Throwable {
Assert.assertEquals(1978, hardcodedParameter);
addMessage(\"Hardcoded parameter is: \"+hardcodedParameter);
addMessage(\"\\nGaussian parameter is: \"+gaussianParameter);
Assert.assertEquals(\"Hello FOSDEM\", scriptedParameter);
addMessage(\"\\nScripted parameter is: \"+scriptedParameter);
}
private int hardcodedParameter;
private double gaussianParameter;
private String scriptedParameter;
}

A really simple JTR runner...
package jtr.test.fosdem;
Your testing logic
import jtr.runners.AbstractRunner;
starts from here
import junit.framework.Assert;
public class FosdemRunner extends AbstractRunner {
@Override
public void test() throws Throwable {
Assert.assertEquals(1978, hardcodedParameter);
addMessage(\"Hardcoded parameter is: \"+hardcodedParameter);
addMessage(\"\\nGaussian parameter is: \"+gaussianParameter);
Assert.assertEquals(\"Hello FOSDEM\", scriptedParameter);
addMessage(\"\\nScripted parameter is: \"+scriptedParameter);
}
private int hardcodedParameter;
private double gaussianParameter;
private String scriptedParameter;
}

A really simple JTR runner...
package jtr.test.fosdem;
Leverage on JUnit assertion
import jtr.runners.AbstractRunner;
APIs
import junit.framework.Assert;
public class FosdemRunner extends AbstractRunner {
@Override
public void test() throws Throwable {
Assert.assertEquals(1978, hardcodedParameter);
addMessage(\"Hardcoded parameter is: \"+hardcodedParameter);
addMessage(\"\\nGaussian parameter is: \"+gaussianParameter);
Assert.assertEquals(\"Hello FOSDEM\", scriptedParameter);
addMessage(\"\\nScripted parameter is: \"+scriptedParameter);
}
private int hardcodedParameter;
private double gaussianParameter;
private String scriptedParameter;
}

A really simple JTR runner...
package jtr.test.fosdem;
import jtr.runners.AbstractRunner;
import junit.framework.Assert;
public class FosdemRunner extends AbstractRunner {
Give end-users a feed-back
@Override
public void test() throws Throwable {
for later inspection
Assert.assertEquals(1978, hardcodedParameter);
addMessage(\"Hardcoded parameter is: \"+hardcodedParameter);
addMessage(\"\\nGaussian parameter is: \"+gaussianParameter);
Assert.assertEquals(\"Hello FOSDEM\", scriptedParameter);
addMessage(\"\\nScripted parameter is: \"+scriptedParameter);
}
private int hardcodedParameter;
private double gaussianParameter;
private String scriptedParameter;
}

A really simple JTR runner...
package jtr.test.fosdem;
import jtr.runners.AbstractRunner;
import junit.framework.Assert;
public class FosdemRunner extends AbstractRunner {
@Override
public void test() throws Throwable {
Instance properties... any
Assert.assertEquals(1978, hardcodedParameter);
addMessage(\"Hardcoded parameter is: \"+hardcodedParameter);
mutator method is required!
addMessage(\"\\nGaussian parameter is: \"+gaussianParameter);
Assert.assertEquals(\"Hello FOSDEM\", scriptedParameter);
addMessage(\"\\nScripted parameter is: \"+scriptedParameter);
}
private int hardcodedParameter;
private double gaussianParameter;
private String scriptedParameter;
}

... and its jtr.xml
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml
how many times to run
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml the runner that has to
run
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml
how many instances?
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
a fixed input value
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
a statistical one...
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
... and a scripted one
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

... and its jtr.xml
<runner runs=\"20\">
<runner-fqn>jtr.test.fosdem.FosdemRunner</runner-fqn>
<instance-count>5</instance-count>
<parameters-assignment-policy>indexed
</parameters-assignment-policy>
<parameters>
<param name=\"sleepTime\" value=\"10\" />
<param name=\"hardcodedParameter\" value=\"1978\"/>
<param name=\"gaussianParameter\" type=\"statistical\"
value=\"Gaussian(5,0.3,8)\"/>
<param name=\"scriptedParameter\" type=\"scripted\"
value=\"helloFOSDEM\"/>
</parameters>
</runner>
and here is the script
<scripts>
<p-scripts>
<p-script name=\"helloFOSDEM\"
body=\"return &quot;Hello FOSDEM&quot;;\"/>
</p-scripts>
</scripts>

Let's run it!
Let's run it!

Write once, run many
and anywhere

Write once, run many
and anywhere
Declarative
concurrent test
execution

Write once, run many
and anywhere
Declarative Declarative
concurrent test distributed test
execution execution

Declarative distribution
Simply add the following snippet to the jtr.xml file:
<nodes>
<node host=\"freddie\" port=\"2000\"/>
<node host=\"barnie\" port=\"2001\"/>
<node host=\"alice\" port=\"2002\"/>
</nodes>
For further details, please check the on-line users' guide.

Some definitions
• JTR Active Node runs the JTR runtime, our
Runners and 3rd party libraries
• JTR Passive Node only runs the JTR runtime

JTR Distributed Testing at work
Step one is the
distribution of
the jtr.xml
configuration
JTR active node JTR passive node
runners and 3rd party
only the JTR runtime
libraries

JTR Distributed Testing at work
Missing class/resource
request
class/resource
response
JTR active node JTR passive node
runners and 3rd party
only the JTR runtime
libraries

Let's run it!
Let'srun it
again!

Thus
no manual deployments
over
passive-nodes

Firewalls?!
Not a problem

What if I'm a JUnit fan?

JTR 5
smoothly integrates
JUnit 4

and much more
• Ant integration
• headless testing
• basic reporting (Excel exporting)

Time is over...
Salvador Dalí. (Spanish, 1904-1989). The Persistence of Memory. 1931. Oil on canvas, 9 1/2 x 13\" (24.1 x 33 cm). Given anonymously. © 2009 Salvador Dalí, Gala-Salvador Dalí Foundation/Artists Rights Society (ARS), New York

THANK YOU!
more on the website!
http://jtrunner.sourceforge.net
Francesco Russo
frusso@users.sourceforge.net