This document describes RandomTest.net, a tool for generating random integration tests. It discusses some limitations of traditional unit and manual testing approaches. RandomTest.net aims to minimize QA effort through test automation using random inputs and assertions to verify results. Developers embed assertions in code and RandomTest.net randomly executes code sections to detect failures. Any failed assertions are collected in a central repository. The tool currently supports C/C++, Java and PHP with more languages to come. It is open source and additional documentation can be found at randomtest.net.

1. http://RandomTest.net
Random Integration Tests
For Your Continuous
Integration Toolchain
By
Dariusz Cieslak
dc at aplikacja.info
RandomTest.net

2. Unit Tests?
●
I strongly believe in Unit Tests efficiency
●
Are great for automation, but:
●
Are hard to implement properly
●
Requires architectural changes (deps!)
●
●
Integration issues not visible there:
performance, resources, …
So then … ?
RandomTest.net

3. QA Team Activity
●
●
I do appreciate our QA teams effort – it's
important part of any project and cannot be
omitted
But: any human activity (including mine) is
expensive – we would like to use them as
effectively as possible
●
How to minimize the effort?
●
→ AUTOMATION!
RandomTest.net

4. Record'n'Play Integration Tests?
●
●
Simple idea: record events in application and
reply them automatically
Have you seen big 100% working UI scripts
after 1 year of development? Me: NOT
●
Fragile due to changes in application
●
Fragile due to environment state
●
→ Requires big effort to maintain
RandomTest.net

5. Contracts!
●
An Idea from Eiffell language:
●
1. Embed assertions in your code
●
●
●
2. Supply runtime coverage somehow (run the
stuff)
3. App will crash on unmet assertion when
found
4. Fix the issue then goto 2
RandomTest.net

6. Code Coverage – How?
●
●
●
Manual tests are important but expensive
(boring?)
Scripted automation – requires big effort for
maintenance
Solution: RANDOMISATION! – cheaper
method to deliver runtime codebase coverage
RandomTest.net

7. Results Verification – How?
●
●
●
●
Do not try to verify output of the randomised
input – it won't work
Assert / Require / Expect / @pre / @post / …
are your friends
Replace static test cases (data – dependant)
with contracts (data – independant)
Collect any failed assertion for aggregations in
central repository
RandomTest.net

8. QA + RandomTest setup
RandomTest.net

9. About Us
●
●
●
●
Aplikacja.info delivers software consulting
services
We're The Continuous Integration Freaks
RandomTest.net supports C/C++, Java, PHP at
the moment, more to come!
Need more information? Please visit
randomtest.net for full source code (LGPL-ed)
and documentation
RandomTest.net

10. About Us
●
●
●
●
Aplikacja.info delivers software consulting
services
We're The Continuous Integration Freaks
RandomTest.net supports C/C++, Java, PHP at
the moment, more to come!
Need more information? Please visit
randomtest.net for full source code (LGPL-ed)
and documentation
RandomTest.net