2005 talk on testing: I know your code doesn't work, because I know my own is broken. Covers CruiseControl & Apache Gump

1. Testing
Steve Loughran
HP Laboratories
Thursday November 17th, 2005

2. your code doesn't work

3. I know this because...

4. my code doesn't work

5. you are allowed to write code
that doesn’t work

6. just don’t ship it

7. how do you get from "broken" to
"go live"?

8. Device Drivers
ship and see
test it!
Test-Driven
∃x:
(∀y∈[1..x]:
f(y)>0)
Formal Methods
prove it works

9. proofs of correctness
+ shows an algorithm really works
+ good for concurrency
+ and distributed things
- makes assumptions about system
- needs lots of maths/CS skills
(Pi-Calculus, The HOL System)
- regression proofs?

10. Test Driven Development
any feature without a test
doesn’t exist

11. JUnit
* unit test framework in Java
(or C++, Python, Ruby, C#, ...)
* simple to learn
* good tool support
* somewhat low-level
* extensions for system testing

13. public class LessSimpleTest extends TestCase {
public LessSimpleTest(String s) {
super(s);
}
public void testAssignment() {
final Date date = new Date();
Event event = new Event(UUID.randomUUID(),
date, "now", "Text");
assertEquals("self equality failed",
event, event);
assertEquals(date, event.getDate());
assertNotNull(date.getID());
}
}

14. System Testing
* deploy, then:
HttpUnit, XmlUnit, Cactus
* needs automated deployment
* Distributed Testing still a research topic
* performance testing by comparing timings w.r.t.
old runs

15. Ant-generated report
scales better; reports published

16. Continuous Integration

17. Apache Gump gump.apache.org
Nightly build of OSS Java apps.
For any project with public CVS/SVN

18. Interop Testing
* Your client against my server
* My client against your server
Issues:
* Blame assignment: a formal test suite makes
blame easier to assign
* Dependent libraries: what if your WS-A doesn’t
interop with others?

19. Test Driven Standards?
any standard without a test
doesn’t exist