The Test Anything Protocol
•Download as KEY, PDF•
3 likes•2,275 views
The Test Anything Protocol or Small Testing Tools Loosely Joined or How to make large automated test suites less of a complete PITA.
Recommended
More Related Content
More from Adrian Howard
More from Adrian Howard (20)
Recently uploaded
Recently uploaded (20)
The Test Anything Protocol
- 1. The Test Anything Protocol or Small Testing Tools Loosely Joined or How to make large automated test suites less of a complete PITA.
- 2. xUnit / declarative testing / load testing / procedural tests / record|replay / TDD / BDD / etc.
- 3. JUnit / SUnit / TestNG / Test::Builder / Test::Unit / jsUnit / PHPUnit / Cucumber / Selenium / Watir / RSpec / JSpec /
- 5. Integrating automated test suites is a PITA
- 6. Everything is tightly coupled
- 7. Writing new test environments is a PITA
- 8. Everybody has to figure out how to produce, consume and report test results
- 9. TAP aims to help
- 10. Test Producer TAP Test Consumer
- 11. BDD xUnit Load Tests TAP TAP TAP Test Consumer
- 12. Test Producer TAP Continuous IDE Reporting Integration
- 13. Small Pieces Loosely Joined
- 14. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
- 15. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
- 16. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
- 17. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
- 18. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
- 19. And there’s more… • Diagnostic output • TAP versions • TODO / SKIP tests • Nested TAP (new-ish) • Structured diagnostics (in dev, in PHPUnit) • Draft IETF Standard (WIP)
- 20. TAP Philosophy • Work on the TAP as a stream • The TAP producer should be pluggable • The TAP consumer should be pluggable • Gracefully handle future upgrades to TAP
- 21. Working well in the Perl world for years and years and years…
- 22. Not just Perl • PHPUnit (PHP) • PLUTO (Oracle PL/SQL) • TAPS (C#) • pgTAP (PostgreSQL) • libtap (C) • tap-functions (sh) • Test.Simple (JavaScript) • etap (Erlang) • PyTAP (Python) • arctap.arc (Arc) • Bacon (Ruby) • libtap++ (C++) • Forth/TAP (Forth) • JTap (Java)
- 23. Smolder
- 27. CPAN Testers
- 29. testanything.org
- 30. You should ask questions now :-) adrianh@quietstars.com twitter.com/adrianh quietstars.com
Editor's Notes
- There are lots of different kinds of automated test you can write
- There are lots of tools for implementing different kinds of testing in the language of your choice
- The problem comes when you have multiple languages and testing models in the same system.
- Different languages, different testing models, different outputs, different inputs.
- Makes integrating different systems hard.
- Writing your own custom piece of automated testing is also a pain.
- And everybody does it differently. The cause of the earlier problem.
- TAP = Test Anything Protocol
- Simple model. A test producer streams out TAP that is read by a test consumer.
- So we can have different kinds of automated test framework all output TAP - and they can all be understood by a single consumer.
- ... and we can have different kinds of test consumer that do interesting things with the results of your tests.
- It’s a good way of building systems.
- This is about the simplest possible example of TAP.
- This is the plan - it says we’re running three tests.
- This indicates a passing test.
- This is a human-readable summary of the passing test
- Failing tests are just prefixed by “not”.
- TAP deals with more than just the simple case
- Some examples of how TAP can be used.
- Takes TAP output
- Shows you passing and failing tests
- Draws you pretty graphs over time (for certain definitions of “pretty”)
- Another example
- Massively distributed system for testing Perl modules from CPAN on multiple platforms, dependency variations, perl versions, etc.
- This is where you should go to find out more about TAP.
- Questions? Just drop me a line.