Your SlideShare is downloading. ×
  • Like
The Test Anything Protocol
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The Test Anything Protocol

  • 2,196 views
Published

The Test Anything Protocol or …

The Test Anything Protocol or
Small Testing Tools Loosely Joined or
How to make large automated test suites less of a complete PITA.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,196
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
21
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

  • There are lots of different kinds of automated test you can write
  • There are lots of tools for implementing different kinds of testing in the language of your choice
  • The problem comes when you have multiple languages and testing models in the same system.
  • Different languages, different testing models, different outputs, different inputs.
  • Makes integrating different systems hard.
  • Writing your own custom piece of automated testing is also a pain.
  • And everybody does it differently. The cause of the earlier problem.
  • TAP = Test Anything Protocol
  • Simple model. A test producer streams out TAP that is read by a test consumer.
  • So we can have different kinds of automated test framework all output TAP - and they can all be understood by a single consumer.
  • ... and we can have different kinds of test consumer that do interesting things with the results of your tests.
  • It’s a good way of building systems.
  • This is about the simplest possible example of TAP.
  • This is the plan - it says we’re running three tests.
  • This indicates a passing test.
  • This is a human-readable summary of the passing test
  • Failing tests are just prefixed by “not”.
  • TAP deals with more than just the simple case



  • Some examples of how TAP can be used.
  • Takes TAP output
  • Shows you passing and failing tests
  • Draws you pretty graphs over time (for certain definitions of “pretty”)
  • Another example
  • Massively distributed system for testing Perl modules from CPAN on multiple platforms, dependency variations, perl versions, etc.
  • This is where you should go to find out more about TAP.
  • Questions? Just drop me a line.

Transcript

  • 1. The Test Anything Protocol or Small Testing Tools Loosely Joined or How to make large automated test suites less of a complete PITA.
  • 2. xUnit / declarative testing / load testing / procedural tests / record|replay / TDD / BDD / etc.
  • 3. JUnit / SUnit / TestNG / Test::Builder / Test::Unit / jsUnit / PHPUnit / Cucumber / Selenium / Watir / RSpec / JSpec /
  • 4. Problem: Using them together
  • 5. Integrating automated test suites is a PITA
  • 6. Everything is tightly coupled
  • 7. Writing new test environments is a PITA
  • 8. Everybody has to figure out how to produce, consume and report test results
  • 9. TAP aims to help
  • 10. Test Producer TAP Test Consumer
  • 11. BDD xUnit Load Tests TAP TAP TAP Test Consumer
  • 12. Test Producer TAP Continuous IDE Reporting Integration
  • 13. Small Pieces Loosely Joined
  • 14. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
  • 15. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
  • 16. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
  • 17. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
  • 18. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
  • 19. And there’s more… • Diagnostic output • TAP versions • TODO / SKIP tests • Nested TAP (new-ish) • Structured diagnostics (in dev, in PHPUnit) • Draft IETF Standard (WIP)
  • 20. TAP Philosophy • Work on the TAP as a stream • The TAP producer should be pluggable • The TAP consumer should be pluggable • Gracefully handle future upgrades to TAP
  • 21. Working well in the Perl world for years and years and years…
  • 22. Not just Perl • PHPUnit (PHP) • PLUTO (Oracle PL/SQL) • TAPS (C#) • pgTAP (PostgreSQL) • libtap (C) • tap-functions (sh) • Test.Simple (JavaScript) • etap (Erlang) • PyTAP (Python) • arctap.arc (Arc) • Bacon (Ruby) • libtap++ (C++) • Forth/TAP (Forth) • JTap (Java)
  • 23. Smolder
  • 24. CPAN Testers
  • 25. testanything.org
  • 26. You should ask questions now :-) adrianh@quietstars.com twitter.com/adrianh quietstars.com