Your SlideShare is downloading. ×
0
The Test Anything Protocol
             or
  Small Testing Tools Loosely Joined
                      or
How to make large...
xUnit / declarative
testing / load testing /
  procedural tests /
record|replay / TDD /
      BDD / etc.
JUnit / SUnit / TestNG /
Test::Builder / Test::Unit /
   jsUnit / PHPUnit /
 Cucumber / Selenium /
  Watir / RSpec / JSpec...
Problem:
Using them together
Integrating automated
 test suites is a PITA
Everything is tightly
     coupled
Writing new test
environments is a PITA
Everybody has to figure
 out how to produce,
 consume and report
      test results
TAP aims to help
Test Producer



    TAP



Test Consumer
BDD       xUnit       Load Tests




TAP       TAP            TAP



      Test Consumer
Test Producer



                  TAP



Continuous
                  IDE         Reporting
Integration
Small Pieces Loosely
       Joined
1..3
ok 1 - we can foo
ok 2 - we can bar
not ok 3 - we can ni
1..3
ok 1 - we can foo
ok 2 - we can bar
not ok 3 - we can ni
1..3
ok 1 - we can foo
ok 2 - we can bar
not ok 3 - we can ni
1..3
ok 1 - we can foo
ok 2 - we can bar
not ok 3 - we can ni
1..3
ok 1 - we can foo
ok 2 - we can bar
not ok 3 - we can ni
And there’s more…
• Diagnostic output
• TAP versions
• TODO / SKIP tests
• Nested TAP (new-ish)
• Structured diagnostics (...
TAP Philosophy

• Work on the TAP as a stream
• The TAP producer should be pluggable
• The TAP consumer should be pluggabl...
Working well in the Perl
 world for years and
  years and years…
Not just Perl
•   PHPUnit (PHP)              •   PLUTO (Oracle PL/SQL)

•   TAPS (C#)                  •   pgTAP (PostgreS...
Smolder
CPAN Testers
testanything.org
You should ask
questions now :-)
          adrianh@quietstars.com
               twitter.com/adrianh
                    q...
The Test Anything Protocol
The Test Anything Protocol
The Test Anything Protocol
The Test Anything Protocol
Upcoming SlideShare
Loading in...5
×

The Test Anything Protocol

2,417

Published on

The Test Anything Protocol or
Small Testing Tools Loosely Joined or
How to make large automated test suites less of a complete PITA.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,417
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

  • There are lots of different kinds of automated test you can write
  • There are lots of tools for implementing different kinds of testing in the language of your choice
  • The problem comes when you have multiple languages and testing models in the same system.
  • Different languages, different testing models, different outputs, different inputs.
  • Makes integrating different systems hard.
  • Writing your own custom piece of automated testing is also a pain.
  • And everybody does it differently. The cause of the earlier problem.
  • TAP = Test Anything Protocol
  • Simple model. A test producer streams out TAP that is read by a test consumer.
  • So we can have different kinds of automated test framework all output TAP - and they can all be understood by a single consumer.
  • ... and we can have different kinds of test consumer that do interesting things with the results of your tests.
  • It’s a good way of building systems.
  • This is about the simplest possible example of TAP.
  • This is the plan - it says we’re running three tests.
  • This indicates a passing test.
  • This is a human-readable summary of the passing test
  • Failing tests are just prefixed by “not”.
  • TAP deals with more than just the simple case



  • Some examples of how TAP can be used.
  • Takes TAP output
  • Shows you passing and failing tests
  • Draws you pretty graphs over time (for certain definitions of “pretty”)
  • Another example
  • Massively distributed system for testing Perl modules from CPAN on multiple platforms, dependency variations, perl versions, etc.
  • This is where you should go to find out more about TAP.
  • Questions? Just drop me a line.
  • Transcript of "The Test Anything Protocol"

    1. 1. The Test Anything Protocol or Small Testing Tools Loosely Joined or How to make large automated test suites less of a complete PITA.
    2. 2. xUnit / declarative testing / load testing / procedural tests / record|replay / TDD / BDD / etc.
    3. 3. JUnit / SUnit / TestNG / Test::Builder / Test::Unit / jsUnit / PHPUnit / Cucumber / Selenium / Watir / RSpec / JSpec /
    4. 4. Problem: Using them together
    5. 5. Integrating automated test suites is a PITA
    6. 6. Everything is tightly coupled
    7. 7. Writing new test environments is a PITA
    8. 8. Everybody has to figure out how to produce, consume and report test results
    9. 9. TAP aims to help
    10. 10. Test Producer TAP Test Consumer
    11. 11. BDD xUnit Load Tests TAP TAP TAP Test Consumer
    12. 12. Test Producer TAP Continuous IDE Reporting Integration
    13. 13. Small Pieces Loosely Joined
    14. 14. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
    15. 15. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
    16. 16. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
    17. 17. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
    18. 18. 1..3 ok 1 - we can foo ok 2 - we can bar not ok 3 - we can ni
    19. 19. And there’s more… • Diagnostic output • TAP versions • TODO / SKIP tests • Nested TAP (new-ish) • Structured diagnostics (in dev, in PHPUnit) • Draft IETF Standard (WIP)
    20. 20. TAP Philosophy • Work on the TAP as a stream • The TAP producer should be pluggable • The TAP consumer should be pluggable • Gracefully handle future upgrades to TAP
    21. 21. Working well in the Perl world for years and years and years…
    22. 22. Not just Perl • PHPUnit (PHP) • PLUTO (Oracle PL/SQL) • TAPS (C#) • pgTAP (PostgreSQL) • libtap (C) • tap-functions (sh) • Test.Simple (JavaScript) • etap (Erlang) • PyTAP (Python) • arctap.arc (Arc) • Bacon (Ruby) • libtap++ (C++) • Forth/TAP (Forth) • JTap (Java)
    23. 23. Smolder
    24. 24. CPAN Testers
    25. 25. testanything.org
    26. 26. You should ask questions now :-) adrianh@quietstars.com twitter.com/adrianh quietstars.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×