Test Driven
Development
  Kirrily Robert
The problem
            Good




Cheap                 Fast
No silver bullet
However, with testing...


• A bit faster
• A bit cheaper
• A bit better
Faster
Time taken to fix bugs
1,000



 750



 500



 250



   0
        Design   Implementation   QA   Post-release
Cheaper
Technical debt


• “We’ll leave it for now”
• Delayed payment plan
• Compound interest
The most powerful
 force in the universe
is compound interest.
      Albert Einstein
Time taken to fix bugs
1,000



 750



 500



 250



   0
        Design   Implementation   QA   Post-release
Easy payment plan


• Don’t go into debt
• Make regular payments
• Pay down the principal
Cheap programmers



• Best programmers 10x as effective
• Testing can close the gap (somewhat)
Better
Software quality



• “Instinctive”
• Hard to measure
Software Kwalitee


• Indicative
• Measurable
• Testable
The solution



• Testing
• Test Driven Development
Testing
Design




         Implement




                     Test
TDD
Design




         Test




                Implement
TDD
Design


         Test


                  Implement



                              Test
TDD

        Design



Test               Test




       Implement
TDD

        Design



Test               Test




       Implement
How to do it

• Design: figure out what you want to do
• Test: write a test to express the design
  • It should FAIL
• Impl...
Design


The subroutine add() takes two arguments and adds
      them together. The result is returned.
Test


use Test::More tests => 1;

is(add(2,2), 4, “Two and two is four”);
FAIL
$ prove -v add.t
add....Undefined subroutine &main::add called at add.t line 3.
# Looks like your test died before it...
Implement

sub add {
  my ($first, $second) = @_;
  return $first + $second;
}
Test

$ prove -v add.t
add....1..1
ok 1 - Two and two is four
ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ...
Wait...


• What if there are fewer than two
  arguments?

• What if there are more than two
  arguments?

• What if the a...
Iterate

        Design



Test               Test




       Implement
Design
• The subroutine add() takes two
  arguments and adds them together. The
  result is returned.

• If fewer than two...
Test
use Test::More tests => 4;

is(add(2,2), 4,
   “Simple case: two and two is four”);

is(add(3), undef,
   “Return und...
FAIL



<insert test failure here>
Implement

sub add {
  my ($first, $second) = @_;
  # insert error-checking here
  return $first + $second;
}
Test

prove -v add.t
add....1..4
ok 1 - Two and two is four
ok 2 - Return undef for < 2 args
ok 3 - Only add first 2 args
...
Effective tests must
   be automated
print “Now calculating shipping...”;
print “Oops, something’s gone wrong...”;
warn “Oops, something’s gone wrong...”;
die “This should never happen!”;
Now calculating shipping...
Let’s see if this works.
Oops, something’s gone wrong...
ERROR: weirdness afoot!?!?
(Contact M...
Write once, run often

• Write tests once
• Keep them somewhere sensible
• Run frequently (one click)
• No human input
• M...
Questions so far?
Testing scenarios


• Public APIs
• Bug fixing/QA
• Legacy code
Public APIs
The steps

         Design



Test                Test




        Implement
API testing tips


• Maintain backward compatibility
• Consider cross-platform issues
• Turn edge cases into tests
Bug fixing
Bug report
     “I’m writing a system to automatically send gift
   baskets to elderly maiden aunts in Timbuktu, using
   ...
“... it says that something’s not
       numeric ... maybe the
 arithmetic code is broken ...”
Steps

• Get some sample code (that breaks)
• Turn it into a test
• Test should fail
• Fix bug
• Test should now pass
Steps

       Bug report



Test                Test




        Bug fix
Sample code
my $basket = Gift::Basket->new();
$basket->add(“flowers”, “chocolate”, “teddy bear”);

my $price = $basket->pr...
Test
my $price = 10.00; # typical gift basket price

my $shipping = multiply($price, 0.15);

is($shipping, 1.5,
   “Shippi...
FAIL
prove -v giftbasket.t
giftbasket....1..2
ok 1 - Shipping price calculated successfully
1.5 isn't numeric at giftbaske...
Fix


• Examine code
• Look for likely problem
• Fix
Examine code
sub add {
    my ($first, $second) = @_;
    return undef unless $first and $second;
    foreach ($first, $se...
Examine code


sub is_numeric {
    my ($number) = @_;
    return int($number) == $number ? 1 : 0;
}
Fix

use Scalar::Util;

sub is_numeric {
    my ($number) = @_;
    return Scalar::Util::looks_like_number($number);
}
Test

$ prove -v giftbasket.t
giftbasket....1..2
ok 1 - Shipping price calculated successfully
ok 2 - Add shipping to pric...
Bug fix tips



• Try to get users to submit tests
• Record bug tracking numbers
Legacy Code
Legacy code
      =
Technical debt
“Does anyone know
 what this does?”
“I don’t suppose we
     have any
documentation for
       that?”
“If we change X it will
  probably break Y.”
“Last time we touched
that, we spent a week
       fixing it.”
Improve legacy code
•Document     •Remove cruft
•Understand   •Standardise
•Clean up     •Strengthen
•Refactor     •Secure
The Steps

          WTF?



Test               Test




          Ah!
The Steps

• Look at legacy code. Be confused.
• Write a test to see if you understand
  • Test FAILS
• Adapt test (iterat...
Legacy code tips


• CAUTION!
• Go very slowly
• Be prepared to back out changes
• Track test coverage
Test coverage


• How much of the code is tested?
• What areas still need testing?
• Where are the greatest risks?
Coverage tools

 Perl    Devel::Cover

Python   Coverage.py

Java        Quilt

 PHP      PHPUnit
Devel::Cover
Testing libraries
• Perl         • Java
• PHP          • Javascript
• Python       • C/C++
• Ruby
Perl
Test::More


• Standard library
• Comes with Perl
• Also on CPAN
lib/Arithmetic.pm
   package Arithmetic;

   use strict;
   use warnings;

   sub add {
      # ...
   }

   sub subtract ...
t/arithmetic.t
use Test::More tests => 5;

use_ok(“Arithmetic.pm”);
can_ok(“Arithmetic.pm”, “add”);

ok(is_numeric(1.23), ...
Other Test::More
         functions

like(“An elephant”, qr/^w+$/,
   “String contains only word chars”);

my $user_agent ...
See also


• http://qa.perl.org/
• Test::More::Tutorial
• http://search.cpan.org/
  • search for “Test”
Python
PyUnit

• “The standard unit testing framework
  for Python”

• http://pyunit.sourceforge.net/
• Included with Python dist...
arithmetic.py


def add(x, y):
    quot;quot;quot;add two numbersquot;quot;quot;
    return x + y
arithmetic_test.py
import arithmetic
import unittest

class AdditionTests(unittest.TestCase):
    knownValues = ( (1, 1, 2...
Run arithmetic_test.py

$ python arithtest.py
.
-----------------------------------------------
Ran 1 test in 0.000s

OK
PHP
PHP


• PHPUnit
 • http://phpunit.de/
• SimpleTest
 • http://simpletest.org/
PHPUnit
<?php
require_once 'PHPUnit/Framework.php';

class ArrayTest extends PHPUnit_Framework_TestCase {
    public funct...
PHPUnit
SimpleTest
<?php
    require_once('simpletest/unit_tester.php');
    require_once('simpletest/reporter.php');
    require_...
SimpleTest
Ruby
Ruby


• Test::Unit
• Yet another JUnit clone
• Part of standard distribution
• http://ruby-doc.org/
Test::Unit
require 'test/unit'

class TestArithmetic < Test::Unit::TestCase

  def test_pass
    assert(true, 'Assertion w...
Run tests
ruby arithtest.rb
Loaded suite arithtest
Started
.F.
Finished in 0.009103 seconds.

  1) Failure:
test_fail(Test...
Java
Java


• JUnit
• Standard library
• Parent of many *Unit test suites
• http://junit.org
Javascript
Javascript


• Several libraries available
• No clear standard
• I like Test.More from JSAN
Test.More


• Port of Perl’s Test::More
• Common output format (TAP)
• http://openjsan.org/
Alternately



• JSUnit
• http://www.jsunit.net/
C/C++
C/C++


• libtap
• outputs TAP
• similar to Perl’s Test::More
test.c
#include <stdio.h>
#include quot;tap.hquot;

int main(void) {
    plan_tests(2);

    ok(1 + 1 == 2, quot;additionq...
Output


1..2
ok 1 - addition
ok 2 - subtraction
CPPUnit



• C++ only
• http://cppunit.sourceforge.net/
CPPUnit

class ComplexNumberTest : public CppUnit::TestCase {
public:
  ComplexNumberTest( std::string name ) : CppUnit::T...
Three families


• XUnit
• TAP
• Miscellaneous
XUnit


• Distinctive markings: “assert”
• Output:
  • ...F.....F...FF...
• JUnit, PyUnit, CPPUnit, JSUnit, etc
TAP


• Distinctive markings: “ok” or “is”
• Output:
  • ok 1 - some comment
• Test::More, Test.More, libtap, etc
Miscellaneous



• eg. xmllint
• wrappers/ports/equivalents often exist
TAP


• Test Anything Protocol
• Standardised test reporting format
• Many languages
• http://testanything.org/
TAP output

1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
# Failed test ‘First line of input vali...
TAP Producers



• Anything that outputs TAP
• eg. Test::More, Test.More, libtap
TAP Consumers



• Anything that reads/parses TAP
• eg. Perl’s Test::Harness
More test libraries


• http://opensourcetesting.org/
• Ada, C/C++, HTML, Java,
  Javascript, .NET, Perl, PHP, Python,
  R...
Questions?
TDD Cookbook
Private code


• Can you test private code?
  • private/protected/hidden/internal
• Should you test private code?
The argument against


• Private code should never be accessed
  externally

• You want to change private code at
  will, ...
The argument for


• Assists in refactoring
• Testing is better than not testing
• If tests break, change/fix them with
  i...
“Black box” testing



• Examine inputs and outputs
• Don’t “see” anything inside
“Glass box” testing



• Look inside the code
• Examine inner workings
Example
sub display_details {
    my ($inventory_id) = @_;
    my $output;

    # 1,000 lines of legacy code ...

    $pri...
Bug report


 “There’s something wrong with the price display. I
occasionally see prices like $123.400, when only two
   d...
Black box


• Comprehend 2000+ lines of legacy code
• Write tests for entire subroutine
• Check price embedded in larger o...
White box



• Extract private method
• Test private method
Extract method

sub _format_price {
   my ($price) = @_;
   $price = ‘$’ . $price;
   if (int($price) == $price) {
      $...
Test
is(_format_price(12), ‘$12.00’,
   ‘12 becomes $12.00’);

is(_format_price(11.99, ‘$11.99’,
   ‘11.99 becomes $11.99’...
Conclusion


• I like glass box testing
• Some people don’t
• They are obviously misguided
• Q.E.D.
External applications


• scripts
• executables
• foreign APIs
External applications



• Do you trust them?
• “Black box” testing
Things to test


• Return values
• Output
• Side effects
Return values


• Works on Unix
 • success == 0
• Otherwise, ???
Output


• Printed output
• Warnings/log messages
• Use pipes/redirection
Side effects



• Eg. changed files
• Risky
Example

is(system($some_command), 0, “Command ran OK”);

ok(! -e $somefile, “File doesn’t exist.”);
is(system($create_fil...
Testing complex systems


• Complex libraries
• Side effects
• Pre-requisites
Example
sub notify {
    my ($user_id) = @_;
    if (my $user = fetch_from_db($user_id)) {
        send_notification_email...
Example



• Requires database access
• Actually sends email
Mocking


• Fake database connection
• Fake email sending
• Only test notify() logic
Mock libraries
 Perl    Test::MockObject

Python     python-mock

Java          jMock

 PHP     PHPUnit (builtin)
Test::MockObject

use OurDB::User;
use Test::MockObject::Extends;

my $user = OurDB::User->fetch($user_id);
$user    = Tes...
python-mock
>>> from mock import Mock
>>> myMock = Mock( {quot;fooquot; : quot;you called fooquot;} )
>>> myMock.foo()
'yo...
Databases


• “Real” data
• Unchanging data
• Set up/tear down
Fixtures


• Known data
• Setup/teardown
• Repeatable tests
Ruby on Rails


• Built-in fixtures
• YAML or CSV format
• Automatically loaded
• Tests occur inside transaction
YAML format
david:
    id: 1
    name: David Heinemeier Hansson
    birthday: 1979-10-15
    profession: Systems developme...
CSV format


id, username, password, stretchable, comments
1, sclaus, ihatekids, false, I like to say quot;quot;Ho! Ho! Ho...
Loading fixtures
require '/../test_helper'
require 'user'

class UserTest < Test::Unit::TestCase

  fixtures :users

  def ...
Websites


• Complex systems
 • Backend/frontend
• Browser dependent
Backend strategy

• Refactor mercilessly
• Separate
  • Database access
  • Business logic
  • Everything you can
• MVC good!
Front-end strategy


• Web pages and links
• Forms
• Javascript
• Browser compatibility
Web links

• Use a web browser / user agent
  • WWW::Mechanize
• Check HTTP GET on each page
  • Status 200
• Follow links...
Forms


• Use web browser / user agent
• HTTP GET / HTTP POST
• Check validation
• Check results of form
Javascript



• Refactor into libraries
• Test using Test.More or similar
Browser testing


• Mimic actual browser use
• Selenium
 • http://openqa.org/selenium/
Selenium

• Firefox / IE / Safari / others
• Linux/Mac/Windows
• Record browser actions
• Save as scripts
• Language integ...
Selenium IDE
Team testing

• Many developers
• Distributed/asynchronous
  development

• Different platforms
• One test strategy
Who’s responsible?
Team responsibility


• Everyone is responsible
• See a problem, fix it
• No code ownership
Individual blame


• Correlate failures with checkins
• “svn blame”
• “Golden fork-up award”
Team testing tips


• Use standard tools
• Document processes
• Nightly smoke test
• Integrate with version control
Questions?
Kirrily Robert
http://infotrope.net/
skud@infotrope.net
Test  Driven  Development  Tutorial
Upcoming SlideShare
Loading in...5
×

Test Driven Development Tutorial

158,087

Published on

A three-hour tutorial from the Open Source Developers' Conference in Brisbane, November 2007. Presented by Kirrily Robert.

Published in: Technology, Business
17 Comments
232 Likes
Statistics
Notes
No Downloads
Views
Total Views
158,087
On Slideshare
0
From Embeds
0
Number of Embeds
55
Actions
Shares
0
Downloads
7,028
Comments
17
Likes
232
Embeds 0
No embeds

No notes for slide

Transcript of "Test Driven Development Tutorial"

  1. 1. Test Driven Development Kirrily Robert
  2. 2. The problem Good Cheap Fast
  3. 3. No silver bullet
  4. 4. However, with testing... • A bit faster • A bit cheaper • A bit better
  5. 5. Faster
  6. 6. Time taken to fix bugs 1,000 750 500 250 0 Design Implementation QA Post-release
  7. 7. Cheaper
  8. 8. Technical debt • “We’ll leave it for now” • Delayed payment plan • Compound interest
  9. 9. The most powerful force in the universe is compound interest. Albert Einstein
  10. 10. Time taken to fix bugs 1,000 750 500 250 0 Design Implementation QA Post-release
  11. 11. Easy payment plan • Don’t go into debt • Make regular payments • Pay down the principal
  12. 12. Cheap programmers • Best programmers 10x as effective • Testing can close the gap (somewhat)
  13. 13. Better
  14. 14. Software quality • “Instinctive” • Hard to measure
  15. 15. Software Kwalitee • Indicative • Measurable • Testable
  16. 16. The solution • Testing • Test Driven Development
  17. 17. Testing Design Implement Test
  18. 18. TDD Design Test Implement
  19. 19. TDD Design Test Implement Test
  20. 20. TDD Design Test Test Implement
  21. 21. TDD Design Test Test Implement
  22. 22. How to do it • Design: figure out what you want to do • Test: write a test to express the design • It should FAIL • Implement: write the code • Test again • It should PASS
  23. 23. Design The subroutine add() takes two arguments and adds them together. The result is returned.
  24. 24. Test use Test::More tests => 1; is(add(2,2), 4, “Two and two is four”);
  25. 25. FAIL $ prove -v add.t add....Undefined subroutine &main::add called at add.t line 3. # Looks like your test died before it could output anything. 1..1 dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED test 1 Failed 1/1 tests, 0.00% okay Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- add.t 255 65280 1 21 Failed 1/1 test scripts. 1/1 subtests failed. Files=1, Tests=1, 0 wallclock secs ( 0.02 cusr + 0.01 csys = 0.03 CPU) Failed 1/1 test programs. 1/1 subtests failed.
  26. 26. Implement sub add { my ($first, $second) = @_; return $first + $second; }
  27. 27. Test $ prove -v add.t add....1..1 ok 1 - Two and two is four ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 cusr + 0.01 csys = 0.03 CPU)
  28. 28. Wait... • What if there are fewer than two arguments? • What if there are more than two arguments? • What if the arguments aren’t numeric?
  29. 29. Iterate Design Test Test Implement
  30. 30. Design • The subroutine add() takes two arguments and adds them together. The result is returned. • If fewer than two arguments are provided, add() will return undef. • If more than two arguments are provided, add() will return the sum of the first two. • If any argument is non-numeric, add() will return undef.
  31. 31. Test use Test::More tests => 4; is(add(2,2), 4, “Simple case: two and two is four”); is(add(3), undef, “Return undef for < 2 args”); is(add(2,2,2), 4, “Only add first 2 args”); is(add(“foo”, “bar”), undef, “Return undef for non-numeric args”);
  32. 32. FAIL <insert test failure here>
  33. 33. Implement sub add { my ($first, $second) = @_; # insert error-checking here return $first + $second; }
  34. 34. Test prove -v add.t add....1..4 ok 1 - Two and two is four ok 2 - Return undef for < 2 args ok 3 - Only add first 2 args ok 4 - Return undef for non-numeric args ok All tests successful.
  35. 35. Effective tests must be automated
  36. 36. print “Now calculating shipping...”;
  37. 37. print “Oops, something’s gone wrong...”;
  38. 38. warn “Oops, something’s gone wrong...”;
  39. 39. die “This should never happen!”;
  40. 40. Now calculating shipping... Let’s see if this works. Oops, something’s gone wrong... ERROR: weirdness afoot!?!? (Contact Mike on ext. 2345) Bailing out, very confused! $
  41. 41. Write once, run often • Write tests once • Keep them somewhere sensible • Run frequently (one click) • No human input • Machine-parsable output
  42. 42. Questions so far?
  43. 43. Testing scenarios • Public APIs • Bug fixing/QA • Legacy code
  44. 44. Public APIs
  45. 45. The steps Design Test Test Implement
  46. 46. API testing tips • Maintain backward compatibility • Consider cross-platform issues • Turn edge cases into tests
  47. 47. Bug fixing
  48. 48. Bug report “I’m writing a system to automatically send gift baskets to elderly maiden aunts in Timbuktu, using Gift::Basket to build the basket and Geo::Names to figure out where the heck Timbuktu is, but I have this problem that whenever I try to calculate the shipping costs, something breaks and it says that something’s not numeric, and I don’t know what’s going on but I think maybe the arithmetic code is broken, or maybe something else. Perhaps Timbuktu doesn’t really exist. Can you help me? I need this fixed by 5pm at the latest. Unless Timbuktu really doesn’t exist, in which case do you think Kathmandu would work instead?”
  49. 49. “... it says that something’s not numeric ... maybe the arithmetic code is broken ...”
  50. 50. Steps • Get some sample code (that breaks) • Turn it into a test • Test should fail • Fix bug • Test should now pass
  51. 51. Steps Bug report Test Test Bug fix
  52. 52. Sample code my $basket = Gift::Basket->new(); $basket->add(“flowers”, “chocolate”, “teddy bear”); my $price = $basket->price(); # shipping is 15% of price my $shipping = multiply($price, 0.15); # code dies horribly on the following line my $total_price = add($price, $shipping);
  53. 53. Test my $price = 10.00; # typical gift basket price my $shipping = multiply($price, 0.15); is($shipping, 1.5, “Shipping price calculated successfully”); is(add($shipping, $price), 11.5, “Add shipping to price”);
  54. 54. FAIL prove -v giftbasket.t giftbasket....1..2 ok 1 - Shipping price calculated successfully 1.5 isn't numeric at giftbasket.t line 16. # Failed test 'Add shipping to price' # at giftbasket.t line 8. # got: undef # expected: '11.5' # Looks like you failed 1 test of 2. not ok 2 - Add shipping to price
  55. 55. Fix • Examine code • Look for likely problem • Fix
  56. 56. Examine code sub add { my ($first, $second) = @_; return undef unless $first and $second; foreach ($first, $second) { unless (is_numeric($_)) { warn quot;$_ isn't numericquot;; return undef; } } return $first + $second; }
  57. 57. Examine code sub is_numeric { my ($number) = @_; return int($number) == $number ? 1 : 0; }
  58. 58. Fix use Scalar::Util; sub is_numeric { my ($number) = @_; return Scalar::Util::looks_like_number($number); }
  59. 59. Test $ prove -v giftbasket.t giftbasket....1..2 ok 1 - Shipping price calculated successfully ok 2 - Add shipping to price ok All tests successful. Files=1, Tests=2, 0 wallclock secs ( 0.03 cusr + 0.01 csys = 0.04 CPU)
  60. 60. Bug fix tips • Try to get users to submit tests • Record bug tracking numbers
  61. 61. Legacy Code
  62. 62. Legacy code = Technical debt
  63. 63. “Does anyone know what this does?”
  64. 64. “I don’t suppose we have any documentation for that?”
  65. 65. “If we change X it will probably break Y.”
  66. 66. “Last time we touched that, we spent a week fixing it.”
  67. 67. Improve legacy code •Document •Remove cruft •Understand •Standardise •Clean up •Strengthen •Refactor •Secure
  68. 68. The Steps WTF? Test Test Ah!
  69. 69. The Steps • Look at legacy code. Be confused. • Write a test to see if you understand • Test FAILS • Adapt test (iteratively) • Test PASSES • Move on to next piece.
  70. 70. Legacy code tips • CAUTION! • Go very slowly • Be prepared to back out changes • Track test coverage
  71. 71. Test coverage • How much of the code is tested? • What areas still need testing? • Where are the greatest risks?
  72. 72. Coverage tools Perl Devel::Cover Python Coverage.py Java Quilt PHP PHPUnit
  73. 73. Devel::Cover
  74. 74. Testing libraries • Perl • Java • PHP • Javascript • Python • C/C++ • Ruby
  75. 75. Perl
  76. 76. Test::More • Standard library • Comes with Perl • Also on CPAN
  77. 77. lib/Arithmetic.pm package Arithmetic; use strict; use warnings; sub add { # ... } sub subtract { # ... } 1;
  78. 78. t/arithmetic.t use Test::More tests => 5; use_ok(“Arithmetic.pm”); can_ok(“Arithmetic.pm”, “add”); ok(is_numeric(1.23), “1.23 is numeric”); is(add(2,2), 4, “2 + 2 = 4”);
  79. 79. Other Test::More functions like(“An elephant”, qr/^w+$/, “String contains only word chars”); my $user_agent = LWP::UserAgent->new(); isa_ok($user_agent, “LWP::UserAgent”); can_ok($user_agent, “get”);
  80. 80. See also • http://qa.perl.org/ • Test::More::Tutorial • http://search.cpan.org/ • search for “Test”
  81. 81. Python
  82. 82. PyUnit • “The standard unit testing framework for Python” • http://pyunit.sourceforge.net/ • Included with Python dist • Port of JUnit
  83. 83. arithmetic.py def add(x, y): quot;quot;quot;add two numbersquot;quot;quot; return x + y
  84. 84. arithmetic_test.py import arithmetic import unittest class AdditionTests(unittest.TestCase): knownValues = ( (1, 1, 2), (2, 2, 4), (0, 0, 0), (-3, -4, -7)) def testAddition(self): for x, y, sum in self.knownValues: result = arithmetic.add(x, y) self.assertEqual(result, sum) unittest.main()
  85. 85. Run arithmetic_test.py $ python arithtest.py . ----------------------------------------------- Ran 1 test in 0.000s OK
  86. 86. PHP
  87. 87. PHP • PHPUnit • http://phpunit.de/ • SimpleTest • http://simpletest.org/
  88. 88. PHPUnit <?php require_once 'PHPUnit/Framework.php'; class ArrayTest extends PHPUnit_Framework_TestCase { public function testNewArrayIsEmpty() { $foo = array(); $this->assertEquals(0, sizeof($foo)); } public function testArrayContainsAnElement() { $foo = array(); $foo[] = 'Element'; $this->assertEquals(1, sizeof($foo)); } } ?>
  89. 89. PHPUnit
  90. 90. SimpleTest <?php require_once('simpletest/unit_tester.php'); require_once('simpletest/reporter.php'); require_once('../classes/log.php'); class TestOfLogging extends UnitTestCase { function TestOfLogging() { $this->UnitTestCase(); } function testCreatingNewFile() { @unlink('/tmp/test.log'); $log = new Log('/tmp/test.log'); $log->message('Should write this to a file'); $this->assertTrue(file_exists('/tmp/test.log')); } } $test = &new TestOfLogging(); $test->run(new HtmlReporter()); ?>
  91. 91. SimpleTest
  92. 92. Ruby
  93. 93. Ruby • Test::Unit • Yet another JUnit clone • Part of standard distribution • http://ruby-doc.org/
  94. 94. Test::Unit require 'test/unit' class TestArithmetic < Test::Unit::TestCase def test_pass assert(true, 'Assertion was true') end def test_fail assert(false, 'Assertion was false.') end def test_arithmetic assert(2 + 2 == 4, '2 plus 2 is 4') end end
  95. 95. Run tests ruby arithtest.rb Loaded suite arithtest Started .F. Finished in 0.009103 seconds. 1) Failure: test_fail(TestArithmetic) [arithtest.rb:10]: Assertion was false. <false> is not true. 3 tests, 3 assertions, 1 failures, 0 errors
  96. 96. Java
  97. 97. Java • JUnit • Standard library • Parent of many *Unit test suites • http://junit.org
  98. 98. Javascript
  99. 99. Javascript • Several libraries available • No clear standard • I like Test.More from JSAN
  100. 100. Test.More • Port of Perl’s Test::More • Common output format (TAP) • http://openjsan.org/
  101. 101. Alternately • JSUnit • http://www.jsunit.net/
  102. 102. C/C++
  103. 103. C/C++ • libtap • outputs TAP • similar to Perl’s Test::More
  104. 104. test.c #include <stdio.h> #include quot;tap.hquot; int main(void) { plan_tests(2); ok(1 + 1 == 2, quot;additionquot;); ok(3 - 2 == 1, quot;subtractionquot;); return exit_status(); }
  105. 105. Output 1..2 ok 1 - addition ok 2 - subtraction
  106. 106. CPPUnit • C++ only • http://cppunit.sourceforge.net/
  107. 107. CPPUnit class ComplexNumberTest : public CppUnit::TestCase { public: ComplexNumberTest( std::string name ) : CppUnit::TestCase( name ) {} void runTest() { CPPUNIT_ASSERT( Complex (10, 1) == Complex (10, 1) ); CPPUNIT_ASSERT( !(Complex (1, 1) == Complex (2, 2)) ); } };
  108. 108. Three families • XUnit • TAP • Miscellaneous
  109. 109. XUnit • Distinctive markings: “assert” • Output: • ...F.....F...FF... • JUnit, PyUnit, CPPUnit, JSUnit, etc
  110. 110. TAP • Distinctive markings: “ok” or “is” • Output: • ok 1 - some comment • Test::More, Test.More, libtap, etc
  111. 111. Miscellaneous • eg. xmllint • wrappers/ports/equivalents often exist
  112. 112. TAP • Test Anything Protocol • Standardised test reporting format • Many languages • http://testanything.org/
  113. 113. TAP output 1..4 ok 1 - Input file opened not ok 2 - First line of the input valid # Failed test ‘First line of input valid’ # at test.pl line 42 # ‘begin data...’ # doesn't match '(?-xism:BEGIN)’ ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet
  114. 114. TAP Producers • Anything that outputs TAP • eg. Test::More, Test.More, libtap
  115. 115. TAP Consumers • Anything that reads/parses TAP • eg. Perl’s Test::Harness
  116. 116. More test libraries • http://opensourcetesting.org/ • Ada, C/C++, HTML, Java, Javascript, .NET, Perl, PHP, Python, Ruby, SQL, Tcl, XML, others
  117. 117. Questions?
  118. 118. TDD Cookbook
  119. 119. Private code • Can you test private code? • private/protected/hidden/internal • Should you test private code?
  120. 120. The argument against • Private code should never be accessed externally • You want to change private code at will, without breaking tests
  121. 121. The argument for • Assists in refactoring • Testing is better than not testing • If tests break, change/fix them with impunity
  122. 122. “Black box” testing • Examine inputs and outputs • Don’t “see” anything inside
  123. 123. “Glass box” testing • Look inside the code • Examine inner workings
  124. 124. Example sub display_details { my ($inventory_id) = @_; my $output; # 1,000 lines of legacy code ... $price = ‘$’ . $price; if (int($price) == $price) { $price = $price . “.00”; } $output .= “Price: $pricen”; # 1,000 more lines of legacy code return $output; }
  125. 125. Bug report “There’s something wrong with the price display. I occasionally see prices like $123.400, when only two digits should appear after the decimal point.”
  126. 126. Black box • Comprehend 2000+ lines of legacy code • Write tests for entire subroutine • Check price embedded in larger output • Curl up under desk, whimper
  127. 127. White box • Extract private method • Test private method
  128. 128. Extract method sub _format_price { my ($price) = @_; $price = ‘$’ . $price; if (int($price) == $price) { $price = $price . “.00”; } return $price; }
  129. 129. Test is(_format_price(12), ‘$12.00’, ‘12 becomes $12.00’); is(_format_price(11.99, ‘$11.99’, ‘11.99 becomes $11.99’); is(_format_price(12.3, ‘$12.30’, ’12.3 becomes $12.30’);
  130. 130. Conclusion • I like glass box testing • Some people don’t • They are obviously misguided • Q.E.D.
  131. 131. External applications • scripts • executables • foreign APIs
  132. 132. External applications • Do you trust them? • “Black box” testing
  133. 133. Things to test • Return values • Output • Side effects
  134. 134. Return values • Works on Unix • success == 0 • Otherwise, ???
  135. 135. Output • Printed output • Warnings/log messages • Use pipes/redirection
  136. 136. Side effects • Eg. changed files • Risky
  137. 137. Example is(system($some_command), 0, “Command ran OK”); ok(! -e $somefile, “File doesn’t exist.”); is(system($create_file_command), 0, “Command ran OK”); ok(-e $somefile, “File exists now.”)
  138. 138. Testing complex systems • Complex libraries • Side effects • Pre-requisites
  139. 139. Example sub notify { my ($user_id) = @_; if (my $user = fetch_from_db($user_id)) { send_notification_email($user->email()); return 1; } return 0; };
  140. 140. Example • Requires database access • Actually sends email
  141. 141. Mocking • Fake database connection • Fake email sending • Only test notify() logic
  142. 142. Mock libraries Perl Test::MockObject Python python-mock Java jMock PHP PHPUnit (builtin)
  143. 143. Test::MockObject use OurDB::User; use Test::MockObject::Extends; my $user = OurDB::User->fetch($user_id); $user = Test::MockObject::Extends->new($user); $user->mock('email', sub { ‘nobody@example.com' });
  144. 144. python-mock >>> from mock import Mock >>> myMock = Mock( {quot;fooquot; : quot;you called fooquot;} ) >>> myMock.foo() 'you called foo' >>> f = myMock.foo >>> f <mock.MockCaller instance at 15d46d0> >>> f() 'you called foo' >>> f( quot;wibblequot; ) 'you called foo' >>>
  145. 145. Databases • “Real” data • Unchanging data • Set up/tear down
  146. 146. Fixtures • Known data • Setup/teardown • Repeatable tests
  147. 147. Ruby on Rails • Built-in fixtures • YAML or CSV format • Automatically loaded • Tests occur inside transaction
  148. 148. YAML format david: id: 1 name: David Heinemeier Hansson birthday: 1979-10-15 profession: Systems development steve: id: 2 name: Steve Ross Kellock birthday: 1974-09-27 profession: guy with keyboard
  149. 149. CSV format id, username, password, stretchable, comments 1, sclaus, ihatekids, false, I like to say quot;quot;Ho! Ho! Ho!quot;quot; 2, ebunny, ihateeggs, true, Hoppity hop y'all 3, tfairy, ilovecavities, true, quot;Pull your teeth, I willquot;
  150. 150. Loading fixtures require '/../test_helper' require 'user' class UserTest < Test::Unit::TestCase fixtures :users def test_count_my_fixtures assert_equal 5, User.count end end
  151. 151. Websites • Complex systems • Backend/frontend • Browser dependent
  152. 152. Backend strategy • Refactor mercilessly • Separate • Database access • Business logic • Everything you can • MVC good!
  153. 153. Front-end strategy • Web pages and links • Forms • Javascript • Browser compatibility
  154. 154. Web links • Use a web browser / user agent • WWW::Mechanize • Check HTTP GET on each page • Status 200 • Follow links on pages
  155. 155. Forms • Use web browser / user agent • HTTP GET / HTTP POST • Check validation • Check results of form
  156. 156. Javascript • Refactor into libraries • Test using Test.More or similar
  157. 157. Browser testing • Mimic actual browser use • Selenium • http://openqa.org/selenium/
  158. 158. Selenium • Firefox / IE / Safari / others • Linux/Mac/Windows • Record browser actions • Save as scripts • Language integration • Java, .Net, Perl, Python, Ruby
  159. 159. Selenium IDE
  160. 160. Team testing • Many developers • Distributed/asynchronous development • Different platforms • One test strategy
  161. 161. Who’s responsible?
  162. 162. Team responsibility • Everyone is responsible • See a problem, fix it • No code ownership
  163. 163. Individual blame • Correlate failures with checkins • “svn blame” • “Golden fork-up award”
  164. 164. Team testing tips • Use standard tools • Document processes • Nightly smoke test • Integrate with version control
  165. 165. Questions?
  166. 166. Kirrily Robert http://infotrope.net/ skud@infotrope.net
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×