german talk about the variety of CPAN testing modules

1. Perl Testing

2. Testphilosophie

3. Testgeschichte
bis 1956 - Debugging oriented
1957–1978 - Demonstration oriented
1979–1982 - Destruction oriented
1983–1987 - Evaluation oriented
1988–2000 - Prevention oriented

4. Testwissenschaft
Smoke -
Unit -
Regression -
white-/Grey-/Black– box

5. Testmafia
V-Modell
XP

6. Testreligion

7. Am Anfang

8. Am Anfang

9. 1987: Perl 1.0
starke Testsuite

10. 1987: Perl 1.0
starke Testsuite
Anfänge von TAP

11. 1987: Perl 1.0
starke Testsuite
Anfänge von TAP
Testscripte benutzten print

12. 1987: Perl 1.0
starke Testsuite
Anfänge von TAP
Testscripte benutzten print
TEST fasste zusammen

13. 1987: Perl 1.0
starke Testsuite
Anfänge von TAP
Testscripte benutzten print
TEST => Test::[Harness|Run]

14. Starte Tests

15. Starte Tests
make test # nur wenn Makefile da ist

16. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl

17. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}

18. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t

19. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t
prove -v # mehr Ausgabe

20. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t
prove --verbose # mehr Ausgabe
prove t xt # + alternative Tests

21. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t
prove --verbose # mehr Ausgabe
prove t xt # + alternative Tests
prove -r # alle dir (keine subdir)

22. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t
prove --verbose # mehr Ausgabe
prove t xt # + alternative Tests
prove -r # alle dir (keine subdir)
prove --jobs N # nutze N Kerne

23. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t
prove --verbose # mehr Ausgabe
prove t xt # + alternative Tests
prove -r # alle dir (keine subdir)
prove --jobs N # nutze N Kerne
prove -s # shuffle–zufällige Reih.

24. Starte Tests
make test # nur wenn Makefile da ist
perl t/datei.pl
perl datei.pl # BEGIN {chdir '..' if -d '../t'}
prove # oberhalb von t
prove --verbose # mehr Ausgabe
prove t xt # + alternative Tests
prove -r # alle dir (keine subdir)
prove --jobs N # nutze N Kerne
prove -s # shuffle–zufällige Reih.

25. Test

26. Test
use Test;
tests =>, todo=>, onfail =>
ok();
skip();
plan(); # Joshua Nathaniel Pritikin

27. sofort austauschen
# use Test;
use Test::Legacy;

28. Test::Legacy::More
# use Test;
use Test::Legacy;

29. Test::Legacy
use Test::Legacy;
use Test::Legacy::More;
# use Test::More import => [ qw(!ok !skip !plan)];

30. Warum?
TAP : Test Anything Protocol

31. Wichtig!

32. Test::Simple

33. Test::Simple
use Test::Simple 'no_plan';
ok( $dies eq $das, 'dies gleich das' );

34. Test::Simple
use Test::Simple 'no_plan';
ok( $dies eq $das, 'dies gleich das' );
Vergleich Meldung (Name)

35. Test::Simple
make test # nur wenn Makefile da ist

36. Test::Simple
use Test::Simple 'no_plan';
ok( $dies eq $das, 'dies gleich das' );
ok( $dies == $das, 'numerisch gleich' );

37. Test::Simple
use Test::Simple tests => 2;
ok( $dies eq $das, 'dies gleich das' );
ok( $dies == $das, 'numerisch gleich' );

38. Test::Simple
package Test::Simple;
use 5.006;
use strict;
our $VERSION = '0.96';
use Test::Builder::Module;
our @ISA = qw(Test::Builder::Module);
our @EXPORT = qw(ok);
my $CLASS = __PACKAGE__;

39. Test::Between
package Test::Between;
use base 'Exporter';
our @EXPORT = qw( is_between);
use Test::Builder;
my $test = Test::Builder->new();
sub is_between($$$;$) {
my ($item, $lower, $upper, $name) = @_;
return ( $test->ok(...) || $test->diag(„...“)
);
}1;

40. Test::More
wie Test::Simple
mehr Augabe

41. Test::More
use Test::More 'no_plan'; # or skip_all => $reason;
ok, is, isnt, like, unlike, cmp_ok, is_deeply
use_ok, require_ok, can_ok, isa_ok, new_ok
SKIP, TODO, BAIL_OUT, subtest
diag, note, explain, pass, fail, done_testing

42. Test::More
ok, is, isnt, like, unlike, cmp_ok, is_deeply
Vergleiche: pos, eq, re, cmp, HoH....

43. Test::More
Module
use_ok, require_ok, can_ok, isa_ok, new_ok

44. Test::More
Steuern:
überspringen, abbrechen, zusammenfassen
SKIP, TODO, BAIL_OUT, subtest

45. Test::More
Ausgabe: Warnungen(STDERR), Zählung
diag, note, explain, pass, fail, done_testing

46. Test::More
use Test::More 'no_plan'; # or skip_all => $reason;
ok, is, isnt, like, unlike, cmp_ok, is_deeply
use_ok, require_ok, can_ok, isa_ok, new_ok
SKIP, TODO, BAIL_OUT, subtest
diag, note, explain, pass, fail, done_testing

47. Test::Most

48. Devel::Cover
$ cover -test
Deleting database /home/pjcj/g/perl/Shell-Source-0.01/cover_db
PERL_DL_NONLAZY=1 /usr/local/pkg/cover/c1/perl-5.8.7/bin/perl "-
MExtUtils::Command::MM"
"-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/bash....ok
...
All tests successful.
Files=6, Tests=24, 17 wallclock secs (16.16 cusr + 1.01 csys = 17.17 CPU)
Reading database from /home/pjcj/g/perl/Shell-Source-0.01/cover_db
------------------------------------------ ------ ------ ------ ------ ------ ------ ------
File stmt bran cond sub pod time total
------------------------------------------ ------ ------ ------ ------ ------ ------ ------
blib/lib/Shell/Source.pm 96.6 65.0 66.7 90.9 n/a 100.0 86.1
Total 96.6 65.0 66.7 90.9 n/a 100.0 86.1
------------------------------------------ ------ ------ ------ ------ ------ ------ ------
Writing HTML output to /home/pjcj/g/perl/Shell-Source0.01/cover_db/coverage.html
…
done.

49. Devel::Cover

50. Test::TestCoverage
use Test::More;
eval {
require Test::Pod::Coverage;
import Test::Pod::Coverage;
};
plan skip_all => "Test::Pod::Coverage 1.08 required
for testing POD coverage" if $@;
all_pod_coverage_ok();

51. Test::Pod::Coverage
use Test::More;
eval {
require Test::Pod::Coverage;
import Test::Pod::Coverage;
};
plan skip_all => "Test::Pod::Coverage 1.08 required
for testing POD coverage" if $@;
all_pod_coverage_ok();

52. Test::If
use Test::More;
use Test::If
# Checked first $ENV{TEST_AUTHOR}, or skip
sub { $ENV{TEST_AUTHOR} },
# Use Test::Pod::Coverage of at least version 1.08
'Test::Pod::Coverage 1.08',
# And want Pod::Coverage at least of version 0.18
'Pod::Coverage 0.18',
;
all_pod_coverage_ok();

53. Test::Pod
use Test::More;
eval {
require Test::Pod;
import Test::Pod;
};
plan skip_all => "Test::Pod missing" if $@;
all_pod_files_ok();

54. Test::Spelling
use Test::Spelling;
use Test::More;
all_pod_files_spelling_ok();

55. Test::Spelling
use Test::Spelling;
use Test::More;
add_stopwords();
my @files = all_pod_files();
pod_files_spelling_ok( @files, "POD file spelling OK" );
pod_file_spelling_ok( $file, "$file spelling OK" );

56. Test::Pod::Spelling::CommonMistakes
use Test::Spelling;
use Test::More;
pod_file_ok( $file, "POD file spelling OK" );
all_pod_files_ok();

57. Test::Strict

58. Test::Kwalitee
use Test::More;
eval {
require Test::Kwalitee;
Test::Kwalitee->import()
};
plan( skip_all => 'Test::Kwalitee not installed; skipping' )
if $@;

59. Test::Kwalitee
● extractable, no_symlinks
● has_readme, have_manifest, have_meta_yml
● have_buildtool, have_changelog, have_tests
● use_strict, proper_libs
● no_pod_errors, have_test_pod, have_test_pod_coverage

60. Test::Distribution
fast deckungsgleich mit Test::Kwalitee

61. Test::Module::Used
use Test::Module::Used;
my $used = Test::Module::Used->new();
$used->ok;
Werden benutze Bibliotheken gefordert?
Werden Geforderte Bibliohteken benutzt?

62. Test::CPANpm
simuliert einen CPAN install
kein ersatz für vorige Module

63. Test::Perl::Critic
use Test::Perl::Critic;
use Test::More tests => 1;
critic_ok($file);
all_critic_ok($dir_1, $dir_2, $dir_N);
all_critic_ok();

64. Test::Most
use Test::More;
use Test::Differences;
use Test::Deep;
use Test::Exception;
use Test::Warn;
mehr ...

65. Test::Most
use Test::More;
use Test::Differences;
use Test::Deep;
use Test::Exception;
use Test::Warn;
mehr ...

66. Test::Most
● die_on_fail bail_on_fail
● restore_fail set_failure_handler
● explain # prove -v
● show
● all_done

67. Test::Most
use Test::More;
use Test::Differences;
use Test::Deep;
use Test::Exception;
use Test::Warn;
mehr ...

68. Test::Differences

69. Test::Differences
use Test::More;
is_deeply([1,2],[1,2,[3]], '… nicht gleich');

70. Test::Differences
use Test::More;
is_deeply([1,2],[1,2,[3]], '… nicht gleich');
# Failed test 'is_deeply'
# at t02-more.t line 12.
# Structures begin differing at:
# $got->[3] = ARRAY(0xae3704)
# $expected->[3] = Does not exist

71. Test::Differences
use Test::More tests => 1;
use Test::Differences;
eq_or_diff([1,2],[1,2,[3]], '… nicht gleich');

72. Test::Differences
use Test::Most tests => 1;
eq_or_diff([1,2],[1,2,[3]], '… nicht gleich');

73. Test::Differences
# Failed test '...nicht gleich' eq_or_diff([1,2],[1,2,[3]]
# at t04-diff.t line 8.
# +----+------+----+------------+
# | Elt|Got | Elt|Expected |
# +----+------+----+------------+
# | 0|[ | 0|[ |
# | 1| 1, | 1| 1, |
# * 2| 2 * 2| 2, *
# | | * 3| [ *
# | | * 4| 3 *
# | | * 5| ] *
# | 3|] | 6|] |
# +----+------+----+------------+

74. Test::Differences
4 Anzeigestile:
* table_diff; #(the default)
* unified_diff;
* oldstyle_diff;
* context_diff;

75. Test::Differences
# Failed test '...nicht gleich' unified_diff;
# at t04-diff.t line 8. eq_or_diff([1,2],[1,2,[3]]
# --- Got
# +++ Expected
# @@ -1,4 +1,7 @@
# [
# 1,
#- 2
# + 2,
#+ [
#+ 3
#+ ]

76. Test::Differences
# Failed test '...nicht gleich' oldstyle_diff;
# at t04-diff.t line 8. eq_or_diff([1,2],[1,2,[3]]
# 3c3,6
#< 2
# ---
# > 2,
#> [
#> 3
#> ]

77. Test::Differences
# *** Got context_diff;
# --- Expected eq_or_diff([1,2],[1,2,[3]]
# ***************
# *** 1,4 ****
# [
# 1,
#! 2
# ]
# --- 1,7 ----
# [
# 1,
#! 2,
#! [
#! 3
#! ]
# ]

78. Test::Differences
use Test::More; # oder Test
use Test::Differences; # use Text::Diff;
eq_or_diff($gegeben, $gesucht, $meldung);
# autoselect:
eq_or_diff_data($gegeben, $gesucht, $meldung);
eq_or_diff_text($gegeben, $gesucht, $meldung);

79. Test::More
use Test::More;
is_deeply([1,2],[1,2,[3]], '… nicht gleich');

80. Test::More
use Test::More;
is_deeply([1,2],[1,2,[3]], '… nicht gleich');
eq_array();
eq_hash();
eq_set();

81. Test::More
use Test::More;
is_deeply([1,2],[1,2,[3]], '… nicht gleich');
eq_array();
eq_hash();
eq_set();

82. Test::Deep
use Test::More 'no_plan';
use Test::Deep;
cmp_deeply($got, $expected, $name); # is_deeply
not ok 1 – test name
# Failed test 'test name'
# at t03-deep.t line 7.
# Compared $data->[3][2][0]
# got : '2'
# expect : '1'

83. Test::Deep
use Test::Most 'no_plan';
cmp_deeply($got, $expected, $name); # is_deeply
eq_deeply($got, $expected); # no test
cmp_bag(@got, @bag, $name); # Reihenfolge egal
cmp_set(@got, @set, $name); # Mehrfache mögl.
cmp_methods(@got, @methods, $name);

84. Test::Deep
use Test::Most 'no_plan';
cmp_deeply($got, $expected, $name); # is_deeply
eq_deeply($got, $expected); # no test
cmp_bag(@got, @bag, $name); # bag wie in P6
cmp_set(@got, @set, $name); # eq_set
cmp_methods(@got, @methods, $name);

85. Aliase
use Test::Deep;
cmp_bag(@got, @bag, $name);
cmp_deeply(@got, bag(@exp), $name);
cmp_set(@got, @set, $name);
cmp_deeply(@got, set(@exp), $name);
cmp_methods($obj, @exp, $name);
cmp_deeply($obj, methods(%exp), $name);

86. kombinierbare Befehle
cmp_deeply(@got, bag(@exp), $name);
cmp_deeply(@got, set(@exp), $name);
cmp_deeply($obj, methods(%exp), $name);

87. kombinierbare Befehle
bag(@exp)
set(@exp)
methods(%exp)

88. kombinierbare Befehle
ignore()
any(@exp) / all(@exp)
bag(@exp)
set(@exp)
super|sub . hash|bag|set . of ()
array_each
str / num / bool / code
re(@exp)
shallow($ref)
methods(@exp)
listmethods(@exp)
noclass(@exp)
useclass(@exp)
isa(@exp)

89. Test::Deep::NoTest
use Test::Deep::NoTest;

90. Test::Exception
use Test::Exception;
throws_ok( {...}, $ausnahmeklasse, $name);
throws_ok( {...}, $regex, $name);
lives_ok( {…}, $name);
dies_ok({...}, $name);

91. Test::Warn
# Welche Warnungen wurden ausgegeben?
use Test::Warn;
warning_is( {...} $warnung, $name);
warnings_are( {...} @warnungen, $name);
warning_like();

92. Test::Most
use Test::More;
use Test::Differences;
use Test::Deep;
use Test::Exception;
use Test::Warn;
mehr … fertig

93. Bundle::Test
Test
Test::Simple
Test::More
Test::Harness
Test::MockObject
Test::Pod
Test::Builder
Test::Warn
Test::Inline
Test::Strict
Test::Distribution
Test::Perl::Critic
Test::Kwalitee
Pod::Spell
Test::Spelling
Devel::Cover
...

94. Test::NoWarnings
use Test::NoWarnings;

95. Test::NoWarnings
use Test::NoWarnings;
# $plan += 1
>ok $plan - no warnings

96. Test::Script
use Test::Script;
script_compiles();
script_runs();

97. Test::Cmd
use Test::Cmd;
my $cmd = Test::Cmd->new ( prog => 'prog.pl',
interpreter => 'script_interpreter',
string => 'identifier_string',
workdir => '',
subdir => 'dir',
match_sub => $code_ref,
verbose => 1, ...);
$cmd->write( # Datei schreiben
$cmd->run( # $? == 0

98. Test::Expect
expect_run (
command => "cpan",
prompt => 'install Kephra',
quit => 'q',
);
expect_like(qr/install UNINST=1 -- OK/, "gut");
expect_handle();

99. Test::Timer
use Test::Timer;
time_ok, time_atmost
time_between
time_atleast
time_between

100. Test::Benchmark
use Test::Timer;
is_faster($times, $sub1, $sub2, $name);
is_n_times_faster($factor, $t, $sub1, $sub2, $name);
is_fastest();

101. Test::ManyParams
use Test::ManyParams;
all_ok { tannu(shift) }, @para, $nachricht;
all_ok { tuwas(@_) }, [ @a, @b], $nachricht;
qw(all any) X~ qw(ok is isnt)
most_ok{ sub(@_) }, [ @a, @b] => $nr, $nachricht;

102. Random
use File::Random;
my $random_gif = random_file(
-dir => $dir,
-check => qr/.gif$/,
-recursive => 1);
use Data::Random;

103. Win32::GuiTest
SendKeys();
MouseMoveAbsPix(100, 200);
SendMouse();

104. Win32::Screenshot

105. Test::Data
use Test::Data qw(Scalar Array Hash Function);

106. Test::Data::Scalar
blessed_ok
defined_ok
undef_ok
greater_than
less_than
length_ok
maxlength_ok
minlength_ok
number_ok

107. Test::LongString
use Test::LongString max => 100; # 50 ist default
is_string
is_string_nows
contains_string
lacks_string
like_string
unlike_string

108. Test::LongString
use Test::LongString max => 100; # 50 ist default
is_string
# Failed test (html-test.t at line 12)
# got: "<!DOCTYPE HTML PUBLIC
"-//W3C//DTD HTML 4.01 Trans"...
# length: 58930
# doesn't match '(?-xism:(perl|cpan).org)'

109. Test::LongString
use Test::LongString max => 100; # 50 ist default
like_string
# Failed test (soliloquy.t at line 15)
# got: "To be, or not to be: that is the question:x{0a} Wh"
...
# length: 1490
# doesn't match '(?-xism:Romeo|Juliet|Mercutio|Tybalt)'

110. Test::Output
stdout_is
stderr_isnt
output_is
combined_is
stdout_like
stdout_unlike

111. Test::Carp
does_carp(&function);
does_carp_that_matches(&function, qr/whoopsy/);

112. Test::Log4perl
Test::Log::Dispatch

113. Test Techniken

114. analyze this

115. Mocking

116. Mocking
Test::MockModule
Test::MockObject
Test::MockObject::Extends
DBD::Mock

117. Test::MockModule
use Module::Name;
use Test::MockModule;
{
my $module = new Test::MockModule('Module::Name');
$module->mock('subroutine', sub { ... });
Module::Name::subroutine(@args); # mocked
}
Module::Name::subroutine(@args); # unverändert

118. Test::MockObject
use

119. Test::MockObject::Extends
use

120. Devel::Hide
use Devel::Hide qw[Test::Most];
use Tests::Most; # error

121. Test::Without::Module
use Test::Without::Module qw[Test::Most];
use Tests::Most; # error

122. Test::Without
run { ... } without 'Net::Server';
run { ... } with 'Net::Server';

123. Organisation

124. Test::Standalone
use Test::Standalone;
=begin test
....
....
=end test
script.pl --test

125. Test::Inline
=begin testing label
...
normaler Testcode
....
....
=end testing

126. Test::Inline
=begin testing label
....
....
=end testing
inline2test.pl

127. Test::Plan
plan tests => 3, need need_module('Kephra'),
need_min_module_version(Wx => 0.9),
need_min_perl_version(5.6);

128. Test::Block
use Test::More 'no_plan';
use Test::Block qw($Plan);
{
local $Plan = 2;
}

129. Test::Group
use Test::More no_plan;
use Test::Group;
test "tausend tests" => sub {
ok(1,'—') for 1..1000;
};
----------------------------------------------------------
ok 1 - tausend tests
1..1

130. Unit-Testing

131. Test::Unit
# starte und räume auf
sub set_up { ... }
sub tear_down { ... }
# eigentliche tests
sub test_... { ... {}
# starte alle tests
create_suite();
run_suite();

132. Test::Unit
use Test::Unit; # 2001

133. Test::Unit::Lite
use Test::Unit::Lite; # 2009

134. Test::SimpleUnit

135. Test::Extreme
use TestConfigs;
use TestAPI;
use TestOutput;
run_tests 'TestConfigs', 'TestAPI', 'TestOutput';

136. Test::Extreme
use Test::Extreme;
sub foo { return 23 };
sub test_foo { assert_equals foo, 23 }

137. Test::Class
package Meine::Test::Klasse;
use base qw(Test::Class);
sub eats : Test(3) {
Meine::Test::Klasse->runtests

138. Test::Class
package Meine::Test::Klasse;
sub eats : Test(3) {
package main;
Meine::Test::Klasse->runtests;

139. Test::Class
BEGIN {
chdir 't' if -d 't';
require Meine::Test::Klasse;
}
Meine::Test::Klasse->runtests;
# Meine/Test/Klasse.pm

140. Test::Class
sub tischlein_deck_dich : Test (startup) {
sub knueppel_aus_dem_sack : Test (shutdown) {
sub neues_tischtuch : Test (setup => 2 ) {
sub abraeumkommando : Test (teardown) {

141. Test::Class::Sugar
use Test::Class::Sugar;

142. Test::Able

143. Test::Able

144. Test::Able
package MyTest;
use Test::Able;
use Test::More;
startup some_startup => sub { ... };
setup some_setup => sub { ... };
test plan => 1, foo => sub { ok( 1 ); };
test bar => sub {
my @runtime_list = 1 .. 42;
$_[ 0 ]->meta->current_method->plan( scalar @runtime_list );
ok( 1 ) for @runtime_list;
};
teardown some_teardown => sub { ... };
shutdown some_shutdown => sub { ... };
MyTest->run_tests;

145. Test::Able
use Test::Able::Helpers qw( shuffle_methods );
$t->meta->test_methods(
[ grep { $_->name !~ /bar/; }
@{ $t->meta->test_methods } ]
);
for ( 1 .. 10 ) {
$t->shuffle_methods;
$t->run_tests;
}

146. Gutes Buch

147. Links
http://www.slideshare.net/lichtkind
search.cpan.org/dist/Test-Simple/lib/Test/Tutorial.pod
.../dist/Test-Able/lib/Test/Able/Cookbook.pm
http://qa.perl.org/testing/
http://www.cpantesters.org/
http://testanything.org
http://docs.racket-lang.org/rackunit

148. Glückliches Ende