1. Getting There from Here:
Unit testing a few generations of code.
Steven Lembark
Workhorse Computing
lembark@wrkhors.com

2. perl 5.8
Fun isn’t it? A generation of nothing.
No improvements in Perl.
No advances in CPAN.
No reason to upgrade.
20 years of nada...
Ask all the programmers stuck with 5.8.

3. perl 5.8
Q: Why are we stuck with 5.8?
A: /usr/bin/perl is 5.8
Q: Why are we stuck with /usr/bin/perl?
A: ???

4. perl 5.8
Q: Why are we stuck with 5.8?
A: /usr/bin/perl is 5.8
Q: Why are we stuck with /usr/bin/perl?
A: Because “it worked”.

5. perl 5.8
Q: Why are we stuck with 5.8?
A: /usr/bin/perl is 5.8
Q: Why are we stuck with /usr/bin/perl?
A: Because something else “might not” work.

6. perl 5.8
Q: Why are we stuck with 5.8?
A: /usr/bin/perl is 5.8
Q: Why are we stuck with /usr/bin/perl?
A: Because we can’t show that 5.X works.
(for some definition of Christmas > 5.8)

7. perl 5.X
Q: Why can’t we show that 5.X works?
A: Because Christmas came after 2002.

8. perl 5.X
Q: Why can’t we show that 5.X works?
A: Because we stopped testing our code.

9. perl 5.X
Q: Why can’t we show that 5.X works?
A: Because they stopped testing our code.

10. perl 5.X
Q: Why can’t we show that 5.X works?
A: Because we have worked around it for way too long.

11. perl 5.X
Q: Why can’t we show that 5.X works?
A: Because we have worked around it for way too long.
perlbrew, all of the other approaches to dodging
/usr/bin/perl.

12. OK how do we fix it?
Prove that “it works”.
With testing.
In one client’s case: Lots and lots of testing.

13. OK how do we fix it?
Prove that “it works”.
With testing.
In one client’s case: Lots and lots of testing.
75_000 modules worth of testing.

14. OK how do we fix it?
Prove that “it works”.
With testing.
In one client’s case: Lots and lots of testing.
75_000 modules worth of testing.
Q: But how did you write 75_000 unit tests?

15. OK how do we fix it?
Prove that “it works”.
With testing.
In one client’s case: Lots and lots of testing.
75_000 modules worth of automated testing.
A: Lazyness.

16. Syntax checks are the first pass.
Unit tests get a bad rap.
“They don’t really test anything.”

17. Syntax checks are the first pass.
Unit tests get a bad rap.
“They don’t really test anything.”
Except whether 75_000 modules compile.
Code that was written on 5.8.
That now to run on 5.20+.

18. How do we test that much code?
Metadata driven testing:
Encode data into a test.
The standard test is data-driven.
Tests each validate one small part of a module.

19. How do we test that much code?
Q: How do you write that many tests?
A: Don’t.

20. How do we test that much code?
Q: How do you write that many tests?
A: Don’t.
Symlinks are your friends.

21. Simple basic test: require_ok
Run “require_ok $module” for every *.pm.
Simple, basic, obvious test for syntax errors.
… missing modules.
… botched system paths.
… all sorts of things.

22. Simple basic test: require_ok
Run “require_ok $module” for every *.pm.
Simple, basic, obvious test for syntax errors.
Nice thing: $module can be a path.
Even an absolute path.

23. Passing a path
“require_ok $path”
One argument.
Q: How do we pass it?
A: In the basename of a test.

24. Basic unit test
use v5.30;
use Test::More;
use File::Basename;
my $base0 = basename $0, '.t';
my $s = substr $base0, 0, 1;
my $path = join ‘/’, split m{[$s]}, $base0;
require_ok $path;
done_testing
__END__

25. Basic unit test
Install
them
using
shell.
#!/bin/bash
dir=$(cd $(dirname $0)/../..; pwd -L);
cd “$dir/t/01-units”;
find $dir/lib -name ‘*pm’ |
while read i
do
base=”$(echo $i | tr ‘/’ ‘~’).t”;
ln -fs ../bin/01-unit_t ./$base;
done

26. Basic unit test
Install
them
using
shell.
From
./t/bin
#!/bin/bash
dir=$(cd $(dirname $0)/../..; pwd -L);
cd “$dir/t/01-units”;
find $dir/lib -name ‘*pm’ |
while read i
do
base=”$(echo $i | tr ‘/’ ‘~’).t”;
ln -fs ../bin/01-unit_t ./$base;
done

27. Basic unit test
Install
them
using
shell.
Path to
basename.
#!/bin/bash
dir=$(cd $(dirname $0)/../..; pwd -L);
cd “$dir/t/01-units”;
find $dir/lib -name ‘*pm’ |
while read i
do
base=”$(echo $i | tr ‘/’ ‘~’).t”;
ln -fs ../bin/01-unit_t ./$base;
done

28. Basic unit test
Now you
know if it
compiles. prove t/01-units;

29. Basic unit test
Now you
know if it
compiles.
A bit
faster.
prove --jobs=8 t/01-units;

30. Basic unit test
Now you
know if it
compiles.
At 3am
without
watching.
args=(--jobs=8 --state=save );
prove ${args[*]} t/01-units;

31. Basic unit test
Now you
know if it
compiles.
At 8am
knowing
what
failed.
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units;

32. Basic unit test
Now you
know if it
compiles.
Missing
module
anyone?
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units
Can't locate Foo/Bar.pm in @INC (you
may need to install the Foo::Bar
module) (@INC contains:
/opt/perl/5.30/lib/site_perl/5.30.1/x8
6_64-linux

33. Basic unit test
Now you
know if it
compiles.
Fine:
Install
them.
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units |
grep ‘you may need to install the’ |
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm ;

34. Basic unit test
Now you
know if it
compiles.
Local
mirror
provides
pre-
verified
modules.
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units |
grep ‘you may need to install the’ |
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm -M$local ;

35. Basic unit test
Now you
know if it
compiles.
Local
library
simplifies
auto-
install.
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units |
grep ‘you may need to install the’ |
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm
-M$local -l$repo --self-contained ;

36. Basic unit test
Start with
a virgin
/opt/perl.
Keep
testing
until it’s
all use-
able.
args=(--jobs=8 --state=save,failed );
prove ${args[*]} t/01-units |
grep ‘you may need to install the’ |
cut -d’(‘ -f2 |
cut -d’ ‘ -f7 |
sort -d |
uniq |
cpanm
-M$local -l$repo --self-contained ;

37. Version control what you install
git submodule add blah://.../site_perl;
perl Makefile.PL INSTALL_BASE=$PWD/site_perl;
cpanm --local-library=$PWD/site_perl;
If anyone asks, you know what non-core modules have
been installed.
If anything breaks, check out the last commit.

38. What else can we do with a path?
Ever fat-finger
a package?
package AcmeWigdit::Config;
use v5.30;

39. What else can we do with a path?
Ever fat-finger
a package?
Paths define
packages.
require_ok $path or skip ... ;
my ($sub)
= $path
=~ m{^.+?/lib/(.+?) [.]pm}x;
my $pkg = $sub =~ s{/}{::}g;
isa_ok $pkg, ‘UNIVERSAL’
or skip “$pkg not ‘UNIVERSAL’”;
$pkg->can( ‘VERSION’ )
or skip “$path missing $pkg”, 1;

40. What else can we do with a path?
Ever fat-finger
a package?
Paths define
packages.
All defined
packages are
UNIVERSAL.
require_ok $path or skip ... ;
my ($sub)
= $path
=~ m{^.+?/lib/(.+?) [.]pm}x;
my $pkg = $sub =~ s{/}{::}g;
isa_ok $pkg, ‘UNIVERSAL’
or skip “$pkg not ‘UNIVERSAL’”;
$pkg->can( ‘VERSION’ )
or skip “$path missing $pkg”, 1;

41. What else can we do with a path?
Ever fat-finger
a package?
Paths define
packages.
VERSION is
UNIVERSAL.
require_ok $path or skip ... ;
my ($sub)
= $path
=~ m{^.+?/lib/(.+?) [.]pm}x;
my $pkg = $sub =~ s{/}{::}g;
is_ok $pkg, ‘UNIVERSAL’
or skip “$pkg not ‘UNIVERSAL’”;
$pkg->can( ‘VERSION’ )
or skip “$path missing $pkg”, 1;

42. Minor issue with paths
Packages are related to paths.
We know that now.
In 2000 not everyone got that.

43. Minor issue with paths
What about:
package <basename>;
use lib <every directory everywhere>;
It works, but there are better ways...

44. Minor issue with paths
One pattern: Overlapping product of evolution.
./lib/AcmeWig/
./lib/AcmeWig/Config.pm AcmeWig::Config

45. Minor issue with paths
One pattern: Overlapping product of evolution.
./lib/AcmeWig/
./lib/AcmeWig/Config.pm AcmeWig::Config

46. Minor issue with paths
One pattern: Overlapping product of evolution.
./lib/AcmeWig/
./lib/AcmeWig/Config.pm AcmeWig::Config
./lib/AcmeWig/Acme/Config.pm Acme::Config

47. Minor issue with paths
One pattern: Overlapping product of evolution.
./lib/AcmeWig/
./lib/AcmeWig/Config.pm AcmeWig::Config
./lib/AcmeWig/Acme/Config.pm Acme::Config
What’s the expected package?

48. Solving nested paths
Say we figure out the paths:
qw
(
./lib/AcmeWig
./lib/AcmeWig/Acme
./lib/AcmeWig/Acme/Plastic/External
./lib/AcmeWig/Acme/Plastic/Internal
./lib/AcmeWig/Acme/Plastic/Wrappers
./lib/AcmeWig/Plastic
./lib/AcmeWig/AcmeWig/External/Plastic
)

49. Solving nested paths
We can obviously iterate them...
qw
(
./lib/AcmeWig
./lib/AcmeWig/Acme
./lib/AcmeWig/Acme/Plastic/External
./lib/AcmeWig/Acme/Plastic/Internal
./lib/AcmeWig/Acme/Plastic/Wrappers
./lib/AcmeWig/Plastic
./lib/AcmeWig/AcmeWig/External/Plastic
)

50. Solving nested paths
Q: But how do we avoid duplicate tests?
qw
(
./lib/AcmeWig
./lib/AcmeWig/Acme
./lib/AcmeWig/Acme/Plastic/External
./lib/AcmeWig/Acme/Plastic/Internal
./lib/AcmeWig/Acme/Plastic/Wrappers
./lib/AcmeWig/Plastic
./lib/AcmeWig/AcmeWig/External/Plastic
)

51. Solving nested paths
A: Sort them by length.
qw
(
./lib/AcmeWig/AcmeWig/External/Plastic
./lib/AcmeWig/Acme/Plastic/External
./lib/AcmeWig/Acme/Plastic/Internal
./lib/AcmeWig/Acme/Plastic/Wrappers
./lib/AcmeWig/Plastic
./lib/AcmeWig/Acme
./lib/AcmeWig
)

52. Solving nested paths
A: Sort them by length & exclude known paths.
qw
(
./lib/AcmeWig/AcmeWig/External/Plastic
./lib/AcmeWig/Acme/Plastic/External
./lib/AcmeWig/Acme/Plastic/Internal
./lib/AcmeWig/Acme/Plastic/Wrappers
./lib/AcmeWig/Plastic
./lib/AcmeWig/Acme
./lib/AcmeWig
)

53. Solving nested paths
my %prune = ();
my $wanted = sub
{
-d && exists $prune{ $File::Find::dir }
? $File::Find::prune = 1
: $File::Find::Path->$install_test_symlink
};
for my $dir ( @dirs_sorted_by_length )
{
find $wanted, $dir;
$prune{ $dir } = ();
}

54. Solving nested paths
my %prune = ();
my $wanted = sub
{
-d && exists $prune{ $File::Find::dir }
? $File::Find::prune = 1
: $File::Find::Path->$install_test_symlink
};
for my $dir ( @dirs_sorted_by_length )
{
find $wanted, $dir;
$prune{ $dir } = (); # don’t revisit
}

55. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.

56. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.
Fix: Double the separator.
~path~to~repo~lib~AcmeWig~Acme~Config.pm.t

57. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.
Fix: Double the separator.
~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t

58. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.
Fix: ‘//’ works fine with require_ok.
~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t
/path/to/repo/lib/AcmeWig//Acme/Config.pm.t

59. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.
Fix: Gives sub-path for module.
~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t
Acme~Config.pm.t

60. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.
Fix: basename with s{~}{::}.
~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t
Acme::Config

61. Testing nested paths
Catch: The test needs to know its ‘root’ directory.
Has to subtract that to get the expected package.
Save the search dir’s length and substr in a ‘~’:
my $l = length $dir;
...
substr $symlink, $l, 0, ‘~’;

62. Skipping test groups
We started with 75_000 files.
Found some dirs we didn’t want to test.
Fix:
my @no_test = qw( … );
my %prune = ();
@prune{ @no_test } = ();

63. Net result
We were able to test 45_000+ files each night.
Found missing modules.
Found outdated syntax.
Managed to get it all working.
Largey with “require_ok”.

64. Knowledge is power
Unit tests are useful.
Provide automated, reproducable results.
No excuse for “It may not work in v5.30.”
We can know.
And fix whatever “it” is.

65. For some definition of Perl5
Perl7 is coming.
Adopting it requires showing that “it works.”
Which means finding where it doesn’t.
Unit testing all of CPAN.
Unit testing all of everything.

66. For some definition of Perl5
Perl7 is coming.
Adopting it requires showing that “it works.”
Which means finding where it doesn’t.
Unit testing all of CPAN.
Unit testing all of everything.
We just have to be lazy about it.

67. Units of your very own!
In case you don’t like pasting from PDF:
https://gitlab.com/lembark/perl5-unit-tests
If anyone wants to work on this we can release it as
App::Testify or Test2::Unitz …
Be nice to have a migration suite for Perl7.

68. Units of your very own!
A wider range of unit tests is shown at:
https://www.slideshare.net/lembark/path-toknowlege
With a little bit of work we could get them all on CPAN.