21st Century
CPAN Testing
Mike Friedman
(friedo)
MongoDB, Inc.
Aug. 27, 2013:
, Inc.
, Inc.
makes
(the database)
, Inc.
makes
(the database)
employs
Mike Friedman
(friedo)
, Inc.
makes
(the database)
employs
Mike Friedman
(friedo)
WAT
What is
MongoDB?
Open
Source
Non-relational
Horizontally
Scalable
Document-
oriented
Fast Database
Fault-tolerant CoolSchemaless
Document-
oriented
Document-
oriented
JSON-like thingy:
{
"foo": "a string",
"bar": 42,
"baz": [ 1, 2, "narf", "poit" ],
"quux": {
"key_1": "w00t.",
"key_2": "y...
{
"foo": "a string",
"bar": 42,
"baz": [ 1, 2, "narf", "poit" ],
"quux": {
"key_1": "w00t.",
"key_2": "you get the idea",
...
{
"foo": "a string",
"bar": 42,
"baz": [ 1, 2, "narf", "poit" ],
"quux": {
"key_1": "w00t.",
"key_2": "you get the idea",
...
{
"foo": "a string",
"bar": 42,
"baz": [ 1, 2, "narf", "poit" ],
"quux": {
"key_1": "w00t.",
"key_2": "you get the idea",
...
First-Class Objects
First-Class Objects
First-Class Objects
Queryable
First-Class Objects
Queryable Indexable
First-Class Objects
Queryable Indexable Updateable
Testing CPAN
in the
21st Century
A lengthy series of bad
ideas and stupid
questions.
Stupid Question No. 1
Stupid Question No. 1
Who here uses CPAN?
Stupid Question No. 2
Stupid Question No. 2
Who here is a CPAN author?
What’s this about,
anyway?
What’s this about,
anyway?
CPANci
A Brief History
A Brief History
•December 18, 1987
A Brief History
•December 18, 1987
•Perl 1.000 released.
A Brief History
•December 18, 1987
•Perl 1.000 released.
•TAP invented.
The Test Anything Protocol
The Test Anything Protocol
1..42
ok 1 the thing looks good!
ok 2
ok 3 $beer isa $drink
not ok 4 too much $beer
not ok 5 $m...
A Brief History
A Brief History
•October 17, 1994
A Brief History
•October 17, 1994
•Perl 5.000 released.
A Brief History
•October 17, 1994
•Perl 5.000 released.
•Perl has a module system.
# from this
require "funcs.pl";
# from this
require "funcs.pl";
# to this
use My::Module;
# but under the hood
BEGIN {
require My::Module;
My::Module->import;
};
A Brief History
A Brief History
•October 26, 1995
A Brief History
•October 26, 1995
•CPAN established.
A Brief History
•October 26, 1995
•CPAN established.
•Perl modules are available.
A Brief History
A Brief History
•May 15, 1997
A Brief History
•May 15, 1997
•Perl 5.004 released.
A Brief History
•May 15, 1997
•Perl 5.004 released.
•CPAN.pm is in the core.
# the dark art
$ perl -MCPAN -e 'install Foo'
A Brief History
A Brief History
•May, 1998
A Brief History
•May, 1998
•CPAN Testers conceived
A Brief History
•May, 1998
•CPAN Testers conceived
•Automated feedback for authors
A Brief History
A Brief History
•November 15, 2003
A Brief History
•November 15, 2003
•Perl 5.6.2 released.
A Brief History
•November 15, 2003
•Perl 5.6.2 released.
•Test::More is in the core.
use Test::More tests => 3;
ok( 42 );
is( $foo, 'my value' );
isnt( 'foo', 'bar' );
A Brief History
A Brief History
•August 6, 2012
A Brief History
•August 6, 2012
•Mike goes to work for 10gen
A Brief History
•August 6, 2012
•Mike goes to work for 10gen MongoDB
Bad Idea No. 1
Bad Idea No. 1
Come up with a cool Perl MongoDB project
to show off atYAPC!
Bad Idea No. 1
Come up with a cool Perl MongoDB project
to show off atYAPC!
It'll be fun!
Bad Idea No. 1
Come up with a cool Perl MongoDB project
to show off atYAPC!
It'll be fun!
promise!
CPAN Testers
CPAN Testers is Wonderful and
Amazing
Disadvantages:
Disadvantages:
Not real time
Disadvantages:
Not real time
Not consistent
Disadvantages:
Not real time
Not consistent
Polluted / Inconsistent
environments
Disadvantages:
Not real time
Not consistent
Polluted / Inconsistent
environments
Not all versions on all platforms
Perl 5.18 runs on
GNU Hurd
Perl 5.18 runs on
GNU Hurd
?
Perl 5.18 runs on
GNU Hurd
?
Perl 5.18 runs on
GNU Hurd
?
I
♥
CPAN
I don't care.
I care about:
I care about:
I care about:
I care about:
I want Continuous Integration
for the entire CPAN.
I want Continuous Integration
for the entire CPAN.
For platforms I care about.
Bad Idea No. 2
CPANci.org
Bad Idea No. 2
Stupid Question No. 3
Stupid Question No. 3
How can we test CPAN without the
disadvantages of CPAN Testers?
Postulate:
Every CPAN distribution must be tested in isolation, on a
virgin Perl installation untouched by human hands.
I release distribution Foo-Awesome-0.0000001
I release distribution Foo-Awesome-0.0000001
use Spiffy::Module;
I release distribution Foo-Awesome-0.0000001
use Spiffy::Module;
Forgotten dependency!
I release distribution Foo-Awesome-0.0000001
use Spiffy::Module;
Forgotten dependency!
What happens?
Scenario  I
The  tester  hath  not  the  missing  
dependency  upon  his  box.
FAIL-
mail
Scenario  II
The  tester  doth  possess  
the  dependency  upon  his  box.
EVERYTHING
IS FINE.
NOT FINE.
Postulate:
Every CPAN distribution must be tested in isolation, on a
virgin Perl installation untouched by human hands.
Postulate:
Every CPAN distribution must be tested in isolation, on a
virgin Perl installation untouched by human hands.
So...
perlbrew
Virtualization
Virtualization
Whoa!
Bad Idea No. 3
•Create an EC2 image
•Create an EC2 image
•Put perlbrew on it
•Create an EC2 image
•Put perlbrew on it
•Install every Perl locally
•Create an EC2 image
•Put perlbrew on it
•Install every Perl locally
•Boot an instance for every
uploaded distribution
•Create an EC2 image
•Put perlbrew on it
•Install every Perl locally
•Boot an instance for every
uploaded distribution
•In...
•Create an EC2 image
•Put perlbrew on it
•Install every Perl locally
•Boot an instance for every
uploaded distribution
•In...
•Create an EC2 image
•Put perlbrew on it
•Install every Perl locally
•Boot an instance for every
uploaded distribution
•In...
Uh-oh.
Stupid Question No. 4
Stupid Question No. 4
How can we do this on one instance?
local::lib
cpanminus
App::cpanminus
App::cpanminus
•Self-contained
App::cpanminus
•Self-contained
•That is, the cpanm script is self-contained
App::cpanminus
•Self-contained
•That is, the cpanm script is self-contained
•via App::FatPacker
App::cpanminus
App::cpanminus
That means the same cpanm can be run by any perl
App::cpanminus
That means the same cpanm can be run by any perl
perlbrew switch master
curl -L http://cpanmin.us/ | perl -...
•Use perlbrew to install a "master" perl
•Use perlbrew to install a "master" perl
•Use it again to install "virgin" perls of every
major version
•Use perlbrew to install a "master" perl
•Use it again to install "virgin" perls of every
major version
•Use the master pe...
•Use perlbrew to install a "master" perl
•Use it again to install "virgin" perls of every
major version
•Use the master pe...
•Use perlbrew to install a "master" perl
•Use it again to install "virgin" perls of every
major version
•Use the master pe...
•Use perlbrew to install a "master" perl
•Use it again to install "virgin" perls of every
major version
•Use the master pe...
•Use perlbrew to install a "master" perl
•Use it again to install "virgin" perls of every
major version
•Use the master pe...
What does that look like?
The "fetcher" grabs the latest distribution URLs from
MetaCPAN's RSS feed.
my $xml = XML::LibXML...
What does that look like?
The "fetcher" grabs the latest distribution URLs from
MetaCPAN's RSS feed.
my $xml = XML::LibXML...
What does that look like?
The "fetcher" grabs the latest distribution URLs from
MetaCPAN's RSS feed.
my $xml = XML::LibXML...
What does that look like?
The "fetcher" grabs the latest distribution URLs from
MetaCPAN's RSS feed.
my $xml = XML::LibXML...
What does that look like?
We retrieve the distribution metadata from
the MetaCPAN JSON API and save it to
MongoDB.
foreach...
What does that look like?
We retrieve the distribution metadata from
the MetaCPAN JSON API and save it to
MongoDB.
foreach...
What does that look like?
We retrieve the distribution metadata from
the MetaCPAN JSON API and save it to
MongoDB.
foreach...
What does that look like?
We retrieve the distribution metadata from
the MetaCPAN JSON API and save it to
MongoDB.
foreach...
What does that look like?
We retrieve the distribution metadata from
the MetaCPAN JSON API and save it to
MongoDB.
foreach...
What does that look like?
We extract the archive in a specific "work" directory
for each perl.
Then use a temp directory fo...
What does that look like?
We extract the archive in a specific "work" directory
for each perl.
Then use a temp directory fo...
What does that look like?
We extract the archive in a specific "work" directory
for each perl.
Then use a temp directory fo...
What does that look like?
We extract the archive in a specific "work" directory
for each perl.
Then use a temp directory fo...
What does that look like?
Use a specific perl binary to run cpanm and
install dependencies, with no tests, to the temp
dire...
What does that look like?
Use a specific perl binary to run cpanm and
install dependencies, with no tests, to the temp
dire...
What does that look like?
Use a specific perl binary to run cpanm and
install dependencies, with no tests, to the temp
dire...
What does that look like?
Use a specific perl binary to run cpanm and
install dependencies, with no tests, to the temp
dire...
What does that look like?
Use a specific perl binary to run cpanm and
install dependencies, with no tests, to the temp
dire...
What does that look like?
Use a specific perl binary to run cpanm and
install dependencies, with no tests, to the temp
dire...
"deps" : {
	 	 "log" : [
	 	 	 {
	 	 	 	 "indent" : 0,
	 	 	 	 "type" : "working-on",
	 	 	 	 "line" : "--> Working on .n"...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Use a specific perl to run each test file,
save the TAP output and any errors, and
use the exit st...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
Parse the TAP output of each test into a
structure which can be saved in MongoDB
eval {
my $pars...
What does that look like?
What does that look like?
	

 {
	

 	

 "number" : 6,
	

 	

 "ok" : true,
	

 	

 "type" : "test",
	

 	

 "desc" : "dept...
What does that look like?
	

 {
	

 	

 "number" : 6,
	

 	

 "ok" : true,
	

 	

 "type" : "test",
	

 	

 "desc" : "dept...
What does that look like?
	

 {
	

 	

 "number" : 6,
	

 	

 "ok" : true,
	

 	

 "type" : "test",
	

 	

 "desc" : "dept...
What does that look like?
	

 {
	

 	

 "number" : 6,
	

 	

 "ok" : true,
	

 	

 "type" : "test",
	

 	

 "desc" : "dept...
What does that mean?
Beautiful Tables!
Final
Thoughts
Final
Thoughts
•Play with new toys.
Final
Thoughts
•Play with new toys.
•Think before you code.
Final
Thoughts
•Play with new toys.
•Think before you code.
•Throw stuff away.
Final
Thoughts
•Play with new toys.
•Think before you code.
•Throw stuff away.
•Have bad ideas.
Final
Thoughts
•Play with new toys.
•Think before you code.
•Throw stuff away.
•Have bad ideas.
•Ask stupid questions.
Final
Thoughts
•Play with new toys.
•Think before you code.
•Throw stuff away.
•Have bad ideas.
•Ask stupid questions.
•Ha...
!
Stupid Questions?
Mike Friedman
(friedo)
friedo@mongodb.com
https://github.com/friedo/cpanci
!
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
21st Century CPAN Testing: CPANci
Upcoming SlideShare
Loading in...5
×

21st Century CPAN Testing: CPANci

535

Published on

Presented at the 2013 Pittsburgh Perl Workshop, this talk discusses the history and technology behind Mike Friedman's CPANci project.

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
535
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "21st Century CPAN Testing: CPANci"

  1. 1. 21st Century CPAN Testing Mike Friedman (friedo) MongoDB, Inc.
  2. 2. Aug. 27, 2013:
  3. 3. , Inc.
  4. 4. , Inc. makes (the database)
  5. 5. , Inc. makes (the database) employs Mike Friedman (friedo)
  6. 6. , Inc. makes (the database) employs Mike Friedman (friedo)
  7. 7. WAT
  8. 8. What is MongoDB?
  9. 9. Open Source Non-relational Horizontally Scalable Document- oriented Fast Database Fault-tolerant CoolSchemaless
  10. 10. Document- oriented
  11. 11. Document- oriented
  12. 12. JSON-like thingy: { "foo": "a string", "bar": 42, "baz": [ 1, 2, "narf", "poit" ], "quux": { "key_1": "w00t.", "key_2": "you get the idea", } }
  13. 13. { "foo": "a string", "bar": 42, "baz": [ 1, 2, "narf", "poit" ], "quux": { "key_1": "w00t.", "key_2": "you get the idea", } }
  14. 14. { "foo": "a string", "bar": 42, "baz": [ 1, 2, "narf", "poit" ], "quux": { "key_1": "w00t.", "key_2": "you get the idea", } }
  15. 15. { "foo": "a string", "bar": 42, "baz": [ 1, 2, "narf", "poit" ], "quux": { "key_1": "w00t.", "key_2": "you get the idea", } } First-Class Objects
  16. 16. First-Class Objects
  17. 17. First-Class Objects
  18. 18. First-Class Objects Queryable
  19. 19. First-Class Objects Queryable Indexable
  20. 20. First-Class Objects Queryable Indexable Updateable
  21. 21. Testing CPAN in the 21st Century
  22. 22. A lengthy series of bad ideas and stupid questions.
  23. 23. Stupid Question No. 1
  24. 24. Stupid Question No. 1 Who here uses CPAN?
  25. 25. Stupid Question No. 2
  26. 26. Stupid Question No. 2 Who here is a CPAN author?
  27. 27. What’s this about, anyway?
  28. 28. What’s this about, anyway? CPANci
  29. 29. A Brief History
  30. 30. A Brief History •December 18, 1987
  31. 31. A Brief History •December 18, 1987 •Perl 1.000 released.
  32. 32. A Brief History •December 18, 1987 •Perl 1.000 released. •TAP invented.
  33. 33. The Test Anything Protocol
  34. 34. The Test Anything Protocol 1..42 ok 1 the thing looks good! ok 2 ok 3 $beer isa $drink not ok 4 too much $beer not ok 5 $me->vomit( 'now' ) ...
  35. 35. A Brief History
  36. 36. A Brief History •October 17, 1994
  37. 37. A Brief History •October 17, 1994 •Perl 5.000 released.
  38. 38. A Brief History •October 17, 1994 •Perl 5.000 released. •Perl has a module system.
  39. 39. # from this require "funcs.pl";
  40. 40. # from this require "funcs.pl"; # to this use My::Module;
  41. 41. # but under the hood BEGIN { require My::Module; My::Module->import; };
  42. 42. A Brief History
  43. 43. A Brief History •October 26, 1995
  44. 44. A Brief History •October 26, 1995 •CPAN established.
  45. 45. A Brief History •October 26, 1995 •CPAN established. •Perl modules are available.
  46. 46. A Brief History
  47. 47. A Brief History •May 15, 1997
  48. 48. A Brief History •May 15, 1997 •Perl 5.004 released.
  49. 49. A Brief History •May 15, 1997 •Perl 5.004 released. •CPAN.pm is in the core.
  50. 50. # the dark art $ perl -MCPAN -e 'install Foo'
  51. 51. A Brief History
  52. 52. A Brief History •May, 1998
  53. 53. A Brief History •May, 1998 •CPAN Testers conceived
  54. 54. A Brief History •May, 1998 •CPAN Testers conceived •Automated feedback for authors
  55. 55. A Brief History
  56. 56. A Brief History •November 15, 2003
  57. 57. A Brief History •November 15, 2003 •Perl 5.6.2 released.
  58. 58. A Brief History •November 15, 2003 •Perl 5.6.2 released. •Test::More is in the core.
  59. 59. use Test::More tests => 3; ok( 42 ); is( $foo, 'my value' ); isnt( 'foo', 'bar' );
  60. 60. A Brief History
  61. 61. A Brief History •August 6, 2012
  62. 62. A Brief History •August 6, 2012 •Mike goes to work for 10gen
  63. 63. A Brief History •August 6, 2012 •Mike goes to work for 10gen MongoDB
  64. 64. Bad Idea No. 1
  65. 65. Bad Idea No. 1 Come up with a cool Perl MongoDB project to show off atYAPC!
  66. 66. Bad Idea No. 1 Come up with a cool Perl MongoDB project to show off atYAPC! It'll be fun!
  67. 67. Bad Idea No. 1 Come up with a cool Perl MongoDB project to show off atYAPC! It'll be fun! promise!
  68. 68. CPAN Testers
  69. 69. CPAN Testers is Wonderful and Amazing
  70. 70. Disadvantages:
  71. 71. Disadvantages: Not real time
  72. 72. Disadvantages: Not real time Not consistent
  73. 73. Disadvantages: Not real time Not consistent Polluted / Inconsistent environments
  74. 74. Disadvantages: Not real time Not consistent Polluted / Inconsistent environments Not all versions on all platforms
  75. 75. Perl 5.18 runs on GNU Hurd
  76. 76. Perl 5.18 runs on GNU Hurd ?
  77. 77. Perl 5.18 runs on GNU Hurd ?
  78. 78. Perl 5.18 runs on GNU Hurd ? I ♥ CPAN
  79. 79. I don't care.
  80. 80. I care about:
  81. 81. I care about:
  82. 82. I care about:
  83. 83. I care about:
  84. 84. I want Continuous Integration for the entire CPAN.
  85. 85. I want Continuous Integration for the entire CPAN. For platforms I care about.
  86. 86. Bad Idea No. 2
  87. 87. CPANci.org Bad Idea No. 2
  88. 88. Stupid Question No. 3
  89. 89. Stupid Question No. 3 How can we test CPAN without the disadvantages of CPAN Testers?
  90. 90. Postulate: Every CPAN distribution must be tested in isolation, on a virgin Perl installation untouched by human hands.
  91. 91. I release distribution Foo-Awesome-0.0000001
  92. 92. I release distribution Foo-Awesome-0.0000001 use Spiffy::Module;
  93. 93. I release distribution Foo-Awesome-0.0000001 use Spiffy::Module; Forgotten dependency!
  94. 94. I release distribution Foo-Awesome-0.0000001 use Spiffy::Module; Forgotten dependency! What happens?
  95. 95. Scenario  I The  tester  hath  not  the  missing   dependency  upon  his  box.
  96. 96. FAIL- mail
  97. 97. Scenario  II The  tester  doth  possess   the  dependency  upon  his  box.
  98. 98. EVERYTHING IS FINE.
  99. 99. NOT FINE.
  100. 100. Postulate: Every CPAN distribution must be tested in isolation, on a virgin Perl installation untouched by human hands.
  101. 101. Postulate: Every CPAN distribution must be tested in isolation, on a virgin Perl installation untouched by human hands. So how do we do that?
  102. 102. perlbrew
  103. 103. Virtualization
  104. 104. Virtualization Whoa!
  105. 105. Bad Idea No. 3
  106. 106. •Create an EC2 image
  107. 107. •Create an EC2 image •Put perlbrew on it
  108. 108. •Create an EC2 image •Put perlbrew on it •Install every Perl locally
  109. 109. •Create an EC2 image •Put perlbrew on it •Install every Perl locally •Boot an instance for every uploaded distribution
  110. 110. •Create an EC2 image •Put perlbrew on it •Install every Perl locally •Boot an instance for every uploaded distribution •Install needed dependencies for the distribution
  111. 111. •Create an EC2 image •Put perlbrew on it •Install every Perl locally •Boot an instance for every uploaded distribution •Install needed dependencies for the distribution •Run the tests and report the results
  112. 112. •Create an EC2 image •Put perlbrew on it •Install every Perl locally •Boot an instance for every uploaded distribution •Install needed dependencies for the distribution •Run the tests and report the results •Shut down the instance
  113. 113. Uh-oh.
  114. 114. Stupid Question No. 4
  115. 115. Stupid Question No. 4 How can we do this on one instance?
  116. 116. local::lib cpanminus
  117. 117. App::cpanminus
  118. 118. App::cpanminus •Self-contained
  119. 119. App::cpanminus •Self-contained •That is, the cpanm script is self-contained
  120. 120. App::cpanminus •Self-contained •That is, the cpanm script is self-contained •via App::FatPacker
  121. 121. App::cpanminus
  122. 122. App::cpanminus That means the same cpanm can be run by any perl
  123. 123. App::cpanminus That means the same cpanm can be run by any perl perlbrew switch master curl -L http://cpanmin.us/ | perl - App::cpanminus ln -s ~/perl5/perlbrew/perls/master/bin/cpanm ./cpanm ~/perl5/perlbrew/perls/perl-5.8.9/bin/perl cpanm ~/perl5/perlbrew/perls/perl-5.10.1/bin/perl cpanm ~/perl5/perlbrew/perls/perl-5.12.5/bin/perl cpanm ~/perl5/perlbrew/perls/perl-5.14.4/bin/perl cpanm ~/perl5/perlbrew/perls/perl-5.16.3/bin/perl cpanm ~/perl5/perlbrew/perls/perl-5.18.0/bin/perl cpanm ~/perl5/perlbrew/perls/perl-5.19.0/bin/perl cpanm
  124. 124. •Use perlbrew to install a "master" perl
  125. 125. •Use perlbrew to install a "master" perl •Use it again to install "virgin" perls of every major version
  126. 126. •Use perlbrew to install a "master" perl •Use it again to install "virgin" perls of every major version •Use the master perl to install everything from CPAN that makes CPANci work
  127. 127. •Use perlbrew to install a "master" perl •Use it again to install "virgin" perls of every major version •Use the master perl to install everything from CPAN that makes CPANci work •For each distribution, create a temp directory
  128. 128. •Use perlbrew to install a "master" perl •Use it again to install "virgin" perls of every major version •Use the master perl to install everything from CPAN that makes CPANci work •For each distribution, create a temp directory •Tell cpanminus to install dependencies there, as if for local::lib
  129. 129. •Use perlbrew to install a "master" perl •Use it again to install "virgin" perls of every major version •Use the master perl to install everything from CPAN that makes CPANci work •For each distribution, create a temp directory •Tell cpanminus to install dependencies there, as if for local::lib •Run tests and report results
  130. 130. •Use perlbrew to install a "master" perl •Use it again to install "virgin" perls of every major version •Use the master perl to install everything from CPAN that makes CPANci work •For each distribution, create a temp directory •Tell cpanminus to install dependencies there, as if for local::lib •Run tests and report results •Delete temp directory, leaving each perl untouched!
  131. 131. What does that look like? The "fetcher" grabs the latest distribution URLs from MetaCPAN's RSS feed. my $xml = XML::LibXML->load_xml( string => $self->ua->get( $self->rss_base )->decoded_content ); my @dists = map { my $u = URI->new( $_->getAttribute( 'rdf:resource' ) ); $u->host( 'api.metacpan.org' ); $u->path( 'v0' . $u->path ); $u; } $xml->findnodes( '//rdf:li' );
  132. 132. What does that look like? The "fetcher" grabs the latest distribution URLs from MetaCPAN's RSS feed. my $xml = XML::LibXML->load_xml( string => $self->ua->get( $self->rss_base )->decoded_content ); my @dists = map { my $u = URI->new( $_->getAttribute( 'rdf:resource' ) ); $u->host( 'api.metacpan.org' ); $u->path( 'v0' . $u->path ); $u; } $xml->findnodes( '//rdf:li' );
  133. 133. What does that look like? The "fetcher" grabs the latest distribution URLs from MetaCPAN's RSS feed. my $xml = XML::LibXML->load_xml( string => $self->ua->get( $self->rss_base )->decoded_content ); my @dists = map { my $u = URI->new( $_->getAttribute( 'rdf:resource' ) ); $u->host( 'api.metacpan.org' ); $u->path( 'v0' . $u->path ); $u; } $xml->findnodes( '//rdf:li' );
  134. 134. What does that look like? The "fetcher" grabs the latest distribution URLs from MetaCPAN's RSS feed. my $xml = XML::LibXML->load_xml( string => $self->ua->get( $self->rss_base )->decoded_content ); my @dists = map { my $u = URI->new( $_->getAttribute( 'rdf:resource' ) ); $u->host( 'api.metacpan.org' ); $u->path( 'v0' . $u->path ); $u; } $xml->findnodes( '//rdf:li' );
  135. 135. What does that look like? We retrieve the distribution metadata from the MetaCPAN JSON API and save it to MongoDB. foreach my $dist( @dists ) { my $fetched_data = decode_json $self->ua->get( $dist )->decoded_content; my %ins = ( _id => 'cpan/' . $name, fetched => DateTime->now, meta => $fetched_data ); $coll->insert( %ins ); }
  136. 136. What does that look like? We retrieve the distribution metadata from the MetaCPAN JSON API and save it to MongoDB. foreach my $dist( @dists ) { my $fetched_data = decode_json $self->ua->get( $dist )->decoded_content; my %ins = ( _id => 'cpan/' . $name, fetched => DateTime->now, meta => $fetched_data ); $coll->insert( %ins ); }
  137. 137. What does that look like? We retrieve the distribution metadata from the MetaCPAN JSON API and save it to MongoDB. foreach my $dist( @dists ) { my $fetched_data = decode_json $self->ua->get( $dist )->decoded_content; my %ins = ( _id => 'cpan/' . $name, fetched => DateTime->now, meta => $fetched_data ); $coll->insert( %ins ); }
  138. 138. What does that look like? We retrieve the distribution metadata from the MetaCPAN JSON API and save it to MongoDB. foreach my $dist( @dists ) { my $fetched_data = decode_json $self->ua->get( $dist )->decoded_content; my %ins = ( _id => 'cpan/' . $name, fetched => DateTime->now, meta => $fetched_data ); $coll->insert( %ins ); }
  139. 139. What does that look like? We retrieve the distribution metadata from the MetaCPAN JSON API and save it to MongoDB. foreach my $dist( @dists ) { my $fetched_data = decode_json $self->ua->get( $dist )->decoded_content; my %ins = ( _id => 'cpan/' . $name, fetched => DateTime->now, meta => $fetched_data ); $coll->insert( %ins ); }
  140. 140. What does that look like? We extract the archive in a specific "work" directory for each perl. Then use a temp directory for building and installing dependencies. foreach my $perl ( @{ $self->perls } ) { my $workdir = catdir $self->home, 'work', $perl, $dist; chdir $workdir; }
  141. 141. What does that look like? We extract the archive in a specific "work" directory for each perl. Then use a temp directory for building and installing dependencies. foreach my $perl ( @{ $self->perls } ) { my $workdir = catdir $self->home, 'work', $perl, $dist; chdir $workdir; }
  142. 142. What does that look like? We extract the archive in a specific "work" directory for each perl. Then use a temp directory for building and installing dependencies. foreach my $perl ( @{ $self->perls } ) { my $workdir = catdir $self->home, 'work', $perl, $dist; chdir $workdir; }
  143. 143. What does that look like? We extract the archive in a specific "work" directory for each perl. Then use a temp directory for building and installing dependencies. foreach my $perl ( @{ $self->perls } ) { my $workdir = catdir $self->home, 'work', $perl, $dist; chdir $workdir; }
  144. 144. What does that look like? Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp directory. Then we parse the cpanm log on stderr my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $pid = open3 $wtr, $rdr, $err, $plbin, $self->cpanm, '--installdeps', '--notest', '-L', $dist_tmp, '.'; my $results = $self->_read_cpanm_deps_log( $err );
  145. 145. What does that look like? Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp directory. Then we parse the cpanm log on stderr my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $pid = open3 $wtr, $rdr, $err, $plbin, $self->cpanm, '--installdeps', '--notest', '-L', $dist_tmp, '.'; my $results = $self->_read_cpanm_deps_log( $err );
  146. 146. What does that look like? Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp directory. Then we parse the cpanm log on stderr my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $pid = open3 $wtr, $rdr, $err, $plbin, $self->cpanm, '--installdeps', '--notest', '-L', $dist_tmp, '.'; my $results = $self->_read_cpanm_deps_log( $err );
  147. 147. What does that look like? Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp directory. Then we parse the cpanm log on stderr my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $pid = open3 $wtr, $rdr, $err, $plbin, $self->cpanm, '--installdeps', '--notest', '-L', $dist_tmp, '.'; my $results = $self->_read_cpanm_deps_log( $err );
  148. 148. What does that look like? Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp directory. Then we parse the cpanm log on stderr my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $pid = open3 $wtr, $rdr, $err, $plbin, $self->cpanm, '--installdeps', '--notest', '-L', $dist_tmp, '.'; my $results = $self->_read_cpanm_deps_log( $err );
  149. 149. What does that look like? Use a specific perl binary to run cpanm and install dependencies, with no tests, to the temp directory. Then we parse the cpanm log on stderr my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $pid = open3 $wtr, $rdr, $err, $plbin, $self->cpanm, '--installdeps', '--notest', '-L', $dist_tmp, '.'; my $results = $self->_read_cpanm_deps_log( $err );
  150. 150. "deps" : { "log" : [ { "indent" : 0, "type" : "working-on", "line" : "--> Working on .n" }, { "line" : "Configuring Lingua-EN-NamedEntity-1.92 ... OKn", "type" : "config", "indent" : 1 }, { "type" : "found-deps", "indent" : 1, "line" : "==> Found dependencies: Lingua::Stem::En, DB_File, LWP::Simplen" }, { "indent" : 1, "type" : "working-on", "line" : "--> Working on Lingua::Stem::Enn" }, { "indent" : 2, "type" : "fetch", "line" : "Fetching http://www.cpan.org/authors/id/S/SN/SNOWHARE/Lingua- Stem-0.84.tar.gz ... OKn" }, What does that look like?
  151. 151. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  152. 152. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  153. 153. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  154. 154. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  155. 155. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  156. 156. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  157. 157. What does that look like? Use a specific perl to run each test file, save the TAP output and any errors, and use the exit status to determine if it passed. foreach my $test( @tests ) { my @test_results; my $plbin = catfile $self->pldir, $perl, 'bin', 'perl'; my $idir = catdir $dist_tmp, 'lib', 'perl5'; my $pid = open3 $wtr, $rdr, $plbin, '-I', $idir, $test; my ( $tap_out, $errors ); { local $/; $tap_out = readline $rdr; $errors = readline $err; } waitpid $pid, 0; my $passed = ( ( $? >> 8 ) == 0 ) true : false ); }
  158. 158. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  159. 159. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  160. 160. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  161. 161. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  162. 162. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  163. 163. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  164. 164. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  165. 165. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  166. 166. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  167. 167. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  168. 168. What does that look like? Parse the TAP output of each test into a structure which can be saved in MongoDB eval { my $parser = TAP::Parser->new( { source => $tap_out } ); while( my $result = $parser->next ) { push @test_results, { text => $result->as_string, ok => ( $result_is_ok ? true : false ), type => $result->type, $result->type eq 'test' ? ( number => $result->number, desc => ( $result->description =~ s/^- //r ) ) : ( ), }; } };
  169. 169. What does that look like?
  170. 170. What does that look like? { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" },
  171. 171. What does that look like? { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" }, •This is JSON
  172. 172. What does that look like? { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" }, •This is JSON •Stored in MongoDB
  173. 173. What does that look like? { "number" : 6, "ok" : true, "type" : "test", "desc" : "depth", "text" : "ok 6 - depth" }, •This is JSON •Stored in MongoDB •But it's also TAP!
  174. 174. What does that mean?
  175. 175. Beautiful Tables!
  176. 176. Final Thoughts
  177. 177. Final Thoughts •Play with new toys.
  178. 178. Final Thoughts •Play with new toys. •Think before you code.
  179. 179. Final Thoughts •Play with new toys. •Think before you code. •Throw stuff away.
  180. 180. Final Thoughts •Play with new toys. •Think before you code. •Throw stuff away. •Have bad ideas.
  181. 181. Final Thoughts •Play with new toys. •Think before you code. •Throw stuff away. •Have bad ideas. •Ask stupid questions.
  182. 182. Final Thoughts •Play with new toys. •Think before you code. •Throw stuff away. •Have bad ideas. •Ask stupid questions. •Have fun.
  183. 183. !
  184. 184. Stupid Questions? Mike Friedman (friedo) friedo@mongodb.com https://github.com/friedo/cpanci !
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×