Making My Own
    CPAN
         brian d foy
Stonehenge Consulting Services
     September 20, 2007
CPAN
• http://www.cpan.org
• Comprehensive Perl Archive Network
• Best, most powerful feature of Perl
• quot;Grokking CPAN...
Goals

• Make my own version of CPAN
 (MyCPAN)

• Freeze MyCPAN
• Make a CPAN for something else
History of CPAN
• Proposed in 1993 as a store for
 everything

• Created in 1995
• PAUSE created to upload
 modules

• Mir...
How CPAN works
• Mirrors content (MIRRORED.FROM)
• Master server in Finland
• Primary ring of mirrors
• Secondary ring of ...
PAUSE
• http://pause.perl.org
• Perl Authors Upload SErver (PAUSE)
• Gateway for adding modules to CPAN
• CPAN Master mirr...
PAUSE modules

• PAUSE puts modules into authors/
• Breaks down by PAUSE ID
 • authors/B/BD/BDFOY/Foo-1.23.tar.gz
• Path d...
PAUSE index files
• PAUSE indexes modules
• Creates index files for CPAN tool
  chain

 • modules/02packages.details.txt.g...
02packages.details.txt


Date::Span   1.123   R/RJ/RJBS/Date-Span-1.123.tar.gz

Date::Tie     0.17   F/FG/FGLOCK/Date-Tie-...
03modlist.data.gz
[
'Fcntl',
'S',
'd',
'c',
'f',
'?',
'Defines fcntl() constants (see File::Lock)',
'JHI',
'4'
],
06perms.txt
WWW::MySpaceBot,KIRSLE,m

WWW::Myspace,GRANTG,m

WWW::Myspace,OALDERS,c

WWW::Myspace::Comment,GRANTG,f

WWW::...
Tool chain
• quot;Third Partyquot; projects
• CPAN Search
• CPAN.pm
• CPANPLUS
• AnnoCPAN
CPAN.pm
• Uses index files to find distro
• Looks for path on mirror
 • http://mirror/...
 • .../authors/
 • .../B/BD/BDFO...
Installing a module

 • Get from first mirror that has it
 • Unpack, run Makefile.PL or Build.PL
 • Discover dependencies
...
MiniCPAN
• A local CPAN mirror
• Only has latest versions, indices
• Point CPAN.pm at local mirror
• Update from network
•...
CPAN::Mini

• The module behind minicpan
• by Ricardo SIGNES
• Configurable
• Programmable
Configure MiniCPAN

 • Edit ~/.minicpanrc
  local:     /MINICPAN

  remote:    http://path/to_mirror

  skip_perl: 1
Updating MiniCPAN

$ minicpan
authors/01mailrc.txt.gz ... updated
modules/02packages.details.txt.gz ... updated
modules/03...
CPAN::Mini filters


• Don’t mirror files you don’t need
• Skip files you want to override
MicroCPAN
  CPAN::Mini->update_mirror(

...,

path_filters     => [


sub { !( $_[0] =~ /BDFOY/ ) }


],

module_filters  ...
Configuring CPAN.pm

  • Edit CPAN/MyConfig.pm directly
   • probably not in library path
   • location set on first use
 ...
Add MiniCPAN
$ cpan

cpan shell -- CPAN exploration and modules
installation (v1.9102)

cpan[2]> o conf urllist unshift fi...
MyCPAN

• Only the modules I want
• Only the versions I want
• Extra, private modules
• Public modules with local patches
...
Private modules

• Local modules for internal use
• Want to install with same tools
• Want to replicate
• Want to distribu...
CPAN::Mini::Inject

• Creates a parallel archive
• Update MiniCPAN first
• Inject parallel archive into
  MiniCPAN

• Upda...
Command line

$ mcpani --add --module Foo::Bar 

 --authorid BDFOY --modversion 0.01 

 --file ./Foo-Bar-0.01.tar.gz
~/.mcpani/config

   local: /MyCPAN
 remote: ftp:///MINICPAN
 repository: /AddThesePrivateModules
 passive: yes
 dirmode: ...
Replacing public distros


• Sometimes I want local patches
 • absent author
 • truculent author
• Don’t want the latest d...
Three steps

• Inject distros into parallel dir
• Update MiniCPAN
• Inject distros into MiniCPAN
BackPAN

• http://backpan.cpan.org
• All modules ever uploaded to PAUSE
• Not integrated with CPAN tool chain
• Which dist...
Finding old distros


• It's all on BackPAN
• Brute force, right now
• Unpack a distro and check
• Repeat until you find it
MyCPAN on a CD
• I have all the bits to carry around
  MyCPAN

 • MiniCPAN
 • Private modules
 • Configure CPAN.pm
• Need ...
Dynamic config
 use CPAN;

 CPAN::Config->load(

 
   be_silent     => 1,

 
   write_report => 0,

 
   );

 CPAN::Shell-...
Future work
• Index all of BackPAN
• Make that searchable
• Easier injecting
• Integrate with CPAN tools
• Perhaps another...
MyCPAN ( LA.pm, September 2007 )
Upcoming SlideShare
Loading in...5
×

MyCPAN ( LA.pm, September 2007 )

4,365

Published on

How CPAN works, how to create a minicpan and microcpan, and future plans for supercpan

Published in: Technology, Art & Photos
1 Comment
1 Like
Statistics
Notes
  • @_brian_d_foy please mention how to create local CPAN (metacpan.org GUI prefered ) repo for only selected module NOT the complete CPAN repo.)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,365
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "MyCPAN ( LA.pm, September 2007 )"

  1. 1. Making My Own CPAN brian d foy Stonehenge Consulting Services September 20, 2007
  2. 2. CPAN • http://www.cpan.org • Comprehensive Perl Archive Network • Best, most powerful feature of Perl • quot;Grokking CPANquot; by Elaine Ashton http://backpan.perl.org/authors/id/H/ HF/HFB/grok-cpan-1.01.pdf
  3. 3. Goals • Make my own version of CPAN (MyCPAN) • Freeze MyCPAN • Make a CPAN for something else
  4. 4. History of CPAN • Proposed in 1993 as a store for everything • Created in 1995 • PAUSE created to upload modules • Mirrors added (ongoing)
  5. 5. How CPAN works • Mirrors content (MIRRORED.FROM) • Master server in Finland • Primary ring of mirrors • Secondary ring of mirrors • Tools work with basic structure
  6. 6. PAUSE • http://pause.perl.org • Perl Authors Upload SErver (PAUSE) • Gateway for adding modules to CPAN • CPAN Master mirror PAUSE • modules/ • authors/
  7. 7. PAUSE modules • PAUSE puts modules into authors/ • Breaks down by PAUSE ID • authors/B/BD/BDFOY/Foo-1.23.tar.gz • Path doesn’t really matter
  8. 8. PAUSE index files • PAUSE indexes modules • Creates index files for CPAN tool chain • modules/02packages.details.txt.gz • modules/03modlist.data.gz • modules/06perms.txt.gz
  9. 9. 02packages.details.txt Date::Span 1.123 R/RJ/RJBS/Date-Span-1.123.tar.gz Date::Tie 0.17 F/FG/FGLOCK/Date-Tie-0.17.tar.gz Date::Tiny 0.02 A/AD/ADAMK/Date-Tiny-0.02.tar.gz
  10. 10. 03modlist.data.gz [ 'Fcntl', 'S', 'd', 'c', 'f', '?', 'Defines fcntl() constants (see File::Lock)', 'JHI', '4' ],
  11. 11. 06perms.txt WWW::MySpaceBot,KIRSLE,m WWW::Myspace,GRANTG,m WWW::Myspace,OALDERS,c WWW::Myspace::Comment,GRANTG,f WWW::Myspace::Data,GRANTG,f WWW::Myspace::Data,OALDERS,c WWW::Myspace::FriendAdder,GRANTG,f
  12. 12. Tool chain • quot;Third Partyquot; projects • CPAN Search • CPAN.pm • CPANPLUS • AnnoCPAN
  13. 13. CPAN.pm • Uses index files to find distro • Looks for path on mirror • http://mirror/... • .../authors/ • .../B/BD/BDFOY/Foo-1.23.tar.gz
  14. 14. Installing a module • Get from first mirror that has it • Unpack, run Makefile.PL or Build.PL • Discover dependencies • Find, get, install dependencies
  15. 15. MiniCPAN • A local CPAN mirror • Only has latest versions, indices • Point CPAN.pm at local mirror • Update from network • Install without network
  16. 16. CPAN::Mini • The module behind minicpan • by Ricardo SIGNES • Configurable • Programmable
  17. 17. Configure MiniCPAN • Edit ~/.minicpanrc local: /MINICPAN remote: http://path/to_mirror skip_perl: 1
  18. 18. Updating MiniCPAN $ minicpan authors/01mailrc.txt.gz ... updated modules/02packages.details.txt.gz ... updated modules/03modlist.data.gz ... updated authors/id/J/JW/JWACH/Apache- FastForward-1.1.tar.gz ... updated
  19. 19. CPAN::Mini filters • Don’t mirror files you don’t need • Skip files you want to override
  20. 20. MicroCPAN CPAN::Mini->update_mirror( ..., path_filters => [ sub { !( $_[0] =~ /BDFOY/ ) } ], module_filters => [ qr/Test/, ], );
  21. 21. Configuring CPAN.pm • Edit CPAN/MyConfig.pm directly • probably not in library path • location set on first use • Start CPAN.pm shell
  22. 22. Add MiniCPAN $ cpan cpan shell -- CPAN exploration and modules installation (v1.9102) cpan[2]> o conf urllist unshift file:///MINICPAN cpan[3]> o conf commit commit: wrote '/Users/brian/Library/Application Support/.cpan/CPAN/MyConfig.pm'
  23. 23. MyCPAN • Only the modules I want • Only the versions I want • Extra, private modules • Public modules with local patches • Internal server or removable media
  24. 24. Private modules • Local modules for internal use • Want to install with same tools • Want to replicate • Want to distribute internally
  25. 25. CPAN::Mini::Inject • Creates a parallel archive • Update MiniCPAN first • Inject parallel archive into MiniCPAN • Update PAUSE index files locally
  26. 26. Command line $ mcpani --add --module Foo::Bar --authorid BDFOY --modversion 0.01 --file ./Foo-Bar-0.01.tar.gz
  27. 27. ~/.mcpani/config local: /MyCPAN remote: ftp:///MINICPAN repository: /AddThesePrivateModules passive: yes dirmode: 0755
  28. 28. Replacing public distros • Sometimes I want local patches • absent author • truculent author • Don’t want the latest distro
  29. 29. Three steps • Inject distros into parallel dir • Update MiniCPAN • Inject distros into MiniCPAN
  30. 30. BackPAN • http://backpan.cpan.org • All modules ever uploaded to PAUSE • Not integrated with CPAN tool chain • Which distro has module version?
  31. 31. Finding old distros • It's all on BackPAN • Brute force, right now • Unpack a distro and check • Repeat until you find it
  32. 32. MyCPAN on a CD • I have all the bits to carry around MyCPAN • MiniCPAN • Private modules • Configure CPAN.pm • Need to discover MyCPAN location
  33. 33. Dynamic config use CPAN; CPAN::Config->load( be_silent => 1, write_report => 0, ); CPAN::Shell->o( qw( conf urllist unshift ), $where_i_found_cpan, );
  34. 34. Future work • Index all of BackPAN • Make that searchable • Easier injecting • Integrate with CPAN tools • Perhaps another CPAN Search

×