• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Reviewing CPAN modules

Reviewing CPAN modules






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Reviewing CPAN modules Reviewing CPAN modules Presentation Transcript

    • Reviewing CPAN modules Neil Bowers 1
    • • Apparently there are 1,900+ Torch/Flashlight apps… 2
    • 3
    • What is CPAN?• Something like a university library? • Lots of useful items, nicely catalogued, easily found Atoolchest full of precision instruments that good programmers can wield to make great things - chromatic• I think more like Rube Goldbergs workshop: • A bunch of useful things, some half-finished ideas, practical jokes, stuff that no longer works, items that shouldnt be there, forgotten things in the corner Holy crap, I forgot about [that module that I wrote]. - Andy Lester 4
    • Review #1Generating passwordsModules that can be used to automatically generate passwords,either random sequences of characters or pronounceable pseudo words 5
    • Modules for generating passwords 6
    • Generating random char strings • 100,000 passwords 7
    • Structure of reviews 8
    • After doing the review• Im now using Crypt::YAPassGen • v0.02 released 2004-05-30• App::Genpass is good • Ive submitted changes to add features I liked in other modules• Ive ended up using Data::Random elsewhere • Because I was aware of it having done this review• Hoping to finally release fix for Crypt::RandPasswd • I was granted co-maint yesterday (23rd Nov 2012) 9
    • Review #2Looking up location of an IP addressModules that can map an IP address to ISO country code,and possibly return additional info like language, county/state, etc 10
    • Modules for getting IP location co-maint to release fixes and doc updates 11
    • Notes• A number of these are commercial • Pay for full data set, or to get unthrottled access to backend service• Performance & coverage tests took the most time • In particular its hard to test for correctness• A number of good options • Competition keeps them on their toes? • Which one you use might depend on what other info you want for an IP• Ended up not having the need, so not using any • But have used them occasionally, now I know about them 12
    • Review #3Spelling out a number in EnglishModules for converting 327 to "three hundred and twenty-seven",-3.25 to "minus three point two five", etc 13
    • Modules for spelling out numbers 14
    • Notes on review #3• Fewer modules meant more time to play • I had fun writing a script to automatically compare results between modules for 1 .. N• First time I hit a module that just doesnt work• Lingua::EN::Numbers • The first module where I remember looking at the code and thinking "hey, nice" • I took over maintenance from Sean Burke to release fixes • And have a todo-list, adding ideas that came up during the review 15
    • My rules of engagement• Treat authors and their code with respect • This might be someones first tentative steps at code sharing• Report all bugs I find • If I can see how to fix them, then submit a fix• Fix and improve documentation • Especially if I had to look at the code to work out how to use it• If appropriate, try and get co-maint to release fixes • Because CPAN is forever • This can be to just mark a module as deprecated & update SEE ALSO• If there are four or more modules for something • It goes onto my backlog of potential reviews 16
    • Observations based on the rules• What encourages me to submit fixes? • Github • Makefile.PL (or more accurately: not dist.ini)• Getting co-maint is relatively costly • People disagreed when I said this at LPW 2011 ("Youre doing it wrong") • I still think this (getting co-maint to fix doc is relatively costly)• Im toying with writing a tool to manage this • I might submit bugs or send email on 10 different modules in a review 17
    • Review #4Parsing User-Agent stringsModules for extracting and inferring information from HTTP User-Agentstrings.Eg to tell how many of your sites users are using IE vs Chrome vs Firefox 18
    • Modules for parsing User-Agent 19
    • Review structure: module section 20
    • Comparison Coverage (test corpus)Speed Review-driven development! 21
    • Curation• Requirements for these modules • Continuing good coverage of (versions of) browsers, bots, libraries, etc • Good enough performance (processing large log files) • Identifying other things (is it a mobile? whats the screen size?)• Working with Olaf Alders on "complete solution" • Currently at the R&D stage  22
    • Review #5Defining constantsModules for defining constants and immutable variables 23
    • Modules for defining constants 24
    • Some firsts with this review• I was prompted to look at the Perl source • To make sure I understood SvREADONLY• Emailed an author for help understanding his code• Enough modules to prompt me to map them out 25
    • Observations• I had been using Readonly • But I clearly hadnt read all the documentation before• Now I use Const::Fast • Latency: all those modules using Readonly …• Surprising variety of designs / approaches • I learned a lot looking at the code for all these modules• But quite a few experiments • "I wonder if you can …" 26
    • Review #6Getting module dependency infoFinding out what modules (and dists) are used by a module or modules 27
    • Tried a new approach• Previously I only published after a lot of work • But I still missed plenty of modules• Decided to experiment with a more lean approach • Found 6 modules, none did what I wanted, so wrote my own • Wrote a quick sketch review and published it • More modules came out of the woodwork every time I updated• I was looking at dependencies for my next review • But ended up pushing this one out sooner 28
    • Modules for getting dependencies 29
    • Module Author• I had a "bug" reported – author is listed as:• But META.json says:• End result was PAUSE::Permissions 30
    • Notes• The 23rd module nearly matched mine!• Very much still in progress• Curation needed: • There are 5 or so distinct module types (out of 26 so far) • But lots of different variations on each theme 31
    • Why so many modules?• Whipupitude• Low barrier to entry for CPAN• Hard to find (all) appropriate modules• Easier to upload than contribute or take over• Playing with Perl is fun (eg constants, dependencies) 32
    • Review #7Making HTTP requestsModules for making HTTP requests (GET, POST, PUT, DELETE, etc)Excluding higher-level modules like WWW::Mechanize 33
    • Modules for making HTTP requests The only module Ive given up on 34
    • Performance on GET requests 35
    • Runtime dependencies Require a C library Requires a C library and Moose! 36
    • What module to use? 37
    • Review #8LUHN checkModules for checking credit card numbers 38
    • Modules for checking CC numbers 39
    • Performance of LUHN modules Moose? 40
    • Review #9Getting a modules pathModules for finding where a module has been installed locally,preferably without having to load the module 41
    • Modules for getting a modules path 42
    • ComparisonNearly three orders of magnitude 43
    • Feature matrix 44
    • Features vs Dependencies 45
    • Appropriate # of dependencies 46
    • ObservationsAnd some mild venting 47
    • What do people use CPAN for?• Sharing code with others!• Configuration management• A cloud-based store for configuration (Task::BeLike::*)• A transport mechanism• A todo list (egBusiness::BankCard)• A blog / dumping ground for ideas• None of the above (ACME::*) 48
    • Version numbers• Version numbers should be designed for humans • PAUSE just wants them to be monotonically increasing• All modules in a dist should have 0.01 • A version number 0.004 • The same version number 1.121660 0.0.20• Dont be afraid of 1.x 0.6.1 5 • 73% of dists on CPAN are version 0.* 0.000 0.001005• our $VERSION = 0.01; v0.0.3 2.4 6632 2011121001 49
    • How can you tell a good module?• Version number • But see Net::HTTP::Tiny v0.01• Recently updated • Some modules are finished• Volume and/or quality of documentation • See Crypt::RandPasswd• Number of bugs • Some big hitters have a long list though…• Number of dependent modules • from different authors• Ratings / +1s• Perl::Critic• Author• All of the above, and more 51
    • How have I changed as a result?• Ive learned new things, & new ways to do things, & …• I do developer releases & pay a lot more attention to CPAN Testers• Im more paranoid• I benchmark and profile• I use IRC, and have had only positive experiences• How I export functions (thank-you Zefram)• How I think about dependencies• Moose / Mouse / Moo• Im a lot more circumspect about releasing code to CPAN• Beginners mind 52
    • My review process• Quickly skim and find initial set of modules• Start writing sections for each module • Write generic SYNOPSIS style example, and evolve it as I go along• Report bugs and documentation issues as I go• First sketch at benchmarking & comparing• More thorough search for modules to include• Email the authors, give them a chance to comment• Publish first version • Iterate as people point out modules that Ive missed 54
    • Its easy – why not give it a go? % mkdir json % cd json % cat > module-list.txt JSON JSON::DWIW JSON::JOM JSON::Parse JSON::Syck JSON::Tiny JSON::Util JSON::XS JSON::YAJL JSON::Streaming::Reader JSON::Streaming::Writer Mojo::JSON Pegex::JSON Test::JSON ^D % cpan-review 55
    • What do I want you to do?• Stop and search CPAN before writing code • Ok, I accept its hard to find (all of the) relevant modules for a task• Just because you wrote it, you dont have to upload it • Only release if its needed for another module, or another user • Put it on github • Write an article about it on blogs.perl.org• Add links to similar modules in SEE ALSO • MetaCPAN will include those in search results • Its helpful for users • Youll make my life easier• Write a helpful ABSTRACT in your NAME section • "a perl interface to foolib" • "the hardest working two letters in Perl" 56
    • Review-related todo list• Related to existing reviews • Fix Crypt::RandPasswd • Configurability for Lingua::EN::Numbers • User-Agent module with Olaf • New constant module: Exporter::Constants • Dependencies: +5 modules to review, my module, curation • HTTP requests: Mojo::UserAgent, async request modules • Changes to Business::CCCheck• New reviews in progress • Hash objects • JSON • Module loading 57
    • Finding modules is hard• Thinking about trying automatic clustering• Help identify groups of modules for review• Possible contribution to MetaCPAN• Automatic tagging of modules • Use keywords field in metadata 58