Reviewing CPAN modules


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Reviewing CPAN modules

  1. 1. Reviewing CPAN modules Neil Bowers 1
  2. 2. • Apparently there are 1,900+ Torch/Flashlight apps… 2
  3. 3. 3
  4. 4. What is CPAN?• Something like a university library? • Lots of useful items, nicely catalogued, easily found Atoolchest full of precision instruments that good programmers can wield to make great things - chromatic• I think more like Rube Goldbergs workshop: • A bunch of useful things, some half-finished ideas, practical jokes, stuff that no longer works, items that shouldnt be there, forgotten things in the corner Holy crap, I forgot about [that module that I wrote]. - Andy Lester 4
  5. 5. Review #1Generating passwordsModules that can be used to automatically generate passwords,either random sequences of characters or pronounceable pseudo words 5
  6. 6. Modules for generating passwords 6
  7. 7. Generating random char strings • 100,000 passwords 7
  8. 8. Structure of reviews 8
  9. 9. After doing the review• Im now using Crypt::YAPassGen • v0.02 released 2004-05-30• App::Genpass is good • Ive submitted changes to add features I liked in other modules• Ive ended up using Data::Random elsewhere • Because I was aware of it having done this review• Hoping to finally release fix for Crypt::RandPasswd • I was granted co-maint yesterday (23rd Nov 2012) 9
  10. 10. Review #2Looking up location of an IP addressModules that can map an IP address to ISO country code,and possibly return additional info like language, county/state, etc 10
  11. 11. Modules for getting IP location co-maint to release fixes and doc updates 11
  12. 12. Notes• A number of these are commercial • Pay for full data set, or to get unthrottled access to backend service• Performance & coverage tests took the most time • In particular its hard to test for correctness• A number of good options • Competition keeps them on their toes? • Which one you use might depend on what other info you want for an IP• Ended up not having the need, so not using any • But have used them occasionally, now I know about them 12
  13. 13. Review #3Spelling out a number in EnglishModules for converting 327 to "three hundred and twenty-seven",-3.25 to "minus three point two five", etc 13
  14. 14. Modules for spelling out numbers 14
  15. 15. Notes on review #3• Fewer modules meant more time to play • I had fun writing a script to automatically compare results between modules for 1 .. N• First time I hit a module that just doesnt work• Lingua::EN::Numbers • The first module where I remember looking at the code and thinking "hey, nice" • I took over maintenance from Sean Burke to release fixes • And have a todo-list, adding ideas that came up during the review 15
  16. 16. My rules of engagement• Treat authors and their code with respect • This might be someones first tentative steps at code sharing• Report all bugs I find • If I can see how to fix them, then submit a fix• Fix and improve documentation • Especially if I had to look at the code to work out how to use it• If appropriate, try and get co-maint to release fixes • Because CPAN is forever • This can be to just mark a module as deprecated & update SEE ALSO• If there are four or more modules for something • It goes onto my backlog of potential reviews 16
  17. 17. Observations based on the rules• What encourages me to submit fixes? • Github • Makefile.PL (or more accurately: not dist.ini)• Getting co-maint is relatively costly • People disagreed when I said this at LPW 2011 ("Youre doing it wrong") • I still think this (getting co-maint to fix doc is relatively costly)• Im toying with writing a tool to manage this • I might submit bugs or send email on 10 different modules in a review 17
  18. 18. Review #4Parsing User-Agent stringsModules for extracting and inferring information from HTTP User-Agentstrings.Eg to tell how many of your sites users are using IE vs Chrome vs Firefox 18
  19. 19. Modules for parsing User-Agent 19
  20. 20. Review structure: module section 20
  21. 21. Comparison Coverage (test corpus)Speed Review-driven development! 21
  22. 22. Curation• Requirements for these modules • Continuing good coverage of (versions of) browsers, bots, libraries, etc • Good enough performance (processing large log files) • Identifying other things (is it a mobile? whats the screen size?)• Working with Olaf Alders on "complete solution" • Currently at the R&D stage  22
  23. 23. Review #5Defining constantsModules for defining constants and immutable variables 23
  24. 24. Modules for defining constants 24
  25. 25. Some firsts with this review• I was prompted to look at the Perl source • To make sure I understood SvREADONLY• Emailed an author for help understanding his code• Enough modules to prompt me to map them out 25
  26. 26. Observations• I had been using Readonly • But I clearly hadnt read all the documentation before• Now I use Const::Fast • Latency: all those modules using Readonly …• Surprising variety of designs / approaches • I learned a lot looking at the code for all these modules• But quite a few experiments • "I wonder if you can …" 26
  27. 27. Review #6Getting module dependency infoFinding out what modules (and dists) are used by a module or modules 27
  28. 28. Tried a new approach• Previously I only published after a lot of work • But I still missed plenty of modules• Decided to experiment with a more lean approach • Found 6 modules, none did what I wanted, so wrote my own • Wrote a quick sketch review and published it • More modules came out of the woodwork every time I updated• I was looking at dependencies for my next review • But ended up pushing this one out sooner 28
  29. 29. Modules for getting dependencies 29
  30. 30. Module Author• I had a "bug" reported – author is listed as:• But META.json says:• End result was PAUSE::Permissions 30
  31. 31. Notes• The 23rd module nearly matched mine!• Very much still in progress• Curation needed: • There are 5 or so distinct module types (out of 26 so far) • But lots of different variations on each theme 31
  32. 32. Why so many modules?• Whipupitude• Low barrier to entry for CPAN• Hard to find (all) appropriate modules• Easier to upload than contribute or take over• Playing with Perl is fun (eg constants, dependencies) 32
  33. 33. Review #7Making HTTP requestsModules for making HTTP requests (GET, POST, PUT, DELETE, etc)Excluding higher-level modules like WWW::Mechanize 33
  34. 34. Modules for making HTTP requests The only module Ive given up on 34
  35. 35. Performance on GET requests 35
  36. 36. Runtime dependencies Require a C library Requires a C library and Moose! 36
  37. 37. What module to use? 37
  38. 38. Review #8LUHN checkModules for checking credit card numbers 38
  39. 39. Modules for checking CC numbers 39
  40. 40. Performance of LUHN modules Moose? 40
  41. 41. Review #9Getting a modules pathModules for finding where a module has been installed locally,preferably without having to load the module 41
  42. 42. Modules for getting a modules path 42
  43. 43. ComparisonNearly three orders of magnitude 43
  44. 44. Feature matrix 44
  45. 45. Features vs Dependencies 45
  46. 46. Appropriate # of dependencies 46
  47. 47. ObservationsAnd some mild venting 47
  48. 48. What do people use CPAN for?• Sharing code with others!• Configuration management• A cloud-based store for configuration (Task::BeLike::*)• A transport mechanism• A todo list (egBusiness::BankCard)• A blog / dumping ground for ideas• None of the above (ACME::*) 48
  49. 49. Version numbers• Version numbers should be designed for humans • PAUSE just wants them to be monotonically increasing• All modules in a dist should have 0.01 • A version number 0.004 • The same version number 1.121660 0.0.20• Dont be afraid of 1.x 0.6.1 5 • 73% of dists on CPAN are version 0.* 0.000 0.001005• our $VERSION = 0.01; v0.0.3 2.4 6632 2011121001 49
  50. 50. How can you tell a good module?• Version number • But see Net::HTTP::Tiny v0.01• Recently updated • Some modules are finished• Volume and/or quality of documentation • See Crypt::RandPasswd• Number of bugs • Some big hitters have a long list though…• Number of dependent modules • from different authors• Ratings / +1s• Perl::Critic• Author• All of the above, and more 51
  51. 51. How have I changed as a result?• Ive learned new things, & new ways to do things, & …• I do developer releases & pay a lot more attention to CPAN Testers• Im more paranoid• I benchmark and profile• I use IRC, and have had only positive experiences• How I export functions (thank-you Zefram)• How I think about dependencies• Moose / Mouse / Moo• Im a lot more circumspect about releasing code to CPAN• Beginners mind 52
  52. 52. My review process• Quickly skim and find initial set of modules• Start writing sections for each module • Write generic SYNOPSIS style example, and evolve it as I go along• Report bugs and documentation issues as I go• First sketch at benchmarking & comparing• More thorough search for modules to include• Email the authors, give them a chance to comment• Publish first version • Iterate as people point out modules that Ive missed 54
  53. 53. Its easy – why not give it a go? % mkdir json % cd json % cat > module-list.txt JSON JSON::DWIW JSON::JOM JSON::Parse JSON::Syck JSON::Tiny JSON::Util JSON::XS JSON::YAJL JSON::Streaming::Reader JSON::Streaming::Writer Mojo::JSON Pegex::JSON Test::JSON ^D % cpan-review 55
  54. 54. What do I want you to do?• Stop and search CPAN before writing code • Ok, I accept its hard to find (all of the) relevant modules for a task• Just because you wrote it, you dont have to upload it • Only release if its needed for another module, or another user • Put it on github • Write an article about it on• Add links to similar modules in SEE ALSO • MetaCPAN will include those in search results • Its helpful for users • Youll make my life easier• Write a helpful ABSTRACT in your NAME section • "a perl interface to foolib" • "the hardest working two letters in Perl" 56
  55. 55. Review-related todo list• Related to existing reviews • Fix Crypt::RandPasswd • Configurability for Lingua::EN::Numbers • User-Agent module with Olaf • New constant module: Exporter::Constants • Dependencies: +5 modules to review, my module, curation • HTTP requests: Mojo::UserAgent, async request modules • Changes to Business::CCCheck• New reviews in progress • Hash objects • JSON • Module loading 57
  56. 56. Finding modules is hard• Thinking about trying automatic clustering• Help identify groups of modules for review• Possible contribution to MetaCPAN• Automatic tagging of modules • Use keywords field in metadata 58