Dealing with Legacy Perl Code - Peter Scott

1. Maintaining Code While Staying Sane Peter Scott O’Reilly School of Technology February 2011
2. Dealing With Legacy Perl <ul><li>Legacy Perl can stink </li></ul><ul><li>Even when you wrote it </li></ul><ul><li>Or especially when you wrote it </li></ul><ul><li>But Why? </li></ul>
3. Why So Many Ugly Perl Programs? <ul><li>Unfortunately, some of those ways stink </li></ul><ul><li>Or, people use more than one way of doing the same thing in the same program </li></ul>
4. The “DWIM” Myth <ul><li>“ Perl programming doesn’t require the same discipline as other languages” </li></ul><ul><li>Indeed; it may require more </li></ul><ul><ul><li>So many WTDI </li></ul></ul><ul><li>Cure: Adopt best practices </li></ul>
5. The “Prototyping Only” Myth <ul><li>“ Perl is too slow and/or unpredictable to be used for serious work” </li></ul><ul><li>Too slow - sometimes, not always when you’d expect it </li></ul><ul><li>Unpredictable - only when programming without discipline or understanding </li></ul><ul><li>Cure: Learn algorithms, profiling, benchmarking </li></ul>
6. The “$@%*!” Myth <ul><li>“ Perl is a write-only language” </li></ul><ul><li>Another product of insufficient discipline </li></ul><ul><li>The dark side of TMTOWTDI </li></ul><ul><li>Cure: Adopt best practices, eschew obfuscation </li></ul>
7. Find the Author(s)! <ul><li>Are they a better programmer than you or worse? </li></ul><ul><li>Especially, better or worse at Perl? </li></ul><ul><li>This helps you evaluate code you don’t understand </li></ul><ul><ul><li>If you find code you don’t understand, it may be wrong </li></ul></ul><ul><ul><li>Or it may be right, and over your head </li></ul></ul><ul><li>What was their background? </li></ul><ul><ul><li>A Shell programmer uses different idioms from a C++ programmer </li></ul></ul>
8. What Are You Dealing With? <ul><li>What was the code optimized for? </li></ul><ul><ul><li>Maintainability </li></ul></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>Brevity </li></ul></ul><ul><ul><li>Job security </li></ul></ul><ul><ul><li>Something else? </li></ul></ul>
9. Maintainability <ul><li># Print words with an even number of letters, AND even </li></ul><ul><li># number of each vowel, AND even position in the input </li></ul><ul><li># (input is a dictionary that has one word per line) </li></ul><ul><li>OUTER: while (<>) </li></ul><ul><li>{ </li></ul><ul><li>next if $. % 2; </li></ul><ul><li>chomp; </li></ul><ul><li>next if length() % 2; </li></ul><ul><li>for my $vowel (qw/a e i o u y/) </li></ul><ul><li>{ </li></ul><ul><li>my @vowels = /$vowel/g; </li></ul><ul><li>next OUTER if @vowels %2; </li></ul><ul><li>} </li></ul><ul><li>print "$_"; </li></ul><ul><li>} </li></ul>
10. Performance <ul><li>while (<>) </li></ul><ul><li>{ </li></ul><ul><li>next if ($. | length() - 1)) % 2; </li></ul><ul><li>next if tr/e// % 2; </li></ul><ul><li>next if tr/a// % 2; </li></ul><ul><li>next if tr/i// % 2; </li></ul><ul><li>next if tr/o// % 2; </li></ul><ul><li>next if tr/u// % 2; </li></ul><ul><li>next if tr/y// % 2; </li></ul><ul><li>print; </li></ul><ul><li>} </li></ul>
11. Brevity <ul><li>#!/usr/bin/perl -ln </li></ul><ul><li>($x=aeiouy)=~s#.#y/$&//|#g;eval("$x$.|y///c")%2&&next;print </li></ul>
12. Job Security <ul><li>@i = map { chop; $x++ %2 ? $_ : () } <>; </li></ul><ul><li>while ($i = shift @i) </li></ul><ul><li>{ </li></ul><ul><li>ord(pack "w/a*", $i) & 1 and next; </li></ul><ul><li>$_ = "$i"; </li></ul><ul><li>$i =~ s/$_(.*)$_/$1/ for qw/a e i o u y/; </li></ul><ul><li>print unless $i =~ /[aeiouy]/; </li></ul><ul><li>} </li></ul>
13. Testing <ul><li>You can’t test too early </li></ul><ul><ul><li>Or too much </li></ul></ul><ul><ul><li>Use Test::More </li></ul></ul><ul><li>Also useful: </li></ul><ul><ul><li>Test::Exception </li></ul></ul><ul><ul><li>Test::Inline </li></ul></ul><ul><ul><li>Test::NoWarnings </li></ul></ul>
14. Tests are Real Programs, Too <ul><li>Don’t abandon good indentation, variable naming, design, etc just because they’re “tests” </li></ul><ul><ul><li>Follow good development practices </li></ul></ul><ul><ul><li>use strict and use warnings in them </li></ul></ul><ul><ul><li>Abstract common code to modules in t/ </li></ul></ul><ul><li>Keep tests small </li></ul><ul><li>They’ll grow anyway </li></ul><ul><li>Refactor as necessary </li></ul><ul><li>It’s fine for tests to prompt for passwords, etc </li></ul>
15. Testing Web Applications <ul><li>Good design would mean you wouldn't have to go through a web server, of course </li></ul><ul><li>Start out simple by using WWW::Mechanize </li></ul><ul><ul><li>Acts like a virtual browser </li></ul></ul><ul><ul><li>Easy to navigate and fill in forms </li></ul></ul>
16. Web Testing Example <ul><li>my $ua = WWW::Mechanize->new; </li></ul><ul><li>my $res = $ua->get("http://www.example.com/"); </li></ul><ul><li>ok( $res->is_success, "Got first page") </li></ul><ul><li>or die $res->message; </li></ul><ul><li>$ua->set_visible($username, $password); </li></ul><ul><li>ok( $ua->submit->is_success, "Logged in" ) </li></ul><ul><li>or die $ua->res->message; </li></ul>
17. Modern Web Testing <ul><li>Now you can use Test::WWW::Mechanize </li></ul><ul><ul><li>$mech->get_ok(...) </li></ul></ul><ul><ul><li>$mech->title_like(...) </li></ul></ul><ul><ul><li>$mech->content_contains(...) </li></ul></ul><ul><ul><li>$mech->follow_link_ok(...) </li></ul></ul><ul><ul><li>$mech->has_tag_like(...) </li></ul></ul><ul><ul><li>etc </li></ul></ul>
18. Layout <ul><li>Code should be pretty to look at </li></ul><ul><li>Add comments where you had to think a lot </li></ul><ul><li>What’s your role? </li></ul><ul><li>Don’t reformat if it doesn’t belong to you </li></ul><ul><li>Use perltidy to fix up even the worst layout </li></ul>
19. Before perltidy <ul><li>for my $word (keys %{$word{$len}}){ </li></ul><ul><li>chop(my $prefix = $word);if ($opt{g}){ </li></ul><ul><li>while( $prefix ){ </li></ul><ul><li>if(my $words=delete$chain{ $prefix} ){ $chain{$word} = [ @$words, $word ]; </li></ul><ul><li>$maxcount=max ($maxcount,@$words+1); last;} </li></ul><ul><li>chop $prefix; } </li></ul><ul><li>}else{ if (my $words = delete </li></ul><ul><li>$chain{$prefix}){$chain{$word} = [@$words, </li></ul><ul><li>$word]; $changed = 1;} }} </li></ul>
20. After perltidy <ul><li>for my $word (keys %{$word{$len}}) { </li></ul><ul><li>chop(my $prefix = $word); </li></ul><ul><li>if ($opt{g}) { </li></ul><ul><li>while ($prefix) { </li></ul><ul><li>if (my $words = delete $chain{$prefix}) { </li></ul><ul><li>$chain{$word} = [@$words, $word]; </li></ul><ul><li>$maxcount = max($maxcount, @$words + 1); </li></ul><ul><li>last; </li></ul><ul><li>} </li></ul><ul><li>chop $prefix; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul><ul><li>else { </li></ul><ul><li>if (my $words = delete $chain{$prefix}) { </li></ul><ul><li>$chain{$word} = [@$words, $word]; </li></ul><ul><li>$changed = 1; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>
21. After perltidy <ul><li>for my $word (keys %{$word{$len}}) </li></ul><ul><li>{ </li></ul><ul><li>chop(my $prefix = $word); </li></ul><ul><li>if ($opt{g}) </li></ul><ul><li>{ </li></ul><ul><li>while ($prefix) </li></ul><ul><li>{ </li></ul><ul><li>if (my $words = delete $chain{$prefix}) </li></ul><ul><li>{ </li></ul><ul><li>$chain{$word} = [@$words, $word]; </li></ul><ul><li>$maxcount = max($maxcount, @$words + 1); </li></ul><ul><li>last; </li></ul><ul><li>} </li></ul><ul><li>chop $prefix; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul><ul><li>else </li></ul><ul><li>{ </li></ul><ul><li>if (my $words = delete $chain{$prefix}) </li></ul><ul><li>{ </li></ul><ul><li>$chain{$word} = [@$words, $word]; </li></ul><ul><li>$changed = 1; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>
22. Analysis <ul><li>Eliminate superfluous code through coverage analysis: </li></ul><ul><ul><li>Devel::Coverage </li></ul></ul><ul><ul><li>Devel::Cover </li></ul></ul><ul><li>Improve speed through profiling: </li></ul><ul><ul><li>Devel::Dprof </li></ul></ul><ul><ul><li>Devel::NYTProf </li></ul></ul>
23. Devel::NYTProf <ul><li>Very new </li></ul><ul><li>Incredibly flexible and accurate </li></ul><ul><li>Terrific reporting </li></ul>
24. Devel::NYTProf
25. What to Look Out For in Inherited Code <ul><li>Apparent level of Perl expertise </li></ul><ul><ul><li>Uses hashes? Regexes? </li></ul></ul><ul><ul><li>Uses parallel arrays/hashes instead of LoLs? </li></ul></ul><ul><ul><li>Calls unnecessary external programs? </li></ul></ul><ul><li>What version of Perl was it apparently developed for? </li></ul><ul><ul><li>Uses my ? Or local ? Uses use ? </li></ul></ul><ul><li>Cargo Cult Perl </li></ul>
26. The Documentation Hound <ul><li>########################################## </li></ul><ul><li># Function name: increment_number </li></ul><ul><li># Author: John Q. Lifer </li></ul><ul><li># Date Created: 1996-07-14 13:45:22 PDT </li></ul><ul><li># Last modified: 2005-03-21 11:09:32 PST </li></ul><ul><li># Inputs: Number </li></ul><ul><li># Outputs: None </li></ul><ul><li># Returns: Input number plus one </li></ul><ul><li># Exceptions: none </li></ul><ul><li># Change history: </li></ul><ul><li># 1996-07-21: Fixed off by one bug - jql </li></ul><ul><li># 2002-10-23: Changed obfuscatory ++ operator - jql </li></ul><ul><li>########################################## </li></ul><ul><li>sub increment_number { </li></ul><ul><li>### Formal parameter list </li></ul><ul><li>my ($num) = @_; </li></ul><ul><li>### Function body </li></ul><ul><li># TODO: Throw exception on missing input, NaN, etc... - jql </li></ul><ul><li>$num = $num + 1; # Add one to $num </li></ul><ul><li>return $num; </li></ul><ul><li>} </li></ul>
27. The Documentation Hound <ul><li>########################################## </li></ul><ul><li># Function name: increment_number </li></ul><ul><li># Author: John Q. Lifer </li></ul><ul><li># Date Created: 1996-07-14 13:45:22 PDT </li></ul><ul><li># Last modified: 2005-03-21 11:09:32 PST </li></ul><ul><li># Inputs: Number </li></ul><ul><li># Outputs: None </li></ul><ul><li># Returns: Input number plus one </li></ul><ul><li># Exceptions: none </li></ul><ul><li># Change history: </li></ul><ul><li># 1996-07-21: Fixed off by one bug - jql </li></ul><ul><li># 2002-10-23: Changed obfuscatory ++ operator - jql </li></ul><ul><li>########################################## </li></ul><ul><li>sub increment_number { </li></ul><ul><li>### Formal parameter list </li></ul><ul><li>my ($num) = @_; </li></ul><ul><li>### Function body </li></ul><ul><li># TODO: Throw exception on missing input, NaN, etc... - jql </li></ul><ul><li>$num = $num + 1; # Add one to $num </li></ul><ul><li>return $num; </li></ul><ul><li>} </li></ul>
28. The Documentation Hound Cure <ul><li>s/^#+//mg </li></ul><ul><li>Move to POD later in this file </li></ul><ul><ul><li>Or maybe another file </li></ul></ul><ul><ul><ul><li>Such as /dev/null </li></ul></ul></ul><ul><ul><li>But keep function/method signatures and descriptions </li></ul></ul><ul><li>Make the code tell the story </li></ul><ul><li>Preserve comments that answer 'Why?' </li></ul>
29. The Documentation Hound Cure <ul><li>For local programs, author and history information can be taken care of by a source code control system </li></ul><ul><li>For distributions: </li></ul><ul><ul><li>Move author information to a README or POD AUTHOR section </li></ul></ul><ul><ul><li>Move history information to change log </li></ul></ul>
30. String Manipulation, BASIC-Style <ul><li>$repl = ' '; </li></ul><ul><li>for ($off = 0; $off < length($str); $off++) { </li></ul><ul><li>$c = substr($str, $off, 1); </li></ul><ul><li>if (index("012345789", $c) < 0) { </li></ul><ul><li>substr($str, $off, 1, $repl); </li></ul><ul><li>$off-- unless $repl; </li></ul><ul><li>$repl = ''; </li></ul><ul><li>} </li></ul><ul><li>else { </li></ul><ul><li>$repl = ' '; </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>
31. … i.e., Without Regexes <ul><li>$str =~ s/+/ /g; </li></ul>
32. Nice Formatting, But… <ul><li>if ($month == "1") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "2") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "3") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "4") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "5") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "6") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "7") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "8") { $month = "0" . $month; } </li></ul><ul><li>if ($month == "9") { $month = "0" . $month; } </li></ul>
33. Nice Formatting, But… <ul><li>$month = sprintf "%02d", $month; </li></ul>
34. Too Much Time on Their Hands <ul><li>$smtp->datasend("To: santa.clausnorth.pole"); </li></ul><ul><li>$smtp->datasend("From: johnnyhome"); </li></ul><ul><li>$smtp->datasend("Subject: I've Been Good"); </li></ul><ul><li>$smtp->datasend(""); </li></ul><ul><li>$smtp->datasend("Dear Santa"); </li></ul><ul><li>$smtp->datasend("For Christmas I would like:"); </li></ul><ul><li>$smtp->datasend(" Perl 6"); </li></ul><ul><li>$smtp->datasend("Thank you"); </li></ul>
35. Too Much Time on Their Hands <ul><li>$smtp->datasend(<<'EOTEXT'); </li></ul><ul><li>To: santa.claus@north.pole </li></ul><ul><li>From: johnny@home </li></ul><ul><li>Subject: I've Been Good </li></ul><ul><li>Dear Santa </li></ul><ul><li>For Christmas I would like: </li></ul><ul><li>Perl 6 </li></ul><ul><li>Thank you </li></ul><ul><li>EOTEXT </li></ul>
36. Too Much Time on Their Hands <ul><li>$smtp->datasend(<<'EOTEXT' =~ /[^]*(.*?)/g); </li></ul><ul><li>To: santa.claus@north.pole </li></ul><ul><li>From: johnny@home </li></ul><ul><li>Subject: I've Been Good </li></ul><ul><li>Dear Santa </li></ul><ul><li>For Christmas I would like: </li></ul><ul><li>Perl 6 </li></ul><ul><li>Thank you </li></ul><ul><li>EOTEXT </li></ul>
37. <ul><li>$smtp->datasend(<<'EOTEXT' =~ /[^]*(.*?)/g); </li></ul><ul><li>EOTEXT </li></ul>Too Much Time on Their Hands To: santa.claus@north.pole From: johnny@home Subject: I've Been Good Dear Santa For Christmas I would like: Perl 6 Thank you <ul><li>Or use, say, Text::Outdent or similar </li></ul>
38. Way Too Much Time On Their hands <ul><li>print "<HTML><HEAD>"; </li></ul><ul><li>print "<TITLE>My Home Page</TITLE>"; </li></ul><ul><li>print "</HEAD><BODY>"; </li></ul><ul><li>print "<H1>My Home Page</H1>"; </li></ul><ul><li>print "<H2>What I Did Last Summer</H2>"; </li></ul><ul><li>print "<H3>by Cuthbert J. Bigglesworth</H3>"; </li></ul><ul><li>print "<P>Me and my dog <I>Fang</I> went down "; </li></ul><ul><li>print "to the river and caught toads.</P>"; </li></ul><ul><li>print "<P>P.S. I also learned Perl.</P>"; </li></ul><ul><li>print "<P>Here is a scalar: <KBD>$x</KBD>.</P>"; </li></ul><ul><li>print "</BODY></HTML>"; </li></ul><ul><li>Use HTML::Template, Text::Template, the Template Toolkit, or Inline::Files </li></ul>
39. <ul><li>my ($sec, $min, $hour, </li></ul><ul><li>$mday, $mon, $year, </li></ul><ul><li>$wday, $yday, $isdst) </li></ul><ul><li>= localtime(time); </li></ul>The Perils of Cut and Paste # But now use only $mon and $mday...
40. <ul><li>my ($mday, $mon) </li></ul><ul><li>=(localtime)[4,3]; </li></ul>The Perils of Cut and Paste
41. <ul><li>my ($mday, $mon) </li></ul><ul><li>= (localtime)[4,3]; </li></ul>The Perils of Cut and Paste
42. <ul><li>use Time::localtime; </li></ul><ul><li>my ($mday, $mon) </li></ul><ul><li>= (localtime->mday, localtime->mon); </li></ul>The Perils of Cut and Paste
43. Scope? What is This Thing You Call Scope? <ul><li>my ($count, $ncount, @recs, @nrecs, %ccname, %ccphone, %ccaddr, %cccity) </li></ul><ul><li>my ($count2, $temp, @vbinfo, @pscan, $is_true); </li></ul><ul><li>my $temp2; </li></ul><ul><li>my $tempcount; </li></ul><ul><li>my $fudgeFactor; </li></ul><ul><li>my ($fname, $lname, $mi, $address1, $address2, $city, $state, $country, $c_code, $phone, $email, $email_valid); </li></ul><ul><li>my ($form1, $form2, $form3, $form4, $form4a, $form4b, $form4b_valid, @subtotals, $preTaxTotal, $postTaxTotal, $shipping, $TotalTotal); </li></ul><ul><li>my ($is_valid, $discount, $mealpref, $likes_pie, $whatisthisfor); </li></ul><ul><li>my ($PI, $PIE) = (3.14159265358979, "cherry"); </li></ul><ul><li>$count2 = 3; </li></ul><ul><li>[...] </li></ul>
44. Scope Ignorance Cure <ul><li>Move variable declaration to latest possible point </li></ul><ul><ul><li>Exception: configuration settings </li></ul></ul><ul><ul><ul><li>Put those in a separate file if appropriate </li></ul></ul></ul><ul><li>Use in-line declarations for loop variables: </li></ul><ul><ul><li>foreach my $dog (@schnauzers) </li></ul></ul><ul><ul><li>while (my $imp = shift @demons) </li></ul></ul><ul><li>You can carry this even further: </li></ul><ul><ul><li>getopts('dq:v', y %Opt); </li></ul></ul><ul><li>Scope Ignorance is frequently combined with Monolithic Madness </li></ul>
45. Monolithic Madness <ul><li>(Visualize 2500 lines of code without the word 'sub') </li></ul><ul><li>Sufferers’ favorite language: JCL </li></ul><ul><li>“ It started out at 30 lines… it just grew” </li></ul><ul><li>“ I know where everything is” </li></ul><ul><ul><li>Of course, no one else does </li></ul></ul>
46. Monolithic Madness Cure <ul><li>Look for variables with short scopes and evaluate the area for subroutine-ness </li></ul><ul><li>If you like Eclipse, try Devel::Refactor and the extract_subroutine method for the EPIC plug-in </li></ul><ul><li>use strict and turn it off over an ever-narrowing scope: </li></ul><ul><ul><li>use strict; </li></ul></ul><ul><ul><li>[...] </li></ul></ul><ul><ul><li>{ </li></ul></ul><ul><ul><li>no strict; </li></ul></ul><ul><ul><li>[...] </li></ul></ul><ul><ul><li>} </li></ul></ul><ul><ul><li>[...] </li></ul></ul>
47. Perl from ??? <ul><li>$#abspaths = $num; </li></ul><ul><li>for ($i=0; $i<$num; $i++) { </li></ul><ul><li>my $newlen = </li></ul><ul><li>$ROOTLEN+1+length($paths[$i]); </li></ul><ul><li>$abspaths[$i] = ' ' x $newlen; </li></ul><ul><li>$abspaths[$i] = sprintf("%s/%s", $ROOT, </li></ul><ul><li>$dirs[$i]); </li></ul><ul><li>} </li></ul>
48. Perl from C <ul><li>abspaths = realloc(abspaths, </li></ul><ul><li>num * sizeof(char*)); </li></ul><ul><li>for ( i=0; i< num; i++) { </li></ul><ul><li>int newlen = </li></ul><ul><li>ROOTLEN+1+ strlen(paths[ i]); </li></ul><ul><li>char temp[newlen]; </li></ul><ul><li>abspaths[ i] = malloc(newlen); </li></ul><ul><li>sprintf(abspaths[i], "%s/%s", ROOT, </li></ul><ul><li>dirs[ i]); </li></ul><ul><li>} </li></ul>
49. Perl from C <ul><li>@abspaths = map { "$ROOT/$_" } @paths; </li></ul>
50. Perl from ??? <ul><li>$file = "matrix.dat"; </li></ul><ul><li>open (FH, ">$file"); </li></ul><ul><li>for ($I = 1, $I <= 4; $I++) { </li></ul><ul><li>$value = $X[$I][$_], </li></ul><ul><li>write (FH) for 1..10; </li></ul><ul><li>} </li></ul><ul><li>format FH = </li></ul><ul><li><<<<<<<<<<<<< </li></ul><ul><li>$value </li></ul><ul><li>. </li></ul><ul><li>close (FH); </li></ul><ul><li>exit "Done"; </li></ul>
51. Perl from FORTRAN <ul><li>ofile = "matrix.dat" </li></ul><ul><li>OPEN (42, FILE=ofile) </li></ul><ul><li>DO 10 I = 1, 4 </li></ul><ul><li>10 WRITE (42,100) (X(I,J),J=1,10) </li></ul><ul><li>100 FORMAT "(10F10.3)" </li></ul><ul><li>CLOSE (42) </li></ul><ul><li>STOP "Done" </li></ul>
52. Perl from FORTRAN <ul><li>open my $fh, '>', $file or die $!; </li></ul><ul><li>for my $i (1 .. 4) { </li></ul><ul><li>printf {$fh} "%10.3f" $X[$i][$_] </li></ul><ul><li>for 1..10; </li></ul><ul><li>print {$fh} ""; </li></ul><ul><li>} </li></ul>
53. Perl from ??? <ul><li>###################### </li></ul><ul><li>#Program name: Report. </li></ul><ul><li>###################### </li></ul><ul><li>sub decipher { </li></ul><ul><li>unpack $fmt, shift; </li></ul><ul><li>} </li></ul><ul><li>$fmt = "A7" . # base </li></ul><ul><li>"x4" . # filler </li></ul><ul><li>"A7"; # bonus </li></ul><ul><li>$rec = <>; </li></ul><ul><li>($base, $bonus) = decipher($rec); </li></ul><ul><li>$salary = $base + $bonus; </li></ul><ul><li>printf "%7.2f", $salary; </li></ul><ul><li>exit; </li></ul>
54. Perl from COBOL <ul><li>IDENTIFICATION DIVISION. </li></ul><ul><li>PROGRAM-ID. Report. </li></ul><ul><li>DATA DIVISION. </li></ul><ul><li>WORKING-STORAGE SECTION. </li></ul><ul><li>01 salary PICTURE 99999V99 </li></ul><ul><li>01 rec </li></ul><ul><li>02 base PICTURE 99999V99 </li></ul><ul><li>02 FILLER PICTURE X(4) </li></ul><ul><li>02 bonus PICTURE 99999V99 </li></ul><ul><li>PROCEDURE DIVISION. </li></ul><ul><li>READ rec </li></ul><ul><li>ADD bonus TO base GIVING salary </li></ul><ul><li>DISPLAY salary </li></ul><ul><li>STOP RUN. </li></ul>
55. Perl from COBOL <ul><li>my $rec = <>; </li></ul><ul><li>my ($bonus, $base) = unpack "A7x4A7", $rec; </li></ul><ul><li>my $salary = $bonus + $base; </li></ul><ul><li>printf "%7.2f", $salary; </li></ul>
56. Perl from ??? <ul><li>#!/usr/bin/perl -l </li></ul><ul><li>print "Think of a number: "; $dummy = <>; </li></ul><ul><li>$I = 1; </li></ul><ul><li>AGAIN: print "Is it ",$I, "?"; </li></ul><ul><li>$A = <>; </li></ul><ul><li>$X = substr($A,0,1); </li></ul><ul><li>if($X eq"Y" or $X eq"y") { goto DONE } </li></ul><ul><li>$I =$I + 1 </li></ul><ul><li>goto AGAIN; </li></ul><ul><li>DONE: exit; </li></ul><ul><li># END </li></ul>
57. Perl from BASIC <ul><li>100 INPUT "Think of a number: ";D$ </li></ul><ul><li>110 LET I = 1 </li></ul><ul><li>120 PRINT "Is it ", I; </li></ul><ul><li>130 INPUT "?"; A$ </li></ul><ul><li>140 LET X$ = LEFT$(A$,1) </li></ul><ul><li>145 IF X$ = "Y" OR X$ = "y" THEN GOTO 149 </li></ul><ul><li>147 LET I = I + 1 </li></ul><ul><li>148 GOTO 120 </li></ul><ul><li>149 STOP </li></ul><ul><li>150 END </li></ul>
58. Perl from BASIC <ul><li>print "Think of a number"; </li></ul><ul><li>my $ans = ''; </li></ul><ul><li>for (my $guess = 1; $ans !~ /^y/i; $guess++) { </li></ul><ul><li>print "Is it $guess? "; </li></ul><ul><li>chomp($ans = <>); </li></ul><ul><li>} </li></ul>
59. <ul><li>read(STDIN, $buffer, $ENV{CONTENT_LENGTH}); </li></ul><ul><li>my @pairs = split(/&/, $buffer); </li></ul><ul><li>push(@pairs, map { split(/&/, $_) } $ENV{QUERY_STRING}); </li></ul><ul><li>push(@pairs, map { split(/&/, $_) } @ARGV); </li></ul><ul><li>foreach my $pair (@pairs) { </li></ul><ul><li>my ($name, $value) = split(/=/, $pair); </li></ul><ul><li>$name =~ tr/+/ /; </li></ul><ul><li>$name =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; </li></ul><ul><li>$value =~ tr/+/ /; </li></ul><ul><li>$value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg; </li></ul><ul><li>[... You know the rest... ] </li></ul>Cargo Cult Perl <ul><li>Stop the insanity! </li></ul>
60. Cargo Cult Perl use CGI;
61. Comment Code Smells <ul><li>Non-O-O: </li></ul><ul><ul><li>Wannabee Objects and Data Clumps* </li></ul></ul><ul><ul><li>Cut And Paste </li></ul></ul><ul><li>O-O: </li></ul><ul><ul><li>Mixed Abstraction Levels </li></ul></ul><ul><ul><li>Time Dependencies* </li></ul></ul><ul><li>* Courtesy of “The Art of Agile Development”, Shore & Warden </li></ul>
62. Line Editing <ul><li>Reduce bloat </li></ul><ul><ul><li>Mothball code that coverage analysis indicates is not called </li></ul></ul><ul><ul><li>Shorten subroutines and main program to one screen’s length at most </li></ul></ul><ul><ul><li>Don’t exceed screen width </li></ul></ul>
63. Consolidate Variables <ul><li>my (%hits_by_client, %hits_by_method, %hits_by_ext, </li></ul><ul><li>%hits_by_protocol, %hits_by_uri); </li></ul><ul><li>$hits_by_client{$client}++; </li></ul><ul><li>$hits_by_method{$method}++; </li></ul><ul><li>$hits_by_ext{$extension}++; </li></ul><ul><li>$hits_by_protocol{$protocol}++; </li></ul><ul><li>$hits_by_uri{$uri}++; </li></ul>
64. Consolidate Variables <ul><li>my %hits; </li></ul><ul><li>$hits{CLIENT}{$client}++; </li></ul><ul><li>$hits{METHOD}{$method}++; </li></ul><ul><li>$hits{EXTENSION}{$extension}++; </li></ul><ul><li>$hits{PROTOCOL}{$protocol}++; </li></ul><ul><li>$hits{URI}{$uri}++; </li></ul>
65. Consolidate Variables <ul><li>my ($client, $method, $extension, $protocol, $uri) = </li></ul><ul><li>($line =~ /^(+) - .../); </li></ul><ul><li>my %hits; </li></ul><ul><li>$hits{CLIENT}{$client}++; </li></ul><ul><li>$hits{METHOD}{$method}++; </li></ul><ul><li>$hits{EXTENSION}{$extension}++; </li></ul><ul><li>$hits{PROTOCOL}{$protocol}++; </li></ul><ul><li>$hits{URI}{$uri}++; </li></ul>
66. Consolidate Variables <ul><li>my @KEYS = qw(CLIENT METHOD EXTENSION PROTOCOL URI); </li></ul><ul><li>my %access; </li></ul><ul><li>@access{@KEYS} = ($line =~ /^(+) - .../); </li></ul><ul><li>my %hits; </li></ul><ul><li>for my $key (@KEYS) { </li></ul><ul><li>$hits{$key}{ $access{$key} }++; </li></ul><ul><li>} </li></ul>
67. Line Editing <ul><li>Remove rote stuff that the computer can figure out for you </li></ul><ul><ul><li>Example: </li></ul></ul><ul><ul><li>$dbh->do("INSERT INTO perf (s, s1a, s1b, x1, x3, y1, y2, y3, zx, zx4, zx4a) VALUES ($s, $s1a, '$s1b', '$x1', $x2, $y1, $y2, $y3, $zx, '$zx4, '$zx4a')"); </li></ul></ul><ul><ul><li>Arrgh </li></ul></ul>
68. Line Editing <ul><li>Try: </li></ul><ul><ul><li>my $sth = $dbh->prepare("INSERT INTO perf (s, s1a, s1b, x1, x3, y1, y2, y3, zx, zx4, zx4a) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)"); </li></ul></ul><ul><ul><li>$sth->execute($s, $s1a, $s1b, $x1, $x3, $y1, $y3, $y2, $zx, $zx4, $zx4a); </li></ul></ul>
69. Line Editing <ul><li>Still too much work, too error-prone. Try: </li></ul><ul><ul><li>$dbh->do( </li></ul></ul><ul><ul><li>make_insert(perf => keys %data), </li></ul></ul><ul><ul><li>undef, values %data); </li></ul></ul><ul><ul><li>sub make_insert </li></ul></ul><ul><ul><li>{ </li></ul></ul><ul><ul><li>my ($table, @cols) = @_; </li></ul></ul><ul><ul><li>"INSERT INTO $table (" </li></ul></ul><ul><ul><li>. join(',' => @cols) . ") VALUES (" </li></ul></ul><ul><ul><li>. join(',' => ('?') x @cols) . ")"; </li></ul></ul><ul><ul><li>} </li></ul></ul>
70. Line Editing <ul><li>This wheel has been invented several times, e.g.: </li></ul><ul><ul><li>use DBIx::Recordset; </li></ul></ul><ul><ul><li># ... </li></ul></ul><ul><ul><li>DBIx::Recordset->insert( { </li></ul></ul><ul><ul><li>'!DataSource' => $dbh, </li></ul></ul><ul><ul><li>'!Table' => 'perf', </li></ul></ul><ul><ul><li>%data } ); </li></ul></ul>
71. Line Editing <ul><li>Get rid of massive strings </li></ul><ul><li>Especially for HTML, use a templating system instead </li></ul><ul><ul><li>HTML::Template works great, even for non-HTML </li></ul></ul><ul><ul><li>So does Text::Template and the Template Toolkit </li></ul></ul>
72. use strict <ul><li>Use it or get sand kicked in your face </li></ul><ul><li>Eliminate all errors in order to get the code to run </li></ul><ul><li>Eliminate unnecessary package variables </li></ul><ul><li>Declare lexical variables explicitly </li></ul><ul><li>Eliminate symbolic references </li></ul><ul><ul><li>They’re hard to maintain anyway and just plain ugly </li></ul></ul><ul><li>Turn strictness off with no strict </li></ul><ul><ul><li>I’ve only ever needed no strict 'refs' </li></ul></ul>
73. use warnings <ul><li>Use it or get sand kicked in your face </li></ul><ul><ul><li>Can use -w instead on older perls </li></ul></ul><ul><ul><li>On newer perls, use -W in testing to force warnings on across all modules </li></ul></ul><ul><li>Leave warnings enabled in production </li></ul><ul><ul><li>But if users might see the warnings, have them sent to you instead </li></ul></ul><ul><ul><li>Trap via $SIG{__WARN__} handler </li></ul></ul>
74. Commonly Neglected Modules <ul><li>Date:: * - look for unnecessary calls to date or cal </li></ul><ul><li>DBI, DBD:: * - look for unnecessary calls to database programs </li></ul><ul><li>LWP:: * - look for unnecessary calls to lynx , wget or GET </li></ul>

Editor's Notes

Thank you for coming. Ask questions before break so I have time to research.

Code written by someone else Or you, long enough ago Say a couple of weeks Why is Perl so susceptible?

Perl’s motto is also a curse Perl is like English If you have William F. Buckley Jr., you also have Homer Simpson

100 line Perl script may not get the same attention to coding standards, documentation, or other methodology as a 1,000 line C program even though it deserves it just as much.

Slow - how often is that really a problem? Odds are that if it’s too slow in Perl it’s going to be too slow in any language. Or, it was written using a poor algorithm to begin with, which is something that likewise you can do in any language.

Books like PBP help with this problem by telling you what not to do as much as what to do. Brian Kernighan: Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are by definition, not smart enough to debug it.

How good a programmer are/were they? How fluent in Perl?

Too many people skip this step and assume they know what they should do. Are original optimization goals still valid?

Think about testing now. Test with same tools as for Test First. Tests inline make code harder to read though.

Sounds obvious, but if tests aren’t fun to write, haven’t followed. Look for more Test:: modules on CPAN.

Could do same as before - save HTML to file and compare. This is better.

Explain how works.

Pretty to look at means easy on the eyes - not in the usual sense but easy to read. Not formatted with Acme::Eyedropts to look like Mona Lisa. Consistent layout is easy with editors - hit TAB in Emacs.

Common indentation style - none at all.

Not quite my style - braces in K&R style to use fewer lines.

Ok… here it is using my style. So what if it takes up more space. How many people prefer this style, anyway?

You probably want to know which parts of the code are being executed. You might want to know how fast those parts are being executed.

Analysis you do with eyeballs. Example app: 2.5 hrs/1GB -> 20min/12MB, code halved, using hashes. Regexes - except may be optimized for performance. So profile.

If you're working in a team, once you've settled on indentation style you ought to settle commenting style. Here you see the result of paying someone by the line. Note that

this is the only thing here that actually does anything. Dealing with the Documentation Hound is an insidious problem because of course many comments *are* useful and on the whole, people don't use enough of them. It’s like talking to a random group of people and saying, “Y’all need to eat more.” There might be a couple of anorexics in the audience, but for most people, that’s not the right message. But the point is that everything in the program, whether it's code or comments, should contribute towards getting the job done or understanding how it works, and anything that doesn't do that is getting in the way of the stuff that does. So some of these comments would be helpful, but the majority of them are just making it harder for you to read the code.

What can you do about it? Firstly, get rid of the ASCII art. If you want a gap, use a blank line. -click- Next, if the comments relate to how to call public methods or functions, then put them in documentation so anyone can see a proper interface document just by running perldoc. But if you're the kind of project that uses comments about the signature of a function, those are fine. Personally, I leave that information to POD. -click- The whole point of this pruning exercise is to expose the code itself. It was pretty obvious in this example that there was no point at all in having a function for adding one to a number since it would be shorter and clearer just to use ++ in line. Usually, you're not going to be able to inline a subroutine, but if you're taking over maintenance of a coding horror this is the first phase in a rewriting campaign and the next step is line by line editing that may shorten the code still further. Ultimately, you'd like every function and method to be no longer than one screen of your editor. Look at Brian Ingerson. His methods are so short that just the line &quot;my $self = shift&quot; accounted for 10% of their line counts, so he wrote a module to put it in automatically. -click- The comments that you really want are any that answer the question, &quot;Why?&quot; That is the whole point of all documentation: Why is this line of code so weird, why does this function have a bizarre name, why does my database not follow normalization rules, why am I using substr() and index() instead of a simpler regular expression, why, why, why? If you come back to a program you wrote six months ago without perfect documentation, I guarantee that the first question you ask when you look at it again is going to begin with &quot;Why&quot;. Anything that answers that question needs to be preserved, enshrined, and embalmed.

I'm not saying you should get rid of all that information about who wrote the program and when, and what changes they made; just that it belongs in the right place. If the program is something you're using in your own environment then you can just use a source code control system. Everything from RCS to Subversion will keep track of all that stuff for you. That's where you'd expect to find that information if you were looking for it, so put it there. If you're making a distribution to send to someone else - such as CPAN - then you can stick that information in a README and a change log file. Okay, moving on to the next example…

Ok, next pop quiz: what does this do? (beat) Beep - time's up.

You just look at that and go, &quot;Oh, of course, it replaces groups of consecutive non-digits with a single space.&quot;

Okay, here we have something I copied here verbatim from where I saw it so you could see how the author daringly defied the normal rules of layout for the sake of conciseness, alignment, and general prettiness, and it certainly is pretty. Too bad it's so darned repetitive. Repetitive code is a coding horror that's like a leech sucking on your brain, because part of your brain is going, &quot;okay, skip all these, they're all the same,&quot; and another, wiser part of your brain is going, &quot;Wait a minute, I have to check to make sure they *are* all the same.&quot; So is this like the regex example - is one of the digits missing? No! But the more cautious among you probably wondered, &quot;Gee, should it include zero in the list? Well, it's a month, and they're usually numbered starting at one, so…&quot; Too much stuff for your brain to think about! You need it for more important things, like how to explain to your wife why you just bought that 50” TV.

So repetitive code is a giant wake-up call that there's a better way of doing something, and of course, here it is. Speaking of repetitive code,

here's another example that's all too common… again, we have someone who's just not impatient enough. Either they like typing, or they like using their editor macros, either way, you've got better things to do than stare at repetitive code making sure all of those things are really the same. Trust me, it's happening, even if you don't realize it - it's the subconscious programmer's brain at work, the same thing that tells you when it's time to eat. Give it something better to do.

This kind of task is exactly why here docs were invented. You just type what you want with the line breaks the way you want them and you don't have to worry about quoting delimiters. &quot;Oh, but Peter, I don't like the text being up against the left margin instead of indented to make it look separate.&quot; Well then,

indent it, if that's what you want to do. It's just a regex away. There are umpteen ways of doing this, and if you want to indent the heredoc terminator as well, <click> you can do that too if you quote it right to begin with, or you can use one of a couple of nifty source code filters that do the job for you. The point is, don't be satisfied with repetitive code.

Especially something heinous like this, which always drives me crazy. Folks, when I'm reading a Perl program, I'm in Perl reading mode, and maybe sometimes in regex reading mode. I'm not in HTML reading mode, and I don't want to be. -click- I prefer not to look at the HTML at all, and there are a lot of ways of just get it out into a separate file or files. If you put it in a separate file then you can use an HTML editor on it - no HTML editor is going to be able to validate HTML that’s intertwingled with Perl like this. My favorite HTML editor is - someone else. Because inevitably you get all kinds of requests to make this font two points bigger, or make this background a little bit pinker, and it’s not my kind of thing. But there are people who get off on that, and I’m only too happy to let them edit HTML without having to look at my code. HTML and Perl are like dogs and cats; they just shouldn't breed together. It's bad enough when the HTML is in a heredoc, but when they're mixed together on the same line like this over and over, it's nuts. If you really want to put the HTML in the same file - say because you are the HTML editor and you won’t want anyone else touching your HTML - then use Inline::Files so you can have it in a nice clearly marked separate section where it can't escape and start molesting the Perl code.

Does this look familiar to anyone? It should, because it is pasted verbatim from the perldoc documentation for localtime. You see this all the time, but then inevitably <click> they only usae a couple of the variables. And that part of your brain that acts like a little Perl interpreter is left wondering what those other variables are for and when they’re going to get used.

Just declare the ones you’re going to use.

And if you don’t like the numbers 4 and 3 here,

well you don't even have to do that either, if you don't mind using a module that's come with Perl since at least version 5.004. The anal-retentive part of me is forced to point out that there’s a small bug here in that if in between the first and the second call to localtime the system clock passes midnight at the end of the last day of the month, then the results will be inconsistent. If that should happen to bother you, then by all means use the previous example with the list slice and symbolic constants set to 4 and 3 so you know what they mean. Next…

How many people have seen a program that starts with a laundry list like this? How many of you liked it? This has already filled up the entire screen and we've hardly gotten to any executable code yet. Of course, the Documentation Hound cures this problem by declaring each one on a separate line with its own comment block preceding it saying what it's for, when it first appeared in the program, what values it's allowed to take, and other stuff that makes this ten times longer than it already is. So what do we do about it?

First and most important, variable declarations should go as late as possible. Remember, part of the reader's brain is going to be occupied with remembering every variable for as long as it's in scope, so push them to the innermost scope levels possible. The one exception to that is the variable that's used for specifying some global configuration setting, like the path to an important directory or the value of Avagadro's constant, or something. The main reason for putting those at the top of the code is so that a lesser programmer who comes in wanting to change one of those things in your program will find it quickly and not go spelunking through the rest of your code where they might break something. -click- You can help keep variables in the innermost scope possible by using the fact that 'my' can appear before a variable just about anywhere in Perl and in particular, you can declare a loop variable at the point you say it's a loop variable so that it'll go out of scope as soon as the loop finishes. -click- In fact, you can put 'my' in some pretty unlikely places. The second argument to the getopts() function exported by Getopt::Std is a reference to a hash to store the options specified by the user on the command line. You can even put the 'my' inside an enreferencing backslash there to save on having to declare it in a separate line. -click- This kind of coding horror is usually a symptom of another kind.

It's not really possible to show an example of this on the screen so you'll have to use your imagination. -click- But I'm sure it's familiar to many of you. (How many people know what JCL is?) -click- This is the usual excuse for how the program got that way. -click- And this is the inevitable defense when you point out that you need a machete to hack through the code.

So what do you do? It's not easy. You have to identify areas where variables are used only over a short part of the program, and then see whether that means tghat that part can be taken out into a subroutine. -click- If you use Eclipse - I'm afraid I haven't yet - then you might want to uses the Devel::Refactor module from CPAN, which can figure out what the subroutine for any given chunk of code should be. That doesn't necessarily mean it's going to be a good choice, mind you. -click- Because Monolithic Madness seldom uses strict, that means you're going to have to add it at some point if you want to stay sane. Or become sane, whichever applies to you. You can take a piecemeal approach to this if you want; first, turn on strictness for the whole program, but then embed the rest of the whole program in a naked block and immediately turn strictness off, which of course is going to be a giant no-op. But now you can start pulling code out of that no strictness block at the beginning and the end and fixing the code to be strict compliant as you go, and keep shrinking the size of the inner no strictness block.

I have taken some creative license with the spacing in these examples for the sake of some entertainment value. So this first example is code written by someone whose favorite language is - what?

Yep, C. This just prepends a root path to a bunch of relative paths. Of course, the C programmer is probably wondering at this point how Perl works without a malloc or realloc function.

Here's how a native Perl speaker would do that same task.

This next example is what might come from someone who was more used to… what?

Yep, FORTRAN. This prints a matrix of numbers to a file. I've tried to pick examples here that are sort of in keeping with the kinds of things those languages are generally used for. So a FORTRAN programmer when they want to print something might start looking for something called FORMAT and figure that's what they need to use for printing formatted values. And… yeessss, technically that's correct, but of course that's not how we'd really do that most of the time.

And here's the native Perl solution, or at least one way of doing it.

Okay, this should be a fairly easy one… this is Perl spoken by a what programmer?

That's right. This being a COBOL program, it just adds two numbers together, but because they're numbers representing money, that's okay.

And here's the native Perl solution. It's not really changed much because the original COBOL implies strict fixed-length formatted records, and we use unpack() for handling those in Perl. Of course, we're more used to writing applications that handle less structured input formats, or more modern structured input formats like XML.

Okay, this last example should be pretty easy, you've got the hang of this by now…

Right, BASIC. Okay, so this is the kind of program I used to write when I was learning BASIC: guess what number someone's thinking of, the hard way.

And here's a native Perl solution. The astute reader will note that since the original program never actually *did* anything with the right answer, in this version, it doesn't even stay in scope once we found it. CHANGE TO USE IO::Prompt

Okay, something a little more serious now. I'm going to assume that all of you have seen this code by now, because it's multiplying around the internet like a bad virus. --click- Of course, this is the code to parse CGI inputs - well, the beginning of the code, anyway, I couldn't bring myself to finish pasting it. This is just one instance of it I found somewhere - it wasn't hard, I just stuck out my foot and this tripped over it - and it's better than many I've seen, but still, isn't

*this* a lot easier to read? Not to mention the fact that it's come bundled with the Perl core since 5.004, and it gets the decoding right, unlike the previous code and virtually every other home-grown solution floating out there. It's just a lot harder than most people think to handle all the possibilities. Of course, it doesn't end with CGI.pm. There are still zillions of people parsing XML and HTML with simple regular expressions, people calling out to programs like 'date' and 'cal' instead of using a Date:: module, and people calling database programs like sqlplus instead of using DBI.

Mixed abstraction levels refers to code of different complexity in a method - it should all look the same level of abstraction. Time dependencies means the caller must call methods in a certain order - this pushes some of the responsibilities of the class onto the caller - create methods managing correct calling sequence

Restructuring is easier the less code you have. I recently compressed subroutine from 200 lines to 12. Ideally all blocks fit on screen (mine is 50 lines high). Code that’s not covered, write test for or delete (have got revision control).

Typical. Bug in this line, what is it? Quote missing after $zx4. Another bug. Says $x2 in the values line where the corresponding column name is x3. Tim wouldn’t settle - read DBI, discover placeholders

Still has a bug. $y2 and $y3 are reversed in the argument list. That's not the only bug. There's one too many question marks in the values line.

Maintaining associations between two sets is a hash. Now every time you need to insert values in a table, just call the subroutine.

Nothing ruins reading like text lump. Lots of print statements -> Here document -> separate file (config file) Usually reason for text is creating some web page etc. Candidates for templating, give work to somone else.

People post code using symref unwittingly. Flamed ‘cos couldn't be using strict. strict subs stops calling sub without parens if no def yet, so define sub earlier, or declare stub, or put empty parens after call. strict vars requires lexicals or effort to use package var.

use warnings is relatively new - so probably see -w. And that's fine. use warnings is lexically scoped, so… -W forces warnings everywhere, regardless of attempts to turn off. So use it, decide, remove, use warnings.

Still zillions of people parsing XML and HTML with regexen, calling out to 'date' and 'cal' & sql+. Some code may have been pasted from third-party modules before was okay to trust CPAN.

Dealing with Legacy Perl Code - Peter Scott

You are reading a preview.

Dealing with Legacy Perl Code - Peter Scott

More Related Content

Viewers also liked

Similar to Dealing with Legacy Perl Code - Peter Scott

More from O'Reilly Media

Featured

Related Books

Related Audiobooks