LPW 2007 - Perl Plumbing

  • 2,901 views
Uploaded on

A survey of development infrastructure tools available for perl.

A survey of development infrastructure tools available for perl.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,901
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
65
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • I'm Mike. I work at Nestoria. We are pleased to again be sponsors of the LPW. I'm going to talk about the things that programmers build that aren't products that help them develop better products. Will draw on our experiences from Nestoria and discuss everything in the possibly familiar context of a LAMP(erl) system. Some of these things you may know intimately, but more likely you have heard of them before, but never really gotten around to using them.

Transcript

  • 1. Perl Plumbing
      • Mike Astle
      • LPW – 1 Dec 2007
  • 2. Perl Plumbing Toolbox
    • Pipe::Flush
    • Pipe::Leak
    • Pipe::Crack
    • Bundle::Pipe
    • Crack::Expose
    • HTML::Spanner
    • Crypt::Drip
    • Data::Bucket
  • 3. Talking Pointz
    • Test::Harness
    • Devel::Cover
    • HTTPD::Bench::ApacheBench?
    • Devel::DProf/Apache::DProf
  • 4. Plumbing == Infrastructure
    • Why spend the time?
    • More effective unit tests
    • Better benchmarking
    • Faster optimization
    • Ever so unsexy
    • Because you should
  • 5. Automated Testing Test::Harness And His Super Friends
  • 6. Testing
    • We've got both kinds of testing - unit >and< blackbox
    • Test::Harness, Test::Simple, Test::More, Test::WWW::Mechanize, ...
    • Mature family of modules with >lots< of features
    • A simple example...
  • 7. Test Example – The Test
    • astle@nerdcore:~$ cat test.t
    • #!/usr/bin/perl -w
    • use Test::More tests => 3;
    • BEGIN { use_ok( ’Acme::Test’ ); }
    • my $obj = new Acme::Test();
    • ok $obj, 'new object';
    • ok $obj->doSomething(), 'doSomething';
  • 8. Test Example – The Output
    • astle@nerdcore:~$ perl test.t
    • 1..3
    • ok 1 - use Acme::Test;
    • ok 2 - new object
    • ok 3 - doSomething
  • 9. Test Example - TAP
    • Test output is in the Test Anything Protocol (TAP) format
    • Easily digested by Test::Harness by way of the prove utility
  • 10. Test Example - prove
    • astle@nerdcore:~$ prove test.t
    • test....ok
    • All tests successful.
    • Files=1, Tests=3, 0 wallclock secs
    • ( 0.02 cusr + 0.00 csys = 0.02 CPU)
  • 11.
    • t/base/config/basics...................................................ok
    • t/base/config/dirs.....................................................ok
    • t/base/config/merging..................................................ok
    • t/base/config/warnings.................................................ok
    • t/base/crypt...........................................................ok
    • t/base/datadumper/basics...............................................ok
    • t/base/dates...........................................................ok
    • t/base/grid_search.....................................................ok
    • t/base/instructions....................................................ok
    • t/base/langutils.......................................................ok
    • t/base/logging/diag....................................................ok
    • t/base/logging.........................................................ok
    • t/base/obfuscate.......................................................ok
    • t/base/ranges..........................................................ok
    • t/base/report..........................................................ok
    • t/base/utils/basics....................................................ok
    • t/blackbox/api/es_realestate/create_url................................ok
    • t/blackbox/api/es_realestate/echo_es...................................ok
    • t/blackbox/api/es_realestate/keywords..................................ok
    • t/blackbox/api/es_realestate/metadata..................................ok
    • t/blackbox/api/es_realestate/search_listings...........................ok
    • t/blackbox/api/uk_realestate/create_url................................ok
    • t/blackbox/api/uk_realestate/echo_uk...................................ok
    • t/blackbox/api/uk_realestate/keywords..................................ok
    • t/blackbox/api/uk_realestate/metadata..................................ok
    • t/blackbox/api/uk_realestate/metadata_coord...........................ok
    • t/blackbox/api/uk_realestate/search_listings...........................ok
    • t/blackbox/frontend/es_realestate/302_or_html..........................ok
    • t/blackbox/frontend/es_realestate/apache...............................ok
  • 12.
    • t/blackbox/frontend/es_realestate/apache...............................ok
    • t/blackbox/frontend/es_realestate/autosuggest..........................ok
    • t/blackbox/frontend/es_realestate/cb...................................ok
    • t/blackbox/frontend/es_realestate/check_404_not_looping................ok
    • t/blackbox/frontend/es_realestate/cookies..............................ok
    • t/blackbox/frontend/es_realestate/demo.................................ok
    • t/blackbox/frontend/es_realestate/dimg.................................ok
    • t/blackbox/frontend/es_realestate/fbml.................................ok
    • t/blackbox/frontend/es_realestate/keep_query_for_redirect..............ok
    • t/blackbox/frontend/es_realestate/kml..................................ok
    • t/blackbox/frontend/es_realestate/map..................................ok
    • t/blackbox/frontend/es_realestate/metrics..............................ok
    • t/blackbox/frontend/es_realestate/redirect_to_www......................ok
    • t/blackbox/frontend/es_realestate/rss..................................ok
    • t/blackbox/frontend/es_realestate/share................................ok
    • t/blackbox/frontend/es_realestate/static_file..........................ok
    • t/blackbox/frontend/es_realestate/timer................................ok
    • t/blackbox/frontend/es_realestate/widget...............................ok
    • t/blackbox/frontend/es_realestate/www_cgi..............................ok
    • t/blackbox/frontend/uk_realestate/302_or_html..........................ok
    • t/blackbox/frontend/uk_realestate/acookie..............................ok
    • t/blackbox/frontend/uk_realestate/apache...............................ok
    • t/blackbox/frontend/uk_realestate/autosuggest..........................ok
    • t/blackbox/frontend/uk_realestate/cb...................................ok
    • t/blackbox/frontend/uk_realestate/check_404_not_looping................ok
    • t/blackbox/frontend/uk_realestate/cluster_image/end_to_end.............ok
    • t/blackbox/frontend/uk_realestate/cookies..............................ok
    • t/blackbox/frontend/uk_realestate/correct_let_to_rent..................ok
    • t/blackbox/frontend/uk_realestate/correct_location.....................ok
  • 13.
    • t/blackbox/frontend/uk_realestate/demo.................................ok
    • t/blackbox/frontend/uk_realestate/dimg.................................ok
    • t/blackbox/frontend/uk_realestate/botnet....ok
    • t/blackbox/frontend/uk_realestate/fbml.................................ok
    • t/blackbox/frontend/uk_realestate/keep_query_for_redirect..............ok
    • t/blackbox/frontend/uk_realestate/kml..................................ok
    • t/blackbox/frontend/uk_realestate/map..................................ok
    • t/blackbox/frontend/uk_realestate/metrics..............................ok
    • t/blackbox/frontend/uk_realestate/read_from_apache_log.................ok
    • t/blackbox/frontend/uk_realestate/redirect_to_www......................ok
    • t/blackbox/frontend/uk_realestate/rss..................................ok
    • t/blackbox/frontend/uk_realestate/share................................ok
    • t/blackbox/frontend/uk_realestate/static_file..........................ok
    • t/blackbox/frontend/uk_realestate/timer................................ok
    • t/blackbox/frontend/uk_realestate/widget...............................ok
    • t/blackbox/frontend/uk_realestate/www_cgi..............................ok
    • t/blackbox/search-api/es_realestate....................................ok
    • t/blackbox/search-api/uk_realestate....................................ok
    • t/cache/basics.........................................................ok
    • t/cache/list_of_all_caches.............................................ok
    • t/compiles_and_use_strict..............................................ok
    • t/database/base/arcgrid_datapoint......................................ok
    • t/database/base/geoid_datapoint........................................ok
    • t/database/base/geoid_summary..........................................ok
    • t/database/base........................................................ok
    • t/database/census......................................................ok
    • t/database/coverage....................................................ok
    • t/database/ctax........................................................ok
    • t/database/get_random_listings.........................................ok
  • 14.
    • t/database/listingdb...................................................ok
    • t/database/localdb.....................................................ok
    • t/database/metricsdb...................................................ok
    • t/database/searchindex/base............................................ok
    • t/database/searchindex/set_comment.....................................ok
    • t/database/users.......................................................ok
    • t/dependencies.........................................................ok
    • t/etl/attrhash.........................................................ok
    • t/etl/convert..........................................................ok
    • t/etl/dropbox..........................................................ok
    • t/etl/get_summaries_since..............................................ok
    • t/etl/images...........................................................ok
    • t/etl/import...........................................................ok
    • t/etl/instance/cleanup.................................................ok
    • t/etl/instance/convert.................................................ok
    • t/etl/instance/import..................................................ok
    • t/etl/instance/process.................................................ok
    • t/etl/instance/receive.................................................ok
    • t/etl/instance.........................................................ok
    • t/etl/object_cache.....................................................ok
    • t/listings/scorer/quality_score_v2.....................................ok
    • t/listings/scorer/random_scorer........................................ok
    • t/listings/scorer/simple_scorer........................................ok
    • t/listings/searchindex.................................................ok
    • t/listings/slice.......................................................ok
    • t/listings/stripe......................................................ok
    • t/listings/striper.....................................................ok
    • t/listings/transformer.................................................ok
    • t/localization/basics..................................................ok
  • 15.
    • t/etl/weather/control.............................ok
    • t/etl/process/realestate/es............................................ok
    • t/etl/process/realestate/uk............................................ok
    • t/etl/process..........................................................ok
    • t/etl/product/realestate/es............................................ok
    • t/etl/product/realestate/realestate....................................ok
    • t/etl/product/realestate/uk............................................ok
    • t/etl/receive..........................................................ok
    • t/etl/servers..........................................................ok
    • t/etl/summary..........................................................ok
    • t/etl/thumbs...........................................................ok
    • t/external_apps/lockrun................................................ok
    • t/geo/cleangeoid.......................................................ok
    • t/geo/clusterutils.....................................................ok
    • t/geo/coord_location...................................................ok
    • t/geo/creategeoid......................................................ok
    • t/geo/find_locations/base..............................................ok
    • t/geo/find_locations/coordinates.......................................ok
    • t/geo/find_locations/es/postcodes......................................ok
    • t/geo/find_locations/es/specials.......................................ok
    • t/geo/find_locations/es/spellcheck.....................................ok
    • t/geo/find_locations/geoids............................................ok
    • t/geo/find_locations/uk/postcodes......................................ok
    • t/geo/find_locations/uk/specials.......................................ok
    • t/geo/find_locations/uk/spellcheck.....................................ok
    • t/geo/find_locations/wordindex.........................................ok
    • t/geo/geocoder/common..................................................ok
    • t/geo/geocoder/coordinates.............................................ok
    • t/geo/geocoder/es/addresses............................................ok
  • 16.
    • t/geo/geocoder/es/cleansing............................................ok
    • t/geo/geocoder/es/hierarchy............................................ok
    • t/geo/geocoder/es/streets..............................................ok
    • t/geo/geocoder/uk/addresses............................................ok
    • t/geo/geocoder/uk/cleansing............................................ok
    • t/geo/geocoder/uk/postcode_attributes..................................ok
    • t/geo/geogrid..........................................................ok
    • t/geo/geoidindex.......................................................ok
    • t/geo/geometry.........................................................ok
    • t/geo/inputformat......................................................ok
    • t/geo/lookupfiles/es...................................................ok
    • t/geo/lookupfiles/uk...................................................ok
    • t/geo/postcodes/es.....................................................ok
    • t/geo/postcodes/uk.....................................................ok
    • t/geo/thirdparty/arcgrid/base..........................................ok
    • t/geo/thirdparty/arcgrid/create........................................ok
    • t/geo/thirdparty/arcgrid/dbfile........................................ok
    • t/geo/thirdparty/arcgrid/parse.........................................ok
    • t/geo/thirdparty/arcgrid/serialize.....................................ok
    • t/geo/thirdparty/arcgrid/testfile......................................ok
    • t/geo/tileoverlay......................................................ok
    • t/geo/utils............................................................ok
    • t/geo/utils_es.........................................................ok
    • t/geo/wordindex........................................................ok
    • t/io/writer/multi/file.................................................ok
    • t/io/writer/multi/memory...............................................ok
    • t/io/writer/single/file................................................ok
    • t/io/writer/single/memory..............................................ok
    • t/link_checker/test_plugins............................................ok
  • 17. And so on...
  • 18.
    • Failed Test Stat Wstat Total Fail List of Failed
    • ------------------------------------------------------------------------------
    • t/blackbox/api/es_realestate/echo_es.t 255 65280 1 1 1
    • t/blackbox/api/uk_realestate/echo_uk.t 255 65280 1 1 1
    • t/listings/scorer/quality_score_v2.t 2 512 57 2 28-29
    • t/localization/check_po_files.t 4 1024 506 4 3-4 425-426
    • t/pod-names.t 1 256 294 1 163
    • t/test-template.t 255 65280 1 2 1
    • 51 tests and 5 subtests skipped.
    • Failed 6/348 test scripts. 10/9515 subtests failed.
    • Files=348, Tests=9515, 845 wallclock secs (624.34 cusr + 18.95 csys = 643.29 CPU)
    • Failed 6/348 test programs. 10/9515 subtests failed.
  • 19. Testing/Staging Server/Cluster
    • This is >not< a development server – no local work
    • As similar as possible to development environment – could be on production hardware running on non-conflicting ports
    • Enough oomph to run test suite in a reasonable amount of time
  • 20. Nightly Build
    • Get the HEAD
      • We use svn up via IPC::RUN3
      • You may prefer SVN::Agent or SVN::Client
    • Check for local changes – no tomfoolery
    • Clean build
      • Update dependencies
      • Rebuild configuration
    • Restart services (e.g. Apache, MySQL)
    • What about test data?
  • 21. Nightly Test Run
    • Only test each revision once – nobody likes to hear the same thing twice
    • We use prove -r (look forward to prove -r -j 4)
    • Send output in a useful format – we do some simple filtering of the prove output
    • Fix problems right away – squeaky wheels eventually get ignored
    • Congratulate yourself when things are good
  • 22. Code Coverage Devel::Cover Your Ass
  • 23. Code Coverage Tools
    • How do you know if you have enough tests?
    • Coverage tools tell you how much of your code is actually executed
    • Devel::Cover is pretty bad ass
    • Still alpha after all these years?
    • Can also be used in mod_perl – we don't do this...yet
  • 24. Run For Cover
    • Call cover on every .t file – output into a single database
    • Ignore list can be a bit trying – trial and error
    • Generate summary stats from a single database
  • 25. Running for Cover
    • Running command: /usr/bin/perl -MDevel::Cover=-ignore,^t/,+ignore,^ext/,+ignore,^/usr/ /home/lokku/code/perl/t/etl/images.t
    • Running command: /usr/bin/perl -MDevel::Cover=-ignore,^t/,+ignore,^ext/,+ignore,^/usr/ /home/lokku/code/perl/t/database/users.t
    • Running command: /usr/bin/perl -MDevel::Cover=-ignore,^t/,+ignore,^ext/,+ignore,^/usr/ /home/lokku/code/perl/t/geo/coord_location.t
  • 26. Ran for Cover
    • ---------------------------- ------ ------ ------ ------ ------ ------ ------
    • File stmt bran cond sub pod time total
    • ---------------------------- ------ ------ ------ ------ ------ ------ ------
    • lib/Lokku/Base/Config.pm 92.9 85.0 66.7 94.7 n/a 5.4 90.8
    • lib/Lokku/Base/Crypt.pm 96.6 85.3 47.8 100.0 n/a 4.7 87.3
    • lib/Lokku/Base/DataDumper.pm 80.0 68.8 66.7 83.3 40.0 1.4 75.0
    • lib/Lokku/Base/Dates.pm 80.6 70.1 63.2 91.3 n/a 10.9 75.2
    • lib/Lokku/Base/GridSearch.pm 100.0 83.3 66.7 100.0 n/a 0.7 95.6
    • ...okku/Base/Instructions.pm 100.0 n/a n/a 100.0 n/a 0.0 100.0
    • lib/Lokku/Base/LangUtils.pm 100.0 n/a n/a 100.0 n/a 0.0 100.0
    • lib/Lokku/Base/Logging.pm 86.6 37.5 38.1 100.0 0.0 0.3 68.9
    • ...se/Logging/DiagLogging.pm 100.0 n/a n/a 100.0 20.0 0.1 89.2
    • ...Base/Logging/NoLogging.pm 66.7 n/a n/a 45.5 12.5 0.0 48.6
    • .../Logging/SimpleLogging.pm 39.3 n/a n/a 36.4 12.5 0.0 34.0
    • lib/Lokku/Base/Obfuscate.pm 98.7 92.3 50.0 100.0 n/a 3.6 96.5
    • lib/Lokku/Base/Ranges.pm 74.4 50.0 40.7 91.7 100.0 1.1 62.4
    • lib/Lokku/Base/Report.pm 82.7 69.4 69.6 94.4 100.0 2.7 80.1
    • lib/Lokku/Base/Utils.pm 68.2 50.0 40.0 85.2 100.0 58.4 63.2
    • lib/Lokku/Cache.pm 26.1 0.0 0.0 33.3 n/a 0.0 23.5
    • lib/Lokku/DB/Base.pm 8.5 0.0 0.0 21.2 24.3 0.1 7.5
    • lib/Lokku/DB/UsersDB.pm 11.6 0.0 0.0 31.9 0.0 0.1 7.4
    • ...okku/Geo/CoordLocation.pm 21.1 3.3 4.3 50.0 n/a 0.0 16.5
    • lib/Lokku/Geo/CreateGeoId.pm 16.6 3.2 0.0 33.3 37.0 0.2 16.3
    • lib/Lokku/Geo/GeoCoder.pm 20.8 0.0 0.0 48.4 87.5 0.1 17.7
    • lib/Lokku/Geo/GeoIdIndex.pm 34.1 9.7 5.9 57.7 90.9 0.2 29.5
    • ...
  • 27. Benchmarking How much do ya HTTPD::Bench::ApacheBench?
  • 28. Benchmarking
    • Running tests against your own code (or services) to measure performance
    • Emphasize comparability
    • Name and version benchmarks
    • Generate benchmarks from real query logs or from your staging server for new features
    • Record context (name, version, hardware, number of clients) in output
  • 29. Benchmarking - Tools
    • We use HTTPD::Bench::ApacheBench
    • A few shortcomings:
      • No per-request headers (e.g. cookies)
      • Could be easier to pick out the slow pokes
  • 30. Benchmarking - Output
    • ---------- Variables ----------
    • Start Time: 1195660491
    • Stop Time: 1195662141
    • Benchmark: frontend-profile
    • Version: version1
    • ---------- Statistics ----------
    • Total Correct Responses: 2931
    • Average: 0.2432
    • Standard Deviation: 0.3950
    • Percentiles:
    • p95: 0.913
    • p98: 1.451
    • p99: 2.075
    • ---------- Errors ----------
  • 31. Benchmarking - Analysis
    • Ignore the average
    • Look at percentiles
    • All SLAs are in nines
    • Good average can be bad
    • Bad average can be good
  • 32. Profiling Not Just For Java Anymore
  • 33. Profiling
    • Profilers are tools for analyzing where your code is spending all of its time
    • My personal fetish
    • Sometimes complicated to set up – but definitely worth it
  • 34. Profiling - Tools
    • We use Devel::DProf
    • Best of Breed?
    • There are others in CPAN
    • Hook into mod_perl with Apache::DProf
  • 35. Profiling - Output
    • DProf File: /home/lokku/common/conf/apache/logs/dprof/10284/tmon.out
    • Total Elapsed Time = 792.3461 Seconds
    • User+System Time = 13.22611 Seconds
    • Exclusive Times
    • %Time ExclSec CumulS #Calls sec/call Csec/c Name
    • 24.0 3.178 3.178 804 0.0040 0.0040 Lokku::URL::SERP::process
    • 17.9 2.371 13.116 172 0.0138 0.0763 Lokku::Website::Base::handler
    • 9.78 1.294 9.353 2506 0.0005 0.0037 HTML::Mason::Request::comp
    • 8.95 1.184 1.292 5806 0.0002 0.0002 Lokku::Template::loc
    • 7.75 1.025 1.129 3783 0.0003 0.0003 Lokku::Website::SERP::__ANON__
    • 3.15 0.417 9.645 4090 0.0001 0.0024 HTML::Mason::Commands::__ANON__
    • 2.80 0.370 5.025 1468 0.0003 0.0034 HTML::Mason::Request::scomp
    • 2.56 0.339 0.339 220 0.0015 0.0015 Lokku::URL::SERP::as_path
    • 1.71 0.226 0.226 1743 0.0001 0.0001 Lokku::Template::url_for
    • 1.70 0.225 0.225 29819 0.0000 0.0000 BerkeleyDB::_tiedHash::NEXTKEY
    • 1.29 0.170 0.170 4 0.0425 0.0425 Lokku::NLP::UK::EN::RealEstate::An
    • alyse::new
    • 1.14 0.151 0.143 260 0.0006 0.0005 Text::Balanced::_match_codeblock
    • 1.00 0.132 0.149 1652 0.0001 0.0001 Text::Balanced::_match_variable
    • 0.73 0.096 0.096 1691 0.0001 0.0001 CGI::Util::rearrange
    • 0.69 0.091 0.091 7748 0.0000 0.0000 URI::_query::query
    • 0.68 0.090 0.090 2 0.0450 0.0450 Lokku::NLP::new
    • 0.65 0.086 0.086 1493 0.0001 0.0001 Lokku::URL::SERP::set
    • 0.60 0.079 0.079 8434 0.0000 0.0000 HTML::Mason::Interp::apply_escapes
    • 0.57 0.075 0.075 5806 0.0000 0.0000 Locale::Maketext::Lexicon::EXISTS
    • 0.57 0.075 0.081 497 0.0002 0.0002 Lokku::NLP::sort_and_convert
    • 0.49 0.065 0.065 1812 0.0000 0.0000 Text::Balanced::_match_quotelike
  • 36. Profiling - Analysis
    • One output file per apache process
      • Code out there to combine, looks dicey
      • Just run one child
    • Look for targets of opportunity
    • Inclusive and Exclusive display modes
    • Some reason to believe that output is not always accurate
  • 37.  
  • 38. Is it all worth it?
    • Hells yes
    • Small teams need good tools – take advantage of the fine work done by others
    • Low one-time cost, low maintenance, big long-term gain
    • Run tests daily – review other data monthly
    • Many ways to do things, but the lazy way is often more efficient – avoid rolling your own
  • 39. Thanks! [email_address]