Wringing Performance out of Perl


Published on

I gave this lightning talk at Yapc 2011. My company uses Perl in a variety of products, some of which have serious performance implications. Here I give a quick overview of some of the tricks we use to squeeze extra performance out of Perl.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Wringing Performance out of Perl

  1. 1.
  2. 2. Wringing Performance out of Perl<br />
  3. 3. Grant Street Group<br />Began as a financial advisor group<br />
  4. 4. Grant Street Group<br />Discovered the Internet in 1997<br />
  5. 5. Grant Street Group<br />Online Auctions of Property Tax Liens<br />Web-Based billing system for tax collectors<br />Conversion of legacy tax-collector databases<br />Online license / vehicle tag renewals<br />Online payment processing<br />Auctions of all types of bonds<br />And lots, lots more!<br />
  6. 6. Tax Lien Auctions<br />
  7. 7. Tax Lien Auctions<br />Absolute feeding frenzy<br />Our bidders threatened to exhaust TIN numbers<br />20 million bidders in 2011<br />More than 30 billion bids altogether<br />Average was a 500,000-way tie<br />About 2,000 auctions closing simultaneously<br />
  8. 8. Tax Lien Auctions<br />How do we award auctions performantly?<br />Random tie-breaking with Crypt::Random<br />Random row-ID plus MySQL = S L O W<br />Turns out we can do it much faster in Perl<br />
  9. 9. Tax Lien Auctions<br />Net result: auction closing takes 20 seconds<br />Breaking 2,000 ties, each 500,000-way<br />Stress-testing indicates can scale by 4x<br />The IRS definitely cannot scale by 4x<br />
  10. 10. Property Tax Online Payments<br />
  11. 11. Property Tax Online Payments<br />Florida residents can pay the property tax<br />Hosted, customized sites per county<br />Largest counties have ~1,000,000 parcels<br />Users are typical Florida residents<br />
  12. 12. Property Tax Online Payments<br />
  13. 13. Property Tax Online Payments<br />
  14. 14. Property Tax Online Payments<br />
  15. 15. Property Tax Online Payments<br />
  16. 16.
  17. 17. Property Tax Online Payments<br />Backend is MySQL and Sphinx<br />Lightning-fast searches with Perl<br />Mapping IDs to table, column, PK<br />Parsing SHOW STATUS LIKE ‘sphinx%’<br />Lots of useful metadata!<br />
  18. 18. Property Tax Online Payments<br />Net results:<br />Sub-second turnaround times<br />9 minute average time on site by payers<br />4 minute average time on site overall<br />
  19. 19. Customer Data Conversion<br />
  20. 20. Customer Data Conversion<br />Largest county in FL is a customer<br />Population ~2.4M people<br />Tax roll of ~900K parcels<br />History of ~5.6M bills across 6 years<br />Full database is large (by our standards)<br />Data files are ~30-50GB<br />Full conversion is ~160 hours, using Perl<br />Might be ~8 hours using pure SQL<br />
  21. 21. Customer Data Conversion<br />Problem is we can’t use pure SQL<br />Ridiculous amounts of business logic<br />Utterly different data models<br />We’re a Perl shop; Perl is our hammer<br />
  22. 22. Customer Data Conversion<br />Hugely parallel data conversion<br />Subdivide conversion into smaller steps<br />Build hash of dependencies between steps<br />Construct DAG of work units in MongoDB<br />Distribute the actual work<br />Run lots of Perl worker processes<br />Workers grab ready work units<br />Perform the work unit sequentially<br />
  23. 23. Customer Data Conversion<br />The end result<br />Total conversion time ~3 hours with 80 workers<br />Nightly reloads now very practical<br />Able to resume incomplete loads<br />
  24. 24. We’re Hiring Telecommuters<br />