Care and Feedingof Large Web Applications     Perrin Harkins
Arcos, a project at Plus ThreeAbout 2.5 years of continuous development2 - 5 developers on the team during that time~79,00...
Arcos, contdCMS with modern AJAX UIE-commerceData warehouse with AJAX query builder GUIE-mail campaign managementAsynchron...
Arcos, contdObject-orientedMVC-ish  Controller classes  Database classes  TemplatesCore modules:  CGI::Application  Class:...
DeploymentHard to generalize.tar.gz filesAlways release full builds, not individual files  Consistent state for production...
The CPAN problem“Just use the CPAN shell”  Fails too often  Installs whatever the latest version is
CPAN installer requirements:Install specific versionsInstall from local mediaAllow locally modified versionsFully automate...
Our solutionFinds all modules in src/ directory of package  (Including ones we hacked)Uses Expect to answer questionsBuild...
Why bundle dependencies?Avoid troubleshooting local problemsIf possible, specify Perl, MySQL, OS, etc.Sometimes reality in...
Next up: binary distributionsCompile takes too long  Still just tar?  RPMs?  PAR?See build system in Krang:  http://krancg...
OH HAI!I HELP WIF URINDENTATION!
UpgradesNeed full automationGot the database part  Schema version number in database  Run all upgrade scripts with appropr...
Configuration●   Highly configurable systems must be highly    configured●   Started with httpd.conf style    –   Config::...
Observations on Configuration●   People ignore options they dont understand    –   If the server starts, it must be ok!●  ...
I CAN HASMAINTENANCE  BRANCH?
Version ControlMany tool choices now  svn, svk, git...  Fight it out and let me know who winsMost projects need at least t...
Very simple version controlMain branch is development  Must build and pass testsWhen making a release from a branch, tag i...
Very simple version control, cntdMaintenance branch from that tag for bug fixes  “2.x” branchOn maintenance release, merge...
Too simple?Not everyone is done at the same timeHard to keep dev branch stable for major changes  Feature branches  Possib...
HALP!TESTS RFAYLIN!
Testing●   You all know the drill●   Local library for common testing tasks    –   Log in mech object    –   Run test SMTP...
Testing tools●   Simple Test::More scripts are easy    –   But scoping becomes an issue    –   Test::Class    –   Test::Bu...
KTHXBYE!
Upcoming SlideShare
Loading in …5
×

Care and Feeding of Large Web Applications

607 views
510 views

Published on

This talk was presented at OSCON 2007 and YAPC::NA 2007.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
607
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Care and Feeding of Large Web Applications

  1. 1. Care and Feedingof Large Web Applications Perrin Harkins
  2. 2. Arcos, a project at Plus ThreeAbout 2.5 years of continuous development2 - 5 developers on the team during that time~79,000 lines of Perl code~4900 lines of SQL (Computed with David Wheelers SLOCCount program)Plenty of HTML, CSS, and JavaScript too~14000 automated tests in 269 files169 CPAN modules
  3. 3. Arcos, contdCMS with modern AJAX UIE-commerceData warehouse with AJAX query builder GUIE-mail campaign managementAsynchronous job queue systemComprehensive reporting
  4. 4. Arcos, contdObject-orientedMVC-ish Controller classes Database classes TemplatesCore modules: CGI::Application Class::DBI HTML::Template
  5. 5. DeploymentHard to generalize.tar.gz filesAlways release full builds, not individual files Consistent state for production system QA installs are accurate copies
  6. 6. The CPAN problem“Just use the CPAN shell” Fails too often Installs whatever the latest version is
  7. 7. CPAN installer requirements:Install specific versionsInstall from local mediaAllow locally modified versionsFully automated “Erase your hard drive? [n]”Install in a local directorySkip the tests Cluster install Some will never work
  8. 8. Our solutionFinds all modules in src/ directory of package (Including ones we hacked)Uses Expect to answer questionsBuilds non-CPAN stuff too Apache mod_perl SWISH-E search engine
  9. 9. Why bundle dependencies?Avoid troubleshooting local problemsIf possible, specify Perl, MySQL, OS, etc.Sometimes reality intervenes Hardware support Office politics
  10. 10. Next up: binary distributionsCompile takes too long Still just tar? RPMs? PAR?See build system in Krang: http://krancgcms.com/
  11. 11. OH HAI!I HELP WIF URINDENTATION!
  12. 12. UpgradesNeed full automationGot the database part Schema version number in database Run all upgrade scripts with appropriate names e.g. 2.0 --> 3.0 means run upgrade/V2_1.pm and upgrade/V3_0.pm Hand-written SQL scripts Test mode that upgrades from old schema firstLVM for rollback in QA
  13. 13. Configuration● Highly configurable systems must be highly configured● Started with httpd.conf style – Config::ApacheFormat – Basic scoping and inheritance
  14. 14. Observations on Configuration● People ignore options they dont understand – If the server starts, it must be ok!● Some comments in a config file == weak documentation● Things you rarely change shouldnt be in the shipped config file
  15. 15. I CAN HASMAINTENANCE BRANCH?
  16. 16. Version ControlMany tool choices now svn, svk, git... Fight it out and let me know who winsMost projects need at least two branches at all times development stable
  17. 17. Very simple version controlMain branch is development Must build and pass testsWhen making a release from a branch, tag it Make a “2.0” tag when you release it
  18. 18. Very simple version control, cntdMaintenance branch from that tag for bug fixes “2.x” branchOn maintenance release, merge all changes on maintenance branch to main branch Tag “2.1” on 2.x branch Merge 2.0 --> 2.1 onto main branch
  19. 19. Too simple?Not everyone is done at the same timeHard to keep dev branch stable for major changes Feature branches Possible integration problemsBeware of complex merges This is not the Linux kernel Could be a people problemAn alternate point of view http://utsl.gen.nz/talks/git-svn/intro.html
  20. 20. HALP!TESTS RFAYLIN!
  21. 21. Testing● You all know the drill● Local library for common testing tasks – Log in mech object – Run test SMTP server● Test data setup and teardown – Keep stack, delete in reverse order – Probably easier to just wipe the db
  22. 22. Testing tools● Simple Test::More scripts are easy – But scoping becomes an issue – Test::Class – Test::Builder ● Improves test failure messages● Test::WWW::Mechanize – Great for testing SSL, Apache config, mod_rewrite – Works fine on JavaScript pages ● But doesnt test the JavaScript ● Selenium does
  23. 23. KTHXBYE!

×