Php com con-2011

Is It Handmade Code If You Use Power Tools? Photo by matsbeads on Etsy Laura Beth Denker (@elblinkin)

Is It Handmade Code If You Use Power Tools? In the beginning...•Deployments happened weekly•Development happened on branches•Release Manager role rotations efo re B

Is It Handmade Code If You Use Power Tools? Deployinator•One button deployment•Eliminated painful weekly deploys • Deployments went up to a handful a day•Eliminated release manager role rotations • If you committed code, it was your responsibility to push the button efo re B

Is It Handmade Code If You Use Power Tools? Voila! Continuous Deployment•Buy-in•One-button deployment process (NO fear!*)•Monitoring and graphs * NO fear in the deploy process, but still some healthy fear in the code changes

Is It Handmade Code If You Use Power Tools?“Confidence is contagious; so is lack ofconfidence.” Vince Lombardi

Is It Handmade Code If You Use Power Tools? Buildbot•Developer written and maintained CI System • Resulted in Test Infrastructure being convoluted with site architecture • Test infrastructure was untested•Tests written in Python • But we’re a PHP shop•Slow and resource intensive • Could not run on a developer VM•Non-deterministic failures • Flaky tests • Assertions in helpers • EC2 connectivity issues

Is It Handmade Code If You Use Power Tools? (Hudson) Jenkins CI•Open source•High community involvement•Highly configurable•Numerous plugins•Eliminated the pitfalls of developer written and maintained CI 21 J u ne

Is It Handmade Code If You Use Power Tools? PyUnit to PHPUnit•~1,500 tests moved to our lingua franca•~3 minutes to run new PHPUnit tests•Eliminated context-switching between languages ly 19 Ju

Is It Handmade Code If You Use Power Tools? Commit Mutex1. Commit code to trunk (maybe merge a development branch into trunk)2. Wait for Buildbot (and Jenkins) to tests a. If tests PASS, continue to Deployinator and begin deployment b. If tests FAIL, kick Buildbot (and Jenkins) because it might be a flake, then do any of the following i. Repeat step (b) until it passes ii. Debug failures and begin fixing trunk iii. Rollback Note: While there are committed changes in trunk that have not been deployed, DO NOT COMMIT

Is It Handmade Code If You Use Power Tools? Why is there a Commit Mutex?1.Branch development2.Tests cannot run in a timely fashion on the developer VMs

Is It Handmade Code If You Use Power Tools? Try Server v.1•Jenkins parameterized build + shell script •Parameters • username • patch file •Shell script • Create a patch file • Create the JSON • Send a request to Jenkins via cURL•Branches NOT SUPPORTED•Eliminated the Commit Mutex

Is It Handmade Code If You Use Power Tools? g us t 19 Au

Is It Handmade Code If You Use Power Tools? Try Server v.1.1•Jenkins parameterized build + PHP script •Parameters • username • patch file •PHP script • Support for creating git-svn patches • Support for dry runs • Support for uploading manually created patches

Is It Handmade Code If You Use Power Tools? Test Classification•Unit•Integration•Functional (System)

Is It Handmade Code If You Use Power Tools? Test Classification•Functional (System) • Provides confidence that a particular use case works• Unit• Integration

Is It Handmade Code If You Use Power Tools? Legacy CI System is Dead! er 15 pte mb Se

Is It Handmade Code If You Use Power Tools? Test Classification• Functional (System)•Integration • Tests interaction between a discrete number of components • A set of interacting classes • Interaction with an external service• Unit

Is It Handmade Code If You Use Power Tools? Test Classification• Functional (System)• Integration•Unit • Fast • Deterministic • NO database • NO file system • NO network • NO threading, forking processes • NO sleep • NO time dependencies • Tests a single object • Best way to obtain line coverage • Reduces debug time

Is It Handmade Code If You Use Power Tools? PHPUnit @group•@group to declare external dependencies•PHPUnit XML Configuration•Jenkins Project for each configuration er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group cachefor tests that use memcache or another cache er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group databasefor tests that use a database like postgres or mysql er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group networkfor tests that talk to external services (in-house or 3rd party) er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group sleepfor tests that call a sleep() function er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group timefor tests that depend on the time of day or day of the year, etc. er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group smokefor tests that execute cURL and regex based tests against an instance ofthe web service under test er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group@group ﬂakyfor tests that fail without code changes er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group There is NO @group for unit tests. er 21 Oc to b

Is It Handmade Code If You Use Power Tools? PHPUnit @group•@group to declare external dependencies•PHPUnit XML Configuration•Jenkins Project for each configuration er 21 Oc to b

Is It Handmade Code If You Use Power Tools? 40 Deploys in a Day•November 18th, 2010 • <5 months after first dedicated Test & Automation engineer • <3 months after releasing Try Server • 1 week before Thanksgiving freeze•34 deploys the day after ber No ve m ! !! 18

Is It Handmade Code If You Use Power Tools? How do we keep the momentum?•The tests ported to PHPUnit are flaky • Re-building intermittent failures is wasting time in the push queue•We are hiring more engineers • We need to deploy as fast, if not faster•We are writing more tests • More tests can take more time, so we need to write faster tests • Started at 1,500 tests, now running >7,000 tests ber ! Cy y ! Mo n da

Is It Handmade Code If You Use Power Tools? Random Data BAD•Fixtures Framework •Written and maintained by developers •Original intent • Generate data to run test instance of the website •Unexpected usage • Flaky and hard to debug integration tests posing as unit tests

Is It Handmade Code If You Use Power Tools? DBUnit for Multiple Databases•Shard-ed database architecture•Our ORM talks to multiple databases • At least the index and a shard•PHPUnit_Extensions_Database_MultipleDatab ases•Increased clarity in old tests ported to PHPUnit • Data explicitly specified in YAML, not at random • Developer needs to know what databases 8test will touch ra em be De c

Is It Handmade Code If You Use Power Tools? Worth Mentioning: Try Server v.2•Master try project in Jenkins • PHP Script executed in Jenkins shell builder • Started all sub-projects over cURL • Pinged all sub-projects for status • Used Jenkins Text Finder plugin to toggle project state • Sent a unified result e-mail•Eliminated problems finding test results be r8 De cem

Is It Handmade Code If You Use Power Tools? Worth Mentioning: Try Server v.2 be r8 De cem

Is It Handmade Code If You Use Power Tools? Why to mock?unit teststest the functionality of a single method in a single object with theexpectations of the collaborating objectstest caseas a suitcase contains suits, a test case contains tests

Is It Handmade Code If You Use Power Tools? PHPUnit Mock Helpers Making it easier to create mocks, should support writing more unit tests• Tweaking prod code• Writing short, tested helpers for internal types• PHPUnit_Extensions_MockObject_Stub_ReturnMapping • UPDATE: Now in PHPUnit as $this->returnValueMap()• Still more to do... ry 31 Ja nua

Is It Handmade Code If You Use Power Tools? PHPUnit Extensions Open-sourced on Github ry 27 Fe b r ua

Is It Handmade Code If You Use Power Tools? Try Server v.3 Jenkins Master Project Plugin Try’d out on Try Server ch 24 M ar

Is It Handmade Code If You Use Power Tools? Pake Now test locally, just like Jenkins! Ap ril 5

Is It Handmade Code If You Use Power Tools? Deployinator Info in Jenkins! Jenkins Master Project Plugin Just like Try Jenkins Deployinator Plugin Special CLI Command ONE Jenkins job per deployment stage Previously, ~15 Jobs that were difficult to correlate with a push pr il 9 A

Is It Handmade Code If You Use Power Tools? On the Big Monitors!

Is It Handmade Code If You Use Power Tools? Still More to Come...•Flaky test resolution•More focused DBUnit Tests•Make pure unit testing even easier•Try to solve whatever comes next... on d !! Bey

Is It Handmade Code If You Use Power Tools? ReferencesCode As Craft http://codeascraft.etsy.com/Deployinator http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment/Divide and Concur http://codeascraft.etsy.com/2011/04/20/divide-and-concur/Etsy PHPUnit Extensions https://github.com/etsy/phpunit-extensionsJenkins CI http://jenkins-ci.org/PHPUnit https://github.com/sebastianbergmann/phpunit/

Is It Handmade Code If You Use Power Tools? Wardrobe Furnished BySpring Daisies Short Patchwork Dresshttp://www.etsy.com/shop/pineapplepop of Nashville, TNAsymmetric Flower and Beads Necklace (Silver)Created by Michelle D’Netto, Software Engineer @ EtsyKnitted DROPS Bolero with Lace Patternhttp://www.garnstudio.com/lang/en/visoppskrift.php?d_nr=127&d_id=24Design by DROPS Design Kni t bu t teKnitted by Laura Beth Denker a l as d at t h , e f o r i t wa s C o n f, the t dre s o o d a r k s :-(

Is It Handmade Code If You Use Power Tools? Our software is artisanally crafted using theancient techniques from the 1960s. All releasesare aged in a handmade filled cave for at least 20 minutes before decanting to the Web. --Noah Sussman

http://elblinkin.info	444
http://bitly.com	30
http://coderwall.com	3

Php com con-2011

by LB Denker on Apr 29, 2011

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 477

Statistics

Php com con-2011 Presentation Transcript