Your SlideShare is downloading. ×
  • Like
  • Save
Solano Labs presented at MassTLC's automated testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Solano Labs presented at MassTLC's automated testing


William Josephson Co-founder at Solano Labs presented at MassTLC's automated testing event on June 12, 2013 at Brightcove, Boston, MA

William Josephson Co-founder at Solano Labs presented at MassTLC's automated testing event on June 12, 2013 at Brightcove, Boston, MA

Published in Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • Avoid testing detailed implementation choicesRe-implementing a feature results in fragility & inflexibilityTrack code quality metrics over timeTrack performance over time at micro-benchmark & system level
  • Deployinator – might make a decent graphic


  • 1. Automated Testing, Continuous Integration & Continuous DeploymentWilliam Josephson, PhDCo - Founder Solano Labs@solanolabsMassTLC Software Development -Automated Testing
  • 2. Some Facts & Figures*High performing organizations:• Deploy code 30 times more frequently.– 8000 times faster than their peers– Deploy multiple times a day, versus an average of once a month.• Have 50 percent fewer failures.• Restore service 12 times faster than their peers.– Fewer failures and faster recovery mean less risk to the business whenchanges are deployed.*Source: Puppet Labs 2013 State of DevOps Report
  • 3. How are they doing this?I don’t have all the answers…But here’s one: Automation
  • 4. Automated Testing Does More Than Find BugsIts Helps Teams Build a Better ProductBug are Common, Difficultto Eliminate & Very CostlyAutomated Testing:Using software to test software• Reduces cost of bugs byreducing time betweenintroduction and discovery• Frees engineers to makelarge changesfearlessly, reducing cycletime• Reduces complexity andrisk at feature integration
  • 5. Automation: Reduces the Cognitive Feedback LoopAutomation shortens thecognitive loop for engineers• Automation delivers results for a build quickly• Shorter turn-around times change howengineers work• Test-Driven Development (TDD) is one logicalextremeWaiting for tests to runis no fun!IncorporateFeedbackRunTestsWriteCodeGet Results
  • 6. What Makes for a Good Automated Test?• Eliminate repetitive manual tests• Yield reproducible results• Cover common and edge/error cases• Ideally either pure unit or integration test• Strive for clarity even when code is gnarly• Test for business requirements
  • 7. Why Not Automate Everything?• Writing tests does have a cost– May not be appropriate for experimental code– May be fragile – UI/mobile automated testing is evolving– Difficult or impossible to express design and human factors• Thorough testing requires commitment and discipline– Writing tests is a short-term pain for a specific individual– Test suite as a whole is valuable to the team over time
  • 8. What do you do with all thoseautomated tests?
  • 9. Continuous Integration: A Discipline for Testing EveryChangeI StrongIsolationDetailedInstrumentationConsistentRuntimeEnvironmentReproducibleResultsComponents that Make for Effective CI:Typical CI Setup:
  • 10. Continuous Deployment: Automatically Release ValidatedChanges• Goal: make release a non-event– Popularized by Web 2.0, but much more widely applicable– Final decision to deploy may still be manual• Valuable to business and engineering sides of the house:– Selectively enabled features – feature flags– Consistent, widespread measurement• Key Ingredients:– Small changes, continuously integrated– Automated infrastructure (DevOps)– Staging/test environment accurately models production
  • 11. Further Reading• Martin Fowler from ThoughtWorks:• Book: Continuous Delivery: Reliable Software (Jez Humble)• IMVU:• Etsy Blog:• Github:•