Solano Labs presented at MassTLC's automated testing

441 views
374 views

Published on

William Josephson Co-founder at Solano Labs presented at MassTLC's automated testing event on June 12, 2013 at Brightcove, Boston, MA

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
441
On SlideShare
0
From Embeds
0
Number of Embeds
50
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Avoid testing detailed implementation choicesRe-implementing a feature results in fragility & inflexibilityTrack code quality metrics over timeTrack performance over time at micro-benchmark & system level
  • Deployinator – might make a decent graphic
  • Solano Labs presented at MassTLC's automated testing

    1. 1. Automated Testing, Continuous Integration & Continuous DeploymentWilliam Josephson, PhDCo - Founder Solano Labs@solanolabsMassTLC Software Development -Automated Testing
    2. 2. Some Facts & Figures*High performing organizations:• Deploy code 30 times more frequently.– 8000 times faster than their peers– Deploy multiple times a day, versus an average of once a month.• Have 50 percent fewer failures.• Restore service 12 times faster than their peers.– Fewer failures and faster recovery mean less risk to the business whenchanges are deployed.*Source: Puppet Labs 2013 State of DevOps Report
    3. 3. How are they doing this?I don’t have all the answers…But here’s one: Automation
    4. 4. Automated Testing Does More Than Find BugsIts Helps Teams Build a Better ProductBug are Common, Difficultto Eliminate & Very CostlyAutomated Testing:Using software to test software• Reduces cost of bugs byreducing time betweenintroduction and discovery• Frees engineers to makelarge changesfearlessly, reducing cycletime• Reduces complexity andrisk at feature integration
    5. 5. Automation: Reduces the Cognitive Feedback LoopAutomation shortens thecognitive loop for engineers• Automation delivers results for a build quickly• Shorter turn-around times change howengineers work• Test-Driven Development (TDD) is one logicalextremeWaiting for tests to runis no fun!IncorporateFeedbackRunTestsWriteCodeGet Results
    6. 6. What Makes for a Good Automated Test?• Eliminate repetitive manual tests• Yield reproducible results• Cover common and edge/error cases• Ideally either pure unit or integration test• Strive for clarity even when code is gnarly• Test for business requirements
    7. 7. Why Not Automate Everything?• Writing tests does have a cost– May not be appropriate for experimental code– May be fragile – UI/mobile automated testing is evolving– Difficult or impossible to express design and human factors• Thorough testing requires commitment and discipline– Writing tests is a short-term pain for a specific individual– Test suite as a whole is valuable to the team over time
    8. 8. What do you do with all thoseautomated tests?
    9. 9. Continuous Integration: A Discipline for Testing EveryChangeI StrongIsolationDetailedInstrumentationConsistentRuntimeEnvironmentReproducibleResultsComponents that Make for Effective CI:Typical CI Setup:
    10. 10. Continuous Deployment: Automatically Release ValidatedChanges• Goal: make release a non-event– Popularized by Web 2.0, but much more widely applicable– Final decision to deploy may still be manual• Valuable to business and engineering sides of the house:– Selectively enabled features – feature flags– Consistent, widespread measurement• Key Ingredients:– Small changes, continuously integrated– Automated infrastructure (DevOps)– Staging/test environment accurately models production
    11. 11. Further Reading• Martin Fowler from ThoughtWorks: martinfowler.com• Book: Continuous Delivery: Reliable Software (Jez Humble)• IMVU: http://timothyfitz.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/• Etsy Blog: codeascraft.com• Github: teach.github.com/articles/lesson-continuous-delivery/• http://info.puppetlabs.com/2013-state-of-devops-report.html

    ×