Boston MeetUp 10.10


Published on

Solano Labs' Presentation - Reference Slide Included

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Boston MeetUp 10.10

  1. 1.       Welcome  to  Automated  Tes/ng  Boston!         Thank  you  Gazelle  for  hos/ng  us  here!   Thank  you  Solano  Labs  for  the  pizza  &  drinks!     Wifi:  GazelleGuestWireless   PW:  internet2012!  
  2. 2. William  Josephson   Co-­‐Founder,  Solano  Labs   @SolanoLabs     Automated  Tes/ng  Boston   Automated  Tes,ng,  Con,nuous  Integra,on  &  Con,nuous  Deployment  
  3. 3. Solano Labs – Lets you Test More, Faster Some Metrics on how Fast we have Sped Up our Customers: •  •  •  •  18 hrs serial, 2 + hrs on Parallel Team City w/ local HW to 13 mins 30 min on Jenkins to 6 mins 5 hrs serial PyPy test suite to 15 mins 5 hour queuing to 5 min max wait time Solano Labs © 2013 Confidential 3
  4. 4. Some Facts & Figures* High performing organizations: •  Deploy code 30 times more frequently. –  8000 times faster than their peers –  Deploy multiple times a day, versus an average of once a month. •  Have 50 percent fewer failures. •  Restore service 12 times faster than their peers. –  Fewer failures and faster recovery mean less risk to the business when changes are deployed. *Source:  Puppet  Labs  2013  State  of  DevOps  Report  
  5. 5. How  are  they  doing  this?   I  don’t  have  all  the  answers…   But  here’s  one:    Automa,on  
  6. 6. Automated Testing Does More Than Find Bugs Its Helps Teams Build a Better Product Bug are Common, Difficult to Eliminate & Very Costly Automated Testing: Using software to test software •  Reduces  cost  of  bugs  by   reducing  ,me  between   introduc,on  and  discovery   •  Frees  engineers  to  make   large  changes  fearlessly,   reducing  cycle  ,me   •  Reduces  complexity  and   risk  at  feature  integra,on  
  7. 7. Automation: Reduces the Cognitive Feedback Loop Automa/on  shortens  the   cogni/ve  loop  for  engineers     Waiting for tests to run is no fun!     •  •  •  Automa,on  delivers  results  for  a  build  quickly   Shorter  turn-­‐around  ,mes  change  how   engineers  work   Test-­‐Driven  Development  (TDD)  is  one  logical   extreme   Incorporate   Feedback   Get  Results   Write   Code   Run   Tests  
  8. 8. What Makes for a Good Automated Test? •  Eliminate repetitive manual tests •  Yield reproducible results •  Cover common and edge/error cases •  Ideally either pure unit or integration test •  Strive for clarity even when code is gnarly •  Test for business requirements
  9. 9. Why Not Automate Everything? •  Writing tests does have a cost –  May not be appropriate for experimental code –  May be fragile – UI/mobile automated testing is evolving –  Difficult or impossible to express design and human factors •  Thorough testing requires commitment and discipline –  Writing tests is a short-term pain for a specific individual –  Test suite as a whole is valuable to the team over time
  10. 10. What  do  you  do  with  all  those     automated  tests?  
  11. 11. Continuous Integration: A Discipline for Testing Every Change Typical CI Setup: Components that Make for Effective CI:            Consistent          Run/me        Environment         I              Strong                  Isola/on                Detailed                Reproducible                Instrumenta/on                Results  
  12. 12. Continuous Deployment: Automatically Release Validated Changes •  Goal: make release a non-event –  Popularized by Web 2.0, but much more widely applicable –  Final decision to deploy may still be manual •  Valuable to business and engineering sides of the house: –  Selectively enabled features – feature flags –  Consistent, widespread measurement •  Key Ingredients: –  Small changes, continuously integrated –  Automated infrastructure (DevOps) –  Staging/test environment accurately models production
  13. 13. Further Reading •  Martin Fowler from ThoughtWorks: •  Book: Continuous Delivery: Reliable Software (Jez Humble) •  IMVU: •  Etsy Blog: •  Github: • •