NYC MeetUp 10.9

355 views

Published on

Solano Labs' Presentation - Reference Slide Included

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
355
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

NYC MeetUp 10.9

  1. 1. William  Josephson   Co-­‐Founder,  Solano  Labs   @SolanoLabs     Automated  Tes,ng  NYC   Automated  Tes,ng,  Con,nuous  Integra,on  &  Con,nuous  Deployment  
  2. 2. Solano Labs – Lets you Test More, Faster Some Metrics on how Fast we have Sped Up our Customers: •  •  •  •  18 hrs serial, 2 + hrs on Parallel Team City w/ local HW to 13 mins 30 min on Jenkins to 6 mins 5 hrs serial PyPy test suite to 15 mins 5 hour queuing to 5 min max wait time Solano Labs © 2013 Confidential 2
  3. 3. Some Facts & Figures* High performing organizations: •  Deploy code 30 times more frequently. –  8000 times faster than their peers –  Deploy multiple times a day, versus an average of once a month. •  Have 50 percent fewer failures. •  Restore service 12 times faster than their peers. –  Fewer failures and faster recovery mean less risk to the business when changes are deployed. *Source:  Puppet  Labs  2013  State  of  DevOps  Report  
  4. 4. How  are  they  doing  this?   I  don’t  have  all  the  answers…   But  here’s  one:    Automa,on  
  5. 5. Automated Testing Does More Than Find Bugs Its Helps Teams Build a Better Product Bug are Common, Difficult to Eliminate & Very Costly Automated Testing: Using software to test software •  Reduces  cost  of  bugs  by   reducing  ,me  between   introduc,on  and  discovery   •  Frees  engineers  to  make   large  changes  fearlessly,   reducing  cycle  ,me   •  Reduces  complexity  and   risk  at  feature  integra,on  
  6. 6. Automation: Reduces the Cognitive Feedback Loop Automa,on  shortens  the   cogni,ve  loop  for  engineers     Waiting for tests to run is no fun!     •  •  •  Automa,on  delivers  results  for  a  build  quickly   Shorter  turn-­‐around  ,mes  change  how   engineers  work   Test-­‐Driven  Development  (TDD)  is  one  logical   extreme   Incorporate   Feedback   Get  Results   Write   Code   Run   Tests  
  7. 7. What Makes for a Good Automated Test? •  Eliminate repetitive manual tests •  Yield reproducible results •  Cover common and edge/error cases •  Ideally either pure unit or integration test •  Strive for clarity even when code is gnarly •  Test for business requirements
  8. 8. Why Not Automate Everything? •  Writing tests does have a cost –  May not be appropriate for experimental code –  May be fragile – UI/mobile automated testing is evolving –  Difficult or impossible to express design and human factors •  Thorough testing requires commitment and discipline –  Writing tests is a short-term pain for a specific individual –  Test suite as a whole is valuable to the team over time
  9. 9. What  do  you  do  with  all  those     automated  tests?  
  10. 10. Continuous Integration: A Discipline for Testing Every Change Typical CI Setup: Components that Make for Effective CI:            Consistent          Run,me        Environment         I              Strong                  Isola,on                Detailed                Reproducible                Instrumenta,on                Results  
  11. 11. Continuous Deployment: Automatically Release Validated Changes •  Goal: make release a non-event –  Popularized by Web 2.0, but much more widely applicable –  Final decision to deploy may still be manual •  Valuable to business and engineering sides of the house: –  Selectively enabled features – feature flags –  Consistent, widespread measurement •  Key Ingredients: –  Small changes, continuously integrated –  Automated infrastructure (DevOps) –  Staging/test environment accurately models production
  12. 12. Further Reading •  Martin Fowler from ThoughtWorks: martinfowler.com •  Book: Continuous Delivery: Reliable Software (Jez Humble) •  IMVU: http://timothyfitz.com/2009/02/10/continuous-deployment-atimvu-doing-the-impossible-fifty-times-a-day/ •  Etsy Blog: codeascraft.com •  Github: teach.github.com/articles/lesson-continuous-delivery/ •  http://info.puppetlabs.com/2013-state-of-devops-report.html •  http://engineering.quora.com/Continuous-Deployment-at-Quora

×