Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Shift Left Testing: Going Beyond Agile


Published on

The concept of “shifting testing left” in the software development lifecycle is not new. Shifting testing from manual to automated and then upstream into engineering is a driving factor in DevOps and agile software development. However, Michael Nauman wonders why test automation, DevOps, and agile software development still frequently fail to deliver on their promises? Aligning and hardening your DevOps and test automation—along with streamlining your agile processes—is critical to your project. Michael shares how AutoCAD’s shifting testing left enabled improvements within their engineering team. Learn how the team increased engineering reliability and velocity, and forced process changes upstream into design and research all the way through to product support. Leave knowing why the concept of separation of concerns with regards to quality is as fundamental as the separation of code quality from product quality. Learn how the AutoCAD web team used process dogma and ruthless prioritization to combat metric idolatry and the host of other evils that hold teams back from fully realizing their potential and going beyond agile.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Shift Left Testing: Going Beyond Agile

  1. 1.           T7   Test  Management   10/5/17  11:15           Shift  Left  Testing:  Going  Beyond  Agile     Presented  by:     Michael  Nauman    Autodesk     Brought  to  you  by:                   350  Corporate  Way,  Suite  400,  Orange  Park,  FL  32073     888-­‐-­‐-­‐268-­‐-­‐-­‐8770  ·∙·∙  904-­‐-­‐-­‐278-­‐-­‐-­‐0524  -­‐  -­‐                
  2. 2. Michael  Nauman   Autodesk     Michael  Nauman  started  out  his  career  doing  computer  aided  design  but  after  a   decade  made  the  leap  from  an  AutoCAD  user  to  a  tester  on  the  test  development   team  for  Autodesk  Inc.’s  flagship  product  AutoCAD.  He  has  spent  the  past  twenty   years  testing  various  versions  of  AutoCAD.  Michael  started  out  as  a  black  box   tester,  dabbled  in  test  automation,  evolved  into  a  testing  lead,  managed  testers   as  a  QA  manager,  and  now  is  the  product  owner  of  AutoCAD  Web.  Michael  is   passionate  about  AutoCAD,  DevOps,  bicycles,  and  surfing.    
  3. 3. 1   © 2017 Autodesk Michael Nauman Software QA Manager STARWEST 2017 Shifted Left: Beyond Agile §  Who I am §  Where I work and what I do §  Why is Shifted Left: Beyond Agile important §  Why Autodesk Shifted Left §  What we learned… What you are going to learn today
  4. 4. 2   Me §  10 years AutoCAD User §  20 years Quality Assurance Autodesk, Inc. §  BA in Geography specializing in Geographic Information Systems and Remote Sensing §  Quality Assurance Lead for AutoCAD Web §  4th STAR Conference What is Autodesk?
  5. 5. 3   What is AutoCAD? THE  GROUND  FLOOR  DRAWING  OFFICE  12.64.  ©  GM  ARCHIVE   Image courtesy Apple Inc. © 2017 Autodesk What is Shifted Left: Beyond Agile?
  6. 6. 4   Software Development Life Cycle (SDLC) Design Development Quality Assurance Research Release Years - Months Waterfall Design Development Quality Assurance Research Release Years - Months
  7. 7. 5   Shifting Left Design Development Quality Assurance Research Release Months - Weeks Quality AssuranceQuality AssuranceQuality Assurance Left Shifted Design DevelopmentResearch Release Weeks Quality Assurance Quality Assurance Quality Assurance Quality Assurance Product Engineering
  8. 8. 6   Beyond… Design Development ProductionResearch Support DevOps Prototyping Surveys Customer Input Weeks - Days QA QA QA QA QA Product Engineering Development Production Support Build Measure Learn Research Prototyping Design QA DevOps QA QA Engineering Product Customer SupportQA QA
  9. 9. 7   © 2017 Autodesk Why is Shift Left important to Autodesk? Why is Shift Left important to Autodesk? §  Business Model Transformation §  Changing Landscape §  Respond to Competition §  Efficiency/Speed §  Profit Image courtesy: Gettyimages Dianne Manson
  10. 10. 8   © 2017 Autodesk How Autodesk is Shifting Left •  Default to Open •  Global Engineering •  Cross Team commitment to Quality •  Senior Leadership Support •  Experience Design VP •  MVP vs. MLP How Autodesk is Shifting Left
  11. 11. 9   Image courtesy How AutoCAD is Shifting Left... Image courtesy Utkarsh Khare Image courtesy © 2017 Autodesk Beyond Agile
  12. 12. 10   Separation of Concerns Engineering Team §  Quality of Code §  Performance Metrics §  Infrastructure §  Builds §  Test Automation §  DevOps §  Monitoring Product Team §  Quality of Product §  Business Metrics §  Customer §  Releases §  Success Criteria §  Analytics §  Support Process Dogma §  Lean Startup vs. Agile Scrum §  Release Cycles §  1 week Sprints §  Planning Monday, Retro Friday, Standup Everyday §  User Stories §  Very small: 1-2 days §  Success Criteria: Gherkin §  Acceptance criteria fails, story rejected, no bug §  Infrastructure & Test Automation…
  13. 13. 11   2.12.0 Develop 2.11.0 2.11.0.d7ec6b92.10.1.715f978 Master 2.10.1 TEST ✖ TEST ✔ TEST ✔ TEST ✔ TEST ✔ TEST ✔ TEST ✔ TEST ✔ DevOps Master Metric Idolatry §  “Nothing is above the scrutiny of why” - Keith Klain §  What problem are we trying to solve? §  What value are we providing to our customer? §  What metrics matter? §  Business vs. Engineering §  Analytics vs. Benchmarks §  Fear Drives Metrics
  14. 14. 12   §  Build Measure Learn = Learning can be painful §  UX, QA, & Tech, ALL debt matters §  Increased Engineering efficency puts pressure on other functional teams §  Experience Design: Quality Injection §  Success Criteria (Gherkin) §  “Environment, Workflow, Dataset” – Hai Qing Wang §  Merging & Promotion: Quality Validation §  CI & DevOps Learnings Learnings, Part Deux §  Right place, right time §  Keep your axe sharp §  “Always proud, never satisfied” - Amar Hanspal
  15. 15. 13   © 2017 Autodesk Questions Autodesk and the Autodesk logo are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document. © 2017 Autodesk. All rights reserved.