Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Winning strategies in Test Automation


Published on

Barely any modern software development project will be initiated without the discussion on test automation. With the demand to deliver software products in higher quality faster, test automation is oftentimes perceived by project leads as the silver bullet, allowing to ‘test everything’ without high overhead in employing skilled workforce. The tools are being discussed more often, than development and maintenance costs of automation frameworks that can support projects from inception into delivery and post-production.

Incorrect approaches that result in choosing invalid test automation strategies are often the cause of much frustration later in the project, when the reality of test automation activities become visible and need to be accounted for.
In this webinar, Anna will discuss her experiences in selecting short term and long term test automation strategies applied within various contexts, various skill sets that are required for running a successful test automation project, and suggest alternatives to full-blown test automation with the insufficient project resources.

* when the test automation is an enabler, and when it’s an impediment to project success
* when to start planning for automation, and how to select the tools and methods most suitable for specific project needs
* how to anticipate investment costs and support long term automation effort across multiple projects
* skills and roles in test automation

Published in: Software
  • Be the first to comment

  • Be the first to like this

Winning strategies in Test Automation

  1. 1. XBOSo&,  Inc.  All  Rights  Reserved.   1    Winning Strategies In Test Automation
  2. 2. XBOSo&   Founded in 2006 Dedicated to software quality Software QA Consulting Software Testing Offices in San Francisco, Beijing and Amsterdam   XBOSo&,  Inc.  All  Rights  Reserved.   2  
  3. 3. House Rules Everyone  except  the  speakers  are  muted     QuesDons  via  the  gotowebinar  control  on  the  right  side  of  your  screen  or   through  TwiIer  @XBOSo&       QuesDons  can  be  asked  throughout  the  webinar  -­‐  we’ll  try  to  answer  them   at  the  end.     You  will  receive  info  on  recording  a&er  the  webinar   XBOSo&,  Inc.  All  Rights  Reserved.   3  
  4. 4. Meet Our Speakers •  CEO and Founder of XBOSoft •  Relevant specialties and passions –  Software quality process, evaluation, measurement and improvement –  Software quality in use / UX design –  Mobile User Experience and usability –  Cycling and travel XBOSo&,  Inc.  All  Rights  Reserved.   4   Philip Lew •  Guest Speaker •  15+ years in Testing and QA management •  International conferences speaker •  STP Community Advisory Board Member •  BBST Instructor •  Passions: •  Cross-functional team collaboration •  Testers’ skills development •  Quality advocacy Anna Royzman Sabrina Gasson •  Marketing Manager of XBOSoft •  Emails you all regularly to join our industry hot topic webinars •  And invites you all to download our latest trends in software testing whitepapers.
  5. 5. MY  EXPERTISE     •  Manager  of  so&ware  testers  and  automaDon   engineers  over  variety  of  projects,  in  both   agile  and  waterfall   •  Leading  AT  SIG  (special  interest  group)  in   Technology  department  of  my  organizaDon   •  15+  years  experience  with  home-­‐grown  test   automaDon  frameworks,  commercial  and   open  source  tools     XBOSo&,  Inc.  All  Rights  Reserved.   5  
  8. 8. EXAMPLES   •  Unit  tesDng  frameworks   •  Test  controllers   •  Recording  and  playback   •  Performance  tesDng   tools   •  Security  tesDng  tools   •  AnalyDcs  collectors   •  Screen  capturing  tools   •  Code  coverage  tools   •  Protocol  tesDng  tools   •  Monitoring  tools   •  Database  tesDng  tools   •  Simulators,  log  helpers,   debuggers     •  Data  populaDon  scripts:   setups,  etc.   •  Keystroke  recording   tools   XBOSo&,  Inc.  All  Rights  Reserved.   8  
  9. 9. AUTOMATION  IS  A  HELPER   – CreaDng  a  ‘safety  net’  of  business-­‐criDcal   workflows  and  logic  that  must  work  every  Dme   – Allowing  to  check  code  on  various  levels:  from   unit  components  to  distributed  systems   – AssisDng  humans  with  humongous  or  repeatable   tasks  (stress  or  load  tesDng,  system  integraDon   tesDng,  data  tesDng,  3rd  party  cerDficaDon  etc.)   – Debugging,  analyzing  or  replicaDng  failures   – Monitoring  running  programs  and  capturing   results  for  later  analysis   XBOSo&,  Inc.  All  Rights  Reserved.   9  
  10. 10. AUTOMATION  IS  A  CODE   •  It’s  created  by  humans:  IT  HAS  BUGS   •  It  needs  development  and  debugging  Dme   •  New/changed  funcDonality  affects  test  suite,   it  requires  maintenance  to  keep  up  to  date   •  It  needs  skills   •  It  has  upfront  costs   •  There  is  always  version  2     XBOSo&,  Inc.  All  Rights  Reserved.   10  
  11. 11. SKILLS  AND  RESPONSIBILITIES   •  Toolsmith  (A  tester?  possibly!)   •  AutomaDon  Engineer  (A  tester?  Not   necessarily!)   •  Scripter  (A  tester?  Not  necessary,  but  it’s  good   to  be  one)   •  Customer  of  automaDon  (Must  be  a  tesDng   expert  and  has  skills  in  tesDng  strategies)     XBOSo&,  Inc.  All  Rights  Reserved.   11  
  12. 12. TEST  AUTOMATION     STRATEGY   •  CriDcal  coverage     •  Low  maintenance:  has  to  be  ready  for  next  iteraDon/ release  with  low  input   •  Easy  to  run  on  demand  (one  buIon  click,  ways  to   select  tests  for  specific  scenarios,  pre-­‐selected  suites   for  different  coverage)   •  Fast  (mulDple  environments,  distributed,  overnight,   etc.)     •  TargeDng  for  ‘0  tolerance’  –  failures  are  acceptable   for  bugs  in  the  product  only   XBOSo&,  Inc.  All  Rights  Reserved.   12  
  13. 13. QUICK  POLL   WHAT  AUTOMATION  TOOLS/FRAMEWORKS   DO  YOU  USE:   A.  LICENCED  TOOLS   B.  OPEN  SOURCE   C.  HOME-­‐GROWN   D.  MIX  OF  THE  ABOVE   E.  NONE   XBOSo&,  Inc.  All  Rights  Reserved.   13  
  14. 14. SELECTING  TOOLS  AND  METHODS   •  What’s  your  long  term  and  short  term  goals?   •  Build,  buy  or  open  source?   •  What  to  look  for  in  a  tool   •  Who  will  be  supporDng  it:     – Do  you  have  specialists?     – Is  that  their  sole  responsibility?   – What  will  happen  when  your  company  has  RIF  or   specialist  leaves?     XBOSo&,  Inc.  All  Rights  Reserved.   14  
  15. 15. COMMON  MISTAKES  TO  AVOID     •  AutomaDng  new  versions  through  GUI  (it  can  work  well  with   legacy  version  for  tesDng  upgrades)   •  Hardcoding  values   •  Trying  to  test  all  aIributes  all  the  Dme.  The  test  failing  on   insignificant  aIribute  is  noise.  All  tests  failing  due  to  one   aIribute  is  more  noise  and  makes  AT  unusable.   •  Product  code  replicas   •  Get  carried  away  with  cool  features  of  less  value   •  StarDng  automaDon  too  early  or  too  late   •  Using  only  one  tool  for  everything   •  Assuming  automaDon  replaces  human  interacDons     XBOSo&,  Inc.  All  Rights  Reserved.   15  
  16. 16. MY ADVICE BASED ON MANY YEARS EXPERIENCE:   XBOSo&,  Inc.  All  Rights  Reserved.   16   •  AT  is  good  when  is  up  to  date;  it  should  be  available  when   team  needs  it   •  Cross-­‐team  integraDon:   •  build  simulators  for  majority  of  tests     •  limit  integraDon  tests  to  the  most  criDcal  paths.     •  Building  a  framework  for  your  own  product:   •  Try  to  test  each  layer  separately  instead  of  tesDng   everything  from  one  layer   •  Try  to  avoid  heavy  UI  automaDon  as  much  as  possible.   •  Take  responsibility!  As  a  tester,  demand  that  the  features   important  to  you  are  prioriDzed  by  automaDon  engineers    
  17. 17. QUICK  POLL   ARE  YOU  WINNING  WITH  YOUR  TEST   AUTOMATION:   A.  YES   B.  NO   C.  WE  DON’T  HAVE  IT   XBOSo&,  Inc.  All  Rights  Reserved.   17  
  18. 18. RISK  MITIGATION  HINTS   •  If    you  don’t  have  a  toolsmith  in    your  team,  consider   lending  from  other  teams  or  asking  programmers   •  Don’t  replace  exploratory  tesDng  with  AT  (it  needs   human  brains)   •  Don’t  replace  usability  tesDng  with  AT  (it  needs   human  emoDons)   •  Always  have  “plan  B”  in  an  emergency  such  as:     •  test  server  failure   •  DB  permission  errors     •  blocking  issue  takes  too  long  to  fix  right  before  release   •  no  electricity   XBOSo&,  Inc.  All  Rights  Reserved.   18  
  19. 19. Post your questions on Twitter and we'll answer them @XBOSoft Join us to keep updated on all our webinars, reports and whitepapers: +xbosoft We post regularly on our blog – check us out! Why not download our free Whitepapers, available here: You will receive an email with information on slides and recording. Any further queries regarding our services or ideas for future webinars please email us! Q+A