Your SlideShare is downloading. ×

Test Automation for Packaged Systems: Yes, You Can!

415

Published on

Today, most businesses are heavily dependent on packaged systems, sometimes called commercial off-the-shelf software, for large parts of their operation. Highly-customizable packages such as BMC’s …

Today, most businesses are heavily dependent on packaged systems, sometimes called commercial off-the-shelf software, for large parts of their operation. Highly-customizable packages such as BMC’s Remedy, Oracle's Maxim, and many others run the show at many of the world’s largest companies. While offering many features and feature options, these packages provide rich software development environments and a “configuration” that is a highly complex programming exercise. Chris Bushell explores why packaged systems, which are just as vulnerable to defects as custom-developed software, have been missing out on the many benefits of early automated testing. Chris argues that it's time for a change. Drawing from hands-on experience working with customized packaged systems, Chris explains that these packages offer ease of customization over testability and offers information on overcoming their limitations. Take away tips to reduce defects, lower testing costs, and improve time to market with your company’s packaged systems.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
415
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. W15 Test Automation 5/1/2013 3:00:00 PM Test Automation for Packaged Systems: Yes, You Can! Presented by: Chris Bushell ThoughtWorks Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Chris Bushell A lead consultant at ThoughtWorks, Chris Bushell has a wide variety of roles including software developer, architect, test automation specialist, agile coach, and trainer. Chris has more than ten years of industry experience working in finance, retail, telecoms, and mobile in the U.K., Australia, and the U.S. Passionate about software development process and technology, he is an advocate of using open source software for building simple solutions to complex problems. Chris is an active member of the local software development community wherever he's based. Contact Chris on Twitter @chrisbushelloz, by email at chris.bushell@thoughtworks.com, or converse with him face-to-face.
  • 3. Test  Automa+on   For  Packaged   Systems: Yes  You  Can! Chris  Bushell Lead  Consultant,  ThoughtWorks
  • 4. What  I  do • Developer/Tech  Lead/Architect/ Agile  coach • Custom  so8ware  development • Focus  on  building  the  right  thing • Focus  on  high  quality • Test  driven
  • 5. How  I  do  it  –  TDD/BDD
  • 6. How  I  do  it  –  Revision  Control
  • 7. How  I  do  it  –  Con+nuous   Integra+on
  • 8. How  I  do  it  –  Open  Source • Light-­‐weight,  powerful • Extensible • Excellent  community • Examples:  xUnit,  Selenium/ WebDriver,  RSpec,  Concordion,   Cucumber  etc. JUnit
  • 9. November  2011
  • 10. In  the  beginning....
  • 11. The  COTS  Con! Highly extensible software “configured” in powerful development environments, comparable in complexity, and susceptibility to defects as anything in the custom software development world
  • 12. The  land  that  +me  forgot
  • 13. Complete  lack  of  test   automa+on “Computers  are  designed  to  do  simple  repeKKve   tasks.  The  second  you  have  humans  doing   repeKKve  tasks,  all  the  computers  get  together   late  at  night  and  laugh  at  you” “Every  Kme  you  do  something  for  the  third  Kme,   automate  it.  You’ll  be  doing  it  a  million  Kmes” -­‐  Neal  Ford
  • 14. Surely  someone  must  be  doing   this  already? • Commercial  tools  exist,  but  tend  to  be  a  poor  fit  for   Agile  environments -­‐ TesKng  viewed  as  a  luxury  (an  add-­‐on) -­‐ Available  to  few  ($$$$  licensing) -­‐ Discourages  collecKve  ownership -­‐ Difficult  to  use  good  engineering  pracKces -­‐ Difficult  to  integrate  with  ConKnuous  IntegraKon   process  (lack  of  scriptability/headless  mode)
  • 15. Lessons  learned  -­‐  what  worked   and  what  didn’t
  • 16. Oracle  Siebel  CRM
  • 17. But  first,  browser  automa+on   basics • HTML  +  DOM • Selectors -­‐  Id,  class,  xpath • AutomaKon  API -­‐ e.g.  Selenium,  WebDriver,  Sahi,  WaKr  etc.
  • 18. Our  biggest  problem • AcKveX • <object> • Impervious  to   tradiKonal   automaKon   toolset
  • 19. All  is  not  lost • Not  all  parts  of  the  applicaKon  used  AcKveX • Some  controls  were  regular  HTML  (albeit  machine   generated) -­‐ WebDriver • Perhaps  we  could  use  some  kind  of  hybrid  approach   to  test  automaKon? -­‐ WebDriver  +  X,  Y  or  Z?
  • 20. Introducing  Sikuli  -­‐  automa+on   using  images
  • 21. An  example  using  Google
  • 22. package com.tw.xconf; import org.sikuli.script.FindFailed; import org.sikuli.script.Screen; import org.sikuli.script.Settings; public class Demo { } public static void main(String[] args) { Screen screen = new Screen(); try { screen.click("images/criteria.png"); screen.type("ThoughtWorks"); screen.click("images/search.png"); } catch (FindFailed e) { e.printStackTrace(); } }
  • 23. Drawbacks  of  Sikuli • Difficult  to  manage  large  numbers  of  images • Can  be  confused  when  different  regions  of  the  screen   look  the  same • Tests  easily  broken  by  small  changes  in  the  UI • ComparaKvely  slow  to  execute  
  • 24. The  story  so  far.... • Ability  to  automate  the  UI -­‐ WebDriver  for  the  regular  HTML -­‐ Sikuli  for  the  AcKveX  controls
  • 25. The  last  hurdle • How  do  we  extract  values  from  the  page  so  we  can   perform  asserKons  on  them?
  • 26. Tried  but  failed • OpKcal  Character  RecogniKon  (OCR) -­‐ Tesseract • White • AutoIt • Win32-­‐Autogui
  • 27. Plan  B •  Oracle  ApplicaKon  TesKng  Suite  (OATS) -­‐ Record  +  playback -­‐ Proprietary  test  automaKon  API
  • 28. Test  automa+on  is  soYware   development • Code  duplicaKon  is  bad,  resulKng  in  fragile  test  suites -­‐ Small  changes  can  break  many  tests  which  requires   significant  maintenance -­‐ Frequently  see  test  automaKon  abandoned  as  a   result  of  this • Record  +  playback  tools  make  it  difficult  to  introduce   abstracKons  to  remove  duplicaKon
  • 29. Oracle  Applica+on  Tes+ng  Suite • A  process  of  learning  through  experimentaKon • Generates  Java  code  under  the  covers  (for  both  Seibel   and  HTML  automaKon) -­‐ Liberate  the  API  from  tooling • Creates  executable  JWG  files  as  test  arKfacts -­‐ Understand  the  format  of  these  files  so  we  can   create  them  programmaKcally
  • 30. Gains •  Ability  to  more  easily  use  good  programming   pracKces  (abstracKons,  code  refactoring  etc) • Ability  to  integrate  test  arKfacts  with  ConKnuous   IntegraKon  process • A  maintainable  test  suite  for  an  Agile  team
  • 31. Remedy  +cket   management   system
  • 32. Great  news! • A  more  tradiKonal  webapp  (albeit  with  many  quirks) • AutomaKon  possible  with  familiar  toolset   (WebDriver,  in  our  case)  
  • 33. Example  quirk  #1:  windows
  • 34. Solu+on:  model  the  windows  a   stack • WindowManagingWebDriver • Allowed  us  to  keep  tests  focused • Also  responsible  for  maximizing  browser  windows   (quirk  1.1)
  • 35. Example  quirk  #2:  mouse   ac+ons  alone  not  enough                public  void  navigateToRelationshipsTab()  {          findElement(RELATIONSHIPS).sendKeys(Keys.ENTER);          findElement(RELATIONSHIPS).click();        }
  • 36. The  importance  of  abstrac+ons • Page  model • Domain  objects  e.g.  Incident • Builders  for  domain  objects • Page  level  gekers/sekers  e.g.  IncidentSeker,   IncidentGeker • Workflows  to  model  sequences  of  interacKons
  • 37. Data  management
  • 38. Data  management
  • 39. Solu+on:  APIs  (again) • AcKon  Request  System  (ARS)  API  allows  creaKon  of   state  e.g.  Incidents  programmaKcally • Simple  key/value  interface • Gave  us  the  ability  to  replace  the  use  of  some  of  our   Workflows  with  faster  running  API  based  Workflows • Objects  created  by  API  based  workflows  could  be   looked  up  in  the  browser
  • 40. The  story  so  far.... • Ability  to  automate  the  UI  using  WebDriver • AbstracKons  in  place  for  robust  tesKng • Ability  to  setup  state  using  fast  API • Growing  regression  test  suite
  • 41. Reverse  Con+nuous  Integra+on
  • 42. Summary • COTS  test  automaKon  is  possible! -­‐ Be  prepared  to  find  challenges/quirks  along  the  way • ConKnuous  IntegraKon  is  possible • APIs  are  your  friends  -­‐  seek  them  out • AbstracKons  are  crucial,  especially  with  machine   generated  code  and  environmental  differences
  • 43. Ques+ons
  • 44. ancisco, California 94104 6 F: +1 415 986 2964 ancisco, California 94104 6 F: +1 415 986 2964 Chris Bushell Chris Bushell 315 Montgomery Street, 16th Floor San Francisco, California 94104 T: +1 415 273 1389 C: +1 415 568 8186 F: +1 415 986 2964 E: chris.bushell@thoughtworks.com 315 Montgomery Street, 16th Floo T: +1 415 273 1389 C: +1 415 E: chris.bushell@thoughtworks.com Chris Bushell Chris Bushell 315 Montgomery Street, 16th Floor San Francisco, California 94104 T: +1 415 273 1389 C: +1 415 568 8186 F: +1 415 986 2964 E: chris.bushell@thoughtworks.com 315 Montgomery Street, 16th Floo T: +1 415 273 1389 C: +1 415 E: chris.bushell@thoughtworks.com Chris Bushell Chris Bushell Lead Consultant Email:  chris.bushell@thoughtworks.com Lead Consultant Twi`er:  @chrisbushelloz Github:  www.github.com/cbushell My  talks:  www.speakerdeck.com/cbushell LinkedIn:  www.linkedin.com/in/chrisbushell Lead Consultant Lead Consultant
  • 45. Credits • Ross  Macbeth,  J3474  :  Old  night  safe,  Belfast,  licensed   under  CreaKve  Commons,  hkp://www.geograph.ie/ photo/2261863 • Alan  Chia,  A pile of Lego blocks, of assorted colours and sizes, licensed  under  CreaKve   Commons,  hkp://en.wikipedia.org/wiki/ File:Lego_Color_Bricks.jpg

×