Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
TESTAR	
  
Test	
  Automa.on	
  at	
  the	
  useR	
  Interface	
  level	
  
estT *
Tanja E.J. Vos
UNIVERSIDAD POLITECNICA ...
Tes$ng	
  at	
  the	
  UI	
  Level	
  
•  UI	
  is	
  where	
  all	
  func$onality	
  comes	
  together	
  
–  Integra$on	...
State	
  of	
  the	
  art	
  in	
  UI	
  tes$ng	
  
•  Capture	
  Replay	
  
–  The	
  tool	
  captures	
  user	
  interac...
State	
  of	
  the	
  art	
  in	
  UI	
  tes$ng	
  
•  Capture	
  Replay	
  
•  Visual	
  tes.ng	
  
–  Based	
  on	
  ima...
State	
  of	
  the	
  art	
  in	
  UI	
  tes$ng	
  
•  Capture	
  Replay	
  
•  Visual	
  tes$ng	
  
	
  
•  Model-­‐based...
START
SUT
SCAN GUI + OBTAIN
WIDGET TREE
DESIRED
LENGTH?
Domain Experts
DERIVE SET OF
USER ACTIONS
SELECT
PROMISING
ACTION
...
TESTAR	
  tool	
  
READY	
  
7	
  
Set	
  path	
  the	
  SUT	
  
estT *
TESTAR	
  tool	
  
SET	
  
8	
  
Filter:	
  
	
  
1)	
  undesirable	
  ac$ons,	
  	
  
i.e.	
  closing	
  the	
  applica$o...
9	
  
GO!	
  
Automated	
  random	
  tes$ng	
  at	
  the	
  UI	
  level	
  
Oracles	
  for	
  free	
  
•  What	
  can	
  we	
  easily	
  detect?	
  
•  Crashes	
  
	
  
	
  
	
  
	
  
	
  
•  Progra...
Cheap	
  Oracles	
  
•  Cri$cal	
  message	
  boxes	
  
	
  
	
  
•  Suspicious	
  stdout	
  /	
  stderr	
  
	
  
	
  
	
 ...
More	
  sophis$ca$on	
  needs	
  work	
  
•  Ac$ons	
  
–  Ac$on	
  detec$on	
  
–  Ac$on	
  selec$on	
  
–  Some$mes	
  a...
Already	
  some	
  empirical	
  evidence	
  	
  
that	
  it	
  works!	
  
•  Lab	
  experiments	
  on	
  real	
  Microsog	...
•  Tanja	
  E.	
  J.	
  Vos	
  
•  email:	
  tvos@pros.upv.es	
  
•  skype:	
  tanja_vos	
  
•  web:	
  hJp://www.pros.upv...
Upcoming SlideShare
Loading in …5
×

Testar

666 views

Published on

Test Automation at the useR interface level

Published in: Software, Technology
  • Be the first to comment

Testar

  1. 1. TESTAR   Test  Automa.on  at  the  useR  Interface  level   estT * Tanja E.J. Vos UNIVERSIDAD POLITECNICA DE VALENCIA DEPARTMENTO DE SISTEMAS INFORMÁTICOS Y COMPUTACIÓN Centro PROS
  2. 2. Tes$ng  at  the  UI  Level   •  UI  is  where  all  func$onality  comes  together   –  Integra$on  /  System  Tes$ng   •  Most  applica$ons  have  UIs   –  Computers,  tables,  smartphones….   •  Faults  that  arise  at  UI  level  are  important   –  These  are  what  your  client  finds  -­‐>  test  from  their  perspec$ve!   •  No  need  for  source  code   –  But  if  we  have  it  even  beJer  ;-­‐)  
  3. 3. State  of  the  art  in  UI  tes$ng   •  Capture  Replay   –  The  tool  captures  user  interac$on  with  the  UI  and  records  a  script  that   can  be  automa$cally  replayed  during  regression  tes$ng   –  UI  change  (at  development  $me  &  at  run  $me)   –  Automated  regression  tests  break   –  Huge  maintenance  problem     •  Visual  Tes$ng     •  Model-­‐based  Tes$ng  
  4. 4. State  of  the  art  in  UI  tes$ng   •  Capture  Replay   •  Visual  tes.ng   –  Based  on  image  recogni$on   –  Easy  to  understand,  no  programming  skills  needed   –  Solves  most  of  maintenance  problem   –  Introduces  addi$onal  problems:   •  Performance  of  image  processing     •  False  posi$ves  and  false  nega$ves   –  the  ambiguity  associated  with  image  locators     –  imprecision  of  image  recogni$on  feeds  into  oracles     •  Model-­‐based  Tes$ng  
  5. 5. State  of  the  art  in  UI  tes$ng   •  Capture  Replay   •  Visual  tes$ng     •  Model-­‐based  tes.ng  -­‐-­‐  TESTAR     –  Based  on  automa$cally  inferred  tree  model  of  the  UI   –  Tests  sequences  are  derived  automa$cally  from  the  model   –  Executed  sequences  can  be  replayed   –  If  UI  changes  so  does  the  model/tests  -­‐>  no  maintenance  of  the  tests   –  Programming  skills  are  needed  to  define  powerful  oracles   •  It  needs  to  be  inves$gated  if  this  is  really  problem….   type: TButton ... rect: [15, 25, 65, 55] hasFocus: true enabled: false title: "Button" Window Button Text SliderMenu MI MI MI MI type: TMenuItem ... title: "File" ABC type: TButton ... rect: [15, 25, 65, 55] hasFocus: true enabled: false title: "Button" MI MI MI MI type: TMenuItem ... title: "File" ABC estT *
  6. 6. START SUT SCAN GUI + OBTAIN WIDGET TREE DESIRED LENGTH? Domain Experts DERIVE SET OF USER ACTIONS SELECT PROMISING ACTION EXECUTE ACTION DETERMINE SEQUENCE FITNESS No Yes Action Definitions Oracle STOP SUT SUT OPTIONAL INSTRUMENTATION LEARN PROMISING ACTIONS promising Actions Replayable Erroneous Sequences ORACLE FOUND POSSIBLE FAULT Yes No estT *
  7. 7. TESTAR  tool   READY   7   Set  path  the  SUT   estT *
  8. 8. TESTAR  tool   SET   8   Filter:     1)  undesirable  ac$ons,     i.e.  closing  the  applica$on     al  the  $me       2)  Undesirable  processes,  for   example  help  panes  in   acrobat,  etc…….   estT *
  9. 9. 9   GO!   Automated  random  tes$ng  at  the  UI  level  
  10. 10. Oracles  for  free   •  What  can  we  easily  detect?   •  Crashes             •  Program  freezes  
  11. 11. Cheap  Oracles   •  Cri$cal  message  boxes       •  Suspicious  stdout  /  stderr         •  Solu$on:  Regular  Expressions:   .*NullPointerExcep$on            .*|[Ee]rror|[Pp]roblem  
  12. 12. More  sophis$ca$on  needs  work   •  Ac$ons   –  Ac$on  detec$on   –  Ac$on  selec$on   –  Some$mes  a  trial/error  process   •  Random  selec$on  =  like  a  child,  just  much  faster   •  Prin$ng,  file  copying  /  moving  /  dele$ng   •  Starts  other  Processes   •  Sandboxing,  rights  management,  dedicated  user  accounts,  disallow  ac$ons   •  Oracles  that  need  programming  
  13. 13. Already  some  empirical  evidence     that  it  works!   •  Lab  experiments  on  real  Microsog  applica$ons  have  shown   that  the  technique  is  scalable,  effec$ve  and  efficient   •  Using  the  tool  in  prac$cal  classes  for  last  year  computer   science  students  has  shown  that  it  is  not  hard  to  learn  to   operate  the  tool   •  3  industrial  case  studies  have  shown  that  real  errors  can  be   found  in  an  efficient  way  
  14. 14. •  Tanja  E.  J.  Vos   •  email:  tvos@pros.upv.es   •  skype:  tanja_vos   •  web:  hJp://www.pros.upv.es/testar   •  telephone:  +34  690  917  971       Do  you  want  to  try?  

×