Practical Sikuli: using screenshots for GUI automation and testing
Upcoming SlideShare
Loading in...5
×
 

Practical Sikuli: using screenshots for GUI automation and testing

on

  • 77,320 views

This's a Sikuli tutorial I gave to the Megason Lab at Harvard Medical School.

This's a Sikuli tutorial I gave to the Megason Lab at Harvard Medical School.

Statistics

Views

Total Views
77,320
Views on SlideShare
76,951
Embed Views
369

Actions

Likes
38
Downloads
1,419
Comments
9

13 Embeds 369

http://bit3725.github.io 187
http://bit3725.github.com 71
http://jotarun.posterous.com 67
http://wiki.synap.co.kr 24
https://jotarun.posterous.com 4
http://iga1976.wordpress.com 3
http://www.verious.com 3
http://www.docshut.com 2
https://twitter.com 2
https://twimg0-a.akamaihd.net 2
http://webcache.googleusercontent.com 2
http://localhost 1
http://posterous.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

15 of 9 Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Recently published a test automation framework that is powered by Sikuli. Check it out here:
    https://github.com/smysnk/sikuli-framework
    Are you sure you want to
    Your message goes here
    Processing…
  • Interesting, I will try it later
    Are you sure you want to
    Your message goes here
    Processing…
  • Interesting tool.. liked it so far
    Are you sure you want to
    Your message goes here
    Processing…
  • @Arye probably you may not set the path for SIKULI &python in environmental variables...
    Are you sure you want to
    Your message goes here
    Processing…
  • why it is not working for me? this is the error:
    [error] Error message: Traceback (most recent call last):
    File 'C:\DOCUME~1\ANUDEL~1\LOCALS~1\Temp\sikuli-tmp6054047037597739699.py', line 2, in
    while not r.exits('1325492826226.png'):
    AttributeError: 'Region' object has no attribute 'exits'
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Practical Sikuli: using screenshots for GUI automation and testing Presentation Transcript

  • 1. PRACTICAL SIKULIUSING SCREENSHOTS FOR GUI AUTOMATION AND TESTING Tsung-Hsiang (Sean) Chang MIT CSAIL Collaborators: Tom Yeh, Rob Miller 1
  • 2. WHAT IS SIKULI? 2
  • 3. WHAT IS SIKULI? 2
  • 4. 2 3 13
  • 5. Low-level API?Mac’s system design? 4
  • 6. EMPTY THE TRASH CAN 5
  • 7. OUTLINE• What Sikuli is• How Sikuli works• Sikuli for GUI automation• Sikuli for GUI testing• Conclusions 6
  • 8. How Sikuli Works 7
  • 9. TEMPLATE MATCHINGclick(                            ) 8
  • 10. TEMPLATE MATCHINGclick(                            ) 8
  • 11. SYSTEM DESIGN 9
  • 12. DEMO 1:EMPTY TRASH CAN 2 3 1 10
  • 13. REGION AND MATCH Regionmatch = region.find( or “text pattern”) 11
  • 14. REGION• basic attributes: x, y, width, height• Match is a Region• Screen is also a Region• chained methods: Screen(0).find(W).click(X) 12
  • 15. KEY METHODS OF REGION• find, findAll, exists, wait, waitVanish• Mouse Actions click, doubleClick, rightClick, hover, dragDrop mouseDown, mouseUp• Keyboard Actions type(text), paste(text) keyDown, keyUp 13
  • 16. EX2: FACEBOOK STALKER 14
  • 17. EX2: FACEBOOK STALKER 14
  • 18. RESTRICT SEARCH REGION 15
  • 19. But the Region constant is bad 16
  • 20. APP CLASS• firefox = App.open(“Firefox”)• firefox.focus(), firefox.close()• firefox.window([n]) • returns the Region of Firefox’s n-th window 17
  • 21. FIND IN CERTAIN APP 18
  • 22. EX3: MUTE SPEAKERS 19
  • 23. EX3: MUTE SPEAKERS 19
  • 24. SPATIAL OPERATORS 20
  • 25. SPATIAL OPERATORS 20
  • 26. SPATIAL OPERATORS (2) 21
  • 27. TEXT MATCHING AND RECOGNITION• click(“OK”), find(“Cancel”)• region.text() - read the text in a region 22
  • 28. VISUAL-EVENT-DRIVEN PROGRAMMING• observes visual events in any Region • Region.onAppear(Pattern, handler) • Region.onVanish(Pattern, handler) • Region.onChange(Pattern, handler) 23
  • 29. VISUAL-EVENT-DRIVEN PROGRAMMING 24
  • 30. VISUAL-EVENT-DRIVEN PROGRAMMING 24
  • 31. VISUAL DICTIONARY 25
  • 32. GUITESTING 26
  • 33. VISUAL ASSERTION• make sure the appearance of visual feedback 27
  • 34. VISUAL ASSERTION• make sure the appearance of visual feedback 27
  • 35. JUNIT INTEGRATION 28
  • 36. SIKULI IN COMMAND LINEusage:Sikuli-­‐IDE  [-­‐-­‐args  <arguments>]  [-­‐h]  [-­‐r  <sikuli-­‐file>]  [-­‐s]  [-­‐t  <sikuli-­‐test-­‐case>]  -­‐-­‐args  <arguments>                          specify  the  arguments  passed  to  Jythons  sys.argv  -­‐h,-­‐-­‐help                                            print  this  help  message  -­‐r,-­‐-­‐run  <sikuli-­‐file>                  run  .sikuli  or  .skl  file  -­‐s,-­‐-­‐stderr                                        print  runtime  errors  to  stderr  instead  of  popping  up  a  message  box  -­‐t,-­‐-­‐test  <sikuli-­‐test-­‐case>      run  .sikuli  as  a  unit  test  case  with  junits  text  UI  runner 29
  • 37. CONCLUSIONS 30
  • 38. PLATFORM INDEPENDENCE• Works on ANY GUI that can be displayed on Windows/Linux/Mac • Virtual machines • Remote desktop • Mobile simulators: Android, iPhone • Web: Flash, HTML+Javascript 31
  • 39. PROGRAM AGAINST UISikuli programs are written against the user interfaceinstead of an API UI: visible, familiar, always exists API: faster, probably more stable 32
  • 40. FUTURE WORK• fast and accurate OCR on screen• Accessibility API integration 33
  • 41. Thanks! http://sikuli.org or, just google “sikuli” New Sikuli X 1.0-rc2 released. Follow us on twitter @sikuliThanks to our funding sources: NSF and Quanta Computer (as a part of TParty project.) 34