Rich GUI Testing Made Easy Functional Testing of Swing and JavaFX GUIs Yvonne Wang Price Guidewire Alex Ruiz Oracle tweet:...
Overall Presentation Goal Learn an easy, quick and natural way to write robust and compact tests for Java-based Rich GUIs:...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Agenda <ul><li>Why testing GUIs is difficult?   </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wr...
Why GUI testing is difficult? <ul><li>Ideally, tests must be automated, but GUIs are designed for humans </li></ul><ul><li...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Requirements for robust GUI testing <ul><li>Being able to simulate user input  </li></ul><ul><li>Having a reliable mechani...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Writing testable GUIs <ul><li>Keep the UI layer as thin as possible </li></ul><ul><li>Use a unique name for GUI components...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Top-3 must-have features in a GUI testing library <ul><li>Intuitive API for test creation </li></ul><ul><li>Concise and ea...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Introducing FEST <ul><li>Supports functional Swing and JavaFX GUI testing </li></ul><ul><li>Website:  http://fest.easytest...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Our demo application: an RSS reader <ul><li>Provides simple business logic </li></ul><ul><li>Its user interface includes: ...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
<ul><li>Failures may be due to: </li></ul><ul><ul><li>Environmental conditions </li></ul></ul><ul><ul><li>A GUI component ...
<ul><li>Failure due to environmental conditions </li></ul><ul><ul><li>Typical example: scheduled anti-virus software scan ...
<ul><li>Failure due to environmental conditions </li></ul><ul><ul><li>FEST can automatically embed a screenshot of the des...
<ul><li>Configuration is easy, short and reuses existing infrastructure </li></ul><ul><li><testng  listeners=&quot;org...S...
<ul><li>Failure due to GUI component not found </li></ul><ul><ul><li>For example, looking for a button with name “ok” in a...
<ul><li>Unable to find component using matcher org.fest.swing.core.NameMatcher[name='label', requireShowing=false]. </li><...
<ul><li>Unable to find component using matcher org.fest.swing.core.NameMatcher[name='label', requireShowing=false]. </li><...
<ul><li>Failure due to more than one GUI component satisfies a lookup condition </li></ul><ul><ul><li>For example, looking...
<ul><li>Let’s image we have a  JFrame  with two  JButtons </li></ul><ul><li>Our test tries to find a  JButton  by type and...
<ul><li>Will throw the exception: </li></ul><ul><li>org.fest.swing.exception.ComponentLookupException: Found more than one...
<ul><li>Will throw the exception: </li></ul><ul><li>org.fest.swing.exception.ComponentLookupException: Found more than one...
Agenda <ul><li>Why testing GUIs is difficult?  </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Wri...
Wait! What about JavaFX?
JavaFX 2.0: Java APIs <ul><li>Java APIs means: </li></ul><ul><ul><li>TestNG/JUnit </li></ul></ul><ul><ul><li>Code coverage...
Roadmap <ul><li>We will continue supporting Swing testing </li></ul><ul><li>Since Swing support is mature, we can focus on...
Conclusion <ul><li>GUI testing is necessary for improving the quality of our applications </li></ul><ul><li>For GUIs, func...
Upcoming SlideShare
Loading in …5
×

Rich GUI Testing: Swing and JavaFX

3,171 views

Published on

JavaOne 2010 presentation about functional testing of rich, java-based user interfaces written in Swing and JavaFX

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,171
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rich GUI Testing: Swing and JavaFX

  1. 1. Rich GUI Testing Made Easy Functional Testing of Swing and JavaFX GUIs Yvonne Wang Price Guidewire Alex Ruiz Oracle tweet: @alexRuiz
  2. 2. Overall Presentation Goal Learn an easy, quick and natural way to write robust and compact tests for Java-based Rich GUIs: Swing and JavaFX
  3. 3. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  4. 4. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  5. 5. Why GUI testing is difficult? <ul><li>Ideally, tests must be automated, but GUIs are designed for humans </li></ul><ul><li>Conventional unit testing is not suitable for testing GUIs: GUI components are usually composed of more than one class </li></ul><ul><li>The room for potential interactions with a GUI is huge </li></ul><ul><li>Conventional test coverage is not enough to cover all user interaction scenarios </li></ul>
  6. 6. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  7. 7. Requirements for robust GUI testing <ul><li>Being able to simulate user input </li></ul><ul><li>Having a reliable mechanism for finding GUI components </li></ul><ul><li>Being able to tolerate changes in </li></ul><ul><ul><li>Component position </li></ul></ul><ul><ul><li>Component size </li></ul></ul><ul><ul><li>Layout </li></ul></ul>
  8. 8. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  9. 9. Writing testable GUIs <ul><li>Keep the UI layer as thin as possible </li></ul><ul><li>Use a unique name for GUI components </li></ul><ul><li>Do not test default component behavior </li></ul><ul><li>Concentrate on testing the expected behavior of your GUI </li></ul>
  10. 10. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  11. 11. Top-3 must-have features in a GUI testing library <ul><li>Intuitive API for test creation </li></ul><ul><li>Concise and easy-to-read API </li></ul><ul><li>Ability to aid developers troubleshoot test failures </li></ul>
  12. 12. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  13. 13. Introducing FEST <ul><li>Supports functional Swing and JavaFX GUI testing </li></ul><ul><li>Website: http://fest.easytesting.org </li></ul><ul><li>Its API provides a fluent interface </li></ul><ul><li>Supports component lookup by name, by type and by custom search criteria </li></ul><ul><li>Open Source project (Apache 2.0 license) </li></ul><ul><li>Supports both TestNG and JUnit </li></ul><ul><li>Simplifies troubleshooting GUI test failures </li></ul>
  14. 14. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  15. 15. Our demo application: an RSS reader <ul><li>Provides simple business logic </li></ul><ul><li>Its user interface includes: </li></ul><ul><ul><li>Complex GUI components like JTree and JTable </li></ul></ul><ul><ul><li>Time-consuming tasks: </li></ul></ul><ul><ul><ul><li>Database access </li></ul></ul></ul><ul><ul><ul><li>Retrieval of web feeds from the Internet </li></ul></ul></ul><ul><ul><ul><li>Drag ‘n drop </li></ul></ul></ul>
  16. 16. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  17. 17. <ul><li>Failures may be due to: </li></ul><ul><ul><li>Environmental conditions </li></ul></ul><ul><ul><li>A GUI component could not be found </li></ul></ul><ul><ul><li>More than one GUI component satisfied the given search criteria </li></ul></ul><ul><ul><li>Programming defect </li></ul></ul>Troubleshooting failures
  18. 18. <ul><li>Failure due to environmental conditions </li></ul><ul><ul><li>Typical example: scheduled anti-virus software scan starts in the middle of our test session </li></ul></ul><ul><ul><li>A screenshot of the desktop at the moment of failure can help us determine if an environmental condition was responsible for the failure </li></ul></ul>Troubleshooting failures
  19. 19. <ul><li>Failure due to environmental conditions </li></ul><ul><ul><li>FEST can automatically embed a screenshot of the desktop in a HTML test report (TestNG or JUnit)! </li></ul></ul>Troubleshooting failures
  20. 20. <ul><li>Configuration is easy, short and reuses existing infrastructure </li></ul><ul><li><testng listeners=&quot;org...ScreenshotOnFailureListener&quot; </li></ul><ul><li>outputDir=&quot;${target.test.results.dir}&quot;> </li></ul><ul><li><classfileset dir=&quot;${target.test.classes.dir}&quot; </li></ul><ul><li>includes=&quot;**/*Test.class&quot; /> </li></ul><ul><li><classpath location=&quot;${target.test.classes.dir}&quot; /> </li></ul><ul><li><classpath location=&quot;${target.classes.dir}&quot; /> </li></ul><ul><li><classpath refid=&quot;test.classpath&quot; /> </li></ul><ul><li></testng> </li></ul>Troubleshooting failures
  21. 21. <ul><li>Failure due to GUI component not found </li></ul><ul><ul><li>For example, looking for a button with name “ok” in a dialog that does not contain such button! </li></ul></ul><ul><ul><li>Having access to the current component hierarchy can help us figure out why a component could not be found </li></ul></ul><ul><ul><li>FEST includes the current component hierarchy when throwing a ComponentLookupException </li></ul></ul>Troubleshooting failures
  22. 22. <ul><li>Unable to find component using matcher org.fest.swing.core.NameMatcher[name='label', requireShowing=false]. </li></ul>Troubleshooting failures
  23. 23. <ul><li>Unable to find component using matcher org.fest.swing.core.NameMatcher[name='label', requireShowing=false]. </li></ul><ul><li>Component hierarchy: </li></ul><ul><li>org.fest.test.MainWindow[name='frame0', title='BasicComponentFinderTest', enabled=true, visible=true, showing=true] </li></ul><ul><li>javax.swing.JRootPane[] </li></ul><ul><li>javax.swing.JPanel[name='null.glassPane'] </li></ul><ul><li>javax.swing.JPanel[name='null.contentPane'] </li></ul><ul><li>javax.swing.JButton[name='button', text='A Button', enabled=true, visible=true, showing=true] </li></ul>Troubleshooting failures
  24. 24. <ul><li>Failure due to more than one GUI component satisfies a lookup condition </li></ul><ul><ul><li>For example, looking up a button with name “ok” in a dialog </li></ul></ul><ul><ul><li>We have accidentally named two buttons with the same name! </li></ul></ul><ul><ul><li>Once again, FEST assists us by providing the list of found components in the thrown ComponentLookupException </li></ul></ul>Troubleshooting failures
  25. 25. <ul><li>Let’s image we have a JFrame with two JButtons </li></ul><ul><li>Our test tries to find a JButton by type and click it: </li></ul><ul><li>FrameFixture frame = new FrameFixture(new MyFrame()); </li></ul><ul><li>frame.show(); </li></ul><ul><li>frame.button().click(); </li></ul>Troubleshooting failures
  26. 26. <ul><li>Will throw the exception: </li></ul><ul><li>org.fest.swing.exception.ComponentLookupException: Found more than one component using matcher org.fest.swing.core.TypeMatcher[type=javax.swing.JButton, requireShowing=false]. </li></ul>Troubleshooting failures
  27. 27. <ul><li>Will throw the exception: </li></ul><ul><li>org.fest.swing.exception.ComponentLookupException: Found more than one component using matcher org.fest.swing.core.TypeMatcher[type=javax.swing.JButton, requireShowing=false]. </li></ul><ul><li>Found: </li></ul><ul><li>javax.swing.JButton[name='first', text='First Button', enabled=true] </li></ul><ul><li>javax.swing.JButton[name='second', text='Second Button', enabled=true] </li></ul>Troubleshooting failures
  28. 28. Agenda <ul><li>Why testing GUIs is difficult? </li></ul><ul><li>Requirements for robust GUI testing </li></ul><ul><li>Writing testable GUIs </li></ul><ul><li>Top-3 must-have features in a GUI testing library </li></ul><ul><li>Introducing FEST, an open source library for GUI testing </li></ul><ul><li>Coding Demos </li></ul><ul><li>Troubleshooting GUI test failures </li></ul><ul><li>Conclusion </li></ul>
  29. 29. Wait! What about JavaFX?
  30. 30. JavaFX 2.0: Java APIs <ul><li>Java APIs means: </li></ul><ul><ul><li>TestNG/JUnit </li></ul></ul><ul><ul><li>Code coverage tools (Cobertura, Clover) </li></ul></ul><ul><ul><li>Code quality tools (FindBugs, Checkstyle) </li></ul></ul><ul><li>Faster development of GUI functional testing tools </li></ul>
  31. 31. Roadmap <ul><li>We will continue supporting Swing testing </li></ul><ul><li>Since Swing support is mature, we can focus on JavaFX testing </li></ul><ul><li>Release date? Unknown </li></ul><ul><li>Please remember: we work on our own spare time </li></ul>
  32. 32. Conclusion <ul><li>GUI testing is necessary for improving the quality of our applications </li></ul><ul><li>For GUIs, functional testing is more suitable than unit testing </li></ul><ul><li>Successful testing strategy: </li></ul><ul><li>Create testable GUIs </li></ul><ul><li>Write robust GUI tests </li></ul><ul><li>Pick the right testing tool for your needs (e.g. FEST) </li></ul><ul><li>Happy testing! </li></ul>

×