Alexandre Iline Rit 2010 Java Fxui


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Note that in reality all the verifications should be done throwg waiting
  • Effectiveness is estimated under assumtion that the testing is thoruogh and there are many tests
  • This is a well accepeted practice nowadays. However it is not really possible without testing. And it is not really possible with manual testing as the turnaround is too long to rely on.
  • This is a well accepeted practice nowadays. However it is not really possible without testing. And it is not really possible with manual testing as the turnaround is too long to rely on.
  • If we get to it, I will be howing figures which explain why do we think so
  • It is not a toy
  • ditto
  • The values of course depend on how the fast product is changing The estimations are for NetBeans – we went throughthis stage Now would happen if the combobox is replaced by color chooser page up test would fail, 'cause it's looking for combobox page down Most importantly ... all tests would fail!
  • Obvious solution
  • Alexandre Iline Rit 2010 Java Fxui

    1. 1. Testing JavaFX UI Tips from JavaFX quality team Alexandre (Shura) Iline Java and JavaFX quality architect Oracle
    2. 2. Agenda <ul><li>UI testing overview
    3. 3. Test automation approaches
    4. 4. Jemmy library
    5. 5. Test automation effectiveness
    6. 6. Test library design approaches </li></ul>
    7. 7. UI testing … by Wikipedia «GUI software testing is the process of testing a product that uses a graphical user interface, to ensure it meets its written specifications.»
    8. 8. UI testing … most often ... «Checking whether usage of a product UI leads to results expected by the the person who performs testing »
    9. 9. <ul><li>Start text editor
    10. 10. Push «File/Open»
    11. 11. Verify file chooser directory
    12. 12. Select some file
    13. 13. Verify editor area content
    14. 14. Verify application title
    15. 15. Verify buttons availabilities
    16. 16. .... </li></ul>
    17. 17. UI Testing Manual Automated Initial step Design test specification. Create tests. Establish regular runs. Ongoing Click, click, click Analize results File bugs Change the specification Change the tests Qualification High, low for test executors High Effectiveness Low High Other Inexpensive to start Continuous quality monitoring , inexpensive to reuse Fun Bo-o-o-ring Much like programming
    18. 18. Continuous build No Yes. Development Success
    19. 19. Continuous build with testing No No Yes Yes. = Compilation successful Success Passed
    20. 20. Automation approach Record && Replay Coding Test creation Inexpensive* Usually just repeating manual test in special environment Must be accompanied by other means Expensive* Consists of programming Test execution Does not depend on approach Test maintanance Higher (in most cases)* Very much depends on test format Lower* Depends on principles of building test library Test analysis Does not depend on approach (*) Much more information closer to the end of this presentation
    21. 21. Jemmy
    22. 22. Jemmy v2 Started as a tool to tests TeamWare UI (1999) Used for NetBeans extensions (2000) Official test tool for NetBeans (2001) Open-source (2001) Jemmy v3 Started in (2008) as a proof of concept experiemnt Extended to support JavaFX (2009) Opensource with support of JavaFX 1.2 (2009) Developed in close-source for 1.3 since then
    23. 23. Jemmy usages <ul><li>V2 (Swing/AWT) </li><ul><li>Internal: Swing, NetBeans, Glassfish
    24. 24. External: </li><ul><li>Used by: Boeing (x2),,,,,,,
    25. 25. Evaluated by:,,,,,, </li></ul></ul><li>V3 (FX) </li><ul><li>Internal: JavaFX, Authoring Tool, JavaStore </li></ul></ul>
    26. 26. Demo... Finally!
    27. 27. Jemmy v3 Covered in this presentation JavaFX AWT/Swing SWT Scenegraph LCDUI JemmyCore Interfaces AWT robot FX robot Hierarchy Input Scene, node containers Scroller, slider, list, text, ... Generic interfaces
    28. 28. UI test Find Do Verify Pass Pass Pass Fail Fail Fail Failure analysis Find next control To perform operation On in Verify that expected State reached Perform necessary actions
    29. 29. Lookup <ul><li>By ID </li><ul><li>Easiest but may not be possible
    30. 30. «open_file_btn» </li></ul><li>By type </li><ul><li>Most common
    31. 31. Button, text, combo box, etc </li></ul><li>By index </li><ul><li>Unavoidable
    32. 32. 2 nd button with text «browse» </li></ul></ul><ul><li>By toString() </li><ul><li>Sucks
    33. 33. projectmanager.Main$Main$Script$1Scene$ObjLit$20@4d16318b </li></ul><li>By text, tooltip, associated label </li><ul><li>Best, if possible
    34. 34. Button with text «browse» </li></ul></ul>
    35. 35. UI could be very complicated
    36. 36. UI could be very complicated
    37. 37. UI could be very complicated
    38. 38. Lookup demo
    39. 39. Interfaces Interface Control types Description Mouse, Keyboard, Drag Enything Low level input Parent Containers, list (for its content) Something you could look within Selectable Toggle button, radio button, combobox, check box, lists, etc A control which provides limited number of choices CaretOwner Text box, scroll bar, slider A control which has a number value which changes within some range Text Text box Editable text container
    40. 40. Interfaces demo
    41. 41. Verifications UI feedback Non UI feedback Dialog displayed Test changed Image updated Progress bar changed position File created Database updated Sunset happened :)
    42. 42. Waiting <ul><li>UI is a multy-thread environment </li></ul>Things happen in background Test code is in another thread <ul><li>Not much could be really verified
    43. 43. Everything should be waited for </li></ul>
    44. 44. Verification demo
    45. 45. Test automation effectiveness
    46. 46. E A – automation effectiveness N R and N C are characteristics for a product. T M is a characteristic of a test suite T D and T S depend on test automation approach Smaller T D and T S - higher the E A . T D + * T S N R T M * N R N C * E A = N C *
    47. 47. Assumptions: T M = 1 engineer*week T S = 0.1 * T M T D = 5 * T M N R = 8
    48. 48. T d or T s – what to minimize T S - if (N C * N R ) is big Multi-platform Compatibility with external products (servers, browsers, ...) Long-living T D - if (N C * N R ) is small Proof of concept Preview
    49. 49. Tests fail every now and then ... … because the tested UI is changed
    50. 50. Tests fail every now and then ... Ah! And also because errors are made ...
    51. 51. End2End demo
    52. 52. T s continuends Time spent on What to do %% of time Allocating failures Use test harness 1% - 5% Anderstandint the failure reason Use test logging, save images, save UI state 10% - 80% Fix the tests Move common code to the library ( * ) 90% - 10% (*) The only way it is different from programming is that there are a lot of tests. More on this later.
    53. 53. Harness TestNG
    54. 54. Understanding failure Logging UI operations Going to push ... button Selecting ... menu UI responces Found list view: … UI hierarchy Scene «…» Group «…» Button «…» UI states Images
    55. 55. Demo Jemmy logging functionality
    56. 56. Fix the tests One UI, many-many tests. Do as few test code changes as possible Ideally … one change in test code for one change in UI code.
    57. 57. Decrease maintenance cost <ul><li>Implement reusable operations in a library </li></ul>openFile(String filename) <ul><li>Decrease implementation dependencies </li></ul>lookup(Button.class, new ByText<Button>( «Low» )) <ul><li>Increasing stability </li></ul>.wrap(0).mouse().click(); Thread.sleep(1000); ...
    58. 58. Demo Refining test.
    59. 59. Library design approaches
    60. 60. Remember the formula? T D + * T S N R T M * N R N C * E A = N C * E A – automation effectiveness To be used for every particular product. N R and N C are unique for a product. T M is a characteristic of a test suite. Smaller T D and T S - higher the E A . Coefficient depend on the way you write your tests
    61. 61. T S mainly consists of time for test modification. When product changes, tests need to be changed accordingly. Many tests! Hundreds. “ Less changes of test code per a change in the product UI.” Ideally ... no more than one. But ... how? You are the coders – you know: Move code to test library.
    62. 62. Application domain model That's ... car catalog of some sort
    63. 63. Application UI
    64. 64. Coordinates <ul><li>click(134,32) //selects some record
    65. 65. click(215,122) //hits “Properties”
    66. 66. sleep(5) //sleeps to let dialog be painted
    67. 67. click(64,182) //expands color combo
    68. 68. click(235,182) //selects Gray
    69. 69. click(235,212) //hit OK </li></ul>T d ~= 1.1 * T m , T s ~= 1 * T m Never tried, but ...
    70. 70. Widgets <ul><li>Find “Car records” frame
    71. 71. Find table
    72. 72. Select “1abc234” cell
    73. 73. Push “Properties” button
    74. 74. Wait for “1abc234” dialog
    75. 75. Select “Gray” color in combo box
    76. 76. Push “OK” </li></ul>
    77. 77. Widgets or coordinates T D ~= 3 * T M , T S ~= .5 * T M
    78. 78. UI Primitives <ul><li>Find car list frame </li></ul>CarListFrame list = new CarListFrame() <ul><li>Open properties dialog for car “1abc234” </li></ul>CarDialog propDialog = list.carProperties(“1abc234”); <ul><li>Set color to gray </li></ul>propDialog.setColor(Color.GRAY); <ul><li>Apply changes </li></ul>propDialog.ok();
    79. 79. T D ~= 5 * T M , T S ~= .2 * T M
    80. 80. Domain model <ul><li>Set color to gray for a car “1abc234” </li></ul>new CarRecord(“1abc234”). setColor(Color.GRAY); Underneath the cover, CarRecord class does all described earlier
    81. 81. T D ~= 7.5 * T M , T S ~= .05 * T M
    82. 82. T d and T s together
    83. 83. T D and T S for N C =3, N R =8, T M =1
    84. 84. E A for N C =3, N R =8
    85. 85. Testing JavaFX UI Tips from JavaFX quality team Alexandre (Shura) Iline Java and JavaFX quality architect Oracle
    86. 86. Links <ul><li> </li></ul>