Alexandre.iline rit 2010 java_fxui_extraPresentation Transcript
Testing JavaFX UI Tips from JavaFX quality team Alexandre (Shura) Iline Java and JavaFX quality architect Oracle
The following is intended to outline our general product direction. It is intended for information purposes, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle's products remains at the sole discretion of Oracle.
UI testing overview
Test automation approaches
Test automation effectiveness
Test library design approaches
UI testing … by Wikipedia «GUI software testing is the process of testing a product that uses a graphical user interface, to ensure it meets its written specifications.»
UI testing … most often ... «Checking whether usage of a product UI leads to results expected by the the person who performs testing »
Start text editor
Verify file chooser directory
Select some file
Verify editor area content
Verify application title
Verify buttons availabilities
UI Testing Manual Automated Initial step Design test specification. Create tests. Establish regular runs. Ongoing Click, click, click Analyze results File bugs Change the specification Change the tests Qualification High, low for test executors High Effectiveness Low High Other Inexpensive to start Continuous quality monitoring , inexpensive to reuse Fun Bo-o-o-ring Much like programming
Continuous build No Yes. Development Success
Continuous build with testing No No Yes Yes. = Compilation successful Success Passed
Automation approach Record && Replay Coding Test creation Inexpensive* Usually just repeating manual test in special environment Must be accompanied by other means Expensive* Consists of programming Test execution Does not depend on approach Test maintenance Higher (in most cases)* Very much depends on test format Lower* Depends on principles of building test library Test analysis Does not depend on approach (*) Much more information closer to the end of this presentation
Jemmy v2 Started as a tool to tests TeamWare UI (1999) Used for NetBeans extensions (2000) Official test tool for NetBeans (2001) Open-source (2001) Jemmy v3 Started in (2008) as a proof of concept experiment Extended to support JavaFX (2009) Open-source with support of JavaFX 1.2 (2009) Developed in close-source for 1.3 since then
Internal: Swing, NetBeans, Glassfish
Used by: Boeing (x2), agfa.com, compuware.com, attix5.com, exgate.tek.com, globo.com, gmx.de, non.agilent.com
Interfaces Interface Control types Description Mouse, Keyboard, Drag Anything Low level input Parent Containers, list (for its content) Something you could look within Selectable Toggle button, radio button, combo box , check box, lists, etc A control which provides limited number of choices CaretOwner Text box, scroll bar, slider A control which has a number value which changes within some range Text Text box Editable text container
Verifications UI feedback Non UI feedback Dialog displayed Text changed Image updated Progress bar changed position File created Database updated Sunset happened :)
UI is a multi -thread environment
Things happen in background Test code is in another thread
Not much could be really verified
Everything should be waited for
Test automation effectiveness
E A – automation effectiveness N R and N C are characteristics for a product. T M is a characteristic of a test suite T D and T S depend on test automation approach Smaller T D and T S - higher the E A . T D + * T S N R T M * N R N C * E A = N C *
Assumptions: T M = 1 engineer*week T S = 0.1 * T M T D = 5 * T M N R = 8
T d or T s – what to minimize T S - if (N C * N R ) is big Multi-platform Compatibility with external products (servers, browsers, ...) Long-living T D - if (N C * N R ) is small Proof of concept Preview
Tests fail every now and then ... … because the tested UI is changed
Tests fail every now and then ... Ah! And also because errors are made ...
T s components Time spent on What to do %% of time Allocating failures Use test harness 1% - 5% Understanding the failure reason Use test logging, save images, save UI state 10% - 80% Fix the tests Move common code to the library ( * ) 90% - 10% (*) The only way it is different from programming is that there are a lot of tests. More on this later.
Understanding failure Logging UI operations Going to push ... button Selecting ... menu UI responses Found list view: … UI hierarchy Scene «…» Group «…» Button «…» UI states Images
Demo Jemmy logging functionality
Fix the tests One UI, many-many tests. Do as few test code changes as possible Ideally … one change in test code for one change in UI code.
Decrease maintenance cost
Implement reusable operations in a library
Decrease implementation dependencies
lookup(Button.class, new ByText<Button>( «Low» ))
.wrap(0).mouse().click(); Thread.sleep(1000); ...
Demo Refining test.
Decrease implementation cost This is not the main target, but still …
Use UI exploration tools
Library design approaches
Remember the formula? T D + * T S N R T M * N R N C * E A = N C * E A – automation effectiveness To be used for every particular product. N R and N C are unique for a product. T M is a characteristic of a test suite. Smaller T D and T S - higher the E A . Coefficient depend on the way you write your tests
T S mainly consists of time for test modification. When product changes, tests need to be changed accordingly. Many tests! Hundreds. “ Less changes of test code per a change in the product UI.” Ideally ... no more than one. But ... how? You are the coders – you know: Move code to test library.
Application domain model That's ... car catalog of some sort
click(134,32) //selects some record
click(215,122) //hits “Properties”
sleep(5) //sleeps to let dialog be painted
click(64,182) //expands color combo
click(235,182) //selects Gray
click(235,212) //hit OK
T d ~= 1.1 * T m , T s ~= 1 * T m Never tried, but ...
Find “Car records” frame
Select “1abc234” cell
Push “Properties” button
Wait for “1abc234” dialog
Select “Gray” color in combo box
Widgets or coordinates T D ~= 3 * T M , T S ~= .5 * T M