The document discusses issues with maintaining automated tests when a website undergoes frequent changes. It proposes generating page object models on every build to account for changes as they happen. This avoids failures from id, type, or navigation path changes. Developers must fix type changes before check-in. With regeneration, tests absorb insignificant changes and the test pass rate improves from identifying issues within 24 hours without noise. The approach provides a model to generate additional tests around standards, errors, forms and security. While reported defects did not decrease, the solution allowed increased velocity and faster feature addition.

1. Page Object
Generation
Lose the maintenance,
increase the productivity

2. The Problem
 Failuresguide our work
 The product is ever-changing
 Developers do not communicate
changes proactively (or at all)
 40% of our time is spent accounting for
these changes
 Who can analyze 300 test failures every
night?

3. What Changes?
 Htmlid (i.e. “Locator”)
 Element type
 Major navigation path
Each will break most automated tests

4. Html id
 Ifthe id changes (e.g. to signinField), the
test will break
 The element will no longer exist

5. Element Type
 Thelink is changed from a Link to Button
 WebDriver will still look for a link

6. Navigation Path
 Pages are added
 Popups are introduced
 Buttons and forms are split up across
pages
 UI Look and feel “reset”
 WebDriver scripts don’t stand a chance—
you just start over

7. The Change Cycle

8. Wish List for a Solution
 Build in a mechanism for change
communication
 Account for the changes AS THEY
HAPPEN, not reactively
 Tighter integration with development
 Pass rate needs to be 97% or better, with
all failures accounted for within 24 hours

9. Page Object Generation
 Generate the pages on every build
 Web Controls are mapped to specific
WebElement types
 If the type of an object changes in a way
that breaks automation, the whole build
fails

10. Let’s See It!

11. What you get
A page, containing every element in the
UI
 Each element is aware of its Tab
 Including every Rich WebElement
 A Fields object, usable by those writing
Test Scripts

12. The New Change Cycle

13. The New Change Cycle
•Significant underlying type (Text Input to Radio Button)
•Insignificant underlying type (Button to Link)
Developer
Change •id changes for localization
•Compiler flags type change—breakage
•Compiler ignores type change (Both are IClickable)
Regenerate
Model •Page auto-updates id change—no breakage
•Tester doesn’t know—fixed before dev check-in
•Tester doesn’t know—Framework “absorbs” the change
Test •Tester doesn’t know—id is s String, stored in one place

14. When change happens
 Html id changes
 Who cares?
 Generation process normalizes names
 Unit test enforces uniqueness
 Element type changes
 Developer fixes prior to check-in
 Navigation path changes
 More rare, but failures show up within 24 hours
 Without all the noise, issues are easier to spot,
analyze, and fix

15. Where do you go now?
 You have a model of your system
 Use it to “analyze itself”
 The all-table test
 Role-based security tests
 Algorithms to
 Iterate through every page looking for
standards compliance
 500 errors
 Forms
 Security issues (code injection, XSS etc)

16. Bottom Line
 We found 7 major regressions in 2008, the
last year of the “old” platform
 12 in 2009 with >500 test cases
 20 in 2010 with >1100 test cases
 More than 50 in 2011 with 1700 test cases
 Customer reported defects did NOT go
down, because…
 Velocity increased so much, features
were added at a much faster clip

17. Questions?