We shall explore the concept of web application testing through the verification of declarative constraints on the contents of its user interface --that is, DOM, CSS and behavioural properties. It turns out that a lot of non GUI (e.g. functional, security) problems can also be verified by observing the application's GUI.

1. Testing Web Applications Through
User Interface Constraints
Sylvain Hallé
Université du Québec à Chicoutimi
CRSNG
NSERC
November 3rd, 2015

2. Exhibit A
Overlapping elements

8. Box has correct
size, if user not
logged in

9. Exhibit B
Elements extending outside
their container

11. Exhibit C
Incorrect z-index

14. Layout bugs don't
impact the application's
functionality...
Wait a minute...

15. Exhibit D
Tone on tone text

17. Exhibit E
Too much escaping

20. Exhibit F
Inaccessible elements

21. Can't scroll to
the left part of
the popup

22. Can't scroll to get the
missing buttons

23. Exhibit G
Form inconsistencies

24. Box pre-filled with 6 characters,
but has maxlength="5"

25. Textbox disappears if window
below specific width (scrolling
won't help)

26. Layout bugs can't probe
the application's
state...
Wait a minute...

27. Close button has no effect...
stuck in the pop-up!

31. Layout bugs are limited
to a single snapshot
of a page...
Wait a minute...

32. Incorrect search results
https://youtu.be/zCg537MZ6Ww
Crazy right click
https://youtu.be/iw7x3H51TFo
Incorrect button state
https://youtu.be/NGbl4vP350s
Summ

33. Experience reveals that many bugs
manifest themselves through their UI
They can be detected by observing
elements of the UI...
...in a single snapshot, or across
a sequence of snapshots

34. A language for expressing "high"-level
oracles over web UIs
- DOM + CSS attributes
- First-order quantification
- Temporal operators
1.

35. For each $x in $(css-selector)
something about $x.
There exists $x in $(css-selector)
such that
something about $x.

36. $x's prop is greater than
$y's prop
$x's height
$x's text
$x's color
$x's prop matches "a string"
...and most DOM/CSS
attributes
...and most common
functions

37. something And something else
Always something
Eventually something
something Until something else
The next time something,
then something else
...in other words, propositional logic + LTL

38. We say that pattern
When something.
Example:
We say that $x is wider than $y
When
$x's width is greater
than $y's width.
User-defined predicates to improve
legibility and create "mini-DSLs"

39. We say that I click on Go when (
There exists $b in $(button) such that (
($b’s text is "Go")
And
($b’s event is "mouseup"))).
Always (
If (I click on Go) Then (
There exists $x in $(.value) such that (
The next time (I click on Go)
Then (
There exists $y in $(.value) such that (
$x’s text equals $y’s text ))))).

40. $(document).mouseUp(function(event) {
var e = arguments.callee;
if ($(event.target).text() === "Go") {
var current_values = [];
$(".value").each(
current_values.push($(this).text());
);
if (e.lastValues !== undefined) {
var found = false;
for (var v in current_values) {
if ($.inArray(v, e.lastValues)) {
found = true;
break;
}
}
}
if (!found)
console.log("Error");
e.lastValues = current_values;}
};);
The same in jQuery

41. An architecture to instrument a web app
and report relevant UI attributes to an
interpreter
2.

42. Server
Application under test
<script ...
"getprobe?uid=39f2ac8">
Probe
creationRequest for probe code
Probe
interaction
Probe code
Probe (JS object)
Probe status report
Probe instructions (optional)
Probe log
Probe repository
Probe
retrieval
Developer
Probe ID
Dashboard
(analytics)
Developer
1
234
5
6
7
8
9
12
10
11
13 16
14
15

43. ... $(ul li) ...
... 's width ...
... $(p#abc) ...
... 's color ...
The JavaScript probe is selective:
it only sends back the attributes
mentioned in the property
only for the elements matching one of
the selectors

44. A mechanism for explaining the violation
of a property when found
3.

47. Producing a witness for a violation does not
require to be coded in any way
It is just a by-product of evaluating the
logical expression
It works for temporal formulas
too!

48. See it in action
https://www.youtu.be/90YitGRRx2w
BEST
TOOL
Hallé, Bergeron, Guérin, Le Breton.
Testing Web Applications Through Layout
Constraints. Proc ICST 2015.
https://bitbucket.org/sylvainhalle/cornipickle

49. cIcI My 2 cents...
"Most browsers"
{ {Desktop
market share
Non-desktop
market share
75%
25%

50. cIcI My 2 cents...
In UI-based bugs, the browser matters
Don't take design choices that limit you from
the start (Selenium, any browser plugin)
If the app runs in browser X, so should your
testing tool
This may mean a lot of work ahead!

51. cIcI My 2 cents...
For each $x in $(selector)
blabla...
There exists $y in $(selector)
blabla...
CSS selectors are fragile. Simple changes to
the layout may break the property

52. cIcI My 2 cents...
...in search of a
carpenter
Hammer and
nails...

53. We still have to write the
properties, so what's the point?*
We're too busy to learn
another tool!*
Using this requires a higher level of
education. We'll have to increase the wages!*
*Really happened
We can already do all this (with a bunch
of home-brewed Python scripts)!*

54. Turn specifications into
sets of test cases
Run Cornipickle in tandem
with a crawler
Hub for sharing
specifications (à la
Greasemonkey)

55. https://lif-labs.github.io
http://lif.uqac.ca
http://leduotang.ca/sylvain