This document discusses using specification by example (SBE) to test rich client GUI applications. It describes using a tool called TextTest that allows writing tests using a domain language and automatically records the GUI interactions and assertions. Tests in TextTest have two parts - a use case section describing actions in domain language terms, and an automatically generated GUI log section capturing screen contents. This allows testing applications by their specifications before code is written and preserves requirements as living documentation through automated regression tests.

1. Specification by Example with GUI tests -
How could that work?
Emily Bache, Bache Consulting & Pagero
Geoff Bache, Jeppesen
@esconfs
#esconfs

3. Specification by Example
with Rich-Client application?
3

4. Jeppesen
 Agile transition underway
 next-generation Crew Management System
4

5. 5

6. Specification by Example
6

7. Specification by Example
 Gojko Adzic’s book
 Collaborative workshops
 Define Example Scenarios BEFORE
build code
7

8. Agile Requirements
User Story + Conversation + Examples
= Requirement
From “Agile Testing” by Crispin, Gregory
8
aka Scenarios

9. Scenario -> Regression test
 BEFORE code is written, it’s a requirement
specification
 AFTER code is written it’s a regression test
Important to preserve the language & format
9
Living Documentation

10. GUI testing
with Record - Replay
10

11. Recorded GUI Tests
 Click around the GUI like a user
would
 Record a script
 Re-play the script to regression
test
11
Assumes the GUI is already built

12. Recorded Scripts
Easy to create, hard to maintain
12

13. Page Object Pattern
13
Test Case Scripts
Hand Coded
Page Objects
Hand Coded
System Under Test

14. GUI testing
with Cucumber, Fitnesse etc
14

15. Using Cucumber
Feature
http://cukes.info
Step Definitions
System Under Test (SUT)

16. Using Fitnesse
16
http://fitnesse.org

17. “Domain Language” Tests
Domain Language
Scenario or Table
Test Fixture
“Glue Code”
“Step Definitions”
System Under Test
17
Hand Coded
Hand Coded

18. Bypass GUI?
Domain Language
Scenario
Test Fixture
Bypass GUI, test
core system directly
GUI
18
Under-the-GUI API

19. 19

20. Demo:
building a new feature
with StoryText
20

21.  Open Source
 Development paid for by Jeppesen
 supports Java Swing, Eclipse RCP, PyGTK, Tkinter, wxPython
 http://texttest.org
21

22. TextTest tests have two parts
Use Case GUI Log
List of user actions:
Domain language
Recorded text:
Contents of GUI
22

23. TextTest - test fixtures
 Create “use case” part of text by Record session in UI
 Name your actions: snippets of domain language
 UI Map - domain language mapped to actions
23

24. Compose a Domain Language
24
 Group several user actions and name them
 Match the language of the specification workshop
 Re-usable terms for many tests

25. Record the Test Fixture/Step definitions
Use Case
(one per test)
“UI Map” + “shortcuts”
(shared by whole suite)
System Under Test
25
Recorded
Configured

26. StoryText tests
Use Case GUI Log
List of user actions:
Domain language
Recorded text:
Contents of GUI
26

27. The GUI log
 “Assert” part of the test
 Auto-generated by
StoryText
 ASCII art represents UI
state
27

28. StoryText tests
Use Case GUI Log
Plain Text: search, store, manipulate, diff, version...
28

29. 29

30. Sample Use-Case :
Undo Service Type changes
30
open default plan and show rosters
change service type of activity JPO 2537 for
Brogan to M
undo
wait for completion of undo
close and discard changes

31. Crew Management System
31

32. Assert Everything by default
 Log of absolutely everything that changes in the GUI
 Each test defines filters to focus on interesting parts
 Often catch errors the test wasn’t designed to find
32

33. Limitations of the approach
 Only supports certain GUI libraries
 Maintainability of tests depends on good test
design
◦ filter GUI log carefully
33

34. Conclusions
 Specification by Example - automating Domain
Language scenarios needs fixture code
◦ TextTest “records” the fixtures
 GUI intensive applications - need to check
carefully what’s on the screen
◦ TextTest takes “screen shots” as ASCII art
34