When testing in agile projects, testers are required to deliver the best service possible to the developers. To be able to deliver this highly needed service to developers, the team needs already “done” features to keep on working, avoid known defects to resurface (reappear) as well as avoid builds to crash. I will present successfully applied methods and techniques used in agile software development projects. The focus on this approach for test automation is to let it become this needed “early” service to both developers and manual testers. By following these methods and techniques, testers will be able to impact the quality (positively) by contributing to test automation and test improvement via better design of test cases for test automation. It is a fundamental matter for high quality products that developers are able to continue keeping the high quality alongside new functionality is added. For this to be a success, test automation needs to be “pushed back” to as close as possible to the developers. In order to keep up with the pace of the developers and to avoid the “Death by Maintenance” syndrome, a proper approach to “early” test automation is required. I will introduce examples of a proper approach as well as some handy test techniques for easing the scripting of tests. The presentation is about how to incorporate successful test automation into an agile project. It is about the processes used daily by developers as well as incorporating our tests to the build processes and hence improving the quality of our products. I will also emphasize the importance of cooperation with the developers and their tool smiths, by pragmatic approaches.

1. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
© Christian Nørlyng and PrettyGoodTesting®
By Christian Nørlyng
Raising The Quality Bar In Agile Teams
By means Of Early Test Automation
1

2. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Who am I?
Test Automation Lead at PrettyGoodTesting
ISTQB certified Test Analyst (adv. level)
9+ years of test experience from
• Small & Large Teams
• Agile & Waterfall Projects
• Medical – Directory – Telecom – Defense – Maritime
2

3. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Agenda
• Mission and Bugs
• Goals, Context and Features for automation
• Structural Test Patterns
• Thinking Automation
• Modeling for Data and more
3

4. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Most Important Mission
"Increase management
confidence in the product."
4

5. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Automation and bugs
Test Automation is looking for
Defects found during regression testing
5

6. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Setting realistic goals
• Rules of thumb are difficult to apply
• Conduct a small pilot project
• Less than a month in duration
• After creating “just enough” infrastructure,
track time to automate each feature
• Compare time to manual testing
• Use these data for subsequent test projects
• Continue to update data
• Automation will become more efficient
after the first few projects
6

7. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Getting the Context
”Features”
”Trade off’s”
”Possibilities”
”Context”
Starting with the
intersection!
7

8. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Picking the first features
”Wants”
”Seems Quick”
”Musts Haves”
”Planned/
Implemented
Features”
Starting with the
intersection!
8

9. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Good Automation Candidates
• Short or simple transactions
• Many data combinations
• Expected results are stable
or easy to generate at runtime
• Tests that are executed regularly
• Tasks that are difficult to do manually
• Highest priority features
9

10. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Poor Automation CandidatesPoor Automation Candidates
• Long or complex transactions
• One-offs
• Unstable or difficult to predict results
• Tests that works across multiple applications
10

11. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Analysing the ”User Action”
Patterns Under Test
• Breaking down the test cases and
system under test into keywords
• Each keyword should represent
a ”unique” set of actions to the system
11

12. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Designing Good Test Cases
• Test cases should have a single objective
• Test cases should result in one of two
dispositions: PASS or FAIL
• Test cases should be independent
• No test case should rely on
the successful completion of another test
• Test cases must start and stop
at a known state
12

13. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
What is ”Structural Test Patterns”?
• A break down method both for supporting
• KeyWords, ActionWords or Primitives
• Can be used for manual
and automated testing
• Can relate manual
and automated testing
• Can ease maintenance of manual
and automated testware
13

14. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
The Principle in short
• By thinking from the bottom up the support
for automation is enabled from the test
design.
• By breaking down the users behaviour into
patterns, the test procedures be simplified.
14

15. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Think Automation
How and what
to automate
Analyzed
GUI/MMI
ActionWords
Structuring
Test Conditions
Domain
knowlegde
Test Design
Structure
Requirements
Modular mindset, during
analyzis and Test design
Test Conditions suitable for automation
Traceable
identification
15

16. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
KeyWord Classification
Navigation
Go from A to B (Start to adjust Time)
Action
Input to system (Set Time)
Verification
The object of the test (Vary Time set)
16

17. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Work Process Description
Requirements
Test Design
Tests using Keywords
AutomatedManual
Modular mindset
during test design
Review
Review
17

18. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Products of Module Based Test Design
Traditional
Sequential
TestCases
A pseudo test case broken down into ActionWords,
each KeyWord is unique to the SUT (Software Under Test)
KW 6
KW 5
KW 4
KW 3
KW 2
KW 1 KW = KeyWord
18

19. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
KW 7
KW 6
KW 3
KW 2.1
KW 1
KW 7
KW 5.1
KW 4
KW 3
KW 2.2
KW 1
KW 7
KW 5.2
KW 4
KW 3
KW 2.3
KW 1
1
2.1 2.2 2.3
3
6 4
5.1
7Saving above 50% of work in just this example.
5.2
KW = KeyWord
An Advantage
19

20. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Why does this work?
• During test analysis, try to break down each
TestCase into unique and identifiable user
action sequences
• A unique sequence, is not to exist elsewhere
• (stick to this…)
• The sequences are to be reused and combined
in any order, representing complete user actions
• Everything implemented is the right
context of the automation project
20

21. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Summary of test design
• Cooperation between Test Designer, Manual Test
Team and Automation Test Team reduces effort, by:
- Thinking Automation First
- Reusing Test Documentation/KeyWords
• KeyWords (aka ActionWords, Modules etc.):
- Promotes flexible & easy maintainable Test Suites
• Exploratory Testing and Model Based Testing
...still to be carried out (not a stand-alone thing)!
21

22. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Where to go from here!
• Data Drive the Test
Use:
Parameters for the Keywords
Keywords as Data
Use Test Techniques and Test
Generation Tools for modelling!
Ex. Decision Tables,
Classification Trees or State Machine Graphing
22

23. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
5.2
6
2.1 2.32.3
7
5.25.1
6 4
3
2.2 2.32.1
11
2.1 2.2 2.3
3
6 4
5.1
7
5.2
Simplified Example
23

24. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Remove the abstraction
• Models can even be
made
in a spread sheet!
• MS Excel (e.g.) can
often
be used as a data source
• Designing test can now
be
done from outside the
test automation tool!!
KeyWord
Parameter
Set
Test 1
KW1 KW1_P1
KW2.2 KW2_P1
KW3 KW3_P1
KW4 KW4_P1
KW5.1 KW5_P1
KW7 KW7_P1
Test 2
KW1 KW1_P1
KW2.2 KW2_P2
KW3 KW3_P1
KW4 KW4_P1
KW5.1 KW5_P1
KW7 KW7_P1
Test 3
KW1 KW1_P1
KW2.2 KW2_P3
KW3 KW3_P1
KW4 KW4_P1
KW5.1 KW5_P1
KW7 KW7_P1
24

25. EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Questions and Comments
Please feel free to contact me!
Christian Nørlyng
Test Automation Lead
Mob: (+45) 3163 0203
CHNO@PrettyGoodTesting.com
http://www.PrettyGoodTesting.com
© Christian Nørlyng and PrettyGoodTesting®
25