Acceptance testing is often thought of as the little brother of system testing and, in many projects, it ends up as a little phase at the end. Having worked in system testing for most of her testing career, Bettina Faldborg found it was a bigger jump than you might think to move to acceptance testing. She had this overwhelming urge to test it ALL. Like most testers, she did not assume that what has been tested before was done well enough, but that is one of the preconditions to accept going into an acceptance test. With regard to acceptance testing Bettina presents four basic principles to remember—know your scope, know your test design techniques, know your testers, and know your suppliers. These four principles can be used in most test phases but using them for acceptance test requires a different spin. Let Bettina be your guide as she shares the lost art of acceptance testing and offers easy to follow advice on how to tackle your next acceptance test.

1.
W2
Test
Techniques
5/10/17
11:30
The
Lost
Art
of
Acceptance
Testing
Presented
by:
Bettina
Faldborg
Capgemini
Sogeti
Denmark
A/S
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Bettina
Faldborg
Bettina
Faldborg
was
a
project
manager
before
she
ended
up
as
a
test
consultant
in
the
testing
field
almost
by
chance.
With
a
passion
for
testing
and
quality,
she
may
have
entered
testing
by
chance
but
quickly
discovered
that
testing
is
the
field
for
her.
With
Capgemini
Sogeti
Denmark
since
2008,
Bettina
has
been
a
test
manager
with
clients
in
the
financial
and
public
sectors.
Her
time
as
a
consultant
has
included
traditional
system
test,
acceptance
test,
and
usability
test.
She
is
currently
a
test
manager
at
a
large
telecom
company.

3. © CapgeminiSogeti
The Lost Art of Acceptance testing
Bettina Lysell Faldborg
StarEast 2017
1

4. © CapgeminiSogeti
Welcome
• Bettina Lysell Faldborg
• With Capgemini Sogeti Denmark
since 2008.
• Test manager
• ISTQB Advanced Test manager,
Prince2, TPI Next
• Live in Denmark in a small village
with my husband and three kids.
• Bettina-Lysell.Faldborg@Capgeminisogeti.dk
2

5. © CapgeminiSogeti
The Lost Art of Acceptance testing
• Profession: Testing
• Mission: Acceptance testing
• The scene: Large government project
• Benefit: Clear requirements
• Concern: Being on the other side of the table
• Challenges: …
• Where do I go from here?
3

6. © CapgeminiSogeti
Agenda
• What is Acceptance testing
• Buildings blocks
• Lessons learned
• Basic principles
4

7. © CapgeminiSogeti
What is Acceptance testing?
5

8. © CapgeminiSogeti
What is acceptance testing
• ISTQB Definition:
Formal testing with respect to user needs,
requirements, and business processes conducted
to determine whether or not a system satisfies
the acceptance criteria and to enable the user,
customers or other authorized entity to
determine whether or not to accept the system.
[IEEE 610]
6

9. © CapgeminiSogeti
What is acceptance testing
• Basic types of acceptance testing
– User Acceptance testing
• The focus is on the functionality and the validation by
the users
7

10. © CapgeminiSogeti
What is acceptance testing
• Basic types of acceptance testing
– Compliance Acceptance testing
• Is conducted against regulations which must be
adhered to, such as government, legal or safety
regulations
8

11. © CapgeminiSogeti
• Basic types of acceptance testing
– Operational Acceptance testing
• A test of whether or not the system meets the requirement
for production – focus is on backup/restore, recovery,
maintenance and other similar situations.
9
What is acceptance testing

12. © CapgeminiSogeti
• Basic types of acceptance testing
– Contract Accepttance testing
• The test is a test of the contract and the acceptance
criterias of the contract.
10
What is acceptance testing

13. © CapgeminiSogeti
Building blocks
11

14. © CapgeminiSogeti
Building blocks
• Starting point is the same
– Development model
– Product risk analysis
– Test strategy
– Test plan
– …
12

15. © CapgeminiSogeti
• Tasks
– Not much different than other test levels
13
Building blocks

16. © CapgeminiSogeti
Lessons learned
14

17. © CapgeminiSogeti 15
Changing your mindset

18. © CapgeminiSogeti 16
System testing
• Test coverage in terms of
parts of system covered or
functionality covered
• Testing all possible
scenarios
• Test depth is both wide and
deep
• Testers are testers by trade
Acceptance testing
• Requirement coverage
• Testing of task oriented
scenarios
• Test depth is primarily
wide
• Testers are testers by
domain
Differences between
ST and AT
Changing your mindset

19. © CapgeminiSogeti
4 basic principles
• Know your scope
• Know your design techniques
• Know your testers
• Know your supplier
17

20. © CapgeminiSogeti 18
Know your scope

21. © CapgeminiSogeti
Know your scope
• Gathering test basis
– Business requirements – ”business questions”
– Business processes
– User experiences
• What are the requirements
– What type of requirements
• Requirement
• Requirement specification
• User stories and acceptance criteria
– How were they risk classified
– What coverage is required
19

22. © CapgeminiSogeti
• What is the goal of the acceptance testing
– Which type is it
– Ask the PO or PM what the expectations are
– Stick to the original mandate unless …
• What is planned for the previous test phases
– What kind of test will be performed
– How long a test period will the test object be under test
– How did it go?
20
Know your scope

23. © CapgeminiSogeti 21
Know your test design techniques

24. © CapgeminiSogeti
Know your test design techniques
• Can design techniques be used in Acceptance
testing?
– YES!!!!
– Remember test coverage vs. requirement coverage
• Which techniques will be used in the previous
phases
– Use different ones for Acceptance testing
– If the test object is not suited for other techniques; go
deeper or wider
22

25. © CapgeminiSogeti
Types and test design techniques
Acceptance testing type Test design techniques
User Acceptance testing Decision table test, Data
combination test,
Exploratory testing, Process
cycle test, Semantic test,
Use case test
Compliance Acceptance testing Exploratory testing,
Semantic test, Use case
test
Operational Acceptance testing Real-Life test
Contract Acceptance testing Checklist, Syntactic test,
Use case test
23

26. © CapgeminiSogeti 24
Know your testers

27. © CapgeminiSogeti
Know your testers
• Who are your testers?
– Professional testers
– Domain testers
– End-users
• Know their weaknesses and strengths
− Find the abide-by-all-rules tester in the group
− Find the critic in the group
− Find the they-had-to-send-one in the group
− Find the secret-tester
25

28. © CapgeminiSogeti
• If the end-users are not available
– Use stand-ins for end-users: Personas
– Ask PO who is this intended for (sometimes not
mentioned in the stakeholder analysis)
• Make it easy for the non-professional testers to
test
– Have a test-kick off, explain the essentials
– Prepare test cases or test charters
– Or help them write them
– Documentation of test performed
– How to report defects
26
Know your testers

29. © CapgeminiSogeti 27
Know your supplier

30. © CapgeminiSogeti
Know your supplier
• Inside or outside own organization?
• Your new best friend: The suppliers test team
– We all know testers know almost everything worth
knowing in a project - especially system testers – tap
into their knowledge
– Share your test cases … EARLY
• You are the product owners and end-users closest
ally
– you make sure they get what they asked for
• We are all in it together
– Remember that everyone is interested in releasing the
best solution into production
28

31. © CapgeminiSogeti
Know your supplier
• Can be a sensitive subject
– Us and them
• But don’t be afraid to call the kettle black
– If the suppliers test is not good enough it is your
responsibility to speak up
– If there are too many defects – stop the test!
(Remember this when writing the test plan)
• If same organization
– be aware of the difference between the system testers
and the acceptance-testers – different mindset and
goals
29

32. © CapgeminiSogeti
Conclusion
• The Lost Art of Acceptance testing
– Change your mindset
– Use your testing toolbox
– Remember the four principles
30

33. © CapgeminiSogeti
Questions
31

34. © CapgeminiSogeti
Thank you!
32