EuroSTAR Software Testing Conference 2008 presentation on An Entity Model for Software Testing by John Kent. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. An Entity Model of
Software Testing:
A Foundation for the Future
John Kent: john.kent@simplytesting.com

2. An Entity Model of
Software Testing:
A Foundation for the Future

3. • Discussions at the Test Retreat
• A work in progress
• Building Test Management Tools
• A Test Entity Model could be used for:
– Software Testing
– All types of systems testing

4. The Need for an Entity Model
• Define the language
• The Software Testing Cannon
– Myers, etc...
– IEEE 829
– BSI 7925
– ISEB
– ISTQB
• For test tools to be built which implement a
real model of testing
• How can we train testers without a clear
picture of testing?
• Future standards, text books and test tools
should be based upon an agreed entity model

5. • Entity Relationship Diagram - ER Diagram
– Entities
• Uses rectangle
– Relationships between entities
• Uses ellipse symbol
• Many shown as
– Entities have Attributes
Entity Relationship Modeling
Singer
Song
Performs
Song

6. Sources
• Standards
– BSI 7925
– IEEE 829
• Training Courses
– ISEB
– ISTQB
• Glossary
• Syllabus
• Experience
• Test Retreat discussions
• Discussions with testers

7. Where We Are Now
• In many organisations tests look like this:
– Test Script
• Test Steps
– Input
– Expected Output
Inputs
Expected
Outcomes

8. IEEE 829 Entity Relationship
Diagram
• No System Specifications
• No Test Conditions
• Relationships ill-defined

9. • Tries to describe all possible relationships –
not just one way of doing things
• We will build the model by:
1. Using a possible relationship example then…
2. Make statements about the relationships

10. The Entities
• Test Entities include:
– Specifications
– Test Conditions
– Test Cases
– Test Scripts
– Test Steps
– Test Execution Schedule
– Test Results
• Not included here
– Defects
– Test Plans
– Test Strategies
– Test Coverage Items

11. Entity: Specification Item
• An indivisible, testable part of a specification
• Also called Test Basis
• Requirement Specification
– User Reqt
– Legal Reqt
– Performance Reqt
– Security Reqt
– etc...
• Design/Functional Specification
• Attributes
– Description
– Risk
– Priority
– …

12. Entity: Test Condition
• Definition
– ISTQB definition
• "An item or event of a component or system that could be verified
by one or more test cases, e.g. a function, transaction, feature,
quality attribute, or structural element."
– 7925 definition – not defined
– Attributes
• Description
• Risk
• Priority
• A statement of required system behavior under certain conditions
• A refinement of the specification (from a test perspective)
• Could be considered to be a specification item

13. Entity: Test Case
• Definition
– ISTQB: “A set of input values, execution preconditions, expected
results and execution postconditions, developed for a particular
objective or test condition, such as to exercise a particular
program path or to verify compliance with a specific
requirement. [After IEEE 610]”
– 7925: “A document providing detailed instructions for the
execution of one or more test cases”
– Attributes
• Described by IEEE 829
• Input
• Expected Output
• Preconditions
• Objective
– Re-use?

14. Entity: Test Script
• Definition
– Also called Test Procedure
– ISTQB: Test Procedure – “A document
specifying a sequence of actions for the
execution of a test. Also known as test script
or manual test script. [After IEEE 829]”
• Consists of Test Steps….
• A sequence of test cases?

15. Entity: Test Step
• Definition
– ISTQB: not defined
– 7925: not defined
• But they do exist – I’ve seen them!

16. Entity: Test Execution Schedule
• Tests run sequence
• ISTQB: “A scheme for the execution of
test procedures. The test procedures are
included in the test execution schedule
in their context and in the order in which
they are to be executed”.

17. Entity: Test Result
• ISTQB: Result:
– “The consequence/outcome of the execution
of a test. It includes outputs to screens,
changes to data, reports, and communication
messages sent out.”.
• 7925: Actual Outcome:
– “The behavior actually produced when the
object is tested under specified conditions”.

18. Relationship: Specifications to Test Conditions
Example – an Asset Management System
Spec: Assets
Spec: Add Assets
Spec: Delete Assets
TCond: Test that cannot delete an Asset when
used in an active work order
TCond: Test that cannot delete an Asset when
used in an active job planA Specification can ‘link’ to
many Test Conditions
TCond: Test that can delete an Asset when
used in an inactive work order
TCond: Test that can delete an Asset when
not used in a job plan or work order
TCond: Test that can delete an Asset when
used in an inactive job plan
Together the Specifications and
Test Conditions can make up a
model of the system under test

19. Relationship: Specifications to Test Conditions(Cont)
Req Spec: Assets
Req Spec: Add Assets
Req Spec: Delete Assets
TCond: Test that cannot delete an Asset when
used in an active work order
TCond: Test that cannot delete an Asset when
used in an active job plan
TCond: Test that can delete an Asset when
used in an inactive job plan
TCond: Test that can delete an Asset when
used in an inactive work order
TCond: Test that can delete an Asset when
not used in a job plan or work order
Design Spec: Assets
Design Spec: Delete Asset
Menu item
A Test condition could link to
many specifications

20. Relationship: Test Conditions to Test Cases
TCond: Test that can delete an Asset when
not used in an active job plan or work order
TCase:
Pre-condition: An asset linked
to an inactive Job Plan
Input: Select ‘Delete Asset’ menu
item
Expected Outcome: Asset
removed from list
TCond: Test that can delete an Asset when
used in an inactive job plan
TCond: Test that Asset is removed from
the asset list when deleted
The Objective (IEEE
829) attribute in this
definition of a Test
Case is the linked
test conditions
A Test Case can link to many
Test Conditions

21. Relationship: Test Conditions to Test Cases(cont)
TCase:
Pre-condition: An asset linked
to an inactive Job Plan
Input: Select ‘Delete Asset’ menu
item
Expected Outcome: Asset
removed from list
TCond: Test that Asset is removed from
the asset list when deleted
TCase:
Pre-condition: An asset linked
to an inactive Work Order
Input: Select ‘Delete Asset’ menu
item
Expected Outcome: Asset
removed from list
TCase:
Pre-condition: An asset not linked
to a Work Order or Job Plan
Input: Select ‘Delete Asset’ menu
item
Expected Outcome: Asset
removed from list
A Test Condition can link to many
Test Cases

22. Relationship: Test Cases to Test Scripts
A Test Script can
consist of many
Test Cases
A Test Case may
appear in many
Test Scripts
Test Script 2:
Test Script 1:
Test Step: 3
Test Case: 2
Test Case: 1
Test Step: 9
Test Case: 2
Test Case: 7
Test Case: 2
This is test case re-use

23. Test Script 1: Asset Tests
Relationship: Test Scripts to Test Steps
Test Step: 3
Pre-condition: An asset linked to inactive JobP
Input: Select ‘Delete Asset’ menu item
Expected Outcome: Asset removed from list
Test Step: 2
Pre-condition: An active job plan
Input: Select ‘Inactivate Asset’ Menu item
Expected Outcome: Asset status=Inactive
Test Step: 1
Select an job plan with
Status= Active
An Asset linked
A Test Step is a
Test Case in a
Test Script

24. Test Script 2:
Test Script 1:
Relationship: Test Steps to Test Execution
Schedules
Test Step: 3
Test Step: 2
Test Step: 1
Test Step: 3
Test Step: 2
Test Step: 1
Test Execution Schedule 1
Script 1 - Test Step 1
Script 2 - Test Step: 2
Script 1 - Test Step: 2
Test Execution Schedule 2
Script 1 - Test Step 2
Script 2 - Test Step: 2
Script 1 - Test Step: 3
A Test Execution
Schedule can contain
many Test Steps
A Test Step can be
used in many Test
Execution Schedules

25. Test Script2:
Test Script1:
Test Execution Schedule 1
Test Script 1
Test Script 8
Test Script 7
Test Execution Schedule 2
Test Script 2
Test Script 7
Test Script 1
A Test Execution
Schedule can contain
many entire Test Scripts
An entire Test Script can
be used in many Test
Execution Schedules
Relationship: Test Scripts to Test Execution
Schedules

26. Relationship: Test Execution Schedules to Test
Results
Test Execution Schedule
Script 1 - Test Step 1
Script 2 - Test Step: 2
Script 1 - Test Step: 2
Test Results: Run 1
Test Results: Run 2
Test Results: Run 3
A Test Execution
Schedule can
appear in many
Test Results

27. The ‘Full’ Test Entity Model
Specification Item
Test Condition
Test Case
Test Script
Test Step
Test Execution Schedule
Test Results

28. The ‘Full’ Test Entity Model
Specification Item
Test Condition
Test Case
Test Script
Test Step
Test Execution Schedule
Test Results
Test Execution Results must
be traceable back through
all of the relationships to the
specifications or
requirements

29. The major difference between the Theory and
Real World is the Real World contains time
and therefore work we do is incomplete
• It takes time to build the entities
• It takes time to build the relationships between entities
• What test entities we build depend on the time we have
• The level of detail in specifications and tests is
dependent upon time
– Requirements are incomplete
– Test conditions are also incomplete

30. Real World Waterfall Model
Code what you can from
specs and make up the rest.
Bugs always included
Specify the Syst Design
incompletely with errors
Produce Requirements
which are incomplete and
slightly wrong
There is an over-emphasis on test
scripts because it is generally
accepted that testers write scripts.
Often we could get greater test
coverage by producing more test
conditions which we could execute
directly and not waste time on
scripting.
Because Requirements are
incomplete we need Test
Conditions to describe the
system – we produce a list
of incomplete Test
Conditions

31. Additions to the ‘Full’ Test Entity Model
Specification Item
Test Condition
Test Case
Test Script
Test Step
Test Execution Schedule
Test Results
We could get
greater test
coverage by
producing more
test conditions
which we could
execute directly.
Writing TCs
gives faster test
coverage
We have seen that TCs are a form
of spec so it would be possible to
execute the specifications directly
in some test projects

32. Simplified Test Entity Model 1
Specification Item
Test Condition
Test Execution Schedule
Test Results
Test conditions
executed directly using
testers knowledge to
provide missing steps

33. Simplified Test Entity Model 2
Test Script
Test Step
Specification Item
Test Condition
Test Execution Schedule
Test Results
Entity Model used in
Quality Center and
Test Director
Many test teams
don’t use this bit
Introduces another
addition to the ‘Full’
Model:
TCond to Test Script
So they cannot
measure test
coverage against
requirements
May be difficult to implement full model
in a database tool – lots of many to
many relationships

34. Simplified Test Entity Model 3
Test Script
Test Step
Test Execution Schedule
Test Results
Specification Item
Test Condition
This is where many test
organisations are
They may be better off doing this

35. Summary

36. Summary
• Full Entity Model Not Completely Defined
– Entities not well defined
– Relationships not well defined
– Attributes not defined – Risk?
– Entities left out, Test Coverage, Test Plan
• Once we understand the full entity model we may
be able to see more/better ways to test
• In the real world we are likely to use a simplified
version of the Entity Model