powerpoint template for testing training

© Allstate Northern Ireland Limited Proprietary and Confidential
Testing Training Material
John Roddy
9th
October 2009

Proprietary and ConfidentialMarch 4, 20152
What is Software Testing?
“…the process of executing a program with the intent to certify its quality”
Mills
“…the process of executing a program with the intent of finding failures /
faults”
Myers
“…the process of exercising software to detect errors & to verify that it
satisfies specified requirements”
“Testing is any activity aimed at evaluating an attribute or capability of a
program or system and determining that it meets required results.”
Bill Hetzel, 1983

March 4, 20153 Proprietary and Confidential
What is Software Testing (continued)?
 Software testing is a process:
The input is often stakeholder requirements
The output is quality information
The process methodically probes the application
from various angles

What is Software Testing (continued)?
Software testing comprises aspects of:
• Engineering
– We must design tests that are effective in identifying software failures
• Literature
– We must understand the stakeholders needs and desires.
– A stakeholder is a person or entity who has a vested interest in the success
of a project. They may be end-users, financial backers, company sponsors
or corporate members.
• Communication
– We must present our findings in a way that our clients can make informed
decisions upon

Get with the Lingo!
Terms: Error - a human
action producing
incorrect result
Fault - a
manifestation of
error in software
Failure - deviation
of software from
expected delivery
or service
Do 100
I=1.10
X
Points to
Points to
produces
produces

Testing Terminology
DEFECT
The departure of a quality characteristic from its specified value that results in a product or
service not satisfying its normal usage requirements
RELIABILITY
The probability that software will not cause the failure of a system for a specified period of time
under specified conditions
QUALITY
The totality of the characteristics of an entity that bears on its ability to satisfy or implied needs
QUALITY ASSURANCE
All those planned actions used to fulfill the requirements for quality.

Misconceptions
Testing is an unnecessary expense
A common development perception
Software testers are ten a penny
Huge difference between good and bad testing
Software testing is not difficult
Most testing activities are not rocket science but …
• Testers have a juggle many things at once
• Make new decisions constantly
• Think methodically and scientifically
• Questions things other people take for granted

Cost of Software Failures
A single failure may incur little cost or millions
In extreme cases software failure may cost LIVES (safety
critical systems – Airline, Nuclear Power)
The cost of failures increases proportionately (tenfold)
with the passing of each successive stage in the system
development process before they are detected
To correct a problem a requirements stage may cost £1
To correct the problem post-implementation may cost
millions

Summary
 The purpose of testing is to find faults. Faults
can be fixed therefore making better software.
Better software is more reliable, less prone to
failures.
 Testing enables us to measure the quality of
the software
 This enables us to understand and manage
the risk to the business

MORNING BREAK

The Test Process
The test process has 5 steps:
 Test Planning
 Test Specification
 Test Execution
 Test Recording
 Checking for Test Completion

Test Planning and Test Specification
Test Planning
• The Test Plan describes how the Test Strategy is implemented
• A project plan for testing
• Defines what is to be tested, how it is to be tested and what is needed
for testing
Test Specification has 3 steps
• Preparation & Analysis
• Building Test Cases
• Defining expected Results

Test Execution Checklist
Test execution schedule / log
Identify which tests are to be run
Test environment primed and ready
Resources ready, willing and able
Back-up and Recovery Processes in place
Any batch runs planned and scheduled
When all the above are in place we are ready to run the tests

Test Recording and Test Completion
Test Recording
 Test Log should record
 Software and Test Versions
 Specifications / Requirements used as Test Base
 Test Timings
 Test Results (Actual Results v Expected Result)
 Any Defect Details
Test Completion
 Have we fulfilled the Test Exit Criteria
 Used to determine when to stop this phase of testing
 Key Functionality Tested
 Test Coverage
 Defect detection rate

The Psychology of Testing
In this session we will:
 Understand what qualities make a good tester
 Look at a testers relationship with developers
 Look at a tester relationship with management
 Understand the issues with testing independence

What makes a Tester?
 Testing is primarily to find faults
 Can be regarded as “destructive”
 Development is “constructive”
 Testing asks questions
 Testers need to ask questions
 A tester needs many qualities

What makes a tester:
 Intellectual Qualities
 Can absorb incomplete data
 Can work with incomplete data
 Can level quickly on manual levels
 Good verbal communication
 Good written communication
 Ability to prioritise

Proprietary and Confidential
 What makes a Tester?
Knowledge
 How projects work
 How computer systems and business needs interact
 Test Techniques
 Testing Processes
 Testing best practices
 To be able to thin outside the box

What makes a good tester?
 More Skills to acquire
 Ability to find faults – planning, preparation & execution
 Ability to understand systems
 Ability to read specifications
 Ability to extract testable functionality
 Ability to work efficiently
 Ability to focus on essentials

 What makes a Tester?
Communication with Developers
 A good relationship is vital
 Developers need to keep testers up-to-date with changes to
application
 Testers need to inform developers of defects to allow fixes to be
applied

What makes a good tester?
 Communication with Management
 Managers need progress reports
 The best way is through Metrics
 Number of Tests planned and prepared
 Number of tests executed to date
 Number of defects raised & fixed
 How long planning and preparation and execution stages take

Testing Independence
 It is important that testing is separate from development
 The developer is likely to confirm adherence not deviation
 The developer will make assumptions
Levels of Independence
 Low – Developers write their own tests
 Medium – tests are written by another developer
 High – Tests are written by an independent body
 Utopia – tests generated automatically

Testers require a particular set of skills
 The desire to break things
 The desire to explore and experiment
 Communication
 Questioning
 Testing requires a different mentally to development
 “Destroying” things rather than creating them
 Testing should be separate from development

LUNCH

Re-Testing and Regression Testing
 Re-Testing is the re-running of failed tests once a fix has been
implemented to check that the fix has worked
 Regression Testing is running a wider test suite to check for
unexpected errors in unchanged code

Re-Testing
 The need for re-testing needs to be planned and
designed for:
 Schedules need to allow for re-testing
 Tests need to be easily re-run
 Test Data needs to be re-usable
The environment needs to be easily restored

Regression Testing
Tests will need to be re-run when checking software upgrades
Regression tests should be run whenever there is a change to
the software or the environment
 Regression tests are executed to prove aspects of a system
have not changed
Regression testing is a vital testing technique
Selecting cases for regression
 Tests for areas that change regularly
Tests of functions that have a high level of faults
Regression Testing is the ideal foundation for Automation

Regression Testing Strategy
 Several factors to determine strategy:
 How many test cases in regression test set?
 What criteria should be used to select them?
 When will regression testing be performed?
 Who’s responsibility is it?
 Why should it be a continuous activity?

Expected Results
 In this session we will:
 Understand the need to define expected results
 Understand where expected results can be found

Expected Results
 Expected Results = Expected Outcomes
 Identify required behaviour
 If the expected outcome of a test is not defined
the actual output may be misinterpreted
 Running a test with only a general concept of the
outcome is fatal
 It is vital the Expected Results are defined with
the tests before they can be used
 You cannot decide whether a test has passed
just because it looks right

Expected Results
 Summary
 Without defining expected results how do you know if a
test has passed or failed?
 Expected Results can be found in the system
specifications and by asking experienced business
users

Prioritisation of Tests
 In this session we will
 Understand why we need to prioritise tests
 Understand how we decide the priority of individual tests

 It is not possible to test everything
 Therefore errors will get through to the Live System
 We must do the best testing possible in the time available
 This means we must prioritize and focus testing on the priorities

 Aspects to Consider
 Severity
 Probability
 Visibility
 Priority of Requirements
 Frequency of Change
 Vulnerability to error
 Technical Complexity
 Time and Resources

 Business Criticality
 What elements of the application are essential to the
success of the organization?
 Customer Factors
 How visible would a failure be?
 What does the customer want?
 Technical Factors
 How complex is it?
 How likely is an error?
 How often does this change?

 Summary
 There will never be enough time to complete all tests
 Therefore those tests that cover those areas deemed most
important (to the business, highest risk) must be tested first
where possible

Models for Testing
There are many approaches to testing.
To models widely use are:
Verification, Validation and Testing (VV&T)
The V-Model
 V-Model
 The most commonly used model in testing
 It represents the software development lifecycle
 Shows the varies stages in development and testing
 Shows the relationship between the various stages

Models for Testing
 VV&T
 Verification
• The process of evaluating a system or component to determine
whether the products of the given development phase satisfies the
conditions imposed at the start of the phase
 Validation
• Determination of the correctness of the products of software
development with respect to the user needs and requirements
 Testing
• The process of exercising software to verify that it satisfies
specified requirements and to detect faults

Testing in the Lifecycle
 Types of Lifecycle
 Sequential Lifecycle – Waterfall, V-Model
 Iterative Lifecycles – Spiral, Pre-planned incremental
delivery
 Rapid Development Models – RAD, DSDM (Agile)
 Evolutionary
 Object Oriented (OO)
 Extreme Programming (XP)

Waterfall Model

A typical development lifecycle V-Model

Rapid Application Development (Agile)

BREAK

High Level Test Planning
 Look at how a test plan is put together
 Understand how it should be used and maintained
 Understand why they are so important to a testing project

 What is a test plan?
“A document describing the scope, approach, resources and
schedule of intended testing activities. It identifies test items,
the features to be tested, the testing tasks, who will do each
task and any risks required contingency planning”
 A project plan for testing
 Covering all aspects of testing
 A “living” document that should change as testing progresses

 IEEE 829 – Standard for Test Documentation
Institute of Electrical & Electronic Engineers
• Standard for Test Documentation
• Includes documentation templates for:
– Test Plan: Plan how the testing will proceed.
– Test Design Specification: Decide what needs to be tested.
– Test Case Specification: Create the tests to be run.
– Test Script: Describe how the tests are run.
– Test Item Transmittal Report: List of items released for testing.
– Test Log: Record the details of tests in time order.
– Test Incident Report: Details events that need to be investigated.
– Test Summary Report: Summarise and evaluate tests.

1. Test plan identifier.
2. Introduction.
3. Test items.
4. Features to be tested.
5. Features not to be tested.
6. Approach.
7. Item pass/fail criteria.
8. Suspension/Resumption.
 9. Test deliverables.
 10. Testing tasks.
 11. Environmental needs.
 12. Responsibilities.
 13. Staffing and training needs.
 14. Schedule.
 15. Risks and contingencies.
 16. Approvals.

High Level Test Planning : Who
 Test Staff
•Who will perform the testing
•How will we address skill gaps
•Contingency planning
 Support Staff
•Who will install operating systems
•Who will set up and manage test databases

High Level Test Planning : What
 Test items
 What components need testing
 Features to be tested
 What features are subject to testing
 Feature not to be tested
 What features are not ready for testing
 What features have already been tested
 What features will be tested at a later level
 Software & Testing Risk
 Contingency planning

High Level Test Planning : When
 Test Schedule
Use of Gant charts
Initially based on activities found in test process
 Test Deliverables
When do they need to be delivered
What is their frequency of delivery
 Work Breakdown
List of tasks required to be performed
All tasks have an associated period of time

High Level Test Planning : How
 Test Environment
 Physical location
 Workspace requirements
 Hardware Requirements
 Software Requirements
 Network Requirements

 Test Approach
Arguably the most important section of a test plan
document
Relates to the details of test case designRelates to the details of test case design
Often categorized by testing typeOften categorized by testing type
SpecifiesSpecifies
 Techniques to be used in designTechniques to be used in design
 Test completion criteriaTest completion criteria
 Measurement techniquesMeasurement techniques
 ApproachApproach

 Summary
 Test plans are project plans for testing
 They identify why testing is needed, what will be tested,
the scope of this phase of testing, what deliverables testing
will provide and what is required to enable testing to
succeed
They are “living” documents that must evolve as the project
progresses

DEMO OF DMEA TEST PLAN

Day 2
Topics:
 Types of Testing
 White Box Testing
 Black Box Testing
 Reviews

Key Testing Phases
 Component Testing
The Testing of individual software components. The main focus is on the internal program
structure
 Component Integration Testing
Process of combining components into larger assembles. Looks at the interaction between
components and their interfaces.
 Functional System Testing
The process of testing an integrated system to verify that it meets specify requirements
 Non-Functional System Testing
Testing of those requirements that do not relate to functionality - performance
 System Integration Testing
Testing performed to expose faults in the interfaces and in the interaction between integrated
systems
 Acceptance Testing
Formal testing conducted to enable a user, customer or other authorized entity whether to
accept a system or component

Acceptance Testing
In this session we will
 Understand what acceptance testing is
 Why you would want to do it
 How you would plan it and prepare for it
 What you need to actually do it
 Understand the different types of acceptance
testing

Acceptance Testing
 What is Acceptance Testing?
“Formal testing conducted to enable a user, customer or
other authorized entity to determine whether to accept a
system or component”

Acceptance Testing
 What is User Acceptance Testing (UAT)?
Exactly what it says it is
 The set-up will represent a working environment
 Users of the end product conducting the tests
 Covers all aspects of the project, not just the system
 Also known as Business Acceptance Testing or Business
Process Testing

Acceptance Testing
 Planning UAT
 Why Plan?
 Things to Consider – what environment is going to be
replicated
 Preparing Tests
 Manual
 Automated

Acceptance Testing
 Why Plan?
 If you don’t, then how do you know you have achieved
what you set out to do
 Avoids repetition
 Test according to code releases
 Makes efficient and effective use of time and resources

Acceptance Testing
 Things to consider:
 Timescales
 Resources
 Availability

Acceptance Testing
 Preparing your tests:
 Take a logical approach
 Identify the business processes
 Build into everyday business scenarios

Acceptance Testing
 Data Requirements
 Copied Environments
 Created Environments
 Running the Tests:
 Order of Tests
 Confidence Checks
 Automated and Manual test runs

Acceptance Testing
 Contract Acceptance Testing
“A demonstration of the acceptance criteria”
 Acceptance Criteria will have been defined in the
contract
 Before the software is accepted it is necessary to
show that it matches its specification as defined in
the criteria

Acceptance Testing
 Other types if Acceptance Testing:
 Alpha Testing – customers do the testing
 Beta Testing
 Factory Acceptance Testing (FAT)

Acceptance Testing
 Summary
 Before software is released it should be
subjected to Acceptance Testing
 User representation in testing is VITAL
 If the product does not pass UAT then a decision
about implementation needs to be made

Functional System Test
 Understand what functional system testing is
 Understand the benefits of functional system
testing

Functional System Testing
 What is functional system testing?
 Testing of the complete system
 Ideally done by an independent test team
 Two type – functional and non-functional

 A Functional Requirement is
“ A requirement that specifies a function that a system
or system component must perform”
 Functional System testing is geared to checking the
function of the system against specification
 My be requirement based or business process based

Functional System Testing
 Testing based on requirements
 Requirements specification used to derive test
case
 System is testing to ensure the requirements are
met

 Testing carried out against Business Processes
 Based on expected use of the system
 Builds use cases – test cases that reflect actual or
expected use of the system

Non-Functional System Testing
 Understand what non-functional System Testing is
 Understand the need for Non-Functional System
Testing
 Look at the various types of Non-Functional System
Testing

Non-Functional System Testing
 Non-Functional System Testing is defined as:
“ Testing of those requirements that do not relate to the
Functionality e.g. Performance, Load, Usability,
Security etc”

Non-Functional System Test
 Security Testing
“Testing whether the system meets specified security requirements”
 Usability Testing
“Testing the ease with which users can learn and use the product”
 Load Testing
“Testing geared to assessing the applications ability to deal with the
expected throughput of data and users
 Stress Testing
“Assesses individual components by exercising them to and beyond
the limits of expected use”
 Performance Testing
“Tests the efficiency of individual components of an application”

 Other Non-Functional System Test
 Volume Testing
 Recovery Testing
 Documentation Testing
 Storage Testing
 Installability Testing

 Summary
 Just because a systems functions have been tested
doesn’t mean that testing is complete
 There are a range of non-functional tests that need to be
performed upon a system

Component Testing
 Understand what Component Testing is
 Look at the Component Testing Process
 Look at the myriad types of Component Testing

Component Testing
 What is Component Testing?
“ The testing of an individual software component.
This is also known as Unit Testing”
 What is a component?
 A minimal software item for which a separate specification
is available

Component Testing
 Component Testing is the lowest form of testing
 At the bottom of the V-Model
 Each component is tested in isolation
 Prior to being integrated into the system
 Involves testing the code itself
 Test the code in the greatest detail
 Usually done by the components developer

Component Testing
 BS7925-2 Standard for Component Testing
 Component Test Process:
 Component Test Planning
 Component Test Specification
 Component Test Execution
 Component Test Recording
 Checking for Component Test Completion

Maintenance Testing
 Look at the challenges that testers face when an
application changes post implementation
 See how to ensure that maintenance applied to
the system does not cause failures

Maintenance Testing
 What is Maintenance Testing?
 Testing of changes to existing, established systems. It is
regression testing in the “Live” environment
 Checking that the fixes have been made and that the
system has not regressed

Maintenance Testing
 Summary
 All established systems need maintenance from time
to time
 The changed code / function will need to be tested
 A regression test will need to be executed to ensure
that the change(s) have been made correctly and that
they have not affected other parts of the system
 Impact Analysis is key

MORNING BREAK

Black and White Box Testing
 Understand the differences between Black and
White Box testing and where they feature in the
testing lifecycle
 Understand how a systematic approach provides
confidence
 Understand how tools can be used to improve
and increase productivity

Black and White Box Testing
 What is Black Box Testing?
“ Test Case selection that is based on an analysis of the specification of a
component without reference to it’s internal workings”
 What is White Box Testing?
“ Test Case selection based on an analysis of the internal structure of a
component”

Black Box Testing
 Concentrates on the Business Function
 Can be used throughout testing cycle
 Dominates the later stages of testing although is relevant
throughout the development lifecycle
 Little / No Knowledge of the underlying code needed

White Box Testing
 Also known as Structural Testing
 Focuses on Lines of Code
 Looks at specific conditions
 Looks at the mechanics of the Application
 Useful in the early stages of testing

Summary
 Black box testing focuses of functionality
 White box testing focuses on code
 A systematic approach is needed for both:
 Tests need to be planned, prepared, executed and verified
 Expected results need to be defined and understood
 Tools can help increase productivity and quality

Black Box Test Techniques
 Who do we need black box test techniques:
 Exhaustive testing is not possible
 Due to the constraints of time, money and resources
 Therefore we must create a sub-set of test
 There must be achievable, but should not reduce coverage
 We should also focus on areas of likely risk
 Those places where mistakes may occur

 Each black box test technique has
 A method
 how to do it
 A test case design approach
 How to create test cases using the approach
 A measurement technique
 Except Random & Syntax
 See BS7925-2 for detailed information

 Equivalence Partitioning
 Boundary Value Analysis
 State Transition
 Cause & Effect Graphing
 Syntax Testing
 Random Testing

Equivalence Partitioning
 Uses a model of the component to partition input
and output values into sets
 Such that each value within a set can be reasonably
expected to be treated in the same manner
 Therefore only one example of that set needs to be
input to or resultant from the component

Equivalence Partitioning
 Equivalence Partitioning – Test Case Design
 Inputs to the component
 Partitions Exercised:
 Valid Partitions
 Invalid Partitions
 Expected Results

BVP and EP
BREAK

Boundary Value Analysis
 Uses a model of the computer to identify the values close
to and on partitions boundaries
 For input and output, valid and invalid data
 Chosen specifically to exercise the boundaries of the
area under test

Boundary Value Analysis – Test Case Design
 Inputs to the component
 Boundaries to be exercised
 Just below
 On
 Just Above
 Expected Results

Example: Loan Amount
Loan Amount:
Invalid 499 500 Valid 9000 9001 Invalid
Valid Amounts: Integers
Conditions Valid
Partitions
Invalid
Partitions
Valid
Boundaries
Invalid
Boundaries
Loan
Amount
500 - 9000 < 500 500
Integers > 9000 9000
Non-integer
32767
-32768

Example: Car Insurance

Input Partitions & Boundaries
March 4, 2015101
Applicant’s Age: 21 25 41 60
0 - 20 21 - 24 25 - 40 41 - 60 >60
Parked
Street
Garage
Convictions
None
Parking
Speeding
Banned
Health
No Issues
Some Issues
Serious Issues

Output Partitions & Boundaries
Outputs
Total Score: 4 8 13 23
4 – 7
cheap
8 – 12
normal
23 – 37
refused
37
13 -22
pricey

Key to Partitions Table
Partition Input/
output
Valid/
Invalid
Description
P1 Input Valid Age = 0-20
P5 Input Valid Age = >60
P6 Input Valid Parked = Street
P7 Input Valid Parked = Garaged
P8 Input Valid Convictions = None
P9 Input Valid Convictions = Parking
P10 Input Valid Convictions = Speeding
P11 Input Valid Convictions = Banned
P12 Input Valid Health = No issues
P13 Input Valid Health = Some issues
P14 Input Valid Health = Serious issues
P15 Output Valid Output = Cheap Insurance
P16 Output Valid Output = Normal Insurance
P17 Output Valid Output = Expensive Insurance
P18 Output Valid Output = Refused

Key to Boundaries Table
Key
ID or
‘Tag’
Input or
Output
Defined
in
Spec?
Descriptive
text
Boundary Input/
output
Valid/
Invalid
Description
B1 Input Valid Input age ‘0’
B6 Output Valid Output total ‘3’

105
Boundary Value Analysis TCs
Test Case 1 2 3 4 5 6
Input Age -1 0 1 20 21 22
Input Parked Garaged Garaged Garaged Garaged Garaged Garaged
Input Convictions None None None None None None
Input Health No issues No issues No issues No issues No issues No issues
Boundary tested B1 (Age=0) B2 (Age=21)
Valid/Invalid I V V V V V
Total - 13 13 13 8 8
Exp. Output - Expensive Expensive Expensive Normal Normal

Boundary Value Analysis TCs
Test Case 1 2 3 4 5 6
Input Age -1 0 1 20 21 22
Input Parked Garaged Garaged Garaged Garaged Garaged Garaged
Input Convictions None None None None None None
Input Health No issues No issues No issues No issues No issues No issues
Boundary tested B1 (Age=0) B2 (Age=21)
Valid/Invalid I V V V V V
Total - 13 13 13 8 8
Exp. Output - Expensive Expensive Expensive Normal Normal

107
Progress =
Number of distinct boundary values executed
Total number of boundary values
* 100%
Progress: Testing Techniques
Progress =
Number of distinct partitions executed
Total number of partitions
* 100%

Why Do Both EP And BVA?
 Invalid partitions maybe easily missed.
 If the test fails, is the whole partition wrong, or
is a boundary in the wrong place…have to test
mid-partition anyway.
 With BVA we are testing extremes. This does
not give confidence for typical scenarios.

LUNCH

State Transition Testing
 May be useful to model the software
 State Transition Diagram shows:
 States the software can occupy
 Transitions between the states
 Events that cause the transitions
 Actions that result from transitions

The Specification
Example
Windows ‘Folder’ Program

State Transition Diagram
Invert Selection
(IS)
Create (C)
Delete (D)
Invert Selection (IS)
0 Files(0F) 1 Selected File(SF) 1 Unselected File (UF)
Invert Selection (IS)

Level 0-Switch Test Cases
Trans# Start
State
Event Action Finish
State
1 0F Create 1 File
Selected
SF
2 0F Invert
Selection
No files
selected
0F
3 UF Invert
Selection
1 File
selected
SF
4 SF Delete 1 File
Deleted
0F
5 SF Invert
Selection
No files
selected
UF

Proprietary and Confidential114
Logical Execution Order
Trans# Start
State
Event Action Finish
State
1 0F Invert
Selection
No files
selected
0F
2 0F Create 1 File
Selected
SF
3 SF Invert
Selection
No files
selected
UF
4 UF Invert
Selection
1 File
selected
SF
5 SF Delete 1 File
Deleted
0F

Level 1-Switch Test Cases
Test Case 1 2 3 4 5 6 7 8 9
Start State 0F SF UF UF SF SF 0F 0F 0F
Input IS IS IS IS D D C C IS
Next State 0F UF SF SF 0F 0F SF SF 0F
Input C IS IS D C IS D IS IS
Finish State SF SF UF 0F SF 0F 0F UF 0F

State Tables
R S CM
S1 AT/S3 N/S1 D/S2
S2 AD/S4 N/S2 T/S1
S3 N/S3 T/S1 N/S3
S4 N/S4 D/S2 N/S4
N PRESENTS NO OUTPUT THEREFORE AN INVALID TRANSITION

Test Basis: Digital Watch
Reset (R)
Set (S) Change Mode (CM)
Welcome to the digital era!

Change
Date (S4)
Display
Date (S2)
Display
Time (S1)
Change
Time (S3)
Reset (R)
Alter time (AT)
set (S)
display time (T)
Reset (R)
Alter Date (AD)
set (S)
display date (D)
Change mode (CM)
display date (D)
Change mode (CM)
display time (T)
State Transition Diagram

Summary – Black Box Techniques
 Black box Testing concentrates on testing the
features of the systems
 Techniques enable us to maximise testing
 Creates an achievable set of tests that offer maximum
coverage
 Ensure possible areas of Risk are tested
 Black box testing is relevant throughout the testing
process

White Box Testing
 Understand what White Box Testing is
 Look at some of the different types of White Box
Testing

White Box Testing
 What is White Box Testing?
 “Test case selection based on an analysis of the
internal structure of a component”
 Also known as Glass Box testing or Clear Box
testing

White Box Testing
 Why do we need White Box Techniques?
 Provide formal structure to testing code
 Enable us to measure how much of a component has
been tested
 Example
 <100 lines of code
 100,000,000,000,000 possible paths
 At 1,000 tests per second would still take 3,170 years to test all paths

White Box Testing
 To plan and design effective cases requires a
knowledge of the
 Programming language usedf
 Database(s) used
 Operating System(s) used
 And ideally knowledge of the code itself

White Box Testing Techniques
 BS7925-2 lists all the white box test techniques
 Statement Testing
 Branch / Decision Testing
 Branch Condition Testing
 Branch Condition Combination Testing
 Modified Condition Decision Testing
 Linear Code Sequence & Jump
 Data Flow Testing

Statement Testing
 “ A test case design technique for a
component in which test cases are designed
to execute statements”
 “ Test cases are designed and run with the
intention of executing every statement in a
component”

Branch / Decision Testing
 A technique used to execute all branches the
code may take based on decisions made
 Test cases designed to ensure all branches &
decision points are covered

EXAMPLES
PRACTICAL EXAMPLES

Summary
 White box testing can be done immediately after
code is written
 Doesn’t need the complete system
 Does need knowledge of the code
 A combination of all techniques are required for a
successful test
 Don’t rely on just one technique
 Control Flow Graphing is a pre-requisitie

DAY 3

Reviews and Test Process
 What is Static Testing?
 “Testing of an object without execution on a computer”
 How is this done
 By reviewing the system deliverables

Reviews and the Test Process
 Why Review?
 To identify errors as soon as possible in the development
lifecycle
 Reviews offer the chance to find errors in the system
specifications
 This should lead to
 Development productivity improvements
 Reduced development time-scales
 Lifetime cost reductions
 Reduced failure levels

 When to review?
 As soon as an object is ready, before it is used as a
product or the basis for the next step in development
 What to review?
 Anything and everything can be reviewed
 Requirements, System and Program Specifications
should be reviewed prior to publication
 System Design deliverables should be reviewed both
in terms of functionality and technical robustness

 What should be reviewed?
 Program Specifications should be reviewed before
construction
 Code should be reviewed before execution
 Test plans should be reviewed before creating tests
 Test Results should be reviewed before implementation

 The cost of failures
 On-going reviews cost approximately 15% of
development budget
 This includes activities such as the review itself, metric
analysis & process improvement
 Reviews are highly cost effective
 Finding and fixing faults in reviews is significantly cheaper
than finding and fixing faults in later stages of testing

Reviews & the Test Process
 The benefits of reviews
 Reviews save time and money
 Development productivity improvements
 People take greater care
 They have more pride in doing good work
 They have a greater understanding of what they are required to
deliver
 Reduced development costs
 Reduced fault levels
 Reduced lifetime costs

Summary
Reviews enable us to ensure that the systems
specification is correct and relates to the user
requirements
 Anything generated by a project can be reviewed
 In order for them to be effective, reviews must be well
managed

Types of Reviews
A process or meeting during which a work
product, or set of work products, is presented
to project personnel, managers, users or other
interested parties for comment or approval’
BS7925-1

Informal Reviews
 Informal Reviews
As the least formal of all reviews, it can occur at any time and is largely unplanned. It will
include conversations at the photocopier, coffee machine and during breaks. It large
consist of questions like ‘ What do you think of this?’ and normal occurs between peers. No
formal documentation is produce and no record is keep of the review.

Walkthroughs
Walkthroughs
A review process in which a professional leads one or
more members of the development team through a
segment of a document that he or she has written while the
other members ask questions and make comments about
technique, style, possible error, violation of development
standards, and other problems.

Technical Reviews
 Technical/Peer Reviews
A formal meeting at which work products, are presented to
interested parties for comments and approval. Participants are
often peers with no management participation. A technical expert
may be present.

Inspections
 Inspections
A formal evaluation technique in which documents are
examined in detail by a person or group other than the
author to detect errors, violations of development
standards, and other problems.

The Inspection Process
Fagan’s Inspection Process
Kick-off
Checking
Logging
Editing
Entry
Exit
Process Improvement
Change Request
Software Development Stage
Next Development Stage

Review Purposes
Purposes:
FaultFinding
Education
C
onsensus
Walkthroughs
Technical/Peer
Reviews
Inspections

Summary
 There are various types of reviews
 Companies must decide which one(s) are best for them
 In order to gain maximum benefit from them they
must be organized and implemented

BREAK

Static Analysis
 Understand what Static Analysis is
 Look at some of the elements of Static Analysis
 Compiler
 Static Analysis Tools
 Data Flow Analysis
 Control Flow Analysis
 Complexity Analysis

Static Analysis
 What is Static Analysis?
 Analysis of the code without dynamic execution
 Attempting to identify errors in the code
 Provides metrics on flow through SUT and its
complexity
 A form of automation – usually done with tools

Static Analysis
 What do we how to find?
 Unreachable code
 Undeclared variables
 Parameter type mismatches
 Uncalled functions and procedures
 Possible array boundary violations

Incident Management
 An “incident” is when testing cannot progress as
planned
 “any significant, unplanned event that occurs during
the testing that requires subsequent investigation
and / or correction
 The application does not function as anticipated
 When actual results differ from expected results
 Components are missing
 They must be raised against documentation, the
SUT, the test environment or the tests themselves

Raising An Incident
 Immediately when a test case fails
• More than one test case may fail due to the same reason,
resulting in duplicate incidents
• But … information is fresh in your mind
 After the execution of the test set
• Allows the one-to-many relationship between incidents and test
cases to be facilitated better
• But … information is not so fresh in the mind

Incident Record

Incident Record
 Other Incident Information
 Name of Tester
 Test Case ID
 Test Specification ID & version
 Test Script ID & version
 Software Build
 Traced Requirements
 Resolution Details (added by developer

Incident Lifecycle
 Lifecycle ensures that an appropriate
process for incident closure is followed
 May be used to monitor and improve testing
 Status and change histories should be
maintained
 Resolution progress should be tracked

Incident Lifecycle
IEEE 1044 – Standard for Software anomalies
Submitted Assigned Opened Resolved Closed
Duplicate
Submit
Assign Open Resolve Valid
Reject
DuplicateDuplicate

Incident Management Systems
 Use a database platform
 Each incident record is stored as a record in
the incident database
 Facilities to ensure incident lifecycle is
followed
 Some test management systems have built in
incident management systems

Quality Center
DEMO of Quality Center

Summary
 An incident is “any significant, unplanned
event that occurs during the testing that
requires subsequent investigation and / or
correction”
 Incidents should be recorded and tracked
 Analysis of incidents will enable us to see
where problems arose and to aid in test
process improvement

LUNCH

Risk-Based Testing
 Currently a software testing buzz word
 Risk determines the testing schedule
 Higher risk test items are tested first
 Higher risk test items are tested well
 Dependant on thorough Risk Management

 Three main activities:
 Identification of risk
– Does it exist?
 Analysis of risk
– How serious could it be?
 Mitigation of risk
– What should we do about it?

Risk Analysis
 Risk Severity = Likelihood x Impact
 Risk Exposure = Severity x Frequency
 If we use our quantitative values for risk then
we are said to be doing ‘Risk-based testing’

Project and Product Risks
Example Project Risks
 Development team in different location from independent testing
team.
 New, unfamiliar technology introduced to the project.
 Project team inexperienced or not mature.
 Unrealistic time constraints.
Example Product Risks
 Failure will cause loss of revenue
 Failure will cause loss of life
 Failure will affect future business

Testing and Risk
 Risk Management affects testing by:
 Defining an ordering of tests
 Suggesting that some tests are more important
than others
 Dictating the depth of testing to be performed
 Directing focus to most important tests (MITs) –
Prioritising Tests

Possible Risk Criterion
 Impact of failure to stakeholders
 Likelihood of failure
 Visibility to the wider world
 Business priority
 Quantity of change undergone
 How error prone
 Criticality to business
 Technical complexity
 Difficulty of testing
 Cost of testing

DAY 4

Test Estimation, Monitoring & Control
 Understand how we estimate how long we need for
testing
 Understand how we monitor the progress of testing
once it has started
 Understand what steps we take to ensure that the
effort progresses as smoothly as possible

Test Estimation
 The same as estimating for any project
 We need to identify the number of tasks to be
performed, the length of each test, the skills and
resources required and the various dependencies
Testing has a high degree of dependency
You cannot test something until it has been delivered
 Faults found need to be fixed and re-tested
 The environment must be available whenever a test is to
be run

Test Estimation
 Factors to consider
 Risk of failure
 Complexity of Code
 Criticality
 Coverage
 Stability of System Under Test

Test Estimation
 Testing is a tool to aid Risk Management
 Consider the cost of the business of failure of a feature
 Test the most important features as soon as possible
 Be prepared to repeat these tests

Test Estimation Techniques
 Expert Based Techniques
 Previous Experience

 Formula Based Techniques
 % of development effort
 Lines of code
 Function Point Analysis
 Test Point Analysis

BREAK

DEMO of Traceability Matrix

DEMO of DMEA Mainframe
Application

DEMO of DMEA Web Application

LUNCH

Revision and Mock Exam
March 4, 2015176

powerpoint template for testing training

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to powerpoint template for testing training

Similar to powerpoint template for testing training (20)

powerpoint template for testing training

Editor's Notes