Software testing basic concepts

© Copyright 2006 FPT Software 1© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2
Software TestingSoftware Testing
Basic ConceptsBasic Concepts

AgendaAgenda
 Duration: 1.5 Hours
 Purpose: Understand basic Test concepts
 Audience: All Members
 Content:
1. Introduction
2. Defect concepts
3. Test process
4. Test plan & test design
5. Testing strategy

IntroductionIntroduction
Objectives:
To arrive at a common understanding of the scope of testing activity
To familiarize the audience with the terminology used in the testing work
Understand the required test skills of developers

Typical Project Blues…Typical Project Blues…

Testing definitionTesting definition
Testing is the process of exercising or evaluating a system or
system component by manual or automated means to verify
that it satisfies specified requirements - (IEEE 83a)
Process:
 Sequence of steps performed for a given purpose. (IEEE)
Software Process
 A set of activities, methods, practices, and transformations
that people use to develop and maintain software and
associated products.(SEI-CMM)

Testing ObjectivesTesting Objectives
Primary: Execute a program with the intent of finding errors to
 Determine whether system meets specifications
 Determine whether system meets user’s needs
Secondary
 Continuously improve the testing process

MisunderstandingsMisunderstandings
about testingabout testing
Testing is debugging
Testing is not the job of a programmer
If programmers were more careful testing would be
unnecessary
Testing never ends
Testing activities start only after the coding is complete
Testing is not a creative task

Theory & RealityTheory & Reality
Theory
 Need sufficient time
 Baselined & freezed requirements
 Automation testing
 Have trained testers: test process, business flow,…
Reality
 Time Pressures to keep delivery date
 Continuously updated requirements
 Don’t know when to stop: Update & update Requirement
continuously (CRs)  Tester updates TC and re-tests
 Manual testing
 Lack of dedicated software test environment: share, steal
 Inexperienced testers without appropriate training

Defect conceptsDefect concepts
Objectives:
Understand the definition of defect
 Understand the cost of a bug to software development
 Understand the common types of defects and their
perceived severity
What is defect?What is defect?
A defect is any error found by testing and reviewing
activities (All errors found by internal reviewer, external
reviewer and customer).

Where defects come from?Where defects come from?
Products
 SRS, DD, Source code, Test cases, etc.
Quality Control
 Review, Test, Audit, Inspection
Processes
 Requirements, Design, Coding, Test, etc.
Other sources:
 Change requests, misunderstanding, carelessness, planning,
etc.

Cost of defectsCost of defects

Defect still exists in the product, change status to ERROR
FSoft Defect lifecycleFSoft Defect lifecycle
Defect classification:
 Open defects
 Closed defects
 Leakage
Log defects Analyze defects Fix defects Re-test
Step 1 Step 2 Step 3 Step 4
What: To log defect into DMS with
clear information: project, module,
stage, QC activity, severity, type,
due date, tester, created date ...
Who: Tester
Defect’s status: ERROR
What: -To analyze the causes of the defect, find-
out solution and assign to person who is
responsible for correcting it.
- To accept the defect if it’s acceptable (by
concession of authority or customer).
Who: Project Leader
Defect’s status: ASSIGNED / ACCEPTED
What: To correct the defect and
perform unit testing to make sure
the defect has been removed.
Who: Developer
Defect’s status:
CORRECTED
What: To retest the function(s)
containing the defect and make sure
that the defect has been corrected
completely
Who: Tester
Defect’s status: CLOSED

Defect status:
• Error
• Assigned
• Corrected
• Cancelled
• Accepted: by PM/PTL/Customer
• Closed: by tester/PM/PTL
Leakage Defects:
• Acceptance test
• After Release test
• After Release review
Fsoft Defect classificationFsoft Defect classification

Defect type
• Functionality
o Requirement misunderstanding
o Feature missing
o Coding logic
o Business logic
• User Interface
• Performance
• Design issue
• Coding standard
Severity: How bad the problem is
(some time disagreement btw. Dev & Tester)
 Fatal: the system is broken
 Serious: can not work around
 Medium: impact the function
 Cosmetic: not impact the function

Defect origin
 Requirement
 Design
 Coding
 Test
 CM
 Document control
Priority
 Immediately
 High priority
 Normal priority
 Low priority
Work product
 Software module
 Software Package
 ADD
 DDD
 Test Cases (UT/IT/ST)
Quality control activity
 Code review
 Unit test
 Integration test
 System test
 Document review
 Inspection
 Audit

Test ProcessTest Process
Objectives:
 Understand the steps followed for a test process
 Understand test planning process
 FSoft Test Process

Tester CompetencyTester Competency
Soft skills (non-technical attributes)
 Discipline and Perseverance
• Testing is repetitive and requests a lot of manual effort
• Have the ability to withstand pressures and workload
• Say ‘no’ to managers when quality is insufficient
 Reading skills: study many docs of specs, design
 Communication & Interpreter skills
• Both Verbal and Written communication
• Diplomatic skill
• Be able to communicate with technical and non-technical
people, engineers, managers, customers
 Negative thinking: foresee things that can go wrong, evaluate risks
 Attitude: ‘test to break’ attitude
 Time management and Effort prioritization

Tester CompetencyTester Competency
Technical skills
 Familiarity with software development architectures,
processes
 Familiarity with testing methodologies
 Very clearly articulate specs, designs, business rules,
inspection reports, configurations, code changes, TP,
TC, bug reports, user manuals,…
 Know how and where to look for bugs
 Know how to report the defects effectively for quickly
reproduce

The Test ProcessThe Test Process
Data Analysis
- Analyze
root
Causes of
defects
- Identify
actions
Test Execution
Perform
testing
Bug
Reporting
- Bug Fixing
- TC Updating
Bug Tracking
(Re-test)
Test
Report
Test Preparation
SRS study
Create
Test
Design
Review
&
Approval
- Develop test
cases, test
scripts
- Prepare test
data
Test
Planning
Create
Test Plan
(by
Test leader)
Definition Solution Construction Transition Termination
Initiation Project life cycle
Test process

Input/Output for testingInput/Output for testing
Input:
• Customer requirements and Acceptance criteria
• Change requests
• Software Requirement Specification (SRS)
• Design documents (ADD, DDD)
• Programs (Modules)
Output:
• Test documents: Test plan, Test cases and procedures,
Test script, Test data
• Defect list
• Test report (Pass/Fail of each TC, Summary report)

FSoft Test resourcesFSoft Test resources
Guideline: Test process & Test guideline
Templates for test documents
 Test Plan
 Test case specification
 Test report
 Defect analysis report
FSOFT tools
 Defect tracking tool: DMS
 Test Effort tracking tool: Timesheet
 Test schedule: FSoft Insight
 Test automation tools
– Rational Robot (Functional & Performance test)
– OpenSTA (Open source)
– Witir (Open source)
FATA: Test Council

Test strategyTest strategy
Objectives
 Understand Unit Testing concept
 Understand Integration Testing concept
 Understand System Testing concept
 Understand Acceptance Testing concept
 Overview of Test types

V-ModelV-Model

Testing stagesTesting stages
There are 4 stages of Testing:
 Unit Test
 Integration Test
 System Test
 Acceptance Test

Unit TestUnit Test
 Purpose: to verify that the component/module
functions properly
 Check:
– internal data structures
– Logic
– boundary conditions for input/output data
 Normally: White box oriented
 Doer: Development team
 Metrics: Test coverage:
– Statement
– Branch
– Path
– Condition

Integration TestIntegration Test
 Purpose:
• To ensure that code is implemented and designed
properly
• to take unit tested modules and build a program
structure that has been dictated by design
 IT only after UT
 Combining the individual components to uncover
errors associated with interfacing
 Normally: Black box oriented
 Doer: independent Test team

System TestSystem Test
 Purpose: To ensure that the system does what the
requirement specifies
 Types
– Function Testing
– Performance Testing
– Installation Testing
 Doer: independent Test team

Acceptance TestAcceptance Test
 Customer’s way to verify that what was wanted is
what is built
 Uncovers more than requirements discrepancies
 Allows the customers to determine what they really
want, whether specified in the document or not.
 New problems may arise
 Customers may decide that the problem as changed
and a different solution is needed
 Doer: Customer/independent Test team

Testing typesTesting types
Functional test
 Function testing
 User Interface testing
 Data and database integrity testing
 Business cycle testing
User Interface: Layout
Performance test
 Performance profiling
 Load testing
 Stress testing
 Volume testing
Access control & Security test
Regression test

Test techniques/methodsTest techniques/methods
White Box Testing
 It guarantees that all independent paths within a module
have been exercised at least once
 Exercise all logical decisions on their true an false sides
 Execute all loops their boundaries and within their
operational bounds
 Exercise internal data structures to assure their validity
Black Box Testing
 Focuses on functional requirement of the software
 Derive sets of input conditions that will fully exercise all
functional requirements for a program
 Not an alternative to white box

Test Plan, Test DesignTest Plan, Test Design
& Test Cases& Test Cases
Objectives
 Understand the principles of test plan & test design
 Understand the concepts for test case design
 Understand the structure of test cases

Test Planning: ObjectivesTest Planning: Objectives
Test planning is to:
 Define Test strategy:
– What to test: list of requirements to test
– How to test (Test techniques)
– Test types
– Test tool, ….
 Define the responsibilities
 Decide test metrics & manage through metrics
 “When to Stop Testing?”
– Not good Unit Test
– Low quality of code,…
– Meet requested Test coverage
– All Test cases are tested and passed

Test PlanTest Plan
 What to do
 Who does it: Test team, Development team
 How to do it: Techniques
 When to do it

Test PlanTest Plan
Test Plan to define:
 Scope of test: stages and types of test
 Risks may affect testing: quantify the loss of the risk
 Training needs for testers
 Requirements to test: what will be tested
 Testing strategy: how will the test be performed
 Completion criteria (Pass/Fail)
 Criteria to stop testing
 Tools to be used for testing
 Testing resource and environment (Responsibilities of testers)
 Test Milestone: schedule of testing activities
 Test deliverables: outputs of testing

When to Stop Testing?When to Stop Testing?
Not good UT: Defects of UT do not meet Defect plan
Low quality of code: too many defects (over the number
identified in planning phase/kick-off meeting)
Does not meet requested Test coverage (by
customer/PM).
It must be identified at planning phase.

Test design structureTest design structure
Specifications
 Basic flow
 Alternative flow
Test Design
 All test cases should be traceable to requirements
 Design framework and rules for test cases (large/average
items)
 Plan number of test cases for normal, abnormal & boundary
data
 Assign testers for module/function/large items

Test DesignTest Design
Structure/Framework for Test cases (TC):
• Modules or functions
• Large items
• GUI
• Functions
• Performance
Number of TC:
 Total
 For each module
 For each type: GUI/Function
 For normal/abnormal/boundary cases

Test CaseTest Case
Test environment
Test condition
A test case includes:
• Action (sample: click on Add button, on a link, input birth
day…)
• Input data (valid/invalid/boundary data)
• Expected result (message, …)
 Good test case:
 Structure of TC is clear and reasonable.
 Follow requirement closely.
 Cover all of cases that can occur.
 Description and test conditions is brief and easy to
understand
 Procedure and expected result is a step by step process

Testing MeasurementTesting Measurement
Objectives
 Understand the need for metrics and measurement in
Testing
 Understand the minimal metrics required for the
measurement of the testing activity

Examples of testing measurement?Examples of testing measurement?
 Product: We’ve tested 80% of the lines of code
 Plan: We’ve run 80% of the test cases
 Results: We’ve discovered 593 defects
 Effort: We’ve worked 80 hours a week on this for 4 months.
We’ve run 7,243 tests
 Quality of Testing: Beta testers have found 30 defects that we
missed. Our regression tests seems ineffective
 History across projects: At this milestone on previous projects,
we had fewer than 12% of the defects found still open. We
should be at that percentage on this product too

How do we measure Testing inHow do we measure Testing in FSOFTFSOFT??
Measurements:
 Defects in DMS
 Test effort in Timesheet
 Defect rate in Fsoft Insight: Weighted defects/ project size (in UCP)
 Test coverage: number of executed test cases/ Total number of test
cases
 Test successful coverage: number of test cases executed
successfully/total number of test cases
 Defect removal efficiency
Metrics to assess tester performance:
 Test effectiveness
• Weighted defects/ Test execution effort
 Leakage
• Weighted defects found after release/ project size

Q & AQ & A

Software testing basic concepts

More Related Content

What's hot

Viewers also liked

Similar to Software testing basic concepts

Recently uploaded

Software testing basic concepts

Editor's Notes