CTFL Module 01

CHAPTER01 -
Fundamentals of testing
Based on ISTQB CTFL Syllabus
Presented by –DavisThomas K

After completing
this lesson, you
will be able to
understand:
1. Fundamentals of testing
2.Terminologies in testing
3. Seven testing principles
4. Fundamental test process

K Level
1)The Fundamentals of testing: (K2)
 Why is testing necessary? (K2)
 What isTesting? (K2)
 Testing Principles (K2)
 FundamentalTest Process (K1)
 The psychology ofTesting (K2)

Share your
thoughts !!!
Why is testing necessary ?
Give few examples from your experience
What is the role of tester in software
development ?

Glossary
 Bug
 Defect
 Error
 Mistake
 Failure
 Fault
 Quality
 Risk

Bug - a bug is a coding error in a computer program
An Error found in the development environment before the product is shipped to the customer.
Defect - Commonly refers to several troubles with the software products, with its external behavior or with
its internal features.
Failure -The inability of a system or component to perform its required functions within specified
performance requirements.
Fault - An incorrect step, process, or data definition in a computer program which causes the program to
perform in an unintended or unanticipated manner. See: bug, defect, error, exception.
 When actual result deviates from the expected result while testing a software application or product
then it results into a defect.
 When a defect reaches the end customer it is called a failure and if the defect is detected internally
and resolved it’s called a defect.
 “A mistake in coding is called error ,error found by tester is called
defect, defect accepted by development team then it is called bug ,build
does not meet the requirements then it Is failure.”

What are the causes of software defects?

What is the cost of defects /Quality?

Question?
How much testing is enough ? [select your answer]
a) Test everything
b) Test nothing
c) Test some of the software

Exam points
Understand--
Testing & risk
Testing & quality
Testing exit criteria
 E.g.) How many test cases are requires to test a 15 input fields
each having a possible 5 value requires ?
Note : -
1.Testing everything is not a solution
2. Exhaustive testing is impossible
3.Test based on Priority / severity and risk
4. How much you need to test based on level of risk
5 . Based on constraints [Time / Budget] , a tester need to
determine how much testing is minimal adequate to cover the risk.

Exit criteria
[Exam point]
 Verify if All tests planned have been run.
 Verify if the level of requirement coverage has been met.
 Verify if there are NO Critical or high severity defects that are left
outstanding.
 Verify if all high risk areas are completely tested.
 Verify if software development activities are completed within the
projected cost.
 Verify if software development activities are completed within the
projected timelines.

what is
Testing?
(K1 / K2)
1.Commonobjectives of
testing
2.Purposeoftesting
 Testingistheprocessconsistingofalllifecyclesactivities,both
staticanddynamic,concernedwithplanning,preparationand
evaluationofsoftwareproductsandrelatedworkproductsto
determinethattheysatisfyspecifiedrequirements,to
demonstratethattheyarefitforpurposeandtodetectdefects.

Testing is the
process
 Testing Is the process consisting of all life cycles activities , both
static and dynamic , concerned with Planning , preparation and
evaluation of software products and related work products to
determine that they satisfy specified requirements , to
demonstrate that they are fit for purpose and to detect defects.
Process: Testing is a process rather than a single activity.
There are a series of activities involved

All LifeCycle
Activities
demonstrate that they are fit for purpose and to detect defects
Testing is a process that’s take place throughout the Software
Development Life Cycle (SDLC).
Chapter 2 , we will study in detail about testing as a process that
take place throughout SDLC

All LifeCycle
Activities
(Cont..)
 What is test basis ?
Test basis is defined as the source of information or the document
that is needed to write test cases and also for test analysis.
 Examples
 Requirement document
 Test Plan
 Business Requirement
Test basis should be well defined and adequately structured so
that one can easily identify test conditions from which test cases
can be derived.

Static and
dynamic
Chapter 3 explains about static and dynamic testing
 Static –Test and find defects without executing code
 Dynamic – Code is executed to demonstrate the result

Planning ,
preparation
and
evaluation
Planning: We need to plan as what we want to do.We control the
test activities, we report on testing progress and the status of the
software under test. [Chapter 5]
 Preparation: We need to choose what testing we will do, by
selecting test conditions and designing test cases. [Chapter 4]
 Evaluation: During evaluation we must check the results and
evaluate the software under test and the completion criteria,
which helps us to decide whether we have finished testing and
whether the software product has passed the tests.

Software
products and
related work
products
Here we don’t just test code, we test the design, requirements

Second part of
definition
Testing Is the process consisting of all life cycles activities , both
Second part of definition describes the
objective for testing

Determine
that they
satisfy
specified
requirements
Testing Is the process consisting of all life cycles activities , both
Here we test software to requirements

Demonstrate
that they are
fit for purpose
Here we check if the software helps the customer to do their
defined task

Detect defects
 Finding defect help us to identify risk
 RCA of defects

Test objective
(K1)
Few test objectives
Share your ideas

Here are few test objectives
1. Finding defects which may get created by the programmer while
developing the software.
2. Gaining confidence in and providing information about the level of
quality and to gain the confidence of the customers by providing them a
quality product.
3.To prevent defects.
4.To make sure that the end result meets the business and user
requirements.
5.To ensure that it satisfies the BRS that is Business Requirement
Specification and SRS that is System Requirement Specifications.

Severity and
Priority
 Severity it defines the impact that a given defect has on the system
Severity types : Critical , Major , Moderate, Minor, Cosmetic
Critical: The defect that results in the termination of the complete system
or one or more component of the system and causes extensive corruption
of the data. [Show stopper] . No alternative method.
Major: The defect that results in the termination of the complete system
or one or more component of the system and causes extensive corruption
of the data.There is a alternative method.
 Moderate: The defect that does not result in the termination, but causes
the system to produce incorrect, incomplete or inconsistent results
Minor: The defect that does not result in the termination and does not
damage the usability of the system and the desired results can be easily
obtained by working around the defects.
 Cosmetic: The defect that is related to the enhancement of the system
where the changes are related to the look and feel of the application.

Severity and
Priority
 Priority defines the order in which we should resolve a defect.
Should we fix it now, or can it wait? .
Priority can be of following types:
 Low: The defect is an irritant which should be repaired, but repair
can be deferred until after more serious defect have been fixed.
 Medium: The defect should be resolved in the normal course of
development activities. It can wait until a new build or version is
created.
 High: The defect must be resolved as soon as possible because
the defect is affecting the application or the product severely.The
system cannot be used until the repair has been done.

Few very important scenarios related to the severity and priority which are asked during the Exam:
High Priority & High Severity: An error which occurs on the basic functionality of the application and will
not allow the user to use the system. (E.g.. A site maintaining the student details, on saving record if it,
doesn’t allow to save the record then this is high priority and high severity bug.)
High Priority & Low Severity: The spelling mistakes that happens on the cover page or heading or title of
an application.
High Severity & Low Priority: An error which occurs on the functionality of the application (for which there
is no workaround) and will not allow the user to use the system but on click of link which is rarely used by the
end user.
Low Priority and Low Severity: Any cosmetic or spelling issues which is within a paragraph or in the report
(Not on cover page, heading, title).

TESTING
PRINCIPLES
(K2)
There are 7 principles
1. Testing shows presence of defects
3. Early testing
4. Defect clustering
5. Pesticide paradox
6.Testing is context depending
7. Absence – of – errors fallacy

Testing
principle #1
1. Testing shows presence of defects
Testing can show that defects are
present.
But cannot prove that there are no
defects.
Testing reduces the probability of
undiscovered defects remaining in the
software
But, even if no defects are found, it is
not a proof of correctness

Testing
principle #2
Testing everything (all combinations of
inputs and preconditions) is not feasible
except for trivial cases.
Instead of exhaustive testing, we use
risks and priorities to focus testing efforts

“Never allow the same
bug to bite you twice.”

Testing
principle #3
3. Early testing
Testing activities should start as early as
possible in the software or system
development life cycle
should be focused on defined objectives.

Testing
principle #4
4. Defect clustering
A small number of modules contain
most of the defects discovered during
prerelease testing or show the most
operational failures

Testing
principle #5
5. Pesticide paradox
If the same tests are repeated over and
over again, eventually the same set of
test cases will no longer find any new
bugs.
To overcome this 'pesticide paradox',
the test cases need to be regularly
reviewed and revised, and new and
different tests need to be written to
exercise different parts of the software or
system to potentially find more defects.

Testing
principle #6
6.Testing is context dependent
Testing is done differently in different
contexts.
For example, safety-critical [Security]
software is tested differently from an e-
commerce site.

Testing
principle #7
7. Absence – of – errors fallacy
Finding and fixing defects does not help if
fallacy the system built is unusable and does
not fulfill the users' needs and expectations.

Before we move in to Fundamentals of testing,
Lets learn some basic concepts

Lets learn
Test Plan
 Test plan is a document that lists all the activities in a QA project,
schedules them, defines the scope of the project, roles &
responsibilities, risks, entry & exit criteria, test objective and
anything else that you can think of
 Example:Test plan gives the information of who is going to test at
what time. For example: Module 1 is going to be tested by “X
tester”. If testerY replaces X for some reason, the test plan has to
be updated.
 TheTest Plan document is usually prepared by theTest Lead or
Test Manager and the focus of the document is to describe what
to test, how to test, when to test and who will do what test.

Lets learn
TestStrategy
 ATest Strategy document is a high level document and normally
developed by project manager.This document defines “Software
Testing Approach” to achieve testing objectives.
 TheTest Strategy is normally derived from the Business
Requirement Specification document.

Ref : PMBOK5
 Business requirement
 Statement of works
 Agreements
 Project charter
 Difference between Project / Product and Portfolio ?

Test PlanVs
TestStrategy
Test Plan Components Test Strategy Components
Test Plan id Scope and Objectives
Introduction Business issues
Test items Roles and responsibilities
Features to be tested Communication
Testing tasks status reporting
Suspension criteria Test automation and tools
Features pass or fail criteria
Testing measurements and
metrices
Test environment
(Entry criteria, Exit criteria)
Risks and mitigation
Test deliverables Defect reporting and tracking
Staff and training needs
Change and configuration
management
Responsibilities / Schedule Training plan
Features not to be tested Test deliverables
Test techniques Industry standards to follow

Lets learn –
TestScenario
 ATest Scenario is any functionality that can be tested. It is also
calledTest Possibility
 Examples test scenarios:
 1.Validate if a new country can be added by the Admin
2.Validate if an existing country can be deleted by the admin
3.Validate if an existing country can be updated

Lets learn –
TestCondition
 Test conditions on the other hand are more specific. It can be
roughly defined as the aim/goal of a certain test.
 Example test condition:
In the previous example, if we were to test the scenario 1,
we can test the following conditions:
1. Enter the country name as “India”(valid )and check for the
addition of the country
 2. Enter a blank and check if the country gets added.
So here details are more precise

Lets learn –
TestCase
 ATest Case is a set of actions executed to verify a particular
feature or functionality of your software application.
 Test scenarios are rather vague and cover a wide range of
possibilities.Testing is all about being very specific. Hence, we
needTest Cases

Lets learn –
Test procedure
 ATest procedure is a combination of test cases based
on a certain logical reason, like executing an end-to end
situation or something to that effect.
 The test cases above are grouped to achieve a certain
target at the end of them
 test procedures have a few test cases combined at any
point of time.

Lets learn –
TestSuite
 Test Suite is the list of all the test cases that have to be
executed as a part of a test cycle or a regression phase
etc.
 There is no logical grouping based on functionality.The
order in which the constituent test cases get executed
may or may not be important.

Lets learn –
Test Basis
 To create test cases you need to look at something to
base your test.This is nothing butTest Basis.
 This test basis could be the actual Application Under
Test (AUT), or may be even by experience but most of
the times, like, in this case, would be based
on documents
 Basically it’s a documentation on which test cases are
based, such as requirements, design specifications,
product risk analysis, architecture and interfaces.

Lets learn –
Test Log
 A test log is nothing but, what are the test cases that
we executed, in what order we executed, who executed
that test cases and what is the status of the test case
(pass/fail).
 These descriptions are documented and called as test
log

Fundamental
test process
We can divide the activities within the
fundamental test process into the following
basic steps:
• planning and control
• analysis and design
• implementation and execution
• evaluating exit criteria and reporting
• test closure activities.

Test planning
and control
 Test planning has following major tasks:
i. To determine the scope and risks and identify the
objectives of testing.
ii.To determine the test approach.
iii.To implement the test policy and/or test strategy
iv.To determine the required test resources like people,
test environments, PCs, etc.
v.To determine the Exit criteria we need to set criteria
such as Coverage criteria.

Test planning
and control
Test control has the following major tasks:
i. To measure and analyze the results of reviews
and testing.
ii. To monitor and document progress, test
coverage and exit criteria.
iii. To provide information on testing.
iv. To initiate corrective actions.
v. To make decisions.

TestAnalysis
and Design
 Test Analysis and Design has following major tasks:
i. To review the test basis.
ii. To identify test conditions.
iii. To design the tests.
iv. To evaluate testability of the requirements and
system.
v. To design the test environment set-up and identify
and required infrastructure and tools.

Test
Implementation
and Execution
Test implementation has the following major
task:
i.To develop and prioritize our test cases by
using techniques and create test data for those
tests.
ii.To create test suites from the test cases for
efficient test execution.

Test
Implementation
and Execution
Test execution has the following major task:
i. To execute test suites and individual test cases
following the test procedures.
ii.To re-execute the tests that previously failed in order
to confirm a fix.This is known as confirmation testing or
re-testing.
iii.To log the outcome of the test execution and record
the identities and versions of the software under tests.

Evaluating Exit
criteria and
Reporting
Evaluating exit criteria has the following major
tasks:
i. To check the test logs against the exit criteria specified
in test planning.
ii. To assess if more test are needed or if the exit criteria
specified should be changed.
iii. To write a test summary report for stakeholders.

TestClosure
activities
Test closure activities have the following major tasks:
i. To check which planned deliverables are actually
delivered and to ensure that all incident reports have
been resolved.
ii.To finalize and archive testware such as scripts, test
environments, etc. for later reuse.
iii.To handover the testware to the maintenance
organization.They will give support to the software.
ivTo evaluate how the testing went and learn lessons for
future releases and projects.

Psychology of
testing
 We are pleased because we found a good bug but how will the
requirement analyst, the designer, developer, project manager
and customer react.
 The people who build the application [dev] may react defensively
and take this reported defect as personal criticism.
 The project manager may be annoyed with everyone for holding
up the project.
 The customer may lose confidence in the product because he can
see defects.
Because testing can be seen as destructive activity we need to
take care while reporting our defects and failures as objectively
and politely as possible.

Q &A
Lets try to answer few questions
based on chapter 01

Question 1 A company recently purchased a commercial off-
the-shelf application to automate their bill-paying process.
They now plan to run an acceptance test against the package
prior to putting it into production.
Which of the following is their most likely reason for testing?
a. To build confidence in the application.
b. b.To detect bugs in the application.
c. c.To gather evidence for a lawsuit.
d. d.To train the users.

Question 2 According to the ISTQB Glossary, the word 'bug'
is synonymous with which of the following words?
a. Incident
b. Defect
c. Mistake
d. Error

Question 3 According to the ISTQB Glossary, a risk relates to
which of the following?
a. Negative feedback to the tester.
b. Negative consequences that will occur.
c. Negative consequences that could occur.
d. Negative consequences for the test object

Question 4 Ensuring that test design starts during the
requirements definition phase is important to enable which of
the following test objectives?
a. Preventing defects in the system.
b. Finding defects through dynamic testing.
c. Gaining confidence in the system.
d. Finishing the project on time.

Question 5 A test team consistently finds between 90% and 95% of the defects
present in the system under test. While the test manager understands that this is
a good defect-detection percentage for her test team and industry, senior
management and executives remain disappointed in the test group, saying that
the test team misses too many bugs. Given that the users are generally happy
with the system and that the failures which have occurred have generally been
low impact, which of the following testing principles is most likely to help the test
manager explain to these managers and executives why some defects are likely
to be missed?
a. Exhaustive testing is impossible
b. Defect clustering
c. Pesticide paradox
d. Absence-of-errors

Question 6 According to the ISTQB Glossary, regression testing is required for
what purpose?
a.To verify the success of corrective actions.
b.To prevent a task from being incorrectly considered completed.
c.To ensure that defects have not been introduced by a modification.
d.To motivate better unit testing by the programmers

Question 7 Which of the following is most important to promote and maintain
good relationships between testers and developers?
a. Understanding what managers value about testing.
b. Explaining test results in a neutral fashion.
c. Identifying potential customer work-arounds for bugs.
d. Promoting better quality software whenever possible.

Question 8 Which of the statements below is the best assessment of how the
test principles apply across the test life cycle?
a.Test principles only affect the preparation for testing.
b.Test principles only affect test execution activities.
c.Test principles affect the early test activities such as review.
d.Test principles affect activities throughout the test life cycle.

Question 9 When is testing complete?
A.When time and budget are exhausted.
B.When there is enough information for sponsors to make an informed decision
about release.
C.When there are no remaining high priority defects outstanding.
D.When every data combination has been exercised successfully.

Question 10 Which of the following is correct?
Debugging is:
A.Testing/checking whether the software performs correctly.
B. Checking that a previously reported defect has been corrected.
C. Identifying the cause of a defect, repairing the code and checking the fix is
correct.
D. Checking that no unintended consequences have occurred as a result of a fix.

Question 11 The five parts of the fundamental test process have a broad
chronological order.Which of the options gives three different parts in the
correct order?
A. Implementation and execution, planning and control, analysis and design.
B. Analysis and design, evaluating exit criteria and reporting, test closure
activities.
C. Evaluating exit criteria and reporting, implementation and execution,.
D. Evaluating exit criteria and reporting, test closure activities, analysis and
design.

Question 12 Which option is part of the ‘implementation and execution’ area
of the fundamental test process?
A. Developing the tests.
B. Comparing actual and expected results.
C.Writing a test summary.
D. Analyzing lessons learnt for future releases.

CTFL Module 01

More Related Content

What's hot

Similar to CTFL Module 01

Recently uploaded

CTFL Module 01

Editor's Notes