2. I. Software Product testing: roles and responsibilities, cycles.
I.I. Understanding the role of Tester -------------------------------------------------------- Slide 3
I.II. Software Product development cycles ------------------------------------------------- Slide 4
I.III. Software Product testing cycle --------------------------------------------------------- Slide 5
I.IV. Software Product testing definition ---------------------------------------------------- Slide 6
II. Software Product testing: documents (artifacts),tools, templates.
II.I. Software Product testing artifacts ------------------------------------------------------- Slide 7
II.II. Software Product testing tools ---------------------------------------------------------- Slide 8
III. Software Product testing: levels, methods, technics, types
III.I. Software Product development/testing is like House building --------------------- Slide 9
III.II. Manual – Automation testing ---------------------------------------------------------- Slide 10
III.III. Black Box, White Box, Gray Box testing ------------------------------------------- Slide 11
III.IV. Unit, Integration, System, Acceptance testing -------------------------------------- Slide 12
III.V. Testing levels, methods, approaches, technics, types scheme --------------------- Slide 13
III.VI. Testing short notions in alphabetical order ------------------------------------------ Slide 14-19
CONTENTS
3. I need a
Software
Product. It must
be qualitative,
correct, complete
and secure!
This is a
Customer These are testers
(Testing team)
We will test this
Software Product!
These are developers
We will develop this
Software Product!
Product
Software
CORRECT
COMPLETE
QUALITATAIVE
SECURE
ROLES AND
RESPONSIBILITIES
I will manage.
Everything
This is a Project Manager
4. SOFTWARE PRODUCT
how to develop?
Software Development methodology
Traditional Software Development Life Cycle
methodology (SDLC)
Agile methodology
Waterfall
Spiral
Incremental
Prototyping
Rapid Application
Development
(RAD)
Scrum
Extreme
Programming
(XP)
Adaptive Software
Development (ASD)
Dynamic System
Development method
(DSDM)
It’s our part
of work!
Developers
5. SOFTWARE PRODUCT
how to test?
We will develop the Software
Product using Waterfall Software
Development Life Cycle
methodology (SDLC)
Requirement
Analysis
Test Planning
Test
Development
Test Execution
Test Reporting
Test Result
Analysis
Defect (bug)
Retesting
Regression
Testing
Test Closure
Software Testing Life Cycle (for Waterfall
Development Model)
Ok! Then we
test the
Software
Product like this
Developers
Testers
Tester!
Don’t create any testing
document, don’t test
anything, don’t do
anything BEFORE
READING THE
REQUIREMENT!!!
REQUIREMENTS – the requirements of
a software product is a description of what
the customer wants the software product
be able to perform.
Requirements Gathering
and Analysis
System
Design
Implementation
Integration
and Testing
Deployment
of System
Maintenance
Waterfall SDLC
6. Software testing is a process of a
software product exploration.
Purpose of this process is finding
ERRORS in a software product (errors
called BUGS)
Testers check if a
Developing/Developed Software
product (Actual Results)
match the Requirements (Expected
Results)
Testers identify QUALITY,
CORRECTNESS, COMPLETENESS,
SECURE of a developed/developing
software product
Testers VERIFY, VALIDATE a
developed/developing software product
SOFTWARE TESTING
SOFTWARE PRODUCT TESTING –
What’s that?
7. What Testing
Documents do we
need to test the
Software Product?
TESTING DOCUMENTS, aka ARTIFACTS
Test Plan
Test Cases
Testing Effort Estimation
Test Case Execution Report
Defect (bug) Report
Test Closure report
Test Metrics
Testers
1. ID, or test case number
2. Steps to reproduce
3. Expected results
4. Status (passed, failed,
blocked, not executable)
I. BUG Characteristics:
1. Title
2. Reporter
3. Product (component)
4. Platform
5. OS
6. Priority
7. Severity
8. Status
9. Assigned to
10. URL
II. BUG description:
1. Summary (short)
2. Detailed description
III. BUG pics
1. Screenshot/s
SOFTWARE PRODUCT TESTING
ARTIFACTS
8. MOST POPUAR DEFECT (BUG, ISSUE) TRACKING TOOLS
BUGZILLA, web-based general purpose bug tracker. It has all the features of essence, convenience and assurance, simple to use. Free and
open source. Website: http://www.bugzilla.org/.
HP Quality Center. HP ALM is an end-to-end test management solution with an integrated bug tracking mechanism within it. HP Quality
Center can be used on Windows platforms with an Internet Explorer browser. In combination with the HP ALM Explorer Add-in, it can be
executed as a normal desktop application. It is commercial and has a free trial. Website: http://www8.hp.com/us/en/software-solutions/quality-
center-quality-management/index.html?jumpid=va_R11374_us/en/large/eb/go_qc
IBM Rational ClearQuest - client-server based web application that supports defect management process. It has an end-to-end, customizable
defect tracking systems. It is commercial and has a free trial. Website: www.ibm.com/software/awdtools/clearquest/
JIRA - used for bug tracking, issue tracking and project management. Supports agile projects. JIRA is a commercial tool but Atlassian
provides JIRA for free to open source projects that meet certain criteria, and organizations that are non-profit, non-government, non-academic,
non-commercial, non-political, and secular. Website: http://www.atlassian.com/software/jira
Mantis Bug Tracker, web-based bug tracking system. MantisBT supports the sending of e-mail notifications upon changes being made to
issues in the system. It also has integration with Twitter via a plugin, to allow a notification to be tweeted when an issue is resolved. Free and
open source. Website: http://www.mantisbt.org/.
Some useful links with Testing Tools:
http://www.ministryoftesting.com/resources/software-testing-tools/ big list of testing tools;
http://www.softwareqatest.com/qatweb1.html list of more then 580 tools;
http://en.wikipedia.org/wiki/Comparison_of_issue-tracking_systems comparison of issue tracking tools.
9. Let’s call a Software Product
– a HOUSE
Developers need to develop a Software Product
=
Developers need to build a House
Testers need to test a Software Product =
Testers need to check if the House meets the
Customer/User of the House expectations
or House
Software Product,
SOFTWARE PRODUCT
CONSEPT
10. SOFTWARE TESTING:
MANUAL,
AUTOMATION
MANUAL testing-
tester plays the role of end user
no automated tools, no scripts
used
when manual testing is preferred
over automation?
little time to automate (ex. a
tight deadline to release the
software
UI (user interface), as it’s
changed very frequently.
AUTOMATION testing
tester writes scripts and uses
automated tools to test the
software.
what should be automated?
automation is good for checking,
verifying, validating,
any tests that need to be repeated
often: ex. regression tests,
agile testing and eXtreme
programming.
Testers
Software Testing
11. Tester pretends to be a
Customer/User of the House.
Tester knows NOTHING about
the house building.
Black-
Box
Testing
Tester doesn’t know the internal structure of
code - this method of software testing is called
Black-Box Testing.
TESTING METHODS
BASED ON KNOWLEDGE OF CODE INTERNAL STRUCTURE
Tester pretends to be a Builder
of the House. Tester knows
EVERYTHING about the house
building.
Tester knows the internal structure of
code - this method of software
testing is called White-Box Testing.
White-
Box
Testing
Tester pretends to be a Student of the
Builder College. Tester knows
SOMETHING about the house
building.
Tester has SOME knowledge of internal
structure of code but NOT DETAILED.
- this method of software testing is
called Gray-Box Testing. This is a
combination of Black Box and White
Box testing methods.
Gray-
Box
Testing
12. The BRICK of the House, Unit, need to be tested by a Builder. The smallest part/function of the Software Product is tested by:
- developers, White Box testing method used + Regression
UNIT TESTING LEVEL
INTEGRATION TESTING
LEVEL
The House, System, need to be tested by a User of this House.
Here, bricks, integrated together, windows, doors, weather conditions,
rooms overloaded with guests ... are tested as a whole System.
Testers validate the software as a whole (software, hardware and network).
Testers use Black-Box testing method. Testers perform:
- Functional testing, Non-Functional testing;
- Regression testing, Smoke testing
The smallest integrated together parts of the Software Product, are
tested as a group by:
1. developers, White Box testing method used ( to ensure description of
major control paths).
2. testers, Black-Box, Gray Box testing methods used.
Testers perform: - Non-Functional testing;
- Regression testing, Smoke testing
The BRICKS, integrated into a WALL, need to be tested as a group by a User of the House.
The House, System, need to be tested from the Customer eyes.
Team of Builders “Will the Customer buy the House? Did we build what the Customer wanted”
Customer: “Will I buy this House. Is it what I wanted?”
Testers validate whether or not a system satisfies the acceptance criteria.
Testers use Black-Box testing method. Testers perform:
- Functional, Non-Functional testing;
- Regression testing, Smoke testing
SYSTEM TESTING
LEVEL
To buy, or
not to buy?
ACCEPTANCE
TESTING LEVEL
TESTING LEVELS:
UNIT, INTEGRATION, SYSTEM, ACCEPTANCE
13. TESTING OF A
SOFTWARE
PRODUCT
Static testing
Dynamic
testing
UNIT
INTEGRATION
SYSTEM
ACCEPTANCE
Documentation
Basic
Path
testing
Branch
testing
Decision
Coverage
testing
Loop
testing
Statement
Coverage
testing
Data
Flow
testing
Control
Flow/Condition
Coverage
testing
Bing Bang
(Usage
Model
testing)
Top
Down
testing
Bottom
Up
testing
Sandwich
testing
UAT
(User
Acceptance
Testing)
Alpha
Testing
Beta
testing
Operational
Acceptance
testing
Contract
Acceptance
testing
Regulation
Acceptance
testing
Functional
testing
Non-Functional
testing
Scalability
testing
Stability
testing
Usability
testing
Vulnerabiity
testing
Recovery
testing
Compliance
testing
Security
testing
Penetration testing
Globalization
testing
Localization
testing
Internalization
testing
(I18n
)
Compatability
testing
Binary
Portability
testing
Browser
Compatability
testing
(BCT)
Certification
testing
Performance
testing
Load
testing
Stress
testing
Volume
testing
Spike
testing
Configuration
testing
Soak
testing
(aka
Endurance,
aka
Longevity,
aka
Aging)
Component testing
* Smoke testing
* Regression testing
* Exploratory testing
Sanity testing
Ad-hoc testing
* GUI testing
These testing works can be
performed at every testing level
Buddy
testing
Pair
testing
Monkey
testing
Black Box testing
Gray Box testing
White Box testing
White Box testing
Black Box testing
Black Box testing
14. TESTING SHORT NOTIONS IN ALPHABETICAL ORDER
• Acceptance testing – testing performed by a tester or end customers to check if the software confirms to business needs and to the
requirements provided earlier.
• Ad-hoc testing – testing with purpose to break the software and find defects with the experience and knowledge they have about
the domain, requirements and functionality of the software. Ad-hoc testing is intended to find defects that were not found by
existing test cases. Ad-hoc testing is quite appropriate to use in any stage of the software development cycle. Ad-hoc testing
includes: Buddy testing, Pair testing, Monkey testing.
• Alpha testing – testing performed by a tester or end customers at development site. The main purpose of alpha testing is to refine
the software product by finding (and fixing) the bugs that were not discovered through previous tests.
• Basic path testing – testing performed to check code for every execution path of code at least once; performed by developers.
• Beta testing – testing performed by a tester or carried out by end customers before releasing or handing over software to end users.
• Binary Portability testing – testing with the purpose to check compatibility of binary or executable files from one hardware and/or
operating system to another.
• Bing Bang testing - testing of all components or modules integrated simultaneously, after that everything is tested as a whole.
• Black Box testing – testing without knowledge of internal code structure.
• Bottom Up testing - testing where the lowest level components are tested first, then used to facilitate the testing of higher level
components. The process is repeated until the component at the top of the hierarchy is tested.
• Branch testing – testing during Unit level testing. The purpose to check code for every branching condition.
15. • Browser Compatibility testing – testing performed for web applications with combination of different browsers and operating
systems.
• Buddy testing - testing in “a buddy pair”, such pair consists of a tester and a developer. A tester and a developer work on
identifying defects in the same module. Buddy testing helps the testers develop better test cases and development team can also
make design changes early. This testing usually happens after unit testing completion.
• Certification testing – testing if a software product is applicable for hardware, different operating systems, browsers.
• Compatibility testing – testing with the purpose to check if the software can be run on different hardware, operating system,
bandwidth, databases, web servers, application servers, hardware peripherals, emulators, different configuration, processor, different
browsers and different versions of the browsers etc.
• Compliance testing – testing with the purpose to check if a software product meets the required standards, government laws,
company policies.
• Component testing – testing of a group of units as code together.
• Configuration testing – testing with the purpose to find optimal configuration settings that can make software perform at its best
for hardware or for a given operating system.
• Contract Acceptance testing – testing of a system against acceptance criteria as documented in a contract, before the system is
accepted.
• Data flow testing – testing aims to explore sequences of events related to the status of variables or data objects. Dataflow Testing
focuses on the points at which variables receive values and the points at which these values are used.
• Decision Coverage testing (aka Branch Coverage testing, aka All-Edges Coverage testing) – testing aims to validate that all the
branches in the code are reached, that no branches lead to any abnormality of the program’s operation.
• Dynamic testing - testing that involves the execution of a software product by giving set of inputs, examined it's output and
compared what is expected.
16. • Exploratory testing - testing conducted to learn the software at the same time to find f errors that seem non-obvious. Usually
done by testers but can be done by other stake holders as well as business analysts, developers, end users. Ad-hoc testing is one of
the Exploratory testing variants.
• Functional testing - testing software against design document, use cases and requirements document.
• Fuzz testing – testing with the purpose to check a software product for failures or error messages that are presented due to the
input errors. Unexpected or random inputs are used.
• Globalization testing - testing validates whether the application is capable for using all over the world and to check whether the
input accepts all the language texts. The purpose of testing is to check whether software can perform properly in any locale or
culture & functioning properly with all types of international inputs and steps to effectively make your product truly global.
• Gray Box testing – testing with some knowledge of internal code but the knowledge is not detailed.
• Graphical User Interface testing (GUI) - testing that involves checking the screens with the controls like menus, buttons, icons,
and all types of bars - tool bar, menu bar, dialog boxes and windows. So GUI testing ensures GUI elements of the software are as
per approved GUI mockups, detailed design documents and functional requirements. GUI testing can be performed at any level of
testing: Unit, Integration, System, Acceptance.
• Installation testing – testing that involves executing software installer in different modes like “express” or “custom” modes and
on different types of environments like virtual operating systems and on different operating systems.
• Integration testing – testing with the aim to check the connectivity among integrated together units/component. Individual
software modules are combined and tested as a group.
• Internationalization testing - testing with the aim to check the extent to which software can support usage of different
languages, different character sets and so on.
17. • Load testing - testing intends to find bottlenecks or issues that prevent software from performing as intended at its peak
workloads. This testing activity is performed to check the behavior of the software under normal and over peak load
conditions. Ideal for automation.
• Localization testing - testing with the purpose to validate whether application is capable enough for using in a particular
location or country.
• Loop testing – testing that concentrates on the validity of loop constructs.
• Monkey testing – testing the product or application without test cases with a goal to break the system, testing is random.
• Negative testing - testing that is intended to break the software by entering incorrect data like incorrect date, time or string or
upload binary file when text files supposed to be upload or enter huge text string for input fields etc.,
• Non-Functional testing – testing of a software product upon the requirements which are non functional in nature related but
important a well such as performance, security, user interface etc.
• Operational Acceptance testing – (aka Operational Readiness testing) testing with the purpose to check a system to ensure
that processes and procedures are in place to allow the system to be used and maintained. This type of testing includes checks
done to back-up facilities, procedures for disaster recovery, training for end users, maintenance procedures, and security
procedures.
• Pair testing – testing of a software product when two testers work on the same machines to find defects. One does the testing
and the other analyzes or reviews the testing. This can be done between a tester and developer or business analyst or between
two testers. Pair testing is done when testers have different levels of a software product knowledge.
• Penetration testing - is done to check how secure software and its environments (hardware, operating system and network)
are when subject to attack by an external or internal intruder.
18. • Performance testing - testing checks how well software works in anticipated and peak workloads. There are very many variants of
Performance testing like Load testing, Stress testing, Volume testing, Soak testing, Configuration testing.
• Positive testing - testing with the purpose to check the application’s functions with valid input and also to verify that the outputs
are correct.
• Recovery testing - testing performed by software testers. Recovery testing performs with the purpose to check how soon and how
efficiently software can recover from software crashes, operating system crashers, and hardware failures.
• Regression testing – testing with the purpose to find defects that got introduced to defect fix(es) or introduction of new feature(s).
Regression testing is ideal for automation. Sanity testing is a variant of Regression.
• Regulation Acceptance testing – testing of a system to ensure it meets governmental, legal and safety standards.
• Sandwich testing – testing combination of Top Down testing with Bottom Up testing.
• Sanity testing – testing with the purpose to evaluate if the software, environment, network, external systems, software environment
as a whole is stable enough to proceed with extensive testing.
• Scalability testing - of a software application for measuring its capability to scale up in terms of any of its non-functional
capability like load supported, the number of transactions, the data volume and so on.
• Security testing – testing checks, how good is software’s authorization mechanism, how strong is authentication, how software
maintains confidentiality of the data, how does the software maintain integrity of the data, what is the availability of the software in
an event of an attack on the software by hackers and malicious programs.
• Smoke testing - testing with the aim to find “show stopper” defects that can prevent testers from testing the application in detail.
Testers check if the new build provided by development team is stable enough i.e., major functionality is working as expected in
order to carry out further or detailed testing.
• Soak testing (aka Endurance testing, aka Longevity testing, aka Aging testing) - testing that is conducted to find errors that
result in degeneration of software performance with continued usage, soak testing may go on for few days or even for few weeks.
Soak testing is extensively done for electronic devices, which are expected to run continuously for days, or months, or years
without restarting or rebooting.
19. • Spike testing - testing with the purpose to check how software responds to workloads that are sent in very short span of time and
which are not constant over a period of time.
• Stability testing – testing with the purpose to check how stable software is when it is subject to loads at acceptable levels, peak loads,
loads generated in spikes, with more volumes of data to be processed.
• Statement Coverage testing – (aka Line Coverage testing, aka Segment Coverage testing). Testing with the purpose to check the
statements executed and where the code is not executed because of blockage. This testing covers only the true conditions.
• Static testing - testing when software code is not executed instead it is reviewed for syntax, commenting, naming convention, size of
the functions. Static testing can be applied for requirements, designs, test cases by using approaches like reviews or walkthroughs.
• Stress testing - is a type of performance testing, in which software is subjected to peak loads and even to a break point to observe how
the software would behave at breakpoint. Stress testing also tests the behavior of the software with insufficient resources like CPU,
Memory, Network bandwidth, Disk space etc. Stress testing enables to check some of the quality attributes like robustness and
reliability.
• System testing – testing with the purpose to find errors related to integration among different applications, services, third party vendor
applications.
• Top Down testing – testing where the top integrated modules are tested and the branch of the module is tested step by step until the
end of the related module.
• Unit testing – testing performed by developers. The developer tests units of source code like statements, branches, functions, methods,
interface. Unit testing is ideal for automation.
• Usability testing – testing performed to understand how user friendly the software is.
• User Acceptance testing (UAT) – testing with the purpose to check the software product with their actual business or real-world
scenarios and to check if the software meets their business requirements.
• Volume testing – testing with the purpose to check the response of the software with different sizes of the data being received or to be
processed by the software.
• Vulnerability testing – testing that involves identifying, exposing the software, hardware or network.
• White Box testing – testing with code structure knowledge.
HAPPY TESTING!
