Software Testing
SAJID SIDI & PANKAJ KHARE
What is Testing?
According to ANSI/IEEE 1059 standard, Testing can be defined as - A process of analysing a
software item to detect the differences between existing and required conditions (that is
defects/errors/bugs) and to evaluate the features of the software item.
Why Testing Required?
Software testing is really required to point out the defects and errors that were made during the
development phases. It's essential since it makes sure of the Customer's reliability and their
satisfaction in the application.
Software Testing
SDLC and STLC
Software Testing
QA, QC & Testing
Quality Assurance Quality Control Testing
QA includes activities that ensure the
implementation of processes,
procedures and standards in context
to verification of developed software
and intended requirements.
It includes activities that ensure the
verification of a developed software
with respect to documented (or not in
some cases) requirements.
It includes activities that ensure the
identification of bugs/error/defects in
a software.
Focuses on processes and procedures
rather than conducting actual testing
on the system.
Focuses on actual testing by executing
the software with an aim to identify
bug/defect through implementation
of procedures and process.
Focuses on actual testing.
Process-oriented activities. Product-oriented activities. Product-oriented activities.
Preventive activities. It is a corrective process. It is a preventive process.
It is a subset of Software Test Life
Cycle (STLC).
QC can be considered as the subset of
Quality Assurance.
Testing is the subset of Quality
Control.
STLC
Software
Testing
Life
Cycle
Software Testing
Test Plan
Software Testing
Testing Models
Waterfall model
V model
Incremental model
Iterative model
Spiral model
Agile model
Software Testing
Agile Model
Software Testing
Types of Testing
Software Testing
Traditional Testing Flow
Definitions
◦ Fault/Defect/Bug – The cause of a failure, e.g., a missing or incorrect piece of code
◦ Error – An intermediate unstable state
◦ Failure – The manifested inability of the program to perform the function required, i.e., a system
malfunction evidenced by incorrect output, abnormal termination or unmet time and space constraints.
Fault  Error  Failure
◦ This chain can recursively iterate: a fault in turn can be caused by the failure of some other interacting
system.
◦ Bug Tracking Tool -A bug tracking system or defect tracking system is a software application that keeps
track of reported software bugs in software development projects. It may be regarded as a type of issue
tracking system. (ex. JIRA, RedMine, Bugnet, Quality Center, Mantis, Bugzilla etc.)
Software Testing
Bug Life Cycle
Software Testing
Emerging Trends
IN SOFTWARE TESTING
Emerging Trends
oMobile application testing (Rapid growth in Mobile testing)
oCloud-based Testing
oTesting in the Agile Development Environment
oContext-driven test
oSecurity Testing
oCrowd sourced testing
Software Testing
Mobile application testing
Challenges
◦ Smooth Downloading
◦ Device Availability
◦ Network Provider
◦ Automation
◦ Devices Vs. Emulator/Simulator
Cloud-based Testing
Cloud testing is a form of software testing in which web applications use cloud computing
environments (a "cloud") to simulate real-world user traffic.
oOn-Demand test execution by third parties
oOnline test execution in a public could
oVirtual/Online user access simulation
oTest Environment are readily available
oLimited Storage Space
oBased on SLA
oTesting Cost
Testing in Agile
Context-driven test
oWhy? What? Where? When? How?
oMonitor major test planning challenges
oClarify your mission
oAnalyze the product
oAnalyze product risk
oDesign the test strategy
oPlan
oShare the plan
Security Testing
oVulnerability Scanning
oPenetration testing
oRisk Assessment
oSecurity Auditing
oEthical hacking
Test Case Design
IN STLC
Test Case Format
Design a Test Scenario
Case Study
REAL IMPACT OF BUG(S) MISSED WHILE TESTING
Heathrow Terminal 5 Opening
Just before the opening of Heathrow’s Terminal 5 in the UK, staff tested the brand new baggage
handling system built to carry the vast amounts of luggage checked in each day. Engineers tested
the system thoroughly before opening the Terminal to the public with over 12,000 test pieces of
luggage. It worked flawlessly on all test runs only to find on the Terminal’s opening day the
system simply could not cope. It is thought that ‘real life’ scenarios such as removing a bag from
the system manually when a passenger had left an important item in their luggage, had caused
the entire system to become confused and shut down. Over the following 10 days some 42,000
bags failed to travel with their owners, and over 500 flights were cancelled.
NASA’s Mars Climate Orbiter
On its mission to Mars in 1998 the Climate Orbiter spacecraft was ultimately lost in space.
Although the failure bemused engineers for some time it was revealed that a sub contractor on
the engineering team failed to make a simple conversion from English units to metric. An
embarrassing lapse that sent the $125 million craft fatally close to Mars’ surface after
attempting to stabilize its orbit too low. Flight controllers believe the spacecraft ploughed into
Mars’ atmosphere where the associated stresses crippled its communications, leaving it hurtling
on through space in an orbit around the sun.
Patriot Missile Error
Sometimes, the cost of a software glitch can’t be measured in dollars. In February of 1991, a U.S.
Patriot missile defence system in Saudi Arabia, failed to detect an attack on an Army barracks. A
government report found that a software problem led to an “inaccurate tracking calculation
that became worse the longer the system operated.” On the day of the incident, the system had
been operating for more than 100 hours, and the inaccuracy was serious enough to cause the
system to look in the wrong place for the incoming missile. The attack killed 28 American
soldiers. Prior to the incident, Army officials had fixed the software to improve the Patriot
system’s accuracy. That modified software reached the base the day after the attack.
Knight’s $440 Million Error
Knight Capital, a firm that specializes in executing trades for retail brokers, took $440m in cash
losses Wednesday due to a faulty test of new trading software.
On the day of 8/1/2012, Knight Capital's software went out and bought at the "market",
meaning it paid ask price and then sold at the bid price--instantly. Over and over and over again.
One of the stocks the program was trading, electric utility Exelon, had a bid/ask spread of 15
cents. Knight Capital was trading blocks of Exelon common stock at a rate as high as 40 trades
per second--and taking a 15 cent per share loss on each round-trip and at the end of the debacle
45 minutes later, Knight Capital had lost $440m and was teetering on the brink of insolvency.
Q & A
Automation
OF SOFTWARE TEST
What is automation testing ?
Test automation is the use of special software (separate from the software being tested) to
control the execution of tests and the comparison of actual outcomes with predicted outcomes.
Test automation can automate some repetitive but necessary tasks in a formalized testing
process already in place, or perform additional testing that would be difficult to do manually.
Test automation is critical for continuous delivery and continuous testing.
Why Automation?
oManual Testing of all work flows, all fields , all negative scenarios is time and cost consuming
oIt is difficult to test for multi lingual sites manually
oAutomation does not require Human intervention. You can run automated test unattended
(overnight)
oAutomation increases speed of test execution
oAutomation helps increase Test Coverage
oManual Testing can become boring and hence error prone.
What to Automate?
oHigh Risk – Business Critical Test cases
oTest cases that are executed repeatedly
oTest cases that are very tedious or difficult to perform manually
oTest cases which are time consuming
What not to Automate?
oTest Cases that are newly designed and not executed manually
oTest Cases for which the requirements are changing frequently
oTest Cases which are executed on ad-hoc basis
Automation Process
Feasibility
oFeature that are important for the business
oScenarios which have large amount of data
oCommon functionalities across applications
oTechnical feasibility
Tool Selection
Test Tool selection largely depends on the technology the Application Under Test is built on. For
example Selenium does not support Desktop application hence it is not suitable for desktop
project . In this case we can use UFT or any other tool which support Desktop application
Test Planning, Design & Development
During this phase, Generally Automation strategy & plan has created, which contains..
o Framework design and its features
o In-Scope and Out-of-scope items of automation
o Automation Script creation
o Schedule and Timeline of scripting and execution
o Deliverables of automation testing
Script Execution
Automation Scripts are executed during this phase. The scripts needs input test data before
there are set to run. Once executed they provide detailed test reports.
Execution can be performed using the automation tool directly or through the Test Management
tool which will invoke the automation tool.
Example: Quality center is the Test Management tool which in turn it will invoke QTP for
execution of automation scripts. Scripts can be executed in a single machine or a group
of machines. The execution can be done during night , to save time.
Maintenance
As new functionalities are added to the System Under Test with successive cycles, Automation
Scripts need to be added, reviewed and maintained for each release cycle. Maintenance
becomes necessary to improve effectiveness of Automation Scripts.
Architecture/Framework
Automation Demo
PRACTICAL DEMONSTRATION ON AUTOMATION TEST USING SELENIUM WEBDRI VER
Q & A
For any further queries and solution, please feel free to contact us.
Sajid Sidi
@: sajid.sidi@gmail.com
M: 9558203909
Pankaj Khare
@: Pankaj.0405@gmail.com
M: 8141287206

Software Testing - Sajid Sidi

  • 1.
  • 2.
    What is Testing? Accordingto ANSI/IEEE 1059 standard, Testing can be defined as - A process of analysing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item. Why Testing Required? Software testing is really required to point out the defects and errors that were made during the development phases. It's essential since it makes sure of the Customer's reliability and their satisfaction in the application. Software Testing
  • 3.
  • 4.
    QA, QC &Testing Quality Assurance Quality Control Testing QA includes activities that ensure the implementation of processes, procedures and standards in context to verification of developed software and intended requirements. It includes activities that ensure the verification of a developed software with respect to documented (or not in some cases) requirements. It includes activities that ensure the identification of bugs/error/defects in a software. Focuses on processes and procedures rather than conducting actual testing on the system. Focuses on actual testing by executing the software with an aim to identify bug/defect through implementation of procedures and process. Focuses on actual testing. Process-oriented activities. Product-oriented activities. Product-oriented activities. Preventive activities. It is a corrective process. It is a preventive process. It is a subset of Software Test Life Cycle (STLC). QC can be considered as the subset of Quality Assurance. Testing is the subset of Quality Control.
  • 5.
  • 6.
  • 7.
    Testing Models Waterfall model Vmodel Incremental model Iterative model Spiral model Agile model Software Testing
  • 8.
  • 9.
  • 10.
  • 11.
    Definitions ◦ Fault/Defect/Bug –The cause of a failure, e.g., a missing or incorrect piece of code ◦ Error – An intermediate unstable state ◦ Failure – The manifested inability of the program to perform the function required, i.e., a system malfunction evidenced by incorrect output, abnormal termination or unmet time and space constraints. Fault  Error  Failure ◦ This chain can recursively iterate: a fault in turn can be caused by the failure of some other interacting system. ◦ Bug Tracking Tool -A bug tracking system or defect tracking system is a software application that keeps track of reported software bugs in software development projects. It may be regarded as a type of issue tracking system. (ex. JIRA, RedMine, Bugnet, Quality Center, Mantis, Bugzilla etc.) Software Testing
  • 12.
  • 13.
  • 14.
    Emerging Trends oMobile applicationtesting (Rapid growth in Mobile testing) oCloud-based Testing oTesting in the Agile Development Environment oContext-driven test oSecurity Testing oCrowd sourced testing Software Testing
  • 15.
    Mobile application testing Challenges ◦Smooth Downloading ◦ Device Availability ◦ Network Provider ◦ Automation ◦ Devices Vs. Emulator/Simulator
  • 16.
    Cloud-based Testing Cloud testingis a form of software testing in which web applications use cloud computing environments (a "cloud") to simulate real-world user traffic. oOn-Demand test execution by third parties oOnline test execution in a public could oVirtual/Online user access simulation oTest Environment are readily available oLimited Storage Space oBased on SLA oTesting Cost
  • 17.
  • 18.
    Context-driven test oWhy? What?Where? When? How? oMonitor major test planning challenges oClarify your mission oAnalyze the product oAnalyze product risk oDesign the test strategy oPlan oShare the plan
  • 19.
    Security Testing oVulnerability Scanning oPenetrationtesting oRisk Assessment oSecurity Auditing oEthical hacking
  • 20.
  • 21.
  • 22.
    Design a TestScenario
  • 23.
    Case Study REAL IMPACTOF BUG(S) MISSED WHILE TESTING
  • 24.
    Heathrow Terminal 5Opening Just before the opening of Heathrow’s Terminal 5 in the UK, staff tested the brand new baggage handling system built to carry the vast amounts of luggage checked in each day. Engineers tested the system thoroughly before opening the Terminal to the public with over 12,000 test pieces of luggage. It worked flawlessly on all test runs only to find on the Terminal’s opening day the system simply could not cope. It is thought that ‘real life’ scenarios such as removing a bag from the system manually when a passenger had left an important item in their luggage, had caused the entire system to become confused and shut down. Over the following 10 days some 42,000 bags failed to travel with their owners, and over 500 flights were cancelled.
  • 25.
    NASA’s Mars ClimateOrbiter On its mission to Mars in 1998 the Climate Orbiter spacecraft was ultimately lost in space. Although the failure bemused engineers for some time it was revealed that a sub contractor on the engineering team failed to make a simple conversion from English units to metric. An embarrassing lapse that sent the $125 million craft fatally close to Mars’ surface after attempting to stabilize its orbit too low. Flight controllers believe the spacecraft ploughed into Mars’ atmosphere where the associated stresses crippled its communications, leaving it hurtling on through space in an orbit around the sun.
  • 26.
    Patriot Missile Error Sometimes,the cost of a software glitch can’t be measured in dollars. In February of 1991, a U.S. Patriot missile defence system in Saudi Arabia, failed to detect an attack on an Army barracks. A government report found that a software problem led to an “inaccurate tracking calculation that became worse the longer the system operated.” On the day of the incident, the system had been operating for more than 100 hours, and the inaccuracy was serious enough to cause the system to look in the wrong place for the incoming missile. The attack killed 28 American soldiers. Prior to the incident, Army officials had fixed the software to improve the Patriot system’s accuracy. That modified software reached the base the day after the attack.
  • 27.
    Knight’s $440 MillionError Knight Capital, a firm that specializes in executing trades for retail brokers, took $440m in cash losses Wednesday due to a faulty test of new trading software. On the day of 8/1/2012, Knight Capital's software went out and bought at the "market", meaning it paid ask price and then sold at the bid price--instantly. Over and over and over again. One of the stocks the program was trading, electric utility Exelon, had a bid/ask spread of 15 cents. Knight Capital was trading blocks of Exelon common stock at a rate as high as 40 trades per second--and taking a 15 cent per share loss on each round-trip and at the end of the debacle 45 minutes later, Knight Capital had lost $440m and was teetering on the brink of insolvency.
  • 28.
  • 29.
  • 30.
    What is automationtesting ? Test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.
  • 31.
    Why Automation? oManual Testingof all work flows, all fields , all negative scenarios is time and cost consuming oIt is difficult to test for multi lingual sites manually oAutomation does not require Human intervention. You can run automated test unattended (overnight) oAutomation increases speed of test execution oAutomation helps increase Test Coverage oManual Testing can become boring and hence error prone.
  • 32.
    What to Automate? oHighRisk – Business Critical Test cases oTest cases that are executed repeatedly oTest cases that are very tedious or difficult to perform manually oTest cases which are time consuming
  • 33.
    What not toAutomate? oTest Cases that are newly designed and not executed manually oTest Cases for which the requirements are changing frequently oTest Cases which are executed on ad-hoc basis
  • 34.
  • 35.
    Feasibility oFeature that areimportant for the business oScenarios which have large amount of data oCommon functionalities across applications oTechnical feasibility
  • 36.
    Tool Selection Test Toolselection largely depends on the technology the Application Under Test is built on. For example Selenium does not support Desktop application hence it is not suitable for desktop project . In this case we can use UFT or any other tool which support Desktop application
  • 37.
    Test Planning, Design& Development During this phase, Generally Automation strategy & plan has created, which contains.. o Framework design and its features o In-Scope and Out-of-scope items of automation o Automation Script creation o Schedule and Timeline of scripting and execution o Deliverables of automation testing
  • 38.
    Script Execution Automation Scriptsare executed during this phase. The scripts needs input test data before there are set to run. Once executed they provide detailed test reports. Execution can be performed using the automation tool directly or through the Test Management tool which will invoke the automation tool. Example: Quality center is the Test Management tool which in turn it will invoke QTP for execution of automation scripts. Scripts can be executed in a single machine or a group of machines. The execution can be done during night , to save time.
  • 39.
    Maintenance As new functionalitiesare added to the System Under Test with successive cycles, Automation Scripts need to be added, reviewed and maintained for each release cycle. Maintenance becomes necessary to improve effectiveness of Automation Scripts.
  • 40.
  • 41.
    Automation Demo PRACTICAL DEMONSTRATIONON AUTOMATION TEST USING SELENIUM WEBDRI VER
  • 42.
    Q & A Forany further queries and solution, please feel free to contact us. Sajid Sidi @: sajid.sidi@gmail.com M: 9558203909 Pankaj Khare @: Pankaj.0405@gmail.com M: 8141287206