1981 IBM releases PC to market Rational software founded Testing conferences in USA Test then code
Cause: A programmer incorrectly transcribed a handwritten formula into computer code, missing a single superscript bar. Software treated normal variations of velocity as if they were serious, causing faulty corrections that sent the rocket off course.
Cause: A bug in the Soviet software failed to filter out false missile detections caused by sunlight reflecting off cloud-tops.
Cause: A software rounding error incorrectly calculated the time, causing the Patriot system to ignore the incoming Scud missile.
Cause: The software calculated radiation dosage based on the order in which data was entered, sometimes delivering a double dose of radiation.
Introduction to Testing Industry
SOFTWARE TESTING INDUSTRY
Purpose of the Presentation ?
To provide some context on the software testing industry
(broader than our day-to-day duties).
Software Testing is …
• Software testing is a process of executing a program or
application with the intent of finding the software bugs
• The principle objective of software testing is to give
confidence in the software. (Unknown Author)
• Testing is an infinite process of comparing the invisible to
the ambiguous in order to avoid the unthinkable happening
to the anonymous (James Bach)
• Software testing is an investigation conducted to provide
stakeholders with information about the quality of the
product or service under test. (Bertrand Meyer)
Money in Software Testing Industry
Pierre Audoin Consultants ( https://www.pac-online.com )
has forecasted the software testing spends to hit
Euro 100 billions by 2014.
IDC and Gartner forecast that the enterprise spending on
independent testing services would cross the benchmark
of USD 30 billions (Euro ~25 billions) by 2015.
Money (Lithuanian GDP)
Estimate for LT GDP 2014 – USD 50 Billions.
Billions of EURO
LT GDP 2014 Independent Testing by
Overall Testing 2014
Money in SW Testing Industry vs
History of Software Testing…
History of Testing [before 1970]
1958 – First independent Test Team at IBM
• Testing is mentioned in “Computer Programming
Fundamentals” book by Jerry Weinberg
• Some of the Testing methods whitepapers are published
• NATO report mentions Software Quality (1968)
History of Testing [1970s]
Cyclomatic complexity, Quality metrics,
structure based testing, code
1979 - Glenford J. Myers in his “The Art
of Software Testing” introduces
separation of debugging from testing and
summarizes the black box testing
History of Testing [1980s]
“Software Testing Techniques”,
Exploratory Testing introduced by
Cem Kaner (1988)
Black-box and White-box testing
techniques are defined
History of Testing [1990s]
Boris Beizer “Bug Taxonomy”, “Pesticide Paradox”
Cem Caner “Testing Computer Software”
Mercury Interactive WinRunner started
First Star conference (1992, Las Vegas)
First EuroStar conference (1993, London)
Tmap published (1995)
ISEB certification created (1998)
James Bach – Context Driven Testing (1999)
History of Testing [2000s …]
James Bach – Rapid Software Testing, Session Based Testing
James Bach, Cem Kaner, Bret Pettichord
“Lessons learned in Software Testing”
ISTQB founded (2002)
IBM Rational Tester (2002), Selenium (2004)
"Schools of Software Testing“, 2003 - Bret Pettichord
uTest – crowdsourcing, 2008
History of Testing [2000s]
Gojko Adzic “Specification by Example” (2011)
Acceptance Test Driven Development (ATDD)
Behavior Driven Testing (BDT)
(TDD -> ATDD -> BDT)
Gherkin (Cucumber specification)
Schools of Software Testing
• Analytical School (structural testing)
• Factory School (requirement traceability)
• QA School (gatekeeping over testing)
• Context Driven School
Some Software Testing-Related Terms
Triangle Problem (1960s)
Pesticide Paradox (early 1990s)
Software Archeology (early 2000s)
• Mariner Bugs Out (1962), Cost: $18.5 million Disaster: The
Mariner 1 rocket with a space probe headed for Venus
diverted from its flight path shortly after launch.
• World War III… Almost (1983), Cost: Nearly all of humanity
Disaster: The Soviet early warning system falsely indicated
the United States had launched five ballistic missiles
• Patriot Fails Soldiers (1991), Cost: 28 soldiers dead, 100
injured. Disaster: During the first Gulf War, an American
Patriot Missile system in Saudi Arabia failed to intercept an
incoming Iraqi Scud missile.
• Cancer Treatment to Die For (2000), Cost: Eight people
dead, 20 critically injured.
A Software Testing Urban Legend
Claim: A flavor of ice cream being transported affects a
car's performance: every time vanilla is the driver's
choice, the car stalls (vapor lock).
(Collected on the Internet, 1998)