Innovative Approaches to Software Dev no good at all

1. SOFTWARE QUALITY
ENGINEERING
Lecture # 1: Overview
Ayesha Kanwal
Department of Computing (DoC), SEECS, NUST

2. Course topics overview
Software quality and quality assurance
Testing concepts, issues, types and techniques
Test activities, management, and automation
Software inspection
Comparison of various QA techniques
Quantifiable quality improvement: defects and risk identification
Quality processed and standards: ISO Standards, CMMI , TQM
Defect analyses
Defect removal models
Software reliability, failures and faults

3. Text book
 Tian, Jeff (2005). Software Quality Engineering:
Testing, Quality Assurance, and Quantifiable
Improvement | ISBN-10: 0471713457 | ISBN-13:
978-0471713456 | Edition: 1

4. Grading Policy
 30% mid term
 10% Assignments
 10% Quizzes
 10% Project
 40% ESE

5. Class Policy
 Assignments:
 Each assignment will count towards the total.
 Late assignments will not be accepted / graded.
 Quiz policy:
 Quizzes may be announced or unannounced.
 No best of quizzes policy.

6. Software errors cost the U.S. economy $60
billion annually in rework, lost productivity and
actual damages. We all know software bugs can be
annoying, but faulty software can also be expensive,
embarrassing, destructive and deadly. Following are
some of the famous software “disasters”:
Why Software Quality Assurance?
6

7. 1. Mariner Bugs Out (1962)
 Disaster: The Mariner 1 rocket with a space probe headed for
Venus diverted from its intended flight path shortly after launch.
Mission Control destroyed the rocket 293 seconds after liftoff.
 Cost: $18.5 million
 Cause: A programmer incorrectly transcribed a handwritten formula
into computer code, missing a single superscript bar. Without the
smoothing function indicated by the bar, the software treated normal
variations of velocity as if they were serious, causing faulty
corrections that sent the rocket off course.
7

8. 2. CIA Gives the Soviets Gas (1982)
 Disaster: Control software went haywire and produced intense
pressure in the Trans-Siberian gas pipeline, resulting in the
largest man-made non-nuclear explosion in Earth’s history.
 Cost: Millions of dollars, significant damage to Soviet economy
 Cause: CIA operatives allegedly planted a bug in a Canadian
computer system purchased by the Soviets to control their gas
pipelines. The purchase was part of a strategic Soviet plan to
steal or secretly obtain sensitive U.S. technology. When the CIA
discovered the purchase, they sabotaged the software so that it
would pass Soviet inspection but fail in operation.
8

9. 3. World War III… Almost (1983)
 Disaster: The Soviet early warning system falsely indicated
the United States had launched five ballistic missiles.
Fortunately the Soviet duty officer had a “funny feeling in my
gut” and reasoned if the U.S. was really attacking they would
launch more than five missiles, so he reported the apparent
attack as a false alarm.
 Cost: Nearly all of humanity
 Cause: A bug in the Soviet software failed to filter out false
missile detections caused by sunlight reflecting off cloud-tops.
9

10. 4. Mars Climate Crasher (1998)
 Disaster: After a 286-day journey from Earth, the
Mars Climate Orbiter fired its engines to push into orbit
around Mars. The engines fired, but the spacecraft fell
too far into the planet’s atmosphere, likely causing it to
crash on Mars.
 Cost: $125 million
 Cause: The software that controlled the Orbiter
thrusters used imperial units (pounds of force), rather
than metric units (Newtons) as specified by NASA.
10

11. INTRODUCTION TO SQE

12. General Expectations
 Good software quality
 People: Consumers vs producers
 quality expectations by consumers
 to be satisfied by producers through software quality
engineering (SQE)
 Deliver software system that...
 does what it is supposed to do
 needs to be “validated”
 does the things correctly
 needs to be “verified”

13. Main tasks for software quality engineering
 Organized into three major topics:
 Software testing, as a primary means to ensure software
quality
 Software quality assurance (SQA), that includes:
 Defect prevention
 Process improvement
 Inspection
 Formal verification
 Fault tolerance etc.
 Measurement and analysis to close the feedback loop for
quality assessment and quantifiable improvement.

14. Testing, quality assurance (QA), and quality
engineering
Testi
ng
Quality Assurance
Software quality engineering

15. What is software quality??
 Conformance to requirements (Crosby)
 Problems:
 What if requirements are wrong?
 How do you know if requirements are being met?
 Fitness For Use (Juran/Gruna)
 Problems:
 How many different ways are there for a customer to
‘use’ a product?
 Customer’s view of Quality
 Perceived value of the product based on price, performance,
reliability, and satisfaction

16. Two perspectives
 “small q”
 Intrinsic product quality
 defect rate - how many bugs, or missing functions
 What is considered a defect to the customer?
 reliability - how often it fails
 “big Q”
 Broader level of quality
 product quality
 process quality
 customer satisfaction

17. Two perspectives
 Will a good ‘q’ guarantee customer satisfaction?
 Can you achieve a good “Q” without a good “q”?

18. Two perspectives
 Will a good “q” guarantee customer satisfaction?
 Issues
 Performance
 Requirements
 Service
 Documentation
 Can you achieve a good “Q” without a good “q”?
 Bugs and poor reliability lead to poor customer
satisfaction

19. Views of quality
Views Transcendental view, quality is hard to define
or describe in abstract terms
User view, quality is fitness for purpose
Manufacturing view, quality means
conformance to process standards
Product view, the focus is on inherent
characteristics in the product
Value-based view, quality is the customers’
willingness to pay for a software

20. ISO-9126 for Quality
 ISO-9 126 (ISO, 2001) provides a hierarchical framework for
quality definition, organized into quality characteristics and sub-
characteristics.
Functionality: Suitability - Accuracy - Interoperability -
Security
Reliability Maturity- Fault tolerance - Recoverability
Usability: Understandability- Learnability
- Operability
Efficiency: Time behavior
- Resource behavior
Maintainability: Analyzability - Changeability
- Stability - Testability
Portability: Adaptability - Installability
- Conformance - Replaceability

21. Defects
 Defect definition – a problem with the software,
either with its external behavior or internal
characteristics
 What does Quality Assurance do?
 Focuses on ‘Correctness’ aspect of quality
 Focuses on defect prevention, detection, removal and
containment
 Focuses on testing activities

22. Correctness
 Many definitions:
 The degree to which a software entity's behavior
matches its specification
 The degree to which a system is free from defects
in its specification, design and implementation.
 The ability of software products to perform their
exact tasks as defined by their specification.
 Free from error, accurate, in accordance with
specifications.

23. Correctness and defects
High quality ≈ low defect
 intuitive notion related to correctness
 quality problem ≈ defect impact
 widely accepted, but need better definitions

24. Error, fault, failure, and defect
 Failure: The inability of a system or component
to perform its required functions within
specified performance requirements.
 Fault: An incorrect step, process, or data
definition in a computer program.
 Error: A human action that produces an
incorrect result.

25. Relation
Dormant faults

26. Cost of removing defects
26
 The cost of removing defects increases
exponentially. A defect caught in requirement and
design phase costs less to fix than an error caught
in the software maintenance cycle.
Cost of defect increases with each phase

27. How do we deal with defects ?
 Defect Prevention
 Defect Detection and Removal
 Defect Containment

28. Defect Prevention
 Eliminating certain error sources, such as
eliminating ambiguities or correcting human
misconceptions
 Fault prevention or blocking by directly
correcting or blocking these missing or incorrect
human actions

29. Defect Prevention through Formal Verification
 Motivation
 Fault present: revealed through testing/inspection/etc.
 Fault absent: formally verify.
 Basic ideas
 Behavior formally specified:
 pre/post conditions, or
 as mathematical functions.
 Verify “correctness":
 intermediate states/steps,
 axioms and compositional rules.
 Approaches: axiomatic/functional/etc

30. Software Defect Prevention Opportunity Tree
IPT, JAD, WinWin,…
PSP, Cleanroom, Dual development,…
Manual execution, scenarios,...
Staffing for dependability
Rqts., Design, Code,…
Interfaces, traceability,…
Checklists
Defect
Prevention
People
practices
Standards
Languages
Prototyping
Modeling & Simulation
Reuse
Root cause analysis

31. Defect detection and removal
 A method for identifying and removing defects at every
phase of the SDLC
 Checkpoints for reviews of the artifacts, i.e., software
requirement specifications, design document, test plan and
test cases, software code.
 Common techniques
 Inspection, faults are discovered
 Testing, failures trace back to faults
 Results of defect removal:
 Reduction in product expense
 Reduction in product development time
 Increase in development team’s productivity
 Increase in product quality

32. Software Defect Detection Opportunity Tree
Completeness checking
Consistency checking
- Views, interfaces, behavior,
pre/post conditions
Traceability checking
Compliance checking
- Models, assertions, standards
Defect
Detection
and Removal
- Rqts.
- Design
- Code
Testing
Reviewing
Automated
Analysis
Peer reviews, inspections
Architecture Review Boards
Pair programming
Requirements & design
Structural
Operational profile
Usage (alpha, beta)
Regression
Value/Risk - based
Test automation

33. Defect containment:
 Fault tolerance
 Fault present in the software but removal is
infeasible or impractical
 Fault tolerance - > contain defects
 Fault tolerance techniques:
 Recovery: rollback and redo
 Block the fault (N-version programming)

34. Defect Impact Reduction Opportunity Tree
Business case analysis
Pareto (80-20) analysis
V/R-based reviews
V/R-based testing
Cost/schedule/quality
as independent variable
Decrease
Defect
Impact,
Size (Loss)
Graceful
Degredation
Value/Risk -
Based Defect
Reduction
Fault tolerance
Self-stabilizing SW
Manual Backup
Rapid recovery

35. Generic ways to deal with defects

36. Class activity (Group of 2)
 Write down two methods (other than those given in
slides) for each of the below with their examples:
 defect prevention
 defect removal
 defect containment

37.  Thank you !!!