Software Quality Reliability Mgmt

1. Software
Quality &
Reliability
Software Engineering
ITS66404

2. Agenda
What is Software Quality
What is Quality Management
Software Quality Assurance
Software Quality Planning
Software Quality Control
Software Quality Metrics
Sample footer text 3/1/20XX 2

3. What is Software
Quality

4. How do you measure the quality of a
car?
+ Provide 5 critical points that you see in a car
Sample footer text 3/1/20XX 4

5. What is Software Quality?
5
+ Software Quality – Software quality is defined as a field of study and
practice that describes the desirable attributes of software products. There are
two main approaches to software quality: defect management and quality
attributes.

6. Why there is a need to Quality
Management?
6
Challenges:
• Development organization has requirements exceeding customer's
specifications (added cost of product development)
• Certain quality characteristics can not be specified in
• unambiguous terms (i.e. maintainability)
• Even if the product conforms to it’s specifications, users may
• not consider it to be a quality product (because users may not be
involved in the development of the requirements)

7. 7
What is Quality Management?
Quality Management – ensuring that required level of product
quality is achieved
• Defining procedures and standards
• Applying procedures and standards to the product and
process
• Checking that procedures are followed
• Collecting and analyzing various quality data
Problems:
• Intangible aspects of software quality can’t be standardized
(i.e elegance and readability)

8. 7/19/2023 8
Software quality management (SQM) is a management process the goal of which is to develop
and manage the quality of a software to make sure the product satisfies the user. SQM is a quality
management culture in an organization where quality is viewed and maintained by everyone.
Software Quality Management (SQM) includes different layer such as Software Quality Assurance
(SQA) layer, Software Quality Plan (SQP) layer and Software Quality Control (SQC) layer.
The Software quality assurance (SQA) is a monitoring process which is used to ensure the quality
in entire software development lifecycle process. It is also a continuous assessment mechanism
which facilitates certain procedures for project development with specific standards along with
documentation. The procedures should be used to assure quality outcome (zero defects) and
project success.
The Software Quality Plan (SQP) is an initial project level quality plan for declaring project
commitment and SQP contain quality goals to be achieved, expected risks and risk management.
The Software Quality Control (SQC) ensures in-process that both SQA and SQP are being
followed by the development teams.

9. What are SQA, SQP, SQC, and SQM?
9
SQA includes all 4 elements…
• Software Quality Assurance – establishment of network of
organizational procedures and standards leading to high-quality
software
2. Software Quality Planning – selection of appropriate procedures
and standards from this framework and adaptation of these to
specific software project
3. Software Quality Control – definition and enactment of processes
that ensure that project quality procedures and standards are being
followed by the software development team
4. Software Quality Metrics – collecting and analyzing quality data to
predict and control quality of the software product being developed

10. Software Quality
Assurance

11. SQA
+ SQA is...
Software - I hope you know what this is!
Quality - "Fitness for purpose"
Assurance - to be assured, be given certainty, freedom from doubt
+ It's a collection of practices that reduce the risk of low-
quality products
+ More than just testing

12. SQA

13. SQA

14. What is Software
Quality Planning (SQP)

15. The SQA plan
+ Prepare an SQA plan for a project at the start of the project
evaluations
audits and reviews
standards applicable
procedures for error tracking/reporting
documents produced by SQA group
feedback provided to project team

16. Why are Standards Important?
16
• Standards provide encapsulation of best, or at least most
appropriate, practice
• Standards provide a framework around which the quality assurance
process may be implemented
• Standards assist in continuity of work when it’s carried out by
different people throughout the software product lifecycle
Standards should not be avoided. If they are too extensive for the
task at hand, then they should be tailored.

17. Software Development Standards
SDS a Simplistic approach
17
In most mature
organizations:
• ISO is not the only
source of SDS
• Process and Product
standards are derived
independently
• Product standards
are not created by
SQA

18. 18
Process Standards – standards that define the process
which should be followed during software development
ISO CMM CMMI
Organizational
Quality Manual
Organizational
SD Process STD’s
IPDS
Project SD Process STD’s
(SDP, IP, Method Sheets)
Project
SQP
Project
SCMP

19. 19
Product Standards – standards that apply to software
product being developed
ISO
STD’s
MIL/ Industry
STD’s
Organizational
Product STD’s
COTS
STD’s
Project Product STD’s (SDP,
IP, Method Sheets)

20. Quality
Standards

21. ISO - 9001 Elements
+ Quality System Requirements
+ Management Responsibility
+ Quality system
+ Contract review
+ Design Control
+ Document control
+ Purchasing
+ Purchaser supplied product
+ Product identification and traceability
+ Process control
+ Inspection and testing
+ Inspection, measuring and test equipment
+ Inspection and test status
+ Software Quality
Responsibilities
+ Management Responsibility
+ Quality system
+ Contract review
+ Design Control
+ Document control
+ Purchasing
+ Product identification and
traceability
+ Process control
+ Inspection and testing
+ Inspection and test status
+ Corrective action
+ Quality records
+ Internal quality audits
+ Training
+ Statistical techniques
21
+ Control of non-conforming product
+ Corrective action
+ Handling, storage, preservation, packaging and
shipping
+ Quality records
+ Internal quality audits
+ Training
+ Servicing
+ Statistical techniques

22. Process and Product Quality
22
• Quality of development process directly affects the
quality of delivered products.
• Software is designed rather then manufactured.

23. Quality Improvement – Six Sigma Process
+ Visualize – Understand how it works now and imagine how it
will work in the future
+ Commit – Obtain commitment to change from the
stakeholders
+ Prioritize – Define priorities for incremental improvements
+ Characterize – Define existing process and define the time
progression for incremental improvements
+ Improve – Design and implement identified improvements
+ Achieve – Realize the results of the change
23

24. Obtaining Quality
+ Quality control
Activities that measure the quality of products produced
E.g. Integration testing, code reviews
+ Quality Assurance
Activities that focus on the process used to create the product
E.g. Ensuring all testing is carried out at each level of the
creation of software

25. Software Quality
Control

26. Methods of Software Quality Control
26
SQC involves overseeing the software development process to ensure that the
procedures and STD’s are being followed
The following activities constitute SQC:
• Quality Reviews - in-process reviews of processes and products
Reviews are the most widely used method of validating the quality of processes and products.
Reviews make quality everyone's responsibility. Quality must be built-in. SQE is responsible
for writing Quality Engineering Records (QERs) documenting their participation in these
reviews.
• Tests - end-result verifications of products. These verifications are conducted after the
software has been developed. Test procedures are followed during conduct of these activities.
SQE is responsible for keeping the logs and some times for writing the test report.
• Quality Audits - in-process verifications of processes. These audits are conducted
periodically (twice a month) to assess compliance to the process STD’s.

27. 7/19/2023 27
Quality Review
Test
Quality Audits

28. Quality Reviews
+ Peer reviews - reviews of processes and products by groups of people. These reviews require pre-
review preparation by all participants. If a participant is not prepared, then the review is not effective.
This type of review requires participation of the SQE, moderator, recorder, author(s), and one or more
critical reviewers. All issues found during these reviews are documented on AR forms.
+ Walkthroughs - reviews of products by groups of people mostly without preparation. For example a
requirements traceability review is a walkthrough. It involves tracing a requirement from customer
requirements to the test procedures. All issues found during these reviews are documented on CAR
forms.
+ Desk inspections - reviews of products by individuals. These reviews involve people reviewing
products by themselves (not in a group) and then submitting their comments to the author(s). The
issues found during these reviews are treated in informal manner.
28

29. Tests
29
• Engineering Dry-run - test conducted by engineering without SQE. These tests
include Unit Tests and engineering dry-runs of the formal tests. These engineering
dry-runs are used to verify correctness and completeness of the test procedures.
Also, these is the final engineering verification of the end-product before sell-off to
SQE. All issues found during these tests are documented on STR forms.
• SQE Dry-run - test conducted by SQE. These tests are used to verify the end-
product before the formal test with the customer. An SQE is sometimes responsible
for writing the test report. However, if a separate test group is available, then SQE is
relived of this obligation. All issues found during these tests are documented on STR
forms.
• TFR - test conducted as “RFR - run-for-record” with the SQE and the customer.
These tests include FAT and SAT. These tests are conducted to sell the end-product
off to the customer. SQE is present at all such tests. All issues found during these
tests are documented on STR forms.

30. Quality Audits
30
• SQE Audits - audits conducted by SQE to verify that the process STD’s are being
followed. Examples of these audits are IPDS compliance, Configuration Control, and
Software Engineering Management. All findings for these audits are documented on
QER forms. The results of the audits are distributed to the next level of management
(above project level). If the issue(s) are not fixed then the findings are elevated to upper
management.
• Independent Audits - audits conducted by ISO generalists or other independent entities
to verify that the process STD’s are being followed. These audits are usually conducted
on a division/facility level. The results of these audits are distributed to upper
management.

31. Defect Detection
31
Formal bug finding activities include Quality Reviews
and Tests
From
Baseline
Capture
System
Requirem
ents
Analysis
Softw
are
Requirem
ents
Analysis
Prelim
inary
Design
Detailed
D
esign
Code
Unit Test
Softw
are
Integration
Softw
are
Q
ualification
Test
S
T
A
G
E
D
E
T
E
C
T
E
D
At Baseline Capture 0
System Requirements Analysis 0 79
Software Requirements Analysis 0 0 1
Preliminary Design 0 6 2 10
Detailed Design 1 0 0 0 42
Code 0 0 0 1 2 37
Unit Test 0 0 0 0 0 0 0
Software Integration 1 0 0 0 4 1 0 0
Software Qualification Test 0 0 0 0 0 0 0 0 0
System Integration 1 0 0 0 4 5 0 0 0
System Test 0 0 0 0 0 0 0 0
Post System Test 0 0 0 0 0 0 0 0 0
93% 33% 91% 81% 86%
93% 11% 95% 79% 74% 0% 36% 0%
44% 2% 6% 27% 22% 0% 0% 0%
Chart Data Last Updated: 10/3/01
S
T
A
G
E
D
E
T
E
C
T
E
D
% Defects Originated In This Phase Out Of All Defects
% Defects Originated in This Phase That Were Contained By This Phase
% Defects Originated in This Phase Plus Defects
That Escaped From Earlier Phases That Were Contained By This Phase

32. Defect Prevention
32
Defect Prevention – establishment of practices that
lower the reliance on defect detection techniques to
find majority of the bugs
• Lessons learned – learning from other peoples experiences and sharing
own experiences with the other projects
• Managing With Metrics – collecting the metrics, understanding it, and
making changes to the product or process based on analysis. Metrics must
be standardized to be effective.
• Risk Analysis – identifying potential risks and opportunities early in the
program and tracking them to realization.
• Build freeze – no changes are made to the code during formal tests.
• Unit-level testing guidelines – test plans and procedures for each UT
• Baseline acceptance criteria – establishment of closure criteria in advance
(i.e. no P1 STRs at FAT TRR)

33. Continuity and Independence of SQA
+ Software Quality Assurance team must be independent in order to take an
objective view of the process and report problems to senior management
directly
+ The standards must be upheld no matter how small the task.
+ Prototyping doesn’t mean no standards. It means tailored standards.
+ Preferable to have an SQA team to assist the Software Engineers doing
quality assurance and quality control activities.
+ SQA team does not (necessarily) work on the project
+ SQA team is there to assist the Software Engineers in producing a high
quality product.
33

34. SQAActivities
+ Assists in selection of the process appropriate to the project
+ Reviews SE activities and work products to verify compliance
with the defined software process
+ Ensures deviations in process and product are reported
correctly using the procedure
+ Records any non-compliance
Should it report to management?

35. What is Reliability?
+ Reliability is a broad concept.
It is applied whenever we expect something to behave in a certain way.
+ Reliability is one of the metrics that are used to measure quality.
+ It is a user-oriented quality factor relating to system operation.
Intuitively, if the users of a system rarely experience failure, the system is considered to be more reliable than one
that fails more often.
+ A system without faults is considered to be highly reliable.
Constructing a correct system is a difficult task.
Even an incorrect system may be considered to be reliable if the frequency of failure is “acceptable.”
+ Key concepts in discussing reliability:
Fault
Failure
Time
35

36. What is Reliability?
+ Failure
A failure is said to occur if the observable outcome of a program execution is different
from the expected outcome.
+ Fault
The adjudged cause of failure is called a fault.
Example: A failure may be cause by a defective block of code.
+ Time
Time is a key concept in the formulation of reliability. If the time gap between two
successive failures is short, we say that the system is less reliable.
Two forms of time are considered.
+ Execution time ()
+ Calendar time (t)
36

37. Definitions of Software Reliability
Software reliability is defined as the probability of failure-free operation of a
software system for a specified time in a specified environment.
+ Key elements of the above definition
Probability of failure-free operation
Length of time of failure-free operation
A given execution environment
+ Example
The probability that a PC in a store is up and running for eight hours without
crash is 0.99.
37

38. Factors Influencing Software Reliability
+ A user’s perception of the reliability of a software depends upon two
categories of information.
The number of faults present in the software. The ways users operate the
system. This is known as the operational profile.
+ The fault count in a system is influenced by the following:
• Size and complexity of code
• Characteristics of the development process used
• Education, experience, and training of development personnel
• Operational environment
38

39. Software Quality
Metrics

40. Classification of Software
Measurement
+ Dynamic metrics – this metrics is collected by measuring
elements during program’s execution. This metrics help to
asses efficiency and reliability of a software product. The
parameters collected can be easily measured (i.e. execution
time, mean time between failures)
+ Static metrics – this metrics is collected by measuring
parameters of the end products of the software development.
This metrics help to asses the complexity, understandability,
and maintainability of a software product. The SLOC size and
ND are the most reliable predictors of understandability,
complexity, and maintainability.

41. The Essential Metrics
+ There is a number of metrics available based on which software quality is
measured. But among them, there are few most useful metrics which are most
essential in software quality measurement. They are –
7/19/2023 41
1. Code Quality
2. Reliability
3. Performance
4. Usability
5. Correctness
6. Maintainability
7. Integrity
8. Security

43. Tools for Software Quality
Management
+ Quality management is an important aspect of business processes. Organizations
must implement quality requirements, e.g., according to standards like ISO 9001.
Existing approaches on business process modeling provide no explicit means to
enforce such requirements. UML Activity Diagrams are a well recognized way of
representing those business processes. In this paper, we present an approach for
enforcing quality requirements in such business processes through the application of
process quality patterns to Activity Diagrams. These patterns are defined using a
pattern description language, being a light-weight extension of UML Activity
Diagrams. Accordingly, such patterns can be used in forward-engineering of
business processes that incorporate quality constraints right from the beginning.
Sample footer text 3/1/20XX 43

44. Activity diagrams
44

45. Activity diagrams
+ Allow you to model complex
behaviour/workflows
+ Similar to flowcharts except they allow parallel
behaviour
+ Each state in the diagram is an activity state
(where some action occurs)
+ Ref. UML Distilled, Martin Fowler
45

46. Why activity diagrams?
+ Model business workflows
+ Identify candidate use cases through examination of workflows
+ Identify pre/post conditions for use cases
+ Model workflows between use cases
+ Model workflows within use cases
+ Model complicated workflows in operations on objects
+ Model lower level details of complex activity diagrams
46
Ref. Schaum's Outlines UML, Bennet, Skelton, Lunn

47. Activity diagrams
+ Activity diagrams describe the flow of activities to achieve
a goal / user requirement.
+ They are an easy form to understand - clients will have no
trouble reading them.
+ Good to identify activities that can take place in parallel
Many systems simply replicate existing people-based systems without thinking about
what needs to be done sequentially and what can be done in parallel.
47

48. Activity diagrams
+ Consist of
Activities
+ exited when the activity is complete
Transitions
+ a path from one activity to another
Synchronisation bars
+ ensures transitions happen in unison
Decision diamonds
+ if statements…
Start and stop markers
+ the boundaries of the activity diagram
48

49. Branching
+ Allow for different paths in a flow
+ Guards are boolean conditions
that
must not overlap (ambiguous)
must cover all possibilities
+ A branch can have more than
leaving paths
+ A branch can be labelled "else" to
catch any other condition
49
Ref: The Unified Modelling Language Guide, p273, 2nd edition

50. Forking and joining
+ Synchronisation bars
reduce the ordering
on activities. Without
them all activities
would have to be
performed
sequentially.
50

51. Swim(ming) lanes
+ The activity diagrams so far
show the actions within a
workflow, but not who does
them.
+ We can show who is
responsible for the actions by
using "swimlanes" - all the
actions within one are done by
one entity
+ Swimlanes ≠ class
+ Swimlanes are any grouping
you think makes sense
51

52. + Activity diagram
for hiring a car.
52

53. Object flows
+ Objects may be
involved in the
flow of control in
an AD.
+ We can show
objects being
created, change of
state etc.
53

54. Activity diagrams
+ An activity can be turned
into a composite activity if
we need more detail.
54

55. Examples 55

56. The Differences
Use Cases
+ A use-case diagram describes
the relationships between actions
and discrete units of a system's
functionality. A use-case
description provides a brief
overview of the purpose of each
use-case and the steps required
to complete that purpose.
Activity Diagram
+ An activity diagram can be used
to expand on a use-case
description. Activity diagrams are
similar to flow charts: they
describe the order of activity and
the branch logic of a process
3/1/20XX 56

57. Thank you
3/1/20XX 57

58. TEAM QUESTION IN CLASS
+ Create an activity diagram for user login into a cloud
system (ie MS Teams).
Sample footer text 3/1/20XX 58