This document provides an overview of software quality assurance. It discusses the evolution of SQA from an initial focus on "code and ship" in the 1960s-1980s to today's emphasis on SQA processes. Key concepts covered include quality, quality control, quality assurance, and the cost of quality. Elements of SQA like activities and models are described. Leading organizations' SQA practices are examined through case studies. The document aims to explain the importance of SQA for software development organizations.

1. A Paper on
Quality Assurance in Software Development
Submitted By:
Sharad Srivastava
12810076
MBA 2nd Year (2012-2014)
DoMs, IIT Roorkee
1|Page

2. Table of Contents
1.
Introduction......................................................................................................................................... 3
2.
Evolution of Software Quality Assurance .......................................................................................... 4
3.
Quality Concepts................................................................................................................................. 5
3.1
3.2
Quality Control ........................................................................................................................... 5
3.3
Quality Assurance ....................................................................................................................... 5
3.4
4.
Quality ........................................................................................................................................ 5
Cost of Quality ............................................................................................................................ 5
Elements of Software Quality Assurance ........................................................................................... 7
4.1
4.2
5.
SQA Activities ............................................................................................................................ 7
Statistical Quality Assurance ...................................................................................................... 8
Software Quality Assurance Models .................................................................................................. 9
5.1
5.2
6.
CMMI ......................................................................................................................................... 9
ISO 9000 ................................................................................................................................... 10
Quality Assurance Practices by Leading Organisations ................................................................... 12
6.1
6.2
TechMahindra ........................................................................................................................... 13
6.3
7.
HCL Technologies .................................................................................................................... 12
A Case Study of Quality Assurance: Tata Consultancy Services ............................................. 14
References......................................................................................................................................... 15
2|Page

3. 1. Introduction
Quality is always an important consideration for software development. Broadly speaking
quality is the degree of conformance to the requirements. In the context of software
development, quality consists of many attributes such as speed, reliability, stability, security
etc. the cumulative effect of these characteristics is what is perceived as a quality. Quality is
also understood as something which meets the user requirement. It should be noted that the
same software might be perceived differently by two users and hence their quality will be
different in the eyes of the users. It is essential for an organization to confirm to the quality
expectations and minimize the chances of error or failures. This process of assurance is
known as Software Quality Assurance. According to IEEE, software quality assurance can be
defined as:
1. Planned and systematic pattern of all actions necessary to provide adequate
confidence that an item or product conforms to established technical requirements.
2. A set of activities designed to evaluate the process by which products are developed
or manufactured.
Software quality assurance (SQA) consists of a means of monitoring the software engineering
processes and methods used to ensure quality. The methods by which this is accomplished
are many and varied, and may include ensuring conformance to one or more standard, such as
ISO 9000 or a model such as CMMI. SQA encompasses the entire software development
process, which includes processes such as requirements definition, software design, coding,
source code control, code reviews, change management, configuration management, testing,
release management, and product integration. SQA is organized into goals, commitments,
abilities, activities, measurements, and verifications. A software quality assurance process
encompasses:
A quality management approach,
Effective software engineering technology (methods and tools),
Formal technical reviews that are applied throughout the software process,
A multitiered testing strategy,
Control of software documentation and the changes made to it,
A procedure to ensure compliance with software development standards (when
applicable), and
Measurement and reporting mechanisms.
Quality assurance is increasingly being given importance by the service providers. As per
World Quality Report 2013-14, the total QA budget has risen from 18% in 2012 to 23% in
2013. Organisations are also adopting standards and best practices to ensure quality. In India
too, quality is given a significant weight by companies and major players such as Infosys or
TCS takes a lead in this front.
3|Page

4. 2. Evolution of Software Quality Assurance
Phase 1: 1960’s Code and Ship
Software began to find its way into systems procured by the US Dept. of Defense (DoD).
These projects were consistently behind schedule, over budget, and had many technical
problems. Frequently, softwares never worked as intended and many projects were cancelled
before anything was delivered. Often when they were unable to understand them and assess
their impact, an ―independent software tester‖ was hired to ―perform additional, unbiased
testing of the software‖. By employing someone who was totally separate from the software
development contractor, the DoD hoped to get a more accurate and objective technical
assessment of the project’s status.
Phase 2: 1970-80’s Code and Ship Crisis
During the 1970’s, as the software development activity began to expand into the private
sector, software development companies were experiencing the same poor results as did
government agencies a decade earlier. Companies had difficulty delivering software within
the constraints of schedule, budget, and quality. Most companies relied on developers to test
their own code and report on issues. During the 1980’s, we experienced what became known
as the ―software crisis‖ – the point in time when spending on software maintenance exceeded
spending on creating new software products. The ―software crisis‖ brought with it a host of
changes and the emergence of SQA. While SQA was viewed as the ―poor stepchild‖ of
software development, many enlightened companies saw measurable benefit from integrating
SQA into the software development process.
Phase 3: 1990’s Code, Test and Ship
By the 1990’s, many software companies had SQA functions within their organizations. Yet,
high-profile software failures continued to occur. The complexity of software development,
along with competitive business pressure, during the 90’s increased significantly. Softwares
were being used in many new areas, especially life-threatening ones. Many people working in
the SQA received little formal training in SQA. SQA engineers were expected to learn their
craft primarily from on-the-job training. Universities failed to recognize that SQA is a
legitimate discipline unto itself and that it requires specialized training.
Phase 4: 2000’s SQA Processes Adopted
The SQA department is a standard role in any major software development lifecycle and
mandated by any standard outsourcing contract for software. Various standards organizations
(such as ISO, IEEE, CMM) have been created to evaluate and improve an organization’s
ability to produce quality software. Universities offer a wide range of SQA courses and areas
of specialization.
4|Page

5. 3. Quality Concepts
3.1
Quality
The American Heritage Dictionary defines quality as ―a characteristic or attribute of
something.‖ As an attribute of an item, quality refers to measurable characteristics – things
we are able to compare to known standards such as length, color, electrical properties, and
malleability. However, software, largely an intellectual entity, is more challenging to
characterize than physical objects. Nevertheless, measures of a program’s characteristic do
exist. These properties include complexity, cohesion, number of function points, lines of
code, and many others.
3.2
Quality Control
Quality control involves the series of inspections, reviews, and tests used throughout the
software process to ensure each work product meets the requirements placed upon it. Quality
control includes a feedback loop to the process that created the work product. The
combination of measurement and feedback allows us to tune the process when the work
products created fail to meet their specifications. This approach views quality control as part
of the manufacturing process. Quality control activities may be fully automated, entirely
manual, or a combination of automated tools and human interaction. A key concept of quality
control is that all work products have defined, measurable specifications to which we may
compare the output of each process. The feedback loop is essential to minimize the defects
produced.
3.3
Quality Assurance
Quality assurance consists of the auditing and reporting functions of management. The goal
of quality assurance is to provide management with the data necessary to be informed about
product quality, thereby gaining insight and confidence that product quality is meeting its
goals. Of course, if the data provided through quality assurance identify problems, it is
management’s responsibility to address the problems and apply the necessary resources to
resolve quality issues.
3.4
Cost of Quality
The cost of quality includes all costs incurred in the pursuit of quality or in performing
quality-related activities. Cost of quality studies are conducted to provide a base- line for the
current cost of quality, identify opportunities for reducing the cost of quality, and provide a
5|Page

6. normalized basis of comparison. The basis of normalization is almost always dollars. Once
we have normalized quality costs on a dollar basis, we have the necessary data to evaluate
where the opportunities lie to improve our processes. Furthermore, we can evaluate the effect
of changes in dollar-based terms. Quality costs may be divided into costs associated with
prevention, appraisal, and failure.
Prevention cost:
Quality Planning
Formal Technical Interviews
Test Equipments
Training
Appraisal Cost:
In-process and Inter-process Inspection
Equipment calibration and maintenance
Testing
Failure Cost:
External Failure
 Complaint Resolution
 Product Return and Replacement
 Helpline Support
 Warranty Work
Internal Failure
 Rework
 Repair
 Failure Mode Analysis
It should be noted that the cost of correcting the defects is inversely proportional to the stage
at which error is detected. This implies that the sooner we identify an error, the lesser would
be the implication and the cost of recovery. On the other hand, if an error is detected late, the
effect could be catastrophic.
6|Page

7. 4. Elements of Software Quality Assurance
Software Quality Assurance encompasses a broad range of concerns and activities that focus
on management of software quality. These can be summarized in the following manner:
Standards
Reviews and audits
Testing
Error/defects collection and analysis
Change management
Education
Vendor management
Security management
Safety
Risk management
4.1
SQA Activities
Software quality assurance is composed of a variety of tasks associated with two different
constituencies—the software engineers who do technical work and an SQA group that has
responsibility for quality assurance planning, oversight, record keeping, analysis, and
reporting. Software engineers address quality (and perform quality assurance and quality
control activities) by applying solid technical methods and measures, conducting formal
technical reviews, and performing well-planned software testing.
The charter of the SQA group is to assist the software team in achieving a high quality end
product. The Software Engineering Institute recommends a set of SQA activities that address
quality assurance planning, oversight, record keeping, analysis, and reporting. These
activities are performed (or facilitated) by an independent SQA group that:
Prepares an SQA plan for a project. The plan is developed during project planning and is
reviewed by all interested parties. Quality assurance activities performed by the software
engineering team and the SQA group are governed by the plan. The plan identifies:
Evaluations to be performed
Audits and reviews to be performed
Standards that are applicable to the project
Procedures for error reporting and tracking
Documents to be produced by the SQA group
The amount of feedback provided to the software project team
The software team selects a process for the work to be performed. The SQA group reviews
the process description for compliance with organizational policy, internal software
standards, externally imposed standards (e.g., ISO-9001), and other parts of the software
7|Page

8. project plan.
Reviews software engineering activities to verify compliance with the defined software
process. The SQA group identifies, documents, and tracks deviations from the process and
verifies that corrections have been made.
Audits designated software work products to verify compliance with those defined as
part of the software process. The SQA group reviews selected work products; identifies,
documents, and tracks deviations; verifies that corrections have been made; and periodically
reports the results of its work to the project manager.
Ensures that deviation in software work and work products are documented and
handled according to a documented procedure. Deviations may be encountered in the
project plan, process description, applicable standards, or technical work products.
Records any noncompliance and reports to senior management. Noncompliance items
are tracked until they are resolved. In addition to these activities, the SQA group coordinates
the control and management of change and helps to collect and analyze software metrics.
4.2
Statistical Quality Assurance
Statistical quality assurance reflects a growing trend throughout industry to become more
quantitative about quality. For software, statistical quality assurance implies the following
steps:
1. Information about software defects is collected and categorized
2. An attempt is made to trace each defect to its underlying cause
3. Using the Pareto principle (80% of the defects can be traced to 20% of all possible
causes), isolate the 20% (the ―vital few‖)
4. Once the vital few causes have been identified, move to correct the problems that
have caused the defects.
This relatively simple concept represents an important step toward the creation of an adaptive
software engineering process in which changes are made to improve those elements of the
process that introduce errors.
Software reliability is defined in statistical terms as "the probability of failure-free operation
of a computer program in a specified environment for a specified time". To illustrate,
program X is estimated to have a reliability of 0.96 over eight elapsed processing hours. In
other words, if program X were to be executed 100 times and require eight hours of elapsed
processing time (execution time), it is likely to operate correctly (without failure) 96 times
out of 100.
8|Page

9. 5. Software Quality Assurance Models
5.1
CMMI
The Capability Maturity Model Integration (CMMI) is a capability maturity model developed
by the Software Engineering Institute, part of Carnegie Mellon University in Pittsburgh,
USA. The CMMI principal is that ―the quality of a system or product is highly influenced by
the process used to develop and maintain it‖. CMMI can be used to guide process
improvement across a project, a division, or an entire organization. CMMI provides:
Guidelines for processes improvement
An integrated approach to process improvement
Embedding process improvements into a state of business as usual
A phased approach to intro- ducing improvements
The CMMI model contains five maturity levels. However appraisal is done only for level 2 to
level 5.
1. Initial (adhoc and chaotic) – Success depends on individual effort as very few
processes are defined. There is no mechanism to ensure if they are used consistently.
The planning use to be ineffective and results unpredictable.
2. Repeatable – It indicates the presence of basic project management processes to track
cost, schedule and functionality. These processes can assist in repeating the success.
The planning and tracking become stable however it still suffers from quality issues.
3. Defined – The software processes for both management as well as engineering are
documented, standardized and integrated into organization wide software processes. It
hints that procedures are in place to ensure that they are followed. Usually project
tailor these processes to develop their own processes. Overall the organization has a
stable foundation for software engineering and management.
4. Managed – At this level, detailed measures of software processes and product quality
are collected. It is done through metrics and prediction can be done. These measures
help establishment of quantitative goals and identification of limits. Exceptional cases
are identified and addressed. Additionally, the challenges of new domains can be
managed.
5. Optimized – It enables continuous process improvement by quantitative feedback
from the processes. The process improvement can be planned, budgeted and becomes
part of the organizational process. This helps in identify and accommodate best
practices.
9|Page

10. Figure 5.1: A pictorial view of CMMI model
5.2
ISO 9000
ISO 9000 is a series of standards, developed and published by the International Organization
for Standardization (ISO), that define, establish, and maintain an effective quality assurance
system for manufacturing and service industries. The ISO 9000 standard is the most widely
known and has perhaps had the most impact of the 13,000 standards published by the ISO. It
serves many different industries and organizations as a guide to quality products, service, and
management. The standard is divided into four parts, which address, respectively, the
following subjects: quality model; external metrics; internal metrics; and quality in use
metrics.
The ISO 9000 series of standards is a set of documents dealing with quality systems that can
be used for external quality assurance purposes. They specify quality system requirements for
use where a contract between two parties requires the demonstration of a supplier's capability
to design and supply a product. The two parties could be an external client and a supplier, or
both could be internal.
ISO 9000, "Quality management and quality assurance standards – Guidelines for selection
and use," clarifies the distinctions and interrelationships between quality concepts and
provides guidelines for the selection and use of a series of international standards on quality
systems that can be used for internal quality management purposes (ISO 9004) and for
external quality assurance purposes (ISO 9001, 9002, and 9003). The quality concepts
addressed by these standards are:
An organization should achieve and sustain the quality of the product or service
produced so as to meet continually the purchaser's stated or implied needs.
10 | P a g e

11. An organization should provide confidence to its own management that the intended
quality is being achieved and sustained.
An organization should provide confidence to the purchaser that the intended quality
is being, or will be, achieved in the delivered product or service provided. When
contractually required, this provision of confidence may involve agreed demonstration
requirements.
ISO 9001, "Quality systems – Model for quality assurance in design/development,
production, installation, and servicing," is for use when conformance to specified
requirements is to be assured by the supplier during several stages, which may include
design, development, production, installation, and servicing. Of the ISO 9000 series, it is the
standard that is pertinent to software development and maintenance.
An organization can be ISO 9000-certified if it successfully follows the ISO 9000 standards
for its industry. In order to be certified, the organization must submit to an examination by an
outside assessor. The assessor interviews staff members to ensure that they understand their
part in complying with the ISO 9000 standard, and the assessor examines the organization's
paperwork to ensure ISO 9000 compliance. The assessor then prepares a detailed report that
describes the parts of the standard the organization missed. The organization then agrees to
correct any problems within a specific time frame. When all problems are corrected, the
organization can then be certified. Today, there are approximately 350,000 ISO 9000certified organizations in over 150 countries.
Table 5.2: Comparison between CMMI and ISO
CMMI
ISO
Five levels of certifications with explicit Minimum requirement to become certified
continuous requirement
with implied continuous improvement
Designed specifically for the software Design to apply to all industry developments
industry
Inwardly focused
Outwardly focused
No yearly recertification process
Yearly recertification
Certified by SEI (developers of CMMI)
Third party certification
11 | P a g e

12. 6. Quality Assurance Practices by Leading Organisations
6.1
HCL Technologies
Quality Assurance is a planned and systematic means for assuring management that the
defined standards, practices, procedures, and methods of the process are applied. (Source:
Software Engineering Institute, Carnegie Mellon University)
The most effective way to manage quality cost is to prevent defects at the early stages of
Project Life Cycle (PLC). As a universal known fact, it is always less expensive to detect &
fix the defect at an earlier stage than fixing it once it has occurred in the production.
HCL enables their customers to have a robust quality system by providing Quality Assurance
Services for all types of projects (Development, Enhancement, Maintenance, Testing and
Production Support). We help the customers to:
Enable projects to detect defects early in the project lifecycle
Establish a Defect Tracking System
Establish Quality Gate (QG) framework
Establish Defect Prevention Methods
Enable Process Improvement initiatives
Improve the Quality of Deliverables
Figure 6.1: HCL Quality Assurance Service Portfolio
12 | P a g e

13. HCL’s consultants are SMEs and have multiple years of process implementation & project
facilitation experience.
Many of HCL’s customers have benefited in:
Improving the ability to manage and prevent the defects to production systems
Reducing rework & cost
Improving productivity
Sustaining the quality of deliverables throughout the project life cycle
Early detection & fixation of defects in the software life cycle
6.2
TechMahindra
As a part of its mature quality processes, Tech Mahindra uses well defined process
measurements to monitor the quality of solutions delivered and ensure continuous
improvement. With a strong focus on process management, Tech Mahindra’s Business
Management System (BMS) integrates business needs and industry best practices to deliver
services that constantly improve:
Customer satisfaction
Productivity and Cycle-Time
Quality of Solutions and Services
Figure 6.2: Integration of Initiatives
Tech Mahindra's BMS is designed to develop solutions that meet client specifications in
accordance with statutory and other industry-wide standards. Tech Mahindra’s quality
leadership conforms to world-class quality standards and models, such as:
SEI-CMMi Level 5 (v 1.3)
ISO 9001:2008
ISO/IEC 27001:2005
ISO 20000-1:2005
13 | P a g e

14. BS 25999-2:2007
TMMi Level 5
The 1st Indian IT services company & 2nd in the World to get the TMMi Level 5
Certification.
6.3
A Case Study of Quality Assurance: Tata Consultancy Services
The Customer: TCS’ client is a leading provider of information and business intelligence for
students, professionals and institutions in the legal, tax, finance and healthcare industries. Its
biggest customers include traditional publishers of legal reference tools, textbooks and
electronic information providers.
Business Challenge: The client’s electronic content delivery and publishing platform
comprise a set of services, standards and process guidelines to support the legal, tax and
regulatory online product development for different business units and delivery value chains.
The company wanted to accelerate its product development through the content delivery
platform to tap into markets across geographies.
Changing customer demand had also led to the need for the simultaneous development of
multiple products across new delivery channels like mobility, portlets, etc. However, it was
unable to reuse the existing test cases for web services as these were changing over the
releases. This led to long manual regression cycles, high automation test re-scripting effort
and large gaps in testing. This also meant that the company was unable to assess and
guarantee the performance on every release.
TCS’ Solution: TCS took charge of the entire platform testing process. We performed
quality assurance activities of the platform UI and web services involving unit, system and
performance testing. Then we developed different approaches for effective automated
regression testing. The test scripts were made reusable to assess the performance on every
release and identify major bottlenecks.
Result: TCS’ automated testing solutions kept pace with the company’s burgeoning
regression testing needs, eliminating the need to increase testing manpower. Using the
automated testing approach, we ensured that the company was able to reuse test cases across
releases and multiple devices and that regression test scripts required minimal maintenance,
enabling multiple releases with a single test case repository. Our testing services have
resulted in annual cost savings with the following:
Automation of 40% regression test cases in web services
Reduction in test effort by 30% with the introduction of web services testing
14 | P a g e

15. 7. References
Books:
Pressman Roger S., ―Software Engineering: A Practitioner's Approach‖, McGraw
Hill, 7e, 2010
Research Paper:
Hendren Jens, ―Quality Control in Software production‖, Seminar on Software
Development Tools, University of Helsinki, 2000
Giese Holger, ―Software Quality Assurance: Introduction‖,
Others:
http://www.csbdu.in/econtent/Software%20Quality%20and%20Testing/UNIT%20I.p
df, accessed on 06th Oct 2013
IEEE Standards for Software Quality Assurance Plans, 730-1998, IEEE Computer
Society, 2002
15 | P a g e