A STUDY ON THE NEED FOR RATING SOFTWARE QUALITY

Western New England University

A Study On The Need For Rating
Software Quality
Although there has been an extensive study over delivering, increasing
and maintaining software quality, there has not been enough aidemémoire on ‘Rating a Software‘s Quality’. This study would project the
literature review thus far and also sculpt the scope and need for the
evolution of a rating system of software quality for the future.

8/28/2012

Under The Guidance Of,

Karthik Murali

Dr. Julie Drzymalski
Asst. Professor
Dept. of Industrial Engineering & Engineering Management
Western New England University

Student ID 131629

EMGT 698 Thesis

August 28, 2012

[A STUDY ON THE NEED FOR RATING SOFTWARE QUALITY]

Table of Contents

Sr. No.

Topic

1.

Page No.

Introduction
1.1 Motivation
1.2 Scope of the Research

3

1.3 Statement of the Problem

2.

2

5

Analysis Approach
2.1 A Graph Showing Previous Researches
2.2 Flowchart depicting SQAD

3.

6
9

Classifications & Reviews
3.1 Critical Quality Metrics

12

3.2 Rating System

16

4.

Need and Scope for Improvement

17

5.

Conclusion

18

Western New England University | EMGT 698 Thesis

1

August 28, 2012


1. INTRODUCTION
1.1

Motivation

“Quality is never an accident; it is always the result of intelligent effort.”
– John Ruskin

We are in the 21st century and this era is being overrun by technology. The top of the line
technological inventions are making things way easier for the masses as the last decade
witnessed the biggest leap in the field of software engineering (Shaw & Clements, 2006).
Businesses are becoming too dependent on the IT industry for rendering efficient and the best
services. There has been a deadlocked competition since then for providing the best quality
software for domestic and industrial purposes.

Research thus far; on delivering, increasing and rating software quality has been more generic
and thereby the frameworks designed fall into the same components. There has not been any
model that exhibits a technique that could rate software quality. The term rating refers to a
scoring system that would assist the customer (end-user) to understand that the software is of
great quality. Different classes of rating would be a yardstick to meticulously measure the
standard of the software application built. These numbers would not only help consumers in their
decision but also act as a foundation for the developers to build better applications in the future.
Consumers would be more comfortable with an easy to understand system that would not only
elevate their interest in the product but also expand the market horizon for the manufacturers
(developers).


2

August 28, 2012


1.2

Scope of the Research

The inclination and the interest of researchers from the past two decades have shown that
software quality has been a comprehensive area of study. Software Quality is a vast framework
but the existing quality process metrics and their generic design has challenged their own
limitations (George, Fleurquin, & Sadou, 2006). There are various quality metrics that have to be
taken into consideration while developing software and thereby assessed with utmost caution.
Some of the vital calibers in the field of Software Engineering are efficiency, maintainability,
portability, reliability, reusability, testability, usability etc. (Fenton, 1996)
However, (Dromey, 1994) has argued in his research titled, “Model for Software Product Quality”
that software metrics can be classified according to levels of attributes. The high level quality
attributes of software are the functionality, reliability and security of a software application.
There are times when these factors overlap and create a problem. If there is a defined Quality
Rating System, these quality metrics will be pre-classified according to the attributes they
possess and eventually it would help in overshadowing the problem of overlapping.
The impact of the software quality has not only affected the consumer market but also the
internal functioning of the developing organizations. From a consumer’s point of view, software
should function flawlessly but it is much beyond that belt of thinking. Any software that is of
good quality relies on metrics that are defined pertaining to the resources used in that design.
(Boehm, Brown, & Lipow, Quantitative Evaluation of Software Quality, 1976)


3

August 28, 2012


There have been more than 50,000 publications in the field of Software Engineering during the
last two decades. But the number of contributions made towards Software Quality Rating is a
minimum in the field. (Microsoft, 2012)
If we notice, today the market allows a product to be dominant only if it is of superior quality
and marked with the right price. Quality of a product and the pricing strategy run parallel with
one another. This has been stated as one of the recent challenges faced by many organizations.
Several models have been defined and designed in order to achieve software quality but all the
research have been more qualitative than quantitative. The concentration was more on process
metrics. Research indicates that the software quality models provide an explicit process building
quality carrying properties into software – which basically dealt with the qualitative attributes of
the measurement factors. (Dromey, 1994) His research aggregates the usage of ISO 9126. It
relates to the evaluation and assessment of the software being a high level framework and giving
room for the software developers to draw a line of quality check on the whole product but it
neither mentions the quantitative weightage of the model nor the evaluation mechanism of
quality.
The most recent publications was regarding the Quamoco Tool (Deissenboeck, Heinemann,
Herrmannsdoerfer, Lochman, & Wagner, 2011) which explains the working of the Quamoco
Tool that was developed using JAVA/Eclipse. It performs quality analysis of the application
depending upon its type. It still leaves the ground of the rating system open for extensive
research since there is no concrete system for rating software’s quality.


4

August 28, 2012

1.3

Statement of the Problem

Software quality is evaluated only on the basis of the metrics used in the application design
(Frakes & Terry, 1996). There have been large differences in areas of software quality since all
process metrics may not be uniform enough for all the software. Whenever a measure is defined
– how and why it has been formulated needs to be clearly expounded (Kearney, Sedlmeyer,
Thompson, Gray, & Adler, 1986). This point has been taken care of by all the developers, still a
clear question beckons, “How can we confirm the assurance about the software’s quality?” The
answer lies in taking a leap forward to model such a system that would help in rating software
quality. It would not just be a measure but a scaling index in itself.
As mentioned (Rosenberg and Hyatt; 1995) there are five basic attributes for quality –
complexity, efficiency, reusability, testability and understandability. These are uniform factors
that are given acute attention when the software is designed.
Figure1

Functional
Testing

Performance
Testing

User
Acceptance
Testing

GUI Based
Testing

Regression
Testing

Security
Testing


5

August 28, 2012


The focus of this research is to model a rating system that would act as a Software Quality Index
– scoring the software in a specific way that would cover and give more information about the
package to the end-user. Testing only, does not mean software quality assurance.

2. ANALYSIS APPROACH

2.1

Domain Trend
Figure2

Domain Trend of the Literature
Advanced Tools for SQA
Software Quality Assurance [Metrics]
Classification
Category

Qualitative/Quantitative Aspects
Software Quality (Basic Concepts)
End

Theoretical Computer Science

Duration
1970

1990

2010

Year

The above graphical representation gives a gist of the domain trend of the literature used for
conducting this research. Software Engineering and Software Quality Assurance (SQA) go hand
in hand.


6

August 28, 2012


Classifications
a. Theoretical Computer Science
The earliest research and publications were mostly related to theoretical computer science.
The concepts and fundamentals revolved around the knowledge that was in the form of
theory and not yet taken into advanced implementation.

b. Software Quality (The Basic Concepts)
The decade of the 90s showed some interesting contributions by many authors to the field
of Software Quality wherein the basic concepts of quality were covered. These concepts
were tailored further to understand quality engineering i.e. quality modeling for achieving
and sustaining software quality.

c. Qualitative & Quantitative Aspects
Following the 90s, the time span of the last decade shows various contributions to field of
software quality – qualitative and quantitative aspects. The publications and researches
during this timeline focused more on the development of models for evaluating software
quality on both aspects but failed to give a concrete model that could rate the quality of a
software.

d. Software Quality Assurance [Metrics]
From the beginning of this new century, organizations and researchers have drifted their
attention span towards the cornerstone of any software’s quality foundation i.e. the


7

August 28, 2012


metrics. There are several metrics that are considered while designing and developing a
software application. This study of the measuring factor gained utmost importance in the
recent decade since there was a strong competition in the software market for delivering
the best quality software.

e. Advanced Tools for SQA
The most recent and the exciting area of research as far as the software quality field goes,
is certainly the advanced tools for software quality assurance (SQA). Many tools have
been packaged for evaluating software quality according to the type of the software but
no design, methodology or technique claims that it can scale the quality of software by
giving a number for a higher and clearer apprehension of the features of the software.

A very recently released executive white paper explains about the ever increasing code growth in
the software projects and the time pressure the developers have to go through to meet the market
demands on time (Rommel & Girard, 2012). This white paper also mentions that the usage of
standard processes and automated testing such as model-based software testing, dynamic
software testing and static analysis tools can help the software engineers to identify and correct
the potential errors and overshadow any future threats to the application.

With the driving change in the need for process automation and to deliver best quality software
with established practices, there arises a need for a system to gauge software quality.


8

August 28, 2012

2.2

Flowchart Depicting SQAD
Figure3


9

August 28, 2012


Explanation
The software product development lifecycle is a uniform model which is adopted by
organizations worldwide that deal with the designing of software products – be it any kind of
application. It is up to the organization as to where and how they include the quality factor to
make sure that their software not only meets the customer and market requirements but also
uplifts the company’s reputation in the business.

Flowchart Details
1. In the initial stages of the software development lifecycle, a team from the software
designing organization goes to the client and gets the requirements. This is also known as
‘Raw Knowledge’ or ‘First Hand Information’. This data is then transformed into the
requirements needed for doing further analysis and generating a prototype design which
would be put to use for checking the match of the transformation of all the gathered
requirements.
2. Once the transformation is done, the prototype is ready, and then begins the interaction
modeling. This stage involves all the necessary steps that would bring out the different
levels of interaction between the application and the client.
3. Once the development team knows the levels of interaction, the next thing they do is
build the design. This would be a full-fledged design as it would be the next step after
structuring the prototype and all the necessary refined information in hand. This design
will exactly be similar to the client’s proposal.


10

August 28, 2012


4. This design is then taken to the client for an approval. Depending upon the approval, the
further steps are taken. If it’s a yes, the team gets back to the advancement in the
development of the application. If it’s a no, the team sits and discusses the modifications
expected on the software with the client and then redesign the prototype.
5. The team starts designing the application that would go live on the client’s platform. This
is one of the stages where the quality factor kicks in. There are a lot of technical attributes
taken into consideration. There will not be a 100% implementation of the quality
assurance factors but this phase will give light to all the mandatory metrics inclusion in
the application. Then starts the restructuring design process. This is also called as the
pretest phase.
6. The Alpha Release Stage – this phase is where the software application’s bare version is
released within the organization who designed it. The software engineers become the end
users and try to understand the way in which the application works. This is done so that
they can scan the software product for bugs and fix them.
7. Beta Release – This release deals with the usage of the software product in the client’s
place. The real end users run the application. They are allowed to get the feel of it and
give a feedback to the designing team and loosen it up if they need to. If any bugs, flaws
found they would be rectified here and thus the software is refined for final use. The final
approval is taken from the client and the software is packaged for the final release.


11

August 28, 2012


8. Quality Metrics –The levels of the alpha and the beta release is also termed as the quality
assurance belt. This is where the software is tailored for best quality. And the metrics of
the software is actually streamlined here. There is always a separate team of people
working on delivering the best quality of the product to the customers.

3. CLASSIFICATION & REVIEWS

3.1

Critical Quality Metrics

The breathtaking development in technology has insisted upon the need for making robust
software. Developers and consumers are becoming more quality conscious when releasing and
buying a software product respectively. This has led to a very intense and a competitive market.

How do we define quality? Is quality equal to reliability? Reliability alone cannot be considered
as a benchmark for rating quality of a software. (Naik & Tripathy, 2008, pp. 471 - 473)

(Rosenberg & Hyatt, 1997) Distinguished authors believe that there are a few uniform metrics
that are defined for achieving software quality. Some of the metrics that are included during the
design can be reused (Frakes & Terry, 1996) in order to improve productivity and quality.
Some of the critical metrics are explained in brief as follows, (Sommerville, 2010, p. 670)


12

August 28, 2012

Figure4

Critical Metrics
Complexity
Efficiency
Functionality
Maintainability
Reusability
Security
Testability
Understandability

1. Complexity
Complexity can be defined from various angles. It could be the lines of codes, the time
taken by the software to transfer to the memory and ready up for execution. Software
complexity is applied to the task span interaction between an application and a
programmer.

2. Efficiency
The efficiency metric of a software product determines the majority of the quality
component. There are various types of software products available in the market, some of
which revolve around huge complex algorithms and terabytes of data on a daily basis. If
the software is not efficient, the end user will eventually stop depending on the electronic
application.


13

August 28, 2012


3. Functionality
This characteristic explains the working of the software and the desired output they have
been programmed for. It also refers to the accuracy at which it functions. Functionality
involves the way in which all modules of the application are related to each other and the
effects of their integration on the overall performance.

4. Maintainability
Maintainability is spread over a wide spectrum. It relates to the controlling of defects
when the software is built and the requirements are transformed into the next phase
designing the software in such a way that it becomes an easy task to maintain it in the
future. Another aim of this metric is to make the software flexible that it easily adjusts to
newer work environments.

5. Reusability
It is a segment of the source code. Reusability depends on how good the code can be
reused to add new functionalities with little tweaks or modifications. The reusable
functions, modules, classes reduce the runtime or the implementation time because they
have already been in the random access memory and re-executing to perform some other
task. This saves time and memory.


14

August 28, 2012


6. Security
Considered to be one of the vital metrics, security of a software has the highest concern
when the application goes live. Data and resources in a software are injected and
extracted frequently in many software; it becomes an exigent task to make sure that the
transfer of data is over a secured platform.

7. Testability
Testing of a software has its own dimensions. Testing flexibility counts when quality of
the software comes into the picture. The easier the testing of the software, the more
optimized and organized is its architecture. If the software allows itself to be tested in as
many ways it can be done so, the chances of detecting defects and clean sweeping them
becomes higher.

8. Understandability
This metric can be classified as one under maintainability. Assessing maintainability may
require the whole designing team to follow the best software engineering practices and
the core attributes, some of which are lines of code, software’s architecture, algorithm
complexity, module pattern, reuse ratio, documentation, portability and hardware
environment.


15

August 28, 2012


3.2

Rating System

The new rating system should be modeled in such a way that it scores the software according to
its design, metric coverage and performance. It should not only help the consumers to make a
buy decision but to also absorb the features and the appropriate usage of the package.

The number that would represent the quality index should comprise of the requirements match,
usage ratio and performance prediction. Requirements match relates to the hardware compliance
in order to help the software application deliver the most. Usage ratio is the interaction rate
between the program and the end-user. Performance prediction indicates the relation between the
hardware and the application and the outcome of their integration and execution.

The Windows Experience Index (Microsoft, What is the Windows Experience Index?, 2012) is a
great example of a typical rating system. It measures the blend of the hardware and software of a
windows based computer and gives a score called as the base score. The higher the base score,
the better the performance of the computer. This score can be used when buying software
applications and games. Some of the games released by Microsoft come with a Windows
Experience Index specification. This makes it easier for the customer to buy applications without
any hesitation. If the index is low, a hardware upgrade helps to scale up that component, thereby
increasing the performance of the whole system (Microsoft, Windows 7 Features - Windows
Experience Index, 2012)


16

August 28, 2012


4.

NEED & SCOPE FOR IMPROVEMENT

This research opens up wider horizons to explore and improvise on the current contributions to
the software quality field. A rating system would certainly revolutionize the current scenario of
software products from both – developer and customer point of view. A model that would derive
the quality index of a software will indeed become a vital cartridge of the software development
life cycle. Quality rating should not refrain itself to just being a score but act as a foundation for
improvements on the software application i.e. smart upgrades and reusability.

The following is a graphical representation of the determinants of software quality (Pressman,
2000, pp. 83 - 85). A rating system may help us to identify more dimensions than just metrics to
ascertain quality of a software.
Figure5

Product

Business
Conditions

Process

Technology

People


17

August 28, 2012


5.

CONCLUSION

This research establishes a foundation that explains the need of a rating system for software
quality. It would be a model that will be quantitative in nature and represent the metric system
and the corresponding determinants and dimensions of software quality as a whole.

Being an era that is dominated by technology, software engineering has gained a noteworthy
impetus. The progress of software quality has also been parallel to the inventions of new
packages in the market. The money value is reaching heights and everyone desires to be sensible
when it comes to expenditure. Quality is a factor that can control time and money. With a very
dynamic rating system, software developers can save time and money simultaneously which
proves the need of the system to be the double the worth.

The rating system model’s methodology of portraying software’s quality as a quantitative
attribute will become a breakthrough and pave way for extensive study and research on the same.

“Quality is the ally of schedule and cost, not their adversary. If we have to sacrifice quality to
meet schedule, it’s because we are doing the job wrong from the very beginning.”
– James A. Ward


18

August 28, 2012


BIBLIOGRAPHY
1.

Bednar, D., & Robertson, D. (n.d.). Software Quality and Standards. SOEC2, 1 36.

2.

Boehm, B. W. (1973). Software and it's Impact - A Quantitative Assessment. 1 52.

3.

Boehm, B. W., Brown, J. R., & Lipow, M. (1976). Quantitative Evaluation of
Software Quality.

4.

Cai, L., Huang, S., & Xie, X. (2011). An Introduction to Software Quality Model
Development. Energy Procedia(13), 8749 - 8758.

5.

Cesar, J., Yu, Y., Liu, L., Eric, S. K., & Mylopoulos, J. (2005). Quality Based
Software Reuse. 1 - 15. Springer.

6.

Chelf, B. (n.d.). Measuring Software Quality: A Study of Open Source Software.
Tech. Report, Coverity, Inc., Department of Homeland Security, San Francisco.

7.

Cognizant. (2010). Software Quality Transformation. Focus on Results, not
Process, 1 - 8. Cognizant © 2010.

8.

Deissenboeck, F., Heinemann, L., Herrmannsdoerfer, M., Lochman, K., &
Wagner, S. (2011). The Quamoco Tool Chain for Quality Modeling and
Assessment. ICSE' 11 (pp. 1 - 3). Honolulu: ACM.

9.

Deissenboeck, F., Wagner, S., Pizka, M., Teuchert, S., & Girard, J. F. (2008). An
Activity Based Quality Model for Maintainability. Munchen, Germany.

10.

Dr. Petrasch, R. (1999). The Definition of Software Quality: A Practical
Approach. FastAbstract ISSRE, 1 - 2.

11.

Dromey, R. (1994). A Model for Software Product Quality. 1 - 35.

12.

Edgren, R., Emilsson, H., & Jansson, M. (n.d.). Software Quality Characteristics.
thetesteye.com v1.1.

13.

ESA, B. (1995). Guide to Software Quality Assurance. Status Report, European
Space Agency, Paris, Paris.


19

August 28, 2012


14.

Etzkorn, L. H., Hughes Jr., W. E., & Davis, C. G. (2001). Automated Reusability
Quality Analysis of OO Legacy Software. Information and Software
Technology(43), 295 - 308.

15.

Fenton, N. (1996). Software Metrics for Quality Control and Assurance. Software
Quality Research Laboratory. McMaster University.

16.

Fitzpatrick, R. (1996). Software Quality: Definitions & Strategic Issues. School of
Computing Report, Staffordshire University, Advanced Research Module.

17.

Frakes, W., & Terry, C. (1996, June). Software Reuse: Metrics and Models. ACM
Computer Surveys, 28(2), 1 - 21.

18.

Galin, D. (2004). Software Quality Assurance: From Theory to Implementation.
Harlow, Essex, England: Pearson Addison Wesley.

19.

Gallin, D., & Patton, R. (n.d.). Introduction to Software Quality Assurance. 1 - 33.

20.

George, B., Fleurquin, R., & Sadou, S. (2006). A Methodological Approach to
Choose Components in Development and Evolution Process. Electronic Notes in
Theoretical Computer Science(166), 27 - 46.

21.

Imam, A., Khoja, S. A., & Shariff, I. (2007). Improving Software Quality - A
Benchmarking Approach. CSREA Press, 1 - 11.

22.

Jordan, A. G., & Provost, E. (2004). Management for Software Quality, Testing &
Industry Development. China's SQ, Testing & Strategy Seminar, (pp. 1 - 48).
Beijing.

23.

Kan, S. H. (2002). Metrics & Models in Software Quality Engineering. Addison Wesley Professional.

24.

Kearney, J. K., Sedlmeyer, L., Thompson, W. B., Gray, M. A., & Adler, M. A.
(1986, November). Software Complexity Measurement. Communications of the
ACM, 20(11), 1044 - 1050.

25.

Kitchenham, B., & Pfleeger, S. L. (1996, January). Software Quality: The Elusive
Target. © IEEE, 13(1), 12 - 21.

26.

Leung, H. K. (2001). Quality Metrics for Intranet Applications. Information &
Management(38), 137 - 152.

27.

Microsoft, ©. (2012, September 13). Domain Trend. Retrieved September 2012,
2012,
from
Microsoft
Academic
Research:
http://academic.research.microsoft.com/DomainTrend?TopDomainId=2


20

August 28, 2012


28.

Microsoft, ©. (2012, September 13). What is the Windows Experience Index?
Retrieved
September
2012,
2012,
from
Microsoft
Windows:
http://windows.microsoft.com/en-US/windows7/What-is-the-WindowsExperience-Index

29.

Microsoft, ©. (2012, September 13). Windows 7 Features - Windows Experience
Index. Retrieved September 13, 2012, from Microsoft Windows:
http://windows.microsoft.com/en-US/windows7/products/features/windowsexperience-index

30.

Naik, K., & Tripathy, P. (2008). Software Testing and Quality Assurance Theory
& Practice (First ed.). August: John Wiley & Sons.

31.

Parallab. (2004). Software Reusability and Efficiency. University of Bergen
(Norway), Bergen Center for Computational Science. Enacts.

32.

Pressman, R. S. (2000). Software Engineering: A Practitioner's Approach (Fifth
ed.). December: McGraw Hill.

33.

Punter, T., & Lami, G. (1998). Factors of Software Quality Evaluation. ESCOMENCRES' 98, (pp. 1 - 11).

34.

Rommel, C., & Girard, A. (2012). Embedded Software & Tools Practice. VDC
Research. Parasoft.

35.

Rosenberg, D. H., & Hyatt, L. E. (1997). Software Quality Metrics for Object
Oriented Environment. Crosstalk Journal, 1 - 7.

36.

Sacha, K. (2005). Evaluation of Software Quality. 1 - 8. Warszawa, Poland.

37.

Shaw, M., & Clements, P. (2006, February). The Golden Age of Software
Architecture: A Comprehensive Survey. CMU-ISRI-06-101, 1 - 14.

38.

Sommerville, I. (2010). Software Engineering (Ninth ed.). March: Pearson.

39.

Tian, J., & Troster, J. (1998). A Comparison of Measurement and the Defect
Characteristics of New and Legacy Software Systems. The Journal of Systems
and Software(44), 135 - 146.

40.

Wallace, D., & Reeker, L. (2001). Software Quality. In NIST. Gaithersburg,
Maryland, USA: © IEEE - Trial Version 1.00.


21

August 28, 2012


APPENDIX 1 – Figures

Figure 1

Different types of testing methodologies used to fix discrepancies in the
software. Information extracted and presented as a SmartArt diagram from
(Pressman, 2000) and (Naik & Tripathy, 2008)

Figure 2

Domain Trend Timeline of the Literature Review powered by the Domain
Trend tool by Microsoft Academic Research

Figure 3

SQAD – Software Quality Assurance and Deployment : Flowchart derived after
studying the SQA Plan (Pressman, 2000, p. 218) and Software Standards
(Sommerville, 2010, p. 657)

Figure 4

SmartArt representation depicting critical metrics involved in the designing of a
software application

Figure 5

SmartArt representation showing the various determinants or dimensions of
software quality


22

A STUDY ON THE NEED FOR RATING SOFTWARE QUALITY

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (19)

Similar to A STUDY ON THE NEED FOR RATING SOFTWARE QUALITY

Similar to A STUDY ON THE NEED FOR RATING SOFTWARE QUALITY (20)

More from Karthik Murali

More from Karthik Murali (13)

Recently uploaded

Recently uploaded (20)

A STUDY ON THE NEED FOR RATING SOFTWARE QUALITY