In this paper we present a quality model for spreadsheets, based on the ISO/IEC 9126 standard that defines a generic quality model for software. To each of the software characteristics defined in the ISO/IEC 9126, we associate an equivalent spreadsheet characteristic. Then, we propose a set of spreadsheet specific metrics to assess the quality of a spreadsheet in each of the defined characteristics. In order to obtain the normal distribution of expected values for a spreadsheet in each of the metrics that we propose, we have executed them against all spreadsheets in the large and widely used EUSES spreadsheet corpus. Then, we quantify each characteristic of our quality model after computing the values of our metrics, and we define quality scores for the different ranges of values. Finally, to automate the atribution of a quality score to a given spreadsheet, according to our quality model, we have integrated the computation of the metrics it includes in both a batch and a web-based tool.

1. A Quality Model for Spreadsheet
Jácome Cunha,João P.Fernandes,Christophe Peixoto,
João Saraiva
1
HASLab / INESC TEC & Universidade do Minho,
Portugal
QUATIC 2012

2. Why do Spreadsheets matter?
95% of all U.S. firms
90% of all analysts in industry
50% of all SSs are the basis for
decisions 2

3. Still...
Economy losses of $10
billion/year!
3

4. A recent spreadsheet error...
4

5. A Portuguese horror story...
5

6. Many more horror stories...
• EuSpriG Original Horror Stories
●
http://www.eusprig.org/stories.htm

7. SpreadSheets as SW Languages
In fact Spreadsheets lack advanced mechanisms
found in modern languages, namely:
Type and Modular Systems
Abstractions
Debuggers/Profilers/Refactors
Mechanisms for Model-Driven
Engineering and Software Evolution!
Best Practice Rules...
7

8. This paper
We propose a quality model for
spreadsheets based on the widely
accepted ISO/IEC 9126 standard for
software product quality.
We introduce a set of domain specific
metrics for spreadsheets used to measure
concrete spreadsheet characteristics.
We calibrate our quality model using a
large repository of real-world
spreadsheets (EUSES corpus)
8

9. Spreadsheets Quality Model:
Definition
9

10. SpreadSheet Quality Model: Definition
We have adapted the ISO/IEC 9126 to
Spreadsheets
10

11. SpreadSheet Quality Model: Definition
For example:
Understandability: There are many ways to
make a spreadsheet more understandable
1- the separation of the input, computation
and output components of the spreadsheet.
2- the use of different background colors
for different types of data cells.
3- a large number of cells makes a
spreadsheet hard to understand.
11

12. SpreadSheet Quality Model: Calibration
To calibrate our model we need to consider a
large repository of spreadsheets: The EUSES
corpus (5606 spreadsheets, organized in 6
categories)
12

13. SpreadSheet Quality Model: Calibration
The Metrics: we reused SIG Spreadsheets
metrics suite
13

14. SpreadSheet Quality Model: Calibration
The idea is to compute the metrics in a large
set of real-word spreadsheet in order to
define metric values that allow us to assign
a five star ranking to a spreadsheet.
Repository: EUSES
Metric: Number of
Cells
14

15. SpreadSheet Quality Model: Calibration
The graphics that represent the aggregation
based on the central tendency of the results
of computing the metric “number of cells”
15

16. SpreadSheet Quality Model: Calibration
Finally, we are able to define the star
ranking intervals for each of the
characteristics considered:
16

17. SpreadSheet Smells: Catalog Validation
We have implemented both a batch and web-
based spreadsheet quality analyzer tools:
17

18. SpreadSheet Quality Model: Evaluation
We have developed an equivalent android-based
application to assess quality of cloud stored
spreadsheets (google docs).
18

19. Conclusions
● We presented a quality model for spreadsheets.
● We have implemented the quality model as three
different tools.
● We have calibrated our model using the tools a
large repository containing real-world spreadsheets.
● But,
this is a preliminary result/model. We need to extend
our metric suite and, thus, to consider more sub-
characteristics of the ISO 9126 std.
19

20. Research Material
● SSaaPP – SpreadSheets as a Programmin
Paradigm (FCT funded)
http://ssaapp.di.uminho.pt
● AutoSeer - Automated Test Oracles for
Software Error Detection (FCT funded)
http://www.fe.up.pt/AutoSeer
20