The document discusses the development and evaluation of a spreadsheet understanding and debugging tool aimed at improving the mental models of users. The tool translates spreadsheet formulas into problem domain narratives, making it easier for authors to locate errors and understand relationships within their spreadsheets. Evaluation results indicated that the tool enhanced usability by increasing the accuracy of error detection and shifting focus from spreadsheet-specific to domain-specific understanding.

1. Visualizing the Problem Domain
for Spreadsheet Users: A Mental
Model Perspective
Bennett Kankuzi, Jorma Sajaniemi
School of Computing, Joensuu Campus
University of Eastern Finland, Finland

2. Outline
•Introduction
•Description of Proposed Tool
•Evaluation of the Tool (Methodology, Results
and Discussion)
•Conclusion
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 2

3. Introduction
•A mental model can be defined as “a mental image of
the world around us that we carry in our heads
depicting only selected concepts and relationships that
represent real systems” (Doyle & Ford, 1998)
– a mental model for a spreadsheet, therefore, does not
carry all possible information, but just those aspects
that the user finds appropriate for the current task
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 3

4. Introduction (cont’d)
•Important to understand spreadsheet authors’ mental
models when doing different spreadsheet process
activities
– to understand why the spreadsheet process is so
error-prone
– to develop the right tools and techniques for
spreadsheet activities
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 4

5. Introduction (cont’d)
•Spreadsheet authors have at least three mental models:
real-world, domain and spreadsheet models (Kankuzi &
Sajaniemi, 2013)
– the real-world model that comprises general knowledge
of the world around us e.g. “motor vehicle”
– the domain model that represents knowledge of the
problem domain and the functionality of the spreadsheet
in problem domain or application terms e.g. “fixed
assets”
– the spreadsheet model that codes the expressions and
data relationships in the spreadsheet e.g. “cell B1”
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 5

6. Introduction (cont’d)
•Research Question:
– Is it possible to develop an easy to use spreadsheet
understanding and debugging tool that relieves
users from spreadsheet details and lets them utilize
more of their mental model of the application
domain and hence improving the mapping between
the domain/real-world mental models and the
spreadsheet mental model?
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 6

7. The Tool
•Translates traditional spreadsheet formulas into problem domain
narratives and highlights referenced cells
– domain terms formed from labels (headers) through spatial layout
information of each cell referenced to in the formula
•Implemented as an MS Excel add-on
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 7

8. Related Tools and Techniques
•In spreadsheets, symbolic names and formula translation have been
used with the hope to clarify the mapping between various levels
of abstraction
– use of named ranges such as in MS Excel and Google Spreadsheets
– some spreadsheet visualization tools also do formula translations e.g.
Spreadsheet Professional
– model-driven spreadsheet development approaches such as
ClassSheet models (Engels & Erwig, 2005) also translate formulas to
more humanized higher level object oriented style formulas
•All these tools and techniques anecdotally assume that symbolic
names and formula translation are useful to spreadsheet authors,
but their usability has not been empirically evaluated nor
psychologically justified
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 8

9. Evaluation of Tool - Overview
•Adapted Nielsen’s usability attributes of learnability, efficiency and
satisfaction (Nielsen, 1994) in the evaluation tasks
– evaluation involved 12 volunteering accountants (one woman and
eleven men) who are frequent users of spreadsheets
– none of the participants had participated earlier in similar studies
– first author visited each participant at their place of work
•Also investigated on effect of tool on the mental models of
spreadsheet authors
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 9

10. Evaluation of Tool - Learnability
•Methodology
– short demo followed by two tasks
•Results
– highlighting task (mark a narrated area on spreadsheet): mean 85%
correct (min 60%, max 100%)
– translation task (convert narration into spreadsheet terms): mean 83%
correct (min 60%, max 100%)
•Discussion
– good enough to proceed to the other evaluation tasks
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 10

11. Evaluation of Tool - Efficiency
•Methodology
– within-subjects design experiment where the task was to locate errors
in a spreadsheet without the tool and with the tool
– two roughly equivalent spreadsheets sourced from EUSES
spreadsheet corpus (Fisher & Rothermel, 2005) seeded with similar
errors adapted from Raffensperger(2005) and Duggirala(2012)
– some error types for seeded errors
• Formula accidentally overwritten with constants (Error Type C)
• Formula missing some range (Error Type D)
• A wrong problem domain formula (Error Type G)
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 11

12. Evaluation of Tool - Efficiency (cont’d)
•Results
•Discussion
– tool generally helps authors to catch more errors in spreadsheets (p =
0.021) although different aspects of the tool may be more helpful for
some error types than others
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 12

13. Evaluation of Tool - Satisfaction
•Methodology
– participants were requested to write down their opinion of the two
scenarios in terms of how they find it easier to locate errors as well as
well as any suggested improvements to the tool
•Results
– eleven out of the twelve participants found the tool helpful in locating
errors
– one participant said that he found the tool confusing as he is used to
the “normal Excel”
•Discussion
– generally, participants found the tool useful
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 13

14. Evaluation of Tool - Effect on Mental Models of
Users
•Methodology
– nine of the twelve participants wrote down explanations for
each of the located errors in the assigned tasks
– explanations were analyzed and classified using an inter-rater
reliability verified adaptation of Good’s program summary
analysis technique in which each object/noun is classified as
spreadsheet specific or domain specific or real-world specific
(Kankuzi & Sajaniemi, 2013)
•e.g. “column D” is classified as spreadsheet specific; ``total
liabilities’’ is classified as domain specific; and “money” is
classified as real-world specific
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 14

15. Evaluation of Tool – Effect on Mental Models
of Users (cont’d)
•Results
p = 0.0001
p = 0.0114
N.S.
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 15

16. Evaluation of Tool - Effect on Mental Models of
Users (cont’d)
•Discussion
– participants used mostly spreadsheet terms when describing an error
in the without tool case while with the tool, the spreadsheet model is
less prominent whereas the share of the domain model increases
– tool, therefore, improves the mapping between the spreadsheet and
domain models which makes understanding and debugging
spreadsheets more efficient (located more errors with the tool)
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 16

17. Conclusion
•Reported on a domain terms visualization tool developed to aid in
spreadsheet comprehension and debugging
– tool was found to be learnable
– tool helped the participants to locate more errors in spreadsheets
– participants also found the tool useful in an error locating task
– tool makes the spreadsheet model to decrease while at the same time
increasing the domain model
– hence we put forward that the tool improves the mapping between
the spreadsheet and domain models which improves performance in
understanding and debugging a spreadsheet
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 17

18. Thank you for your attention!
www.uef.fi

19. References
1. B. Kankuzi and J. Sajaniemi, “An Empirical Study of Spreadsheet Authors’
Mental Models in Explaining and Debugging Tasks,” in 2013 IEEE Symposium on
Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 2013, pp. 15–18.
2. J. Nielsen, Usability Engineering. Boston: AP Professional, 1994
3. M. Wertheimer, A Source Book of Gestalt Psychology. London: Routledge & Kegan
Paul, 1938.
4. M. Fisher and G. Rothermel, “The EUSES spreadsheet corpus: a shared resource
for supporting experimentation with spreadsheet dependability mechanisms,” in
Proceedings of the First Workshop on End-User Software Engineering, ser. WEUSE I.
New York, NY, USA: ACM, 2005, pp. 1–5.
5. J. Sajaniemi, “Modeling spreadsheet audit: A rigorous approach to automatic
visualization,” Journal of Visual Languages & Computing, vol. 11, no. 1, pp. 49–
82, 2000.
6. J. S. Davis, “Tools for spreadsheet auditing,” International Journal of Human-
Computer Studies, vol. 45, no. 4, pp. 429–442, 1996.
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 19

20. References (cont’d)
5. R. Desimone and J. Duncan, “Neural Mechanisms of Selective Visual Attention,”
Annual Review of Neurosciences, vol. 18, pp. 193–222, 1995.
6. P. Duggirala, Excel Auditing Functions [Spreadsheet Risk Management – Part 3
of 4], 2012, accessed December 2012. [Online]. Available:
http://chandoo.org/wp/2012/01/18/excel-auditing-functions/
7. G. Engels and M. Erwig, “ClassSheets: automatic generation of spreadsheet
applications from object-oriented specifications,” in Proceedings of the 20th
IEEE/ACM International Conference on Automated Software Engineering. ACM,
2005, pp. 124–133.
8. J. F. Raffensperger, The Art of the Spreadsheet, 2008, accessed December 2012.
[Online]. Available: http://john.raffensperger.org/john/ArtOfTheSpreadsheet/
9. J. K. Doyle and D. N. Ford, “Mental Models Concepts for System Dynamics
Research,” System Dynamics Review, vol. 14, no. 1, pp. 3–29, 1998.
Kankuzi, Sajaniemi VL/HCC 2014 1.8.2014 20