Your SlideShare is downloading. ×
0
An Empirical Study on End-users Productivity Using             Model-based SpreadsheetsLaura Beckwith           J´come Cun...
Agenda1   Introduction2   Model-based Spreadsheets     Inferring Models     Refactored Spreadsheet     Visual Spreadsheet3...
Introduction    Much research has been done to prevent spreadsheet end-user errors    Model-Driven Engineering (MDE) is a ...
Introduction    Much research has been done to prevent spreadsheet end-user errors    Model-Driven Engineering (MDE) is a ...
Original Spreadsheet - originalA project management systemTabular spreadsheet (database like spreadsheet)   J´come Cunha (...
Functional Dependencies     A     B means that the values in column A uniquely determine values     in column B     If two...
Relational Model    From the spreadsheet data (also from formulas) we can infer    functional dependencies    From these d...
Refactored Spreadsheet - refactored    We use the relational model to generate a new refactored spreadsheet    (respecting...
Visual Spreadsheet - visual    From functional dependencies we can generate edit assistance    A    B: A columns become gr...
The Entire Picture                        Reasoning                                               Visual SS               ...
Research QuestionsRQ1 Do end users introduce fewer errors when they use one of themodel-based spreadsheets versus the orig...
Study Design    Background questionnaire, followed by a list of tasks and a post    questionnaire    No initial tutorial w...
Global Effectiveness                                       original   refactored   visual                          dishes  ...
Effectiveness: Inserting Data                                       original   refactored   visual                         ...
Effectiveness: Editing Data                                       original   refactored   visual                          d...
Effectiveness: Computing Statistics                                       original   refactored   visual                   ...
Efficiency                                       original   refactored   visual                          dishes         35  ...
Answers to Research QuestionsRQ1 Do end users introduce fewer errors when they use one of themodel-based spreadsheets vers...
Conclusions    We did not give a tutorial to participants    We believe better results could be achieve with initial train...
Thank you for your attention!J´come Cunha (UMinho) a                         EuSpRIG 2011   19 / 19
Upcoming SlideShare
Loading in...5
×

Talk at EUSPRIG '11

2,945

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,945
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Talk at EUSPRIG '11"

  1. 1. An Empirical Study on End-users Productivity Using Model-based SpreadsheetsLaura Beckwith J´come Cunha a Jo˜o P. Fernandes a Jo˜o Saraiva a HCIResearcher, Denmark, beckwith@hciResearcher.com Universidade do Minho, Portugal, {jacome,jpaulo,jas}@di.uminho.pt Universidade do Porto, Portugal, jpaulo@fe.up.pt EuSpRIG 2011 July 14 – 15 J´come Cunha (UMinho) a EuSpRIG 2011 1 / 19
  2. 2. Agenda1 Introduction2 Model-based Spreadsheets Inferring Models Refactored Spreadsheet Visual Spreadsheet3 Study Design4 Results Effectiveness Efficiency5 Conclusions J´come Cunha (UMinho) a EuSpRIG 2011 2 / 19
  3. 3. Introduction Much research has been done to prevent spreadsheet end-user errors Model-Driven Engineering (MDE) is a promising approach We have created a set of different model-based spreadsheets J´come Cunha (UMinho) a EuSpRIG 2011 3 / 19
  4. 4. Introduction Much research has been done to prevent spreadsheet end-user errors Model-Driven Engineering (MDE) is a promising approach We have created a set of different model-based spreadsheets Need to evaluate their impact J´come Cunha (UMinho) a EuSpRIG 2011 3 / 19
  5. 5. Original Spreadsheet - originalA project management systemTabular spreadsheet (database like spreadsheet) J´come Cunha (UMinho) a EuSpRIG 2011 4 / 19
  6. 6. Functional Dependencies A B means that the values in column A uniquely determine values in column B If two values two rows in column A are equal, then the corresponding values in B are also equalFor example, suppose A = {A1 , A2 } and B = {B1 , B2 }: A1 A2 B1 B2 A1 A2 B1 B2 a1 a2 b1 b2 a1 a2 b1 b2 a1 a3 b1 b1 a1 a3 b1 b1 a1 a2 b1 b2 a1 a2 b2 b2 (a) Table inducing A B. (b) Table not inducing A B. J´come Cunha (UMinho) a EuSpRIG 2011 5 / 19
  7. 7. Relational Model From the spreadsheet data (also from formulas) we can infer functional dependencies From these dependencies, we can compute a relational database schema Reasoning about FDs Original SS A⇀B Infer Infer RDB RDB CD⇀E FDs Schema Schema ... J´come Cunha (UMinho) a EuSpRIG 2011 6 / 19
  8. 8. Refactored Spreadsheet - refactored We use the relational model to generate a new refactored spreadsheet (respecting the relational model) Each entity separated in a different table Combo boxes used to list allowed values (from other tables) J´come Cunha (UMinho) a EuSpRIG 2011 7 / 19
  9. 9. Visual Spreadsheet - visual From functional dependencies we can generate edit assistance A B: A columns become green combo boxes and B red Similar to original Combo boxes help user introducing data J´come Cunha (UMinho) a EuSpRIG 2011 8 / 19
  10. 10. The Entire Picture Reasoning Visual SS about FDsOriginal SS Generate Visual SS A⇀B Infer CD⇀E FDs ... Infer RDB Schema RDB Schema Refactored SS Generate Refactored SSGoal: compare efficiency and effectiveness of original, refactored and visual J´come Cunha (UMinho) a EuSpRIG 2011 9 / 19
  11. 11. Research QuestionsRQ1 Do end users introduce fewer errors when they use one of themodel-based spreadsheets versus the original unmodified one?RQ2 Are end users more efficient using the model-based spreadsheets?RQ3 Do particular models lead to fewer errors in particular tasks? J´come Cunha (UMinho) a EuSpRIG 2011 10 / 19
  12. 12. Study Design Background questionnaire, followed by a list of tasks and a post questionnaire No initial tutorial was given to the participants 38 participants: 25 females and 11 males, all students Tasks for inserting and editing data and computing statistics 3 different spreadsheet applications (projects, dishes and properties) 3 different models (original, refactored, visual) Each participant received 3 spreadsheets, each of a different application/model J´come Cunha (UMinho) a EuSpRIG 2011 11 / 19
  13. 13. Global Effectiveness original refactored visual dishes 86% 76% 78% projects 73% 68% 78% properties 75% 64% 62% Globally, models do not seem very competitive Further analyses are necessary J´come Cunha (UMinho) a EuSpRIG 2011 12 / 19
  14. 14. Effectiveness: Inserting Data original refactored visual dishes 91% 90% 81% projects 76% 60% 75% properties 86% 67% 68% Models always lose But very similar to original J´come Cunha (UMinho) a EuSpRIG 2011 13 / 19
  15. 15. Effectiveness: Editing Data original refactored visual dishes 91% 82% 82% projects 54% 62% 50% properties 65% 98% 48% refactored in general was better Because there is less data to edit J´come Cunha (UMinho) a EuSpRIG 2011 14 / 19
  16. 16. Effectiveness: Computing Statistics original refactored visual dishes 52% 37% 57% projects 19% 76% 13% properties 44% 57% 51% Again, refactored was better And again, less data do analyze J´come Cunha (UMinho) a EuSpRIG 2011 15 / 19
  17. 17. Efficiency original refactored visual dishes 35 32 28 projects 39 40 41 properties 37 36 40 Models are competitive In further analyzes we discovered that users took more time understanding model, specially refactored J´come Cunha (UMinho) a EuSpRIG 2011 16 / 19
  18. 18. Answers to Research QuestionsRQ1 Do end users introduce fewer errors when they use one of themodel-based spreadsheets versus the original unmodified one? We believe there is potential for model-based spreadsheets But a deep knowledge seems to be necessary to be effectiveRQ2 Are end users more efficient using model-based spreadsheets? Model-based spreadsheet can help in some cases We believe better knowledge about models could improve this resultRQ3 Do particular models lead to fewer errors in particular tasks? For inserting data, this was not the case The models helped editing data and in computing statistics J´come Cunha (UMinho) a EuSpRIG 2011 17 / 19
  19. 19. Conclusions We did not give a tutorial to participants We believe better results could be achieve with initial training Nevertheless, our model-based spreadsheet are competitive Better for editing and statistics We plan to run a new study similar to this, but with an initial tutorial J´come Cunha (UMinho) a EuSpRIG 2011 18 / 19
  20. 20. Thank you for your attention!J´come Cunha (UMinho) a EuSpRIG 2011 19 / 19
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×