1
Spreadsheets are models too
Richard Paige, Dimitris Kolovos, Nicholas Matragkas
Department of Computer Science
University of York
@richpaige, @kolovos, @nmatragkas
2MDE
•  Aims to improve the quality and efficiency
of the software development process
– Promotes models to first-class citizens
– Reduces the need for human compilers
– Models are purposeful descriptions that are
processable by automated tools.
•  Not restricted to a particular modelling
technology / model representation format
Skip Ad>
3Models?
EMF
XML XSD
Spreadsheets
Visio RDBMS
4The MDE Community
EMF
XML XSD
Spreadsheets
Visio
RDBMS
5Everyone else
EMF
XML XSD
Spreadsheets
Visio RDBMS
6EMF Consolidation?
•  Benefit from modern modelling languages,
DSLs, language workbenches, tools.
•  Benefit from use of generic tools for model
management (e.g., for transformation,
validation, migration).
•  Benefit from consolidated technology
platforms.
– Possibly better tool interoperability.
7Spreadsheets for SE
•  Versatile and intuitive
•  Fill gaps in software development process
–  When no specialised tool exist for the job
–  When specialised tools are too expensive/
complicated
–  When information needs to be collected from non-
programmers
•  Some problems are inherently tabular (e.g.,
certification/assurance arguments, control
law modelling, data modelling, traceability).
Skip Ad>
8Spreadsheet = Model
•  Spreadsheets are models with interesting
(valuable) properties.
– Structure is largely read-only
– Fixed implicit metamodel vs variable explicit
metamodel (with EMF/Ecore)
•  Model editing vs metamodel then model editing.
– Specialised rather than generic tools.
•  Excel/Google Spreadsheets interface vs. Eclipse
interface…
9
Treat Spreadsheets Like
(EMF) Models?
•  What are the benefits?
– Use generic tools (transformation, validation,
merging, comparison) on spreadsheets.
– Support richer heterogeneity in the MDE
space (not just EMF!).
– Better support for early stages of SE: bridge
between early requirements and structured
modelling.
– Support for large legacy models, e.g., 200
sheet x 500 row sheets, never re-engineered.
10
Treat Spreadsheets Like
(EMF) Models (2)
•  What are the benefits?
– No need to build DS(M)Ls for tabular
problems.
– Synergy with existing skills.
– Exploit MDE tools for pattern detection, repair/
update, ….
•  Sometimes a spreadsheet is all that’s
needed; EMF and DSLs are overkill.
11How?
•  Injectors:
– Generate models (in EMF) from
spreadsheets.
– May need to generate spreadsheets
from models, i.e., some form of
(partially) bidirectional transformations.
– Specification blow-up?
12How?
•  Model management operations for
spreadsheets.
– i.e., implement typical MDE operations on
spreadsheets, using spreadsheet APIs,
Google Spreadsheet Service etc.
– Transformation, code generation, validation,
etc.
•  Badly reinvented wheels?
– Weigh against maintaining traditional
spreadsheet interface/interaction model.
13How?
•  Provide a spreadsheet driver for MDE
languages/tools for
–  Model querying
–  Model-to-model transformation
–  Model validation
–  Model-to-text transformation
–  Model comparison
–  Model merging
•  ... so that spreadsheets can be used in any
step of an MDE process
14Basic Idea?
•  Spreadsheet -> Model
•  Worksheet -> Type
•  Column -> Property
•  Row -> Model element
modules
MSD,HCI
F
MSD,RQE
E
mt506
supervisor
mt506
4
jd5023 SmithJane 22
jd501 232 ThompsonJoe
D
age
C
lastname
B
firstname
A
id1
Student Staff
ED
teaches
C
lastname
B
firstname
A
id1
Module Mark
15eclipse.org/Epsilon
16Example (Query)
modules
MSD,HCI
F
MSD,RQE
E
mt506
supervisor
mt506
4
jd5023 SmithJane 22
jd501 232 ThompsonJoe
D
age
C
lastname
B
firstname
A
id1
Student Staff
E
dj5123 JacksonDaniel HCI
mt506 MSD,RQE2 ThomasMatthew
D
teaches
C
lastname
B
firstname
A
id1
Student Staff
Module
Module
Student Staff Module
Human Computer Interaction Spring4 HCI
D
RQE3 SpringRequirements Engineering
MSD2 AutumnModelling and System Design
C
term
B
title
A
id1
Mark
Mark
Mark
Student.allInstances->
select(s | s.age >= 18).println();
17Example (M2T)
modules
MSD,HCI
F
MSD,RQE
E
mt506
supervisor
mt506
4
jd5023 SmithJane 22
jd501 232 ThompsonJoe
D
age
C
lastname
B
firstname
A
id1
Student Staff
E
dj5123 JacksonDaniel HCI
mt506 MSD,RQE2 ThomasMatthew
D
teaches
C
lastname
B
firstname
A
id1
Student Staff
Module
Module
Student Staff Module
Human Computer Interaction Spring4 HCI
D
RQE3 SpringRequirements Engineering
MSD2 AutumnModelling and System Design
C
term
B
title
A
id1
Mark
Mark
Mark
<ul>
[%for (s in Student.allInstances){%]
<li> [%=s.firstname] [%=s.lastname]
[%}%]
</ul>
18Example (Validation)
context Mark {
constraint WithinRange {
check: self.mark <= 100 and
self.mark >= 0
message: “Mark ” + self.mark +
“ must be between 0-100”
}
}
Student Staff Module
Human Computer Interaction Spring4 HCI
D
RQE3 SpringRequirements Engineering
MSD2 AutumnModelling and System Design
C
term
B
title
A
id1
Mark
E
jd5023 74ICAR
jd5012 62TPOP
DC
mark
B
module
A
student1
Student Staff Module Mark
19How?
•  Develop a native spreadsheets driver for
Epsilon’s pluggable type system (EMC)
•  Pros
– No intermediate artefacts
– No stale data
– Direct updates to the spreadsheet
– Can leverage native querying capabilities
20Query Translation
•  The implementation of primitive query
operations can be overloaded.
•  The Google Spreadsheets driver rewrites
(EOL) queries at runtime, e.g., to:
– https://spreadsheets.google.com/feeds/list/tb-
<mark-worksheet-guid>/od6/private/full?
sq=mark>70
•  Search is performed on the server
– Composite queries are supported
21Conclusions
•  How do we interface MDE approaches with
spreadsheets?
–  Cannot/will not be re-engineered!
–  Need to use “real” models in combination with
spreadsheets (and other “inferior” tech).
•  Currently working on extending Google
Spreadsheets work with annotations.
–  Health care data project (York+TU/e+Lille).
•  Driver available at:
https://code.google.com/p/epsilonlabs/

Spreadsheets are models too - Richard Paige at Sems 2014

  • 1.
    1 Spreadsheets are modelstoo Richard Paige, Dimitris Kolovos, Nicholas Matragkas Department of Computer Science University of York @richpaige, @kolovos, @nmatragkas
  • 2.
    2MDE •  Aims toimprove the quality and efficiency of the software development process – Promotes models to first-class citizens – Reduces the need for human compilers – Models are purposeful descriptions that are processable by automated tools. •  Not restricted to a particular modelling technology / model representation format Skip Ad>
  • 3.
  • 4.
    4The MDE Community EMF XMLXSD Spreadsheets Visio RDBMS
  • 5.
  • 6.
    6EMF Consolidation? •  Benefitfrom modern modelling languages, DSLs, language workbenches, tools. •  Benefit from use of generic tools for model management (e.g., for transformation, validation, migration). •  Benefit from consolidated technology platforms. – Possibly better tool interoperability.
  • 7.
    7Spreadsheets for SE • Versatile and intuitive •  Fill gaps in software development process –  When no specialised tool exist for the job –  When specialised tools are too expensive/ complicated –  When information needs to be collected from non- programmers •  Some problems are inherently tabular (e.g., certification/assurance arguments, control law modelling, data modelling, traceability). Skip Ad>
  • 8.
    8Spreadsheet = Model • Spreadsheets are models with interesting (valuable) properties. – Structure is largely read-only – Fixed implicit metamodel vs variable explicit metamodel (with EMF/Ecore) •  Model editing vs metamodel then model editing. – Specialised rather than generic tools. •  Excel/Google Spreadsheets interface vs. Eclipse interface…
  • 9.
    9 Treat Spreadsheets Like (EMF)Models? •  What are the benefits? – Use generic tools (transformation, validation, merging, comparison) on spreadsheets. – Support richer heterogeneity in the MDE space (not just EMF!). – Better support for early stages of SE: bridge between early requirements and structured modelling. – Support for large legacy models, e.g., 200 sheet x 500 row sheets, never re-engineered.
  • 10.
    10 Treat Spreadsheets Like (EMF)Models (2) •  What are the benefits? – No need to build DS(M)Ls for tabular problems. – Synergy with existing skills. – Exploit MDE tools for pattern detection, repair/ update, …. •  Sometimes a spreadsheet is all that’s needed; EMF and DSLs are overkill.
  • 11.
    11How? •  Injectors: – Generate models(in EMF) from spreadsheets. – May need to generate spreadsheets from models, i.e., some form of (partially) bidirectional transformations. – Specification blow-up?
  • 12.
    12How? •  Model managementoperations for spreadsheets. – i.e., implement typical MDE operations on spreadsheets, using spreadsheet APIs, Google Spreadsheet Service etc. – Transformation, code generation, validation, etc. •  Badly reinvented wheels? – Weigh against maintaining traditional spreadsheet interface/interaction model.
  • 13.
    13How? •  Provide aspreadsheet driver for MDE languages/tools for –  Model querying –  Model-to-model transformation –  Model validation –  Model-to-text transformation –  Model comparison –  Model merging •  ... so that spreadsheets can be used in any step of an MDE process
  • 14.
    14Basic Idea? •  Spreadsheet-> Model •  Worksheet -> Type •  Column -> Property •  Row -> Model element modules MSD,HCI F MSD,RQE E mt506 supervisor mt506 4 jd5023 SmithJane 22 jd501 232 ThompsonJoe D age C lastname B firstname A id1 Student Staff ED teaches C lastname B firstname A id1 Module Mark
  • 15.
  • 16.
    16Example (Query) modules MSD,HCI F MSD,RQE E mt506 supervisor mt506 4 jd5023 SmithJane22 jd501 232 ThompsonJoe D age C lastname B firstname A id1 Student Staff E dj5123 JacksonDaniel HCI mt506 MSD,RQE2 ThomasMatthew D teaches C lastname B firstname A id1 Student Staff Module Module Student Staff Module Human Computer Interaction Spring4 HCI D RQE3 SpringRequirements Engineering MSD2 AutumnModelling and System Design C term B title A id1 Mark Mark Mark Student.allInstances-> select(s | s.age >= 18).println();
  • 17.
    17Example (M2T) modules MSD,HCI F MSD,RQE E mt506 supervisor mt506 4 jd5023 SmithJane22 jd501 232 ThompsonJoe D age C lastname B firstname A id1 Student Staff E dj5123 JacksonDaniel HCI mt506 MSD,RQE2 ThomasMatthew D teaches C lastname B firstname A id1 Student Staff Module Module Student Staff Module Human Computer Interaction Spring4 HCI D RQE3 SpringRequirements Engineering MSD2 AutumnModelling and System Design C term B title A id1 Mark Mark Mark <ul> [%for (s in Student.allInstances){%] <li> [%=s.firstname] [%=s.lastname] [%}%] </ul>
  • 18.
    18Example (Validation) context Mark{ constraint WithinRange { check: self.mark <= 100 and self.mark >= 0 message: “Mark ” + self.mark + “ must be between 0-100” } } Student Staff Module Human Computer Interaction Spring4 HCI D RQE3 SpringRequirements Engineering MSD2 AutumnModelling and System Design C term B title A id1 Mark E jd5023 74ICAR jd5012 62TPOP DC mark B module A student1 Student Staff Module Mark
  • 19.
    19How? •  Develop anative spreadsheets driver for Epsilon’s pluggable type system (EMC) •  Pros – No intermediate artefacts – No stale data – Direct updates to the spreadsheet – Can leverage native querying capabilities
  • 20.
    20Query Translation •  Theimplementation of primitive query operations can be overloaded. •  The Google Spreadsheets driver rewrites (EOL) queries at runtime, e.g., to: – https://spreadsheets.google.com/feeds/list/tb- <mark-worksheet-guid>/od6/private/full? sq=mark>70 •  Search is performed on the server – Composite queries are supported
  • 21.
    21Conclusions •  How dowe interface MDE approaches with spreadsheets? –  Cannot/will not be re-engineered! –  Need to use “real” models in combination with spreadsheets (and other “inferior” tech). •  Currently working on extending Google Spreadsheets work with annotations. –  Health care data project (York+TU/e+Lille). •  Driver available at: https://code.google.com/p/epsilonlabs/