The document proposes a benchmark for comparing UML and OCL analysis tools. It includes 4 example models addressing concepts like class invariants, disjoint subclasses, and modeling integers as objects. The benchmark evaluates tools on 11 functionalities using the example models, such as checking invariant consistency, deriving consequences from constraints, and instantiating classes and associations while satisfying constraints. The benchmark aims to compare tool functionality using agreed-upon models contributed by various tool builders. It is intended as input for tool builders to emphasize the strengths of their tools. Further areas of interest for additional models are identified, and contributions from other tool builders are sought to expand the benchmark.

1. Initiating a Benchmark for
UML and OCL Analysis Tools
Martin Gogolla, Fabian Büttner, Jordi Cabot
University of Bremen, Germany
AtlanMod, Ecole des Mines de Nantes - INRIA, LINA, France

2. Context: Model-Based (Software) Engineering
here focus on UML and OCL
Model: UML class diagram + OCL constraints
Various tools for analyzing such models exist
- Test-oriented tools (USE, EMF2toCSP, mOdCL, ...)
- Proof-oriented tools (HOL-OCL, KeY, ...)
Tools offer various functionalities
- building valid/invalid object diagrams, manually or
automatically; valid: explicit and inherent constraints
- debugging invalid object diagrams by revealing
failing constraints, constraint parts or model elements
- prooving consequences from stated constraints
- revealing classes or associations which cannot be populated
- ...
Aim of planned benchmark (here 1st version)
- Compare tool functionality with an agreed set of
example models
- Include models and proposals from other tool builders

3. Benchmark (currently): 4 models, 11 functionalities/questions
- Civil Status
- Writes Reviews
- Disjoint Subclasses
- Objects As Integers
? Consistent Invariants
? Independence
? Consequences
? Large State
? Instantiate Nonempty Class
? Instantiate Nonempty Assoc
? Instantiate Invariant Ignore
? Instantiate Disjoint Inheritance
? Instantiate Multiple Inheritance
? Object Represents Integer
? Integer Represents Object

4. Model: Civil Status

5. Model: Civil Status
context Person
inv attributesDefined: name<>null and civstat<>null and
gender<>null
inv nameCapitalThenSmallLetters:
let small:Set(String)=
Set{'a','b','c','d','e','f','g','h','i','j','k','l','m',
'n','o','p','q','r','s','t','u','v','w','x','y','z'} in
let capital:Set(String)=
Set{'A','B','C','D','E','F','G','H','I','J','K','L','M',
'N','O','P','Q','R','S','T','U','V','W','X','Y','Z'} in
capital->includes(name.substring(1,1)) and
Set{2..name.size}->forAll(i |
small->includes(name.substring(i,i)))
inv nameIsUnique: Person.allInstances->forAll(self2|
self<>self2 implies self.name<>self2.name)
inv femaleHasNoWife: gender=#female implies wife->isEmpty
inv maleHasNoHusband: gender=#male implies husband->isEmpty
inv hasSpouse_EQ_civstatMarried: (spouse()<>null)=(civstat=#married)

6. Benchmark (currently): 4 models, 11 functionalities/questions
- Civil Status
- Writes Reviews
- Disjoint Subclasses
- Objects As Integers
? Consistent Invariants
? Independence
? Consequences
? Large State
? Instantiate Nonempty Class
? Instantiate Nonempty Assoc
? Instantiate Invariant Ignore
? Instantiate Disjoint Inheritance
? Instantiate Multiple Inheritance
? Object Represents Integer
? Integer Represents Object

7. Model: Writes Reviews

8. Model: Writes Reviews
context Researcher inv oneManuscript:
self.manuscript->size=1
context Researcher inv oneSubmission:
self.submission->size=1
context Researcher inv noSelfReviews:
self.submission->excludes(self.manuscript)
context Paper inv paperLength:
self.wordCount < 10000
context Paper inv authorsOfStudentPaper:
self.studentPaper=self.author->exists(x | x.isStudent)
context Paper inv noStudentReviewers:
self.referee->forAll(r | not r.isStudent)
context Paper inv limitsOnStudentPapers:
Paper.allInstances->exists(p | p.studentPaper) and
Paper.allInstances->select(p | p.studentPaper)->size < 5

9. Benchmark (currently): 4 models, 11 functionalities/questions
- Civil Status
- Writes Reviews
- Disjoint Subclasses
- Objects As Integers
? Consistent Invariants
? Independence
? Consequences
? Large State
? Instantiate Nonempty Class
? Instantiate Nonempty Assoc
? Instantiate Invariant Ignore
? Instantiate Disjoint Inheritance
? Instantiate Multiple Inheritance
? Object Represents Integer
? Integer Represents Object

10. Model: Disjoint Subclasses
context b:B inv disjointBC: C.allInstances->forAll(c|b<>c)

11. Benchmark (currently): 4 models, 11 functionalities/questions
- Civil Status
- Writes Reviews
- Disjoint Subclasses
- Objects As Integers
? Consistent Invariants
? Independence
? Consequences
? Large State
? Instantiate Nonempty Class
? Instantiate Nonempty Assoc
? Instantiate Invariant Ignore
? Instantiate Disjoint Inheritance
? Instantiate Multiple Inheritance
? Object Represents Integer
? Integer Represents Object

12. Model: Objects As Integers

13. Model: Objects As Integers
VALID
INVALID

14. context Int
inv acyclicPredSucc:
predPlus()->union(succPlus())->excludes(self)
context Zero
inv zeroNotLinkedToZero:
not predPlus()->union(succPlus())->exists(i|
i.oclIsTypeOf(Zero))
inv zeroNotLinkedToNegAndPos:
not predPlus()->union(succPlus())->exists(n,p|
n.oclIsTypeOf(Neg) and p.oclIsTypeOf(Pos))
context Neg
inv negNotLinkedToPos:
not predPlus()->union(succPlus())->exists(p|
p.oclIsTypeOf(Pos))
inv negLinkedToZeroBySucc:
succPlus()->exists(z|z.oclIsTypeOf(Zero))
context Pos
inv posNotLinkedToNeg:
not predPlus()->union(succPlus())->exists(n|
n.oclIsTypeOf(Neg))
inv posLinkedToZeroByPred:
predPlus()->exists(z|z.oclIsTypeOf(Zero))
Model: Objects As Integers

15. Benchmark (currently): 4 models, 11 functionalities/questions
- Civil Status
- Writes Reviews
- Disjoint Subclasses
- Objects As Integers
? Consistent Invariants
? Independence
? Consequences
? Large State
? Instantiate Nonempty Class
? Instantiate Nonempty Assoc
? Instantiate Invariant Ignore
? Instantiate Disjoint Inheritance
? Instantiate Multiple Inheritance
? Object Represents Integer
? Integer Represents Object

16. Possible topics to be emphasized in further models
- local constraints (single class constraints)
- global constraints (many classes involved)
- easy tractable constraints
- hard constraints (NP hard problems)
- symmetry problems
- integer arithmetic
- real arithmetic
- string calculations
- redundant constraints
- sparse models (few links)
- recursive operations
- null value and collection operations
- large models, large instances

17. Planned benchmark
- Currently 1st version
- Compare tool functionality with an agreed set of
example models
- Include models and proposals from other tool builders
emphasizing the strength of their tool
- Used as input for the remaining tool builders

18. ... waiting for contributions ...
Thanks for your attention!