Assure Ecommerce and Retail Operations Uptime with ThousandEyes
Verifying ATL transformations using SMT solvers
1. On verifying ATL transformations using
‘off-the-shelf’ SMT solvers
Fabian Buttner1 , Marina Egea2 , Jordi Cabot1
¨
1 AtlanMod, ´
INRIA / Ecole de Mines de Nantes
2 ATOS Research, Madrid
MODELS 2012
2. Our Approach
Motivation
Model transformations should be correct
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
3. Our Approach
Motivation
Model transformations should be correct
This should be verified automatically
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
4. Our Approach
Motivation
Model transformations should be correct
This should be verified automatically
Our approach
Addresses partial correctness of ATL
transformations w.r.t. OCL
pre-/postconditions
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
5. Our Approach
Motivation
Model transformations should be correct
This should be verified automatically
Our approach
Addresses partial correctness of ATL
transformations w.r.t. OCL
pre-/postconditions
Translates this into a first-order logic
problem
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
6. Our Approach
Motivation
Model transformations should be correct
This should be verified automatically
Our approach
Addresses partial correctness of ATL
transformations w.r.t. OCL
pre-/postconditions
Translates this into a first-order logic
problem
Employs SMT solvers to check it
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
7. Our Approach
Motivation
Model transformations should be correct
This should be verified automatically
Our approach
Addresses partial correctness of ATL
transformations w.r.t. OCL
pre-/postconditions
Translates this into a first-order logic
problem
Employs SMT solvers to check it
Puts no upper bounds on the model
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
8. Our Approach
Motivation
Model transformations should be correct
This should be verified automatically
Our approach
Addresses partial correctness of ATL
transformations w.r.t. OCL
pre-/postconditions
Translates this into a first-order logic
problem
Employs SMT solvers to check it
Puts no upper bounds on the model
Works well for our case studies
c AtlanMod – atlanmod-contact@mines-nantes.fr 2/19
9. Outline
1 Transformation correctness
c AtlanMod – atlanmod-contact@mines-nantes.fr 3/19
10. Outline
1 Transformation correctness
2 Deriving a FOL specification
c AtlanMod – atlanmod-contact@mines-nantes.fr 3/19
11. Outline
1 Transformation correctness
2 Deriving a FOL specification
3 Checking it using SMT solvers
c AtlanMod – atlanmod-contact@mines-nantes.fr 3/19
12. Outline
1 Transformation correctness
2 Deriving a FOL specification
3 Checking it using SMT solvers
4 Conclusions
c AtlanMod – atlanmod-contact@mines-nantes.fr 3/19
13. Outline
1 Transformation correctness
2 Deriving a FOL specification
3 Checking it using SMT solvers
4 Conclusions
c AtlanMod – atlanmod-contact@mines-nantes.fr 4/19
14. Two Metamodels
ER metamodel -- CONSTRAINTS (PRECONDITIONS)
-- unique schema names
context ERSchema inv:
ERSchema.allInstances()->forall(s1,s2|
1 1
ERSchema s1<>s2 implies s1.name<>s2.name)
* entities * relships -- entity names are unique in schema
Entity type RelshipEnd ends Relship
-- relship names are unique in schema
name : String 1 * name : String 2..* 1 name : String
0..1 {xor} 0..1
-- disjoint entity and relship names
ERAttribute -- attr names are unique in entity
attrs attrs
name : String -- attr names are unique in relship
* isKey : Boolean *
-- entities have a key
REL metamodel -- CONSTRAINTS (POSTCONDITIONS)
1
-- relations have a key
RELSchema context Relation inv:
self.attrs->exists(a | a.isKey)
Relation 1..*
1 -- schema names are unique
name : String relations
-- relation names are unique in schema
RELAttribute -- attribute names unique in relation
1..* name : String
attrs isKey : Boolean
c AtlanMod – atlanmod-contact@mines-nantes.fr 5/19
15. An ATL Transformation
ER
REL
ER2REL
module ER2REL; create OUT : REL from IN : ER;
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name<-s.name) }
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name,
schema<-s.schema) }
-- not shown: R2R, EA2A, RA2A, RA2AK ...
c AtlanMod – atlanmod-contact@mines-nantes.fr 6/19
16. An ATL Transformation
ER
REL
ER2REL
module ER2REL; create OUT : REL from IN : ER;
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name<-s.name) }
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name,
schema<-s.schema) }
-- not shown: R2R, EA2A, RA2A, RA2AK ...
Execution semantics of ATL:
(i) Match source patterns
c AtlanMod – atlanmod-contact@mines-nantes.fr 6/19
17. An ATL Transformation
ER
REL
ER2REL
module ER2REL; create OUT : REL from IN : ER;
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name<-s.name) }
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name,
schema<-s.schema) }
-- not shown: R2R, EA2A, RA2A, RA2AK ...
Execution semantics of ATL:
(i) Match source patterns; (ii) create target objects
c AtlanMod – atlanmod-contact@mines-nantes.fr 6/19
18. An ATL Transformation
ER
REL
ER2REL
module ER2REL; create OUT : REL from IN : ER;
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema ( name <- s.name ) }
rule E2R { from s : ER!Entity
to t : REL!Relation ( name<-s.name ,
schema<-s.schema ) }
-- not shown: R2R, EA2A, RA2A, RA2AK ...
Execution semantics of ATL:
(i) Match source patterns; (ii) create target objects; (iii) bind properties
c AtlanMod – atlanmod-contact@mines-nantes.fr 6/19
19. Checking Correctness
We are interested in Hoare-style partial correctness of
T : MI → MF with respect to pre- and postconditions.
c AtlanMod – atlanmod-contact@mines-nantes.fr 7/19
20. Checking Correctness
We are interested in Hoare-style partial correctness of
T : MI → MF with respect to pre- and postconditions.
Our approach:
Systematically derive a FOL specification (a set of
assertions) from T : MI → MF and its pre- and
postconditions that is logically valid iff the transformation is
correct w.r.t. these conditions
c AtlanMod – atlanmod-contact@mines-nantes.fr 7/19
21. Checking Correctness
We are interested in Hoare-style partial correctness of
T : MI → MF with respect to pre- and postconditions.
Our approach:
Systematically derive a FOL specification (a set of
assertions) from T : MI → MF and its pre- and
postconditions that is logically valid iff the transformation is
correct w.r.t. these conditions
Check validity using an SMT solver (Z3, Yices)
c AtlanMod – atlanmod-contact@mines-nantes.fr 7/19
22. Checking Correctness
We are interested in Hoare-style partial correctness of
T : MI → MF with respect to pre- and postconditions.
Our approach:
Systematically derive a FOL specification (a set of
assertions) from T : MI → MF and its pre- and
postconditions that is logically valid iff the transformation is
correct w.r.t. these conditions
Check validity using an SMT solver (Z3, Yices)
Remarks:
We consider core subsets of OCL and declarative ATL
c AtlanMod – atlanmod-contact@mines-nantes.fr 7/19
23. Checking Correctness
We are interested in Hoare-style partial correctness of
T : MI → MF with respect to pre- and postconditions.
Our approach:
Systematically derive a FOL specification (a set of
assertions) from T : MI → MF and its pre- and
postconditions that is logically valid iff the transformation is
correct w.r.t. these conditions
Check validity using an SMT solver (Z3, Yices)
Remarks:
We consider core subsets of OCL and declarative ATL
The proofs we get are valid without model bounds
c AtlanMod – atlanmod-contact@mines-nantes.fr 7/19
24. Checking Correctness
We are interested in Hoare-style partial correctness of
T : MI → MF with respect to pre- and postconditions.
Our approach:
Systematically derive a FOL specification (a set of
assertions) from T : MI → MF and its pre- and
postconditions that is logically valid iff the transformation is
correct w.r.t. these conditions
Check validity using an SMT solver (Z3, Yices)
Remarks:
We consider core subsets of OCL and declarative ATL
The proofs we get are valid without model bounds
The problem is undecidable in general
c AtlanMod – atlanmod-contact@mines-nantes.fr 7/19
25. Outline
1 Transformation correctness
2 Deriving a FOL specification
3 Checking it using SMT solvers
4 Conclusions
c AtlanMod – atlanmod-contact@mines-nantes.fr 8/19
26. FOL Semantics of MMs
Encode metamodels and constraints as a specification in
first-order logic (based on [Clavel et al., 2009])
Classes, associations, attributes generate
predicates and functions over object identifiers
Constraints generate assertions
Interpretations of the FOL specification correspond to valid
instances of the metamodel
c AtlanMod – atlanmod-contact@mines-nantes.fr 9/19
27. Translation of ER
Predicates: Relship( ),
RelshipEnd ends Relship RelshipEnd( ),
name : String 2..*
ends( , )
Functions: name( )
Assertion:
∀x . Relship(x) → ∃y, z . RelshipEnd(y ) ∧ ends(x, y )∧
RelshipEnd(z) ∧ ends(x, z) ∧ y = z
context ERSchema inv:
ERSchema.allInstances()->forall(s1,s2|
s1<>s2 implies s1.name<>s2.name)
Assertion:
∀x, y . ERSchema(x) ∧ ERSchema(y) ∧ x = y →
name(x) = name(y)
c AtlanMod – atlanmod-contact@mines-nantes.fr 10/19
28. FOL Semantics of ATL
Encode transformation rules in first-order logic
Matched rules become functions over object identifiers
The semantics of the rules is translated into assertions
Interpretations of the FOL specification correspond to valid
traces of the transformation
c AtlanMod – atlanmod-contact@mines-nantes.fr 11/19
29. Translation of ER2REL
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name <- s.name)}
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name, schema<-s.schema)}
rule R2R { from s : ER!Relship
to t : REL!Relation (name<-s.name, schema<-s.schema)}
c AtlanMod – atlanmod-contact@mines-nantes.fr 12/19
30. Translation of ER2REL
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name <- s.name)}
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name, schema<-s.schema)}
rule R2R { from s : ER!Relship
to t : REL!Relation (name<-s.name, schema<-s.schema)}
Matching semantics
∀e . Entity(e) → ∃t . Relation(t) ∧ E2R(e) = t
∀e . Relship(rh) → ∃t . Relation(t) ∧ R2R(rh) = t
c AtlanMod – atlanmod-contact@mines-nantes.fr 12/19
31. Translation of ER2REL
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name <- s.name)}
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name, schema<-s.schema)}
rule R2R { from s : ER!Relship
to t : REL!Relation (name<-s.name, schema<-s.schema)}
Creation semantics
∀t . Relation(t) → (∃e . Entity(e) ∧ E2R(e) = t) ∨
(∃rh . Relship(rh) ∧ R2R(rh) = t)
c AtlanMod – atlanmod-contact@mines-nantes.fr 12/19
32. Translation of ER2REL
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name <- s.name)}
rule E2R { from s : ER!Entity
to t : REL!Relation ( name<-s.name , schema<-s.schema)}
rule R2R { from s : ER!Relship
to t : REL!Relation (name<-s.name, schema<-s.schema)}
Simple property bindings
∀e, t . Entity(e) ∧ Relation(t) ∧ E2R(e) = t →
name(e) = name(t)
c AtlanMod – atlanmod-contact@mines-nantes.fr 12/19
33. Translation of ER2REL
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name <- s.name)}
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name, schema<-s.schema )}
rule R2R { from s : ER!Relship
to t : REL!Relation (name<-s.name, schema<-s.schema)}
Resolved property bindings
∀e, t . (Entity(e) ∧ Relation(t) ∧ E2R(e) = t) →
(∀p . ERSchema(p) ∧ erschema(e, p) →
∃s . RELSchema(s) ∧ relschema(t, s) ∧ resolve1 (p, s)) ∧
(∀s . RELSchema(s) ∧ relschema(t, s) →
∃p . ERSchema(p) ∧ erschema(e, p) ∧ resolve1 (p, s))
resolve1 (x, y) =def .
(ERSchema(x) ∧ RELSchema(y ) ∧ S2S(x) = y) ∨
(Entity(x) ∧ Relation(y ) ∧ E2R(x) = y ) ∨
(Relship(x) ∧ Relation(y ) ∧ R2R(x) = y)
c AtlanMod – atlanmod-contact@mines-nantes.fr 12/19
34. Translation of ER2REL
rule S2S { from s : ER!ERSchema
to t : REL!RELSchema (name <- s.name)}
rule E2R { from s : ER!Entity
to t : REL!Relation (name<-s.name, schema<-s.schema)}
rule R2R { from s : ER!Relship
to t : REL!Relation (name<-s.name, schema<-s.schema)}
(and some more details...)
The paper describes the complete translation
Two complete examples are available online
http://www.emn.fr/z-info/atlanmod/index.php/MODELS_2012_SMT
c AtlanMod – atlanmod-contact@mines-nantes.fr 12/19
35. Verification
Definition
Let T = {r 1 , . . . , rn } be an ATL model transformation [. . . ].
Then, T is correct with respect to preconditions {ς1 . . . ςl } and
postconditions {τ1 , . . . , τw } if and only if, upon termination of T ,
for every τi , i = 1, . . . , w, the following formula always holds:
l n
ocl2fol(ςj ) ∧ atl2fol(rj ) ⇒ ocl2fol(τi )
j=1 j=1
c AtlanMod – atlanmod-contact@mines-nantes.fr 13/19
36. Verification
Definition
Let T = {r1 , . . . , rn } be an ATL model transformation [. . . ].
Then, T is correct with respect to preconditions {ς1 . . . ςl } and
postconditions {τ1 , . . . , τw } if and only if, upon termination of T ,
for every τi , i = 1, . . . , w, the following formula is unsatisfiable
l n
ocl2fol(ςj ) ∧ atl2fol(rj ) ∧¬(ocl2fol(τi ))
j=1 j=1
c AtlanMod – atlanmod-contact@mines-nantes.fr 13/19
37. Outline
1 Transformation correctness
2 Deriving a FOL specification
3 Checking it using SMT solvers
4 Conclusions
c AtlanMod – atlanmod-contact@mines-nantes.fr 14/19
38. SMT solvers
Automatic Boolean SAT solving + Theories
Uninterpreted functions
Arithmetic
...
Support for quantifiers (incomplete procedures)
> 15 implementations (SMT-COMP)
Standardized language and libraries (SMT-LIB)
c AtlanMod – atlanmod-contact@mines-nantes.fr 15/19
39. Employing the solver
Feeding our first-order logic specification to the SMT solver:
Both Z3 and Yices can be used in the verification
c AtlanMod – atlanmod-contact@mines-nantes.fr 16/19
40. Employing the solver
Feeding our first-order logic specification to the SMT solver:
Both Z3 and Yices can be used in the verification
The specification file mirrors our formalization one-to-one
c AtlanMod – atlanmod-contact@mines-nantes.fr 16/19
41. Employing the solver
Feeding our first-order logic specification to the SMT solver:
Both Z3 and Yices can be used in the verification
The specification file mirrors our formalization one-to-one
Solving
Z3 solves our examples fully automatically using
Model-based Quantifier Instantiation
Yices sometimes requires Lemmas
c AtlanMod – atlanmod-contact@mines-nantes.fr 16/19
42. Some Observations
Proofs found automatically for ER2REL by Z3 and Yices∗
Preconditions Postcondition Unsat core
(total = 69)
pre1 (ERSchema.name) post1 (RELSchema.name) 4
E::schema[1..1], RS::schema[1..1] R::schema[1..1] 9
pre2, pre3, pre4, post2 (Relation.name) 16
E::schema[1..1], RS::schema[1..1]
RSE:relship[1..1] RA::relation[1..1] 11
pre4, RSE::type[1..1], RS::ends[2..*] post4 (RELAttr.isKey) 14
... ... ...
c AtlanMod – atlanmod-contact@mines-nantes.fr 17/19
43. Some Observations
Proofs found automatically for ER2REL by Z3 and Yices∗
Preconditions Postcondition Unsat core
(total = 69)
pre1 (ERSchema.name) post1 (RELSchema.name) 4
E::schema[1..1], RS::schema[1..1] R::schema[1..1] 9
pre2, pre3, pre4, post2 (Relation.name) 16
E::schema[1..1], RS::schema[1..1]
RSE:relship[1..1] RA::relation[1..1] 11
pre4, RSE::type[1..1], RS::ends[2..*] post4 (RELAttr.isKey) 14
... ... ...
For all our examples:
Implied constraints can be found fast and automatically
c AtlanMod – atlanmod-contact@mines-nantes.fr 17/19
44. Some Observations
Proofs found automatically for ER2REL by Z3 and Yices∗
Preconditions Postcondition Unsat core
(total = 69)
pre1 (ERSchema.name) post1 (RELSchema.name) 4
E::schema[1..1], RS::schema[1..1] R::schema[1..1] 9
pre2, pre3, pre4, post2 (Relation.name) 16
E::schema[1..1], RS::schema[1..1]
RSE:relship[1..1] RA::relation[1..1] 11
pre4, RSE::type[1..1], RS::ends[2..*] post4 (RELAttr.isKey) 14
... ... ...
For all our examples:
Implied constraints can be found fast and automatically
Unsat cores are nice to understand the implications
c AtlanMod – atlanmod-contact@mines-nantes.fr 17/19
45. Some Observations
Proofs found automatically for ER2REL by Z3 and Yices∗
Preconditions Postcondition Unsat core
(total = 69)
pre1 (ERSchema.name) post1 (RELSchema.name) 4
E::schema[1..1], RS::schema[1..1] R::schema[1..1] 9
pre2, pre3, pre4, post2 (Relation.name) 16
E::schema[1..1], RS::schema[1..1]
RSE:relship[1..1] RA::relation[1..1] 11
pre4, RSE::type[1..1], RS::ends[2..*] post4 (RELAttr.isKey) 14
... ... ...
For all our examples:
Implied constraints can be found fast and automatically
Unsat cores are nice to understand the implications
Counter examples are much harder and often time out
c AtlanMod – atlanmod-contact@mines-nantes.fr 17/19
46. Some Observations
Proofs found automatically for ER2REL by Z3 and Yices∗
Preconditions Postcondition Unsat core
(total = 69)
pre1 (ERSchema.name) post1 (RELSchema.name) 4
E::schema[1..1], RS::schema[1..1] R::schema[1..1] 9
pre2, pre3, pre4, post2 (Relation.name) 16
E::schema[1..1], RS::schema[1..1]
RSE:relship[1..1] RA::relation[1..1] 11
pre4, RSE::type[1..1], RS::ends[2..*] post4 (RELAttr.isKey) 14
... ... ...
For all our examples:
Implied constraints can be found fast and automatically
Unsat cores are nice to understand the implications
Counter examples are much harder and often time out
Bounded-model search seems more useful here
(c.f. [Troya et al., JOT, 2011], [Buttner et al., ICFEM, 2012])
¨
c AtlanMod – atlanmod-contact@mines-nantes.fr 17/19
47. Outline
1 Transformation correctness
2 Deriving a FOL specification
3 Checking it using SMT solvers
4 Conclusions
c AtlanMod – atlanmod-contact@mines-nantes.fr 18/19
48. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
49. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
50. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
The approach complements bounded search approaches
for counter example finding
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
51. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
The approach complements bounded search approaches
for counter example finding
Future work:
Provide improved tooling
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
52. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
The approach complements bounded search approaches
for counter example finding
Future work:
Provide improved tooling
Extend the supported sets of ATL and OCL
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
53. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
The approach complements bounded search approaches
for counter example finding
Future work:
Provide improved tooling
Extend the supported sets of ATL and OCL
Verify larger case studies
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
54. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
The approach complements bounded search approaches
for counter example finding
Future work:
Provide improved tooling
Extend the supported sets of ATL and OCL
Verify larger case studies
Proofs of executability
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19
55. Conclusions
Summary:
Partial correctness of declarative ATL transformations w.r.t.
pre- and postconditions can be nicely formulated in FOL
Modern SMT solvers seem to be quite good to
automatically find proofs in this setting (but beware:
incomplete procedure)
The approach complements bounded search approaches
for counter example finding
Future work:
Provide improved tooling
Extend the supported sets of ATL and OCL
Verify larger case studies
Proofs of executability
Identify decidable fragments of ATL transformations
c AtlanMod – atlanmod-contact@mines-nantes.fr 19/19