SlideShare a Scribd company logo
1 of 45
Download to read offline
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,       Comparing the Effectiveness of Reasoning
  Robinson,
Chechik, Salay       Formalisms for Partial Models
Introduction

Designer
Uncertainty       Pooya Saadatpanah, Michalis Famelis, Jan Gorzny,
Modeling            Nathan Robinson, Marsha Chechik, Rick Salay
Uncertainty

Property
Checking                          University of Toronto
Process

Verification
Technologies                   September 30th, 2012
Experiments
                                     MoDeVVa’12
Results

Conclusion




                                                                     1 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                          Uncertainty in software modeling
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay
                 • Uncertainty: pervasive in MDE

Introduction

Designer
Uncertainty      • Models with uncertainty:
Modeling            • Represent choice among many possibilities
Uncertainty
                    • Can be refined to many different classical models
Property
Checking
Process

Verification
Technologies     • Our goal:
Experiments
                       Handle models with uncertainty in MDE
Results                without having to remove it [MoDeVVa’11].
Conclusion




                                                                        2 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Existing Work
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                 3 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 In This Paper
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                 4 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                           Outline
Partial Models

Saadatpanah,
   Famelis,      Introduction
   Gorzny,
  Robinson,
Chechik, Salay
                 What is uncertainty?
Introduction

Designer         How to represent uncertainty with partial models (MAVO).
Uncertainty

Modeling
Uncertainty      Process for checking properties
Property
Checking
Process          Alternative verification technologies
Verification
Technologies
                 Experiments
Experiments

Results
                 Results
Conclusion

                 Conclusion

                                                                            5 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                  Introduction to Uncertainty
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay   What the designer knows.
Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                                6 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                 Introduction to Uncertainty
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay   What the designer does not know.
Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                               6 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                 Introduction to Uncertainty
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay   What the designer does not know.
Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                               6 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                 Introduction to Uncertainty
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay
                 What the designer does not know.

Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                               6 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                Uncertainty: a Set of Possible
Partial Models

Saadatpanah,
                                                 Refinements.
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty
                 If we remove all uncertainty, we have a concrete refinement.
Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                                               7 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                            Modeling Uncertainty with Partial
Partial Models

Saadatpanah,
                                                     Models
   Famelis,
   Gorzny,
  Robinson,      Explicating uncertainty in a partial model.
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies     In a refinement, a May element is optional.
Experiments      In a refinement, a Set element can be multiplied to many copies.
Results          In a refinement, a Variable element can be unified with some other.
Conclusion       In a refinement, an Open world model can be expanded with some
                 other elements.
                 Partial models: Syntactic annotations of the points of uncertainty.

                                                                                       8 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                            Modeling Uncertainty with Partial
Partial Models

Saadatpanah,
                                                     Models
   Famelis,
   Gorzny,
  Robinson,      Explicating uncertainty in a partial model.
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies     In a refinement, a May element is optional.
Experiments      In a refinement, a Set element can be multiplied to many copies.
Results          In a refinement, a Variable element can be unified with some other.
Conclusion       In a refinement, an Open world model can be expanded with some
                 other elements.
                 Partial models: Syntactic annotations of the points of uncertainty.

                                                                                       8 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                            Modeling Uncertainty with Partial
Partial Models

Saadatpanah,
                                                     Models
   Famelis,
   Gorzny,
  Robinson,      Explicating uncertainty in a partial model.
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies     In a refinement, a May element is optional.
Experiments      In a refinement, a Set element can be multiplied to many copies.
Results          In a refinement, a Variable element can be unified with some other.
Conclusion       In a refinement, an Open world model can be expanded with some
                 other elements.
                 Partial models: Syntactic annotations of the points of uncertainty.

                                                                                       8 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                            Modeling Uncertainty with Partial
Partial Models

Saadatpanah,
                                                     Models
   Famelis,
   Gorzny,
  Robinson,      Explicating uncertainty in a partial model.
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies     In a refinement, a May element is optional.
Experiments      In a refinement, a Set element can be multiplied to many copies.
Results          In a refinement, a Variable element can be unified with some other.
Conclusion       In a refinement, an Open world model can be expanded with some
                 other elements.
                 Partial models: Syntactic annotations of the points of uncertainty.

                                                                                       8 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                            Modeling Uncertainty with Partial
Partial Models

Saadatpanah,
                                                     Models
   Famelis,
   Gorzny,
  Robinson,      Explicating uncertainty in a partial model.
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies     In a refinement, a May element is optional.
Experiments
                 In a refinement, a Set element can be multiplied to many copies.
Results
                 In a refinement, a Variable element can be unified with some other.
Conclusion
                 In a refinement, an Open world model can be expanded with some
                 other elements.
                 Partial models: Syntactic annotations of the points of uncertainty.

                                                                                       8 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                           Outline
Partial Models

Saadatpanah,
   Famelis,      Introduction
   Gorzny,
  Robinson,
Chechik, Salay
                 What is uncertainty?
Introduction

Designer         How to represent uncertainty with partial models (MAVO).
Uncertainty

Modeling
Uncertainty      Process for checking properties
Property
Checking
Process          Alternative verification technologies
Verification
Technologies
                 Experiments
Experiments

Results
                 Results
Conclusion

                 Conclusion

                                                                            9 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Property Checking in Partial
Partial Models

Saadatpanah,
                                     Models
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                10 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Property Checking in Partial
Partial Models

Saadatpanah,
                                     Models
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                10 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Property Checking in Partial
Partial Models

Saadatpanah,
                                     Models
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                10 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Property Checking in Partial
Partial Models

Saadatpanah,
                                     Models
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                10 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                     Verification Technologies I
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
                 • Alloy
Chechik, Salay
                      • Lightweight formal methods
Introduction          • Model finder based on SAT
Designer
                      • First order logic specifications expressed in relational logic
Uncertainty           • Grounded to CNF representation
Modeling              • Finds counter examples
Uncertainty

Property
Checking
Process
                 • Constraint Satisfaction Problem (Minizinc/Flatzinc)
Verification
Technologies         • Assign value to variables to satisfy all constraints
Experiments          • Constraint modeling language
Results              • Easily translatable to the form required by other CSP
Conclusion             solvers



                                                                                   11 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                   Verification Technologies II
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay   • Satisfiability Modulo Theory (Z3)
                     • Constraint satisfaction search with richer theories
Introduction
                     • Theorem prover
Designer
Uncertainty          • Check the satisfiability logical formulas
Modeling
Uncertainty

Property
Checking
Process          • Answer Set Programming (Clingo=Gringo+Clasp)
Verification          • Answer set solvers
Technologies
                     • Conflict-driven nogood learning
Experiments
                     • Normal logic programs
Results

Conclusion




                                                                             12 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                           Outline
Partial Models

Saadatpanah,
   Famelis,      Introduction
   Gorzny,
  Robinson,
Chechik, Salay
                 What is uncertainty?
Introduction

Designer         How to represent uncertainty with partial models (MAVO).
Uncertainty

Modeling
Uncertainty      Process for checking properties
Property
Checking
Process          Alternative verification technologies
Verification
Technologies
                 Experiments
Experiments

Results
                 Results
Conclusion

                 Conclusion

                                                                            13 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Experimental Setup
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                      14 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Experimental Setup
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                      15 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                   Random Input Generation
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,      • Meta-model : directed graphs
Chechik, Salay
                    • Minimal meta-model
Introduction        • A few constraints
Designer            • Most difficult one for solvers
Uncertainty

Modeling
Uncertainty      • Randomly decorated with MAVO annotations.
Property
Checking
Process          • Parameters are based on real case studies.
Verification          • Graph density
Technologies
                     • Percentage of MAVO annotated elements
Experiments
                     • Percentages of M-, S- and V-annotated elements
Results

Conclusion
                 • 3 Model Size : Small, Medium, Large, X-Large


                                                                        16 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Experimental Setup
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                      17 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                         Relational Encoding I
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay   • Relational Algebra used in DBMSs
Introduction

Designer         • Directly translatable into the different formalisms
Uncertainty

Modeling
Uncertainty
                 • Intermediate representation
Property
Checking              • FOL semantics of MAVO
Process
                      • Reasoning formalisms
Verification
Technologies

Experiments
                 • Meaningful comparison
Results
                    • Most efficient encoding in each formalism : impossible!
Conclusion
                    • Solution : common encoding



                                                                              18 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                               Relational Encoding II
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay
                 (Very) high level overview.

Introduction
                   • The metamodel encoded
Designer             as a schema.
Uncertainty
                   • Partial model FOL
Modeling
Uncertainty          semantics encoded as
Property
Checking
                     constraints over the
Process              schema.
Verification
Technologies       • Creating a concrete
Experiments          refinement populates the
Results              database.
Conclusion




                                                                        19 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Experimental Setup
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                      20 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                   Translation To Formalisms
Partial Models
                 • Alloy
Saadatpanah,
   Famelis,           • Relations : Alloy signatures
   Gorzny,
  Robinson,
                      • Instances : Atoms
Chechik, Salay        • MAVO constraints : quantified predicates over signatures
                      • Bound is required
Introduction

Designer
                 • CSP
Uncertainty           • Relations : Finite set of Integers
Modeling              • Instances : Integers
Uncertainty
                      • MAVO constraints : cardinality and intersection of sets
Property
Checking              • Bound is required
Process
                 • SMT
Verification           • Relations : Uninterpreted boolean functions
Technologies
                      • Instances : Abstract values
Experiments
                      • MAVO constraints : Quantified logic over truth table of
Results
                        functions
Conclusion
                 • ASP
                      • Program rules for both instances and relations
                      • Bound is required
                                                                              21 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Experimental Setup
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                      22 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                Properties Checked
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,        • Inspired by real metamodel constraints.
  Robinson,
Chechik, Salay
                  • No transitive closure, since it is expensive to check.
Introduction

Designer
Uncertainty
                 P1: There exists a node with a self-loop.
Modeling
Uncertainty      P2: All nodes have outgoing edges.
Property
Checking         P3: All nodes have outgoing or incoming edges.
Process

Verification      P4: For all pairs of nodes n1 , n2 there exists at most one edge e
Technologies                        e
                     such that n1 → n2
Experiments

Results          P5: For every pair of nodes n1 , n2 , n1 = n2 there exist two edges
                                           e1            e2
Conclusion            e1 , e2 such that n1 → n2 and n2 → n1 .



                                                                                      23 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Experimental Setup
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                      24 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                     Experimental Parameters
Partial Models

Saadatpanah,     • Bound (2, 4, 6)
   Famelis,
   Gorzny,           • Solvers (except SMT) use bound for grounding expressions
  Robinson,
Chechik, Salay         to atoms.
                     • How many times can an ’S’-annotated element be
Introduction
                       replicated in a refinement.
Designer
Uncertainty
                 • Repetitions
Modeling
Uncertainty          • 5 times
Property
Checking
                 • Cutoff time/memory
Process
                     • less than 10 minutes
Verification
Technologies
                     • less than 5 gigabyte
Experiments
                     • otherwise timeout
Results          • What we measure
Conclusion
                    • How long does it take for each solver to return an answer
                    • A score out of 1200
                    • if timeout : zero!

                                                                              25 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Findings
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                            26 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Findings
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                            26 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                 Findings
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                            26 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                              SMT the champion?
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay     • Unaffected by bounds!
Introduction       • Works at higher level of abstraction
Designer                   (Theory of uninterpreted functions.)
Uncertainty

Modeling
                   • Unaffected by expensive grounding phase.
Uncertainty

Property
Checking
Process

Verification      Caveat:
Technologies

Experiments
                   • SMT can theoretically return “I don’t know”.
Results            • (However: we didn’t observe that.’)
Conclusion




                                                                    27 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                              Threats to Validity
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,       Randomly generated graphs.
  Robinson,
Chechik, Salay     • Tuned the generator with realistic graph properties.
Introduction       • Values of graph properties from case studies.
Designer
Uncertainty

Modeling
Uncertainty      Fairness of comparisons.
Property           • Common encoding to level the field.
Checking
Process

Verification
Technologies
                 Choice of specific reasoning engines
Experiments

Results
                   • When available: winners of competitions.
Conclusion         • CSP: most convenient input language.



                                                                            28 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                         Summary
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay
                 Research Question
                 What is the most efficient formalism for verifying models
Introduction

Designer
                 containing uncertainty?
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification
Technologies

Experiments

Results

Conclusion




                                                                           29 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                             Summary
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction

Designer
Uncertainty

Modeling
Uncertainty

Property
Checking
Process

Verification      Other results:
Technologies
                   • Framework for running experiments (now full tool support).
Experiments

Results
                   • Random generator for arbitrary type graphs.
Conclusion         • Relational encoding.
                   • Translations of the RA encoding to different formalisms.


                                                                                  29 / 32
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                     Future Work
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay
                 • Implement symmetry breaking in the SMT encoding.
Introduction

Designer
Uncertainty

Modeling
Uncertainty      • Experiment with properties that require transitive closure.
Property
Checking
Process

Verification
Technologies

Experiments      • Experiment with partial models containing OW.
Results

Conclusion




                                                                            30 / 32
Questions?
Comparing the
 Effectiveness
 of Reasoning
Formalisms for
                                                                       Bibliography I
Partial Models

Saadatpanah,
   Famelis,
   Gorzny,
  Robinson,
Chechik, Salay


Introduction     Famelis, M., Ben-David, S., Chechik, M., and Salay, R. (2011).
                 “Partial Models: A Position Paper”.
Designer
                 In Proceedings of MoDeVVa’11, pages 1–6.
Uncertainty
                 Famelis, M., Chechik, M., and Salay, R. (2012).
Modeling
Uncertainty      “Partial Models: Towards Modeling and Reasoning with Uncertainty”.
                 In Proceedings of ICSE’12.
Property
Checking         Salay, R., Famelis, M., and Chechik, M. (2012).
Process          “Language Independent Refinement using Partial Modeling”.
                 In Proceedings of FASE’12.
Verification
Technologies

Experiments

Results

Conclusion




                                                                                        32 / 32

More Related Content

Viewers also liked

Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Michalis Famelis
 
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...Michalis Famelis
 
Zebra Direct Sensing MIP-CPT-HPT-LIF
Zebra Direct Sensing MIP-CPT-HPT-LIFZebra Direct Sensing MIP-CPT-HPT-LIF
Zebra Direct Sensing MIP-CPT-HPT-LIFZEBRA Environmental
 
Modeling and Solving Decision Optimization Problems
Modeling and Solving Decision Optimization ProblemsModeling and Solving Decision Optimization Problems
Modeling and Solving Decision Optimization ProblemsJacob Feldman
 
The Concurrent Constraint Programming Research Programmes -- Redux (part2)
The Concurrent Constraint Programming Research Programmes -- Redux (part2)The Concurrent Constraint Programming Research Programmes -- Redux (part2)
The Concurrent Constraint Programming Research Programmes -- Redux (part2)Pierre Schaus
 

Viewers also liked (6)

Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...
 
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
 
Large scalecplex
Large scalecplexLarge scalecplex
Large scalecplex
 
Zebra Direct Sensing MIP-CPT-HPT-LIF
Zebra Direct Sensing MIP-CPT-HPT-LIFZebra Direct Sensing MIP-CPT-HPT-LIF
Zebra Direct Sensing MIP-CPT-HPT-LIF
 
Modeling and Solving Decision Optimization Problems
Modeling and Solving Decision Optimization ProblemsModeling and Solving Decision Optimization Problems
Modeling and Solving Decision Optimization Problems
 
The Concurrent Constraint Programming Research Programmes -- Redux (part2)
The Concurrent Constraint Programming Research Programmes -- Redux (part2)The Concurrent Constraint Programming Research Programmes -- Redux (part2)
The Concurrent Constraint Programming Research Programmes -- Redux (part2)
 

Similar to Comparing the Effectiveness of Reasoning Formalisms for Partial Models

Partial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with UncertaintyPartial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with UncertaintyMichalis Famelis
 
Testing Harmony Design Patterns & Anti-Patterns Unveiled
Testing Harmony Design Patterns & Anti-Patterns UnveiledTesting Harmony Design Patterns & Anti-Patterns Unveiled
Testing Harmony Design Patterns & Anti-Patterns UnveiledKnoldus Inc.
 
Do you have a #bug? Your unit tests are not well planned
Do you have a #bug? Your unit tests are not well plannedDo you have a #bug? Your unit tests are not well planned
Do you have a #bug? Your unit tests are not well plannedJosé San Román A. de Lara
 
The Process of Process Modeling
The Process of Process ModelingThe Process of Process Modeling
The Process of Process ModelingDirk Fahland
 
Testing survey by_directions
Testing survey by_directionsTesting survey by_directions
Testing survey by_directionsTao He
 
STAT7440StudentIMLPresentationJishan.pptx
STAT7440StudentIMLPresentationJishan.pptxSTAT7440StudentIMLPresentationJishan.pptx
STAT7440StudentIMLPresentationJishan.pptxJishanAhmed24
 
Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...
Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...
Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...SEAA 2022
 
Linking CSCL script design patterns: connections between assessment
Linking CSCL script design patterns: connections between assessmentLinking CSCL script design patterns: connections between assessment
Linking CSCL script design patterns: connections between assessmentevilfer
 
The process of process modeling by Hajo Reijers
The process of process modeling by Hajo ReijersThe process of process modeling by Hajo Reijers
The process of process modeling by Hajo ReijersCONFENIS 2012
 
The Test Case as Executable Example
The Test Case as Executable ExampleThe Test Case as Executable Example
The Test Case as Executable ExampleArie van Deursen
 
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
Annotated Bibliography  .Guidelines Annotated Bibliograph.docxAnnotated Bibliography  .Guidelines Annotated Bibliograph.docx
Annotated Bibliography .Guidelines Annotated Bibliograph.docxjustine1simpson78276
 
Model validation strategies ftc 2018
Model validation strategies ftc 2018Model validation strategies ftc 2018
Model validation strategies ftc 2018Philip Ramsey
 
Hybrid models
Hybrid modelsHybrid models
Hybrid modelshreijers
 
Evaluating the efficiency of using a search-based automated model merge techn...
Evaluating the efficiency of using a search-based automated model merge techn...Evaluating the efficiency of using a search-based automated model merge techn...
Evaluating the efficiency of using a search-based automated model merge techn...Ankica Barisic
 
RESEARCH in software engineering
RESEARCH in software engineeringRESEARCH in software engineering
RESEARCH in software engineeringIvano Malavolta
 
1How to Perform ExperimentsBasic Concepts CSCI .docx
1How to Perform ExperimentsBasic Concepts CSCI .docx1How to Perform ExperimentsBasic Concepts CSCI .docx
1How to Perform ExperimentsBasic Concepts CSCI .docxdrennanmicah
 
Business Process Modeling What Is Possible Vs What Is Useful
Business Process Modeling   What Is Possible Vs What Is UsefulBusiness Process Modeling   What Is Possible Vs What Is Useful
Business Process Modeling What Is Possible Vs What Is UsefulDenis Gagné
 

Similar to Comparing the Effectiveness of Reasoning Formalisms for Partial Models (20)

Partial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with UncertaintyPartial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with Uncertainty
 
Testing Harmony Design Patterns & Anti-Patterns Unveiled
Testing Harmony Design Patterns & Anti-Patterns UnveiledTesting Harmony Design Patterns & Anti-Patterns Unveiled
Testing Harmony Design Patterns & Anti-Patterns Unveiled
 
Mm design principles
Mm design principlesMm design principles
Mm design principles
 
Do you have a #bug? Your unit tests are not well planned
Do you have a #bug? Your unit tests are not well plannedDo you have a #bug? Your unit tests are not well planned
Do you have a #bug? Your unit tests are not well planned
 
Test for AI model
Test for AI modelTest for AI model
Test for AI model
 
The Process of Process Modeling
The Process of Process ModelingThe Process of Process Modeling
The Process of Process Modeling
 
Testing survey by_directions
Testing survey by_directionsTesting survey by_directions
Testing survey by_directions
 
STAT7440StudentIMLPresentationJishan.pptx
STAT7440StudentIMLPresentationJishan.pptxSTAT7440StudentIMLPresentationJishan.pptx
STAT7440StudentIMLPresentationJishan.pptx
 
Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...
Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...
Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-G...
 
Linking CSCL script design patterns: connections between assessment
Linking CSCL script design patterns: connections between assessmentLinking CSCL script design patterns: connections between assessment
Linking CSCL script design patterns: connections between assessment
 
The process of process modeling by Hajo Reijers
The process of process modeling by Hajo ReijersThe process of process modeling by Hajo Reijers
The process of process modeling by Hajo Reijers
 
The Test Case as Executable Example
The Test Case as Executable ExampleThe Test Case as Executable Example
The Test Case as Executable Example
 
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
Annotated Bibliography  .Guidelines Annotated Bibliograph.docxAnnotated Bibliography  .Guidelines Annotated Bibliograph.docx
Annotated Bibliography .Guidelines Annotated Bibliograph.docx
 
Model validation strategies ftc 2018
Model validation strategies ftc 2018Model validation strategies ftc 2018
Model validation strategies ftc 2018
 
Hybrid models
Hybrid modelsHybrid models
Hybrid models
 
Evaluating the efficiency of using a search-based automated model merge techn...
Evaluating the efficiency of using a search-based automated model merge techn...Evaluating the efficiency of using a search-based automated model merge techn...
Evaluating the efficiency of using a search-based automated model merge techn...
 
Managing machine learning
Managing machine learningManaging machine learning
Managing machine learning
 
RESEARCH in software engineering
RESEARCH in software engineeringRESEARCH in software engineering
RESEARCH in software engineering
 
1How to Perform ExperimentsBasic Concepts CSCI .docx
1How to Perform ExperimentsBasic Concepts CSCI .docx1How to Perform ExperimentsBasic Concepts CSCI .docx
1How to Perform ExperimentsBasic Concepts CSCI .docx
 
Business Process Modeling What Is Possible Vs What Is Useful
Business Process Modeling   What Is Possible Vs What Is UsefulBusiness Process Modeling   What Is Possible Vs What Is Useful
Business Process Modeling What Is Possible Vs What Is Useful
 

More from Michalis Famelis

Transformations of Models Containing Uncertainty
Transformations of Models Containing UncertaintyTransformations of Models Containing Uncertainty
Transformations of Models Containing UncertaintyMichalis Famelis
 
Research Questions for Validation and Verification in the Context of Model-Ba...
Research Questions for Validation and Verification in the Context of Model-Ba...Research Questions for Validation and Verification in the Context of Model-Ba...
Research Questions for Validation and Verification in the Context of Model-Ba...Michalis Famelis
 
MAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model UncertaintyMAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model UncertaintyMichalis Famelis
 
Greece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist eventGreece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist eventMichalis Famelis
 
Uncertainty Management With Partial Models
Uncertainty Management With Partial ModelsUncertainty Management With Partial Models
Uncertainty Management With Partial ModelsMichalis Famelis
 
The Semantics of Partial Model Transformations
The Semantics of Partial Model TransformationsThe Semantics of Partial Model Transformations
The Semantics of Partial Model TransformationsMichalis Famelis
 

More from Michalis Famelis (6)

Transformations of Models Containing Uncertainty
Transformations of Models Containing UncertaintyTransformations of Models Containing Uncertainty
Transformations of Models Containing Uncertainty
 
Research Questions for Validation and Verification in the Context of Model-Ba...
Research Questions for Validation and Verification in the Context of Model-Ba...Research Questions for Validation and Verification in the Context of Model-Ba...
Research Questions for Validation and Verification in the Context of Model-Ba...
 
MAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model UncertaintyMAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model Uncertainty
 
Greece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist eventGreece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist event
 
Uncertainty Management With Partial Models
Uncertainty Management With Partial ModelsUncertainty Management With Partial Models
Uncertainty Management With Partial Models
 
The Semantics of Partial Model Transformations
The Semantics of Partial Model TransformationsThe Semantics of Partial Model Transformations
The Semantics of Partial Model Transformations
 

Recently uploaded

AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...SOFTTECHHUB
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformWSO2
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governanceWSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceIES VE
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...caitlingebhard1
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 

Recently uploaded (20)

AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational Performance
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 

Comparing the Effectiveness of Reasoning Formalisms for Partial Models

  • 1. Comparing the Effectiveness of Reasoning Formalisms for Partial Models Saadatpanah, Famelis, Gorzny, Comparing the Effectiveness of Reasoning Robinson, Chechik, Salay Formalisms for Partial Models Introduction Designer Uncertainty Pooya Saadatpanah, Michalis Famelis, Jan Gorzny, Modeling Nathan Robinson, Marsha Chechik, Rick Salay Uncertainty Property Checking University of Toronto Process Verification Technologies September 30th, 2012 Experiments MoDeVVa’12 Results Conclusion 1 / 32
  • 2. Comparing the Effectiveness of Reasoning Formalisms for Uncertainty in software modeling Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay • Uncertainty: pervasive in MDE Introduction Designer Uncertainty • Models with uncertainty: Modeling • Represent choice among many possibilities Uncertainty • Can be refined to many different classical models Property Checking Process Verification Technologies • Our goal: Experiments Handle models with uncertainty in MDE Results without having to remove it [MoDeVVa’11]. Conclusion 2 / 32
  • 3. Comparing the Effectiveness of Reasoning Formalisms for Existing Work Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 3 / 32
  • 4. Comparing the Effectiveness of Reasoning Formalisms for In This Paper Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 4 / 32
  • 5. Comparing the Effectiveness of Reasoning Formalisms for Outline Partial Models Saadatpanah, Famelis, Introduction Gorzny, Robinson, Chechik, Salay What is uncertainty? Introduction Designer How to represent uncertainty with partial models (MAVO). Uncertainty Modeling Uncertainty Process for checking properties Property Checking Process Alternative verification technologies Verification Technologies Experiments Experiments Results Results Conclusion Conclusion 5 / 32
  • 6. Comparing the Effectiveness of Reasoning Formalisms for Introduction to Uncertainty Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay What the designer knows. Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 6 / 32
  • 7. Comparing the Effectiveness of Reasoning Formalisms for Introduction to Uncertainty Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay What the designer does not know. Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 6 / 32
  • 8. Comparing the Effectiveness of Reasoning Formalisms for Introduction to Uncertainty Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay What the designer does not know. Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 6 / 32
  • 9. Comparing the Effectiveness of Reasoning Formalisms for Introduction to Uncertainty Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay What the designer does not know. Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 6 / 32
  • 10. Comparing the Effectiveness of Reasoning Formalisms for Uncertainty: a Set of Possible Partial Models Saadatpanah, Refinements. Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty If we remove all uncertainty, we have a concrete refinement. Property Checking Process Verification Technologies Experiments Results Conclusion 7 / 32
  • 11. Comparing the Effectiveness of Reasoning Formalisms for Modeling Uncertainty with Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Explicating uncertainty in a partial model. Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies In a refinement, a May element is optional. Experiments In a refinement, a Set element can be multiplied to many copies. Results In a refinement, a Variable element can be unified with some other. Conclusion In a refinement, an Open world model can be expanded with some other elements. Partial models: Syntactic annotations of the points of uncertainty. 8 / 32
  • 12. Comparing the Effectiveness of Reasoning Formalisms for Modeling Uncertainty with Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Explicating uncertainty in a partial model. Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies In a refinement, a May element is optional. Experiments In a refinement, a Set element can be multiplied to many copies. Results In a refinement, a Variable element can be unified with some other. Conclusion In a refinement, an Open world model can be expanded with some other elements. Partial models: Syntactic annotations of the points of uncertainty. 8 / 32
  • 13. Comparing the Effectiveness of Reasoning Formalisms for Modeling Uncertainty with Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Explicating uncertainty in a partial model. Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies In a refinement, a May element is optional. Experiments In a refinement, a Set element can be multiplied to many copies. Results In a refinement, a Variable element can be unified with some other. Conclusion In a refinement, an Open world model can be expanded with some other elements. Partial models: Syntactic annotations of the points of uncertainty. 8 / 32
  • 14. Comparing the Effectiveness of Reasoning Formalisms for Modeling Uncertainty with Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Explicating uncertainty in a partial model. Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies In a refinement, a May element is optional. Experiments In a refinement, a Set element can be multiplied to many copies. Results In a refinement, a Variable element can be unified with some other. Conclusion In a refinement, an Open world model can be expanded with some other elements. Partial models: Syntactic annotations of the points of uncertainty. 8 / 32
  • 15. Comparing the Effectiveness of Reasoning Formalisms for Modeling Uncertainty with Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Explicating uncertainty in a partial model. Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies In a refinement, a May element is optional. Experiments In a refinement, a Set element can be multiplied to many copies. Results In a refinement, a Variable element can be unified with some other. Conclusion In a refinement, an Open world model can be expanded with some other elements. Partial models: Syntactic annotations of the points of uncertainty. 8 / 32
  • 16. Comparing the Effectiveness of Reasoning Formalisms for Outline Partial Models Saadatpanah, Famelis, Introduction Gorzny, Robinson, Chechik, Salay What is uncertainty? Introduction Designer How to represent uncertainty with partial models (MAVO). Uncertainty Modeling Uncertainty Process for checking properties Property Checking Process Alternative verification technologies Verification Technologies Experiments Experiments Results Results Conclusion Conclusion 9 / 32
  • 17. Comparing the Effectiveness of Reasoning Formalisms for Property Checking in Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 10 / 32
  • 18. Comparing the Effectiveness of Reasoning Formalisms for Property Checking in Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 10 / 32
  • 19. Comparing the Effectiveness of Reasoning Formalisms for Property Checking in Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 10 / 32
  • 20. Comparing the Effectiveness of Reasoning Formalisms for Property Checking in Partial Partial Models Saadatpanah, Models Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 10 / 32
  • 21. Comparing the Effectiveness of Reasoning Formalisms for Verification Technologies I Partial Models Saadatpanah, Famelis, Gorzny, Robinson, • Alloy Chechik, Salay • Lightweight formal methods Introduction • Model finder based on SAT Designer • First order logic specifications expressed in relational logic Uncertainty • Grounded to CNF representation Modeling • Finds counter examples Uncertainty Property Checking Process • Constraint Satisfaction Problem (Minizinc/Flatzinc) Verification Technologies • Assign value to variables to satisfy all constraints Experiments • Constraint modeling language Results • Easily translatable to the form required by other CSP Conclusion solvers 11 / 32
  • 22. Comparing the Effectiveness of Reasoning Formalisms for Verification Technologies II Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay • Satisfiability Modulo Theory (Z3) • Constraint satisfaction search with richer theories Introduction • Theorem prover Designer Uncertainty • Check the satisfiability logical formulas Modeling Uncertainty Property Checking Process • Answer Set Programming (Clingo=Gringo+Clasp) Verification • Answer set solvers Technologies • Conflict-driven nogood learning Experiments • Normal logic programs Results Conclusion 12 / 32
  • 23. Comparing the Effectiveness of Reasoning Formalisms for Outline Partial Models Saadatpanah, Famelis, Introduction Gorzny, Robinson, Chechik, Salay What is uncertainty? Introduction Designer How to represent uncertainty with partial models (MAVO). Uncertainty Modeling Uncertainty Process for checking properties Property Checking Process Alternative verification technologies Verification Technologies Experiments Experiments Results Results Conclusion Conclusion 13 / 32
  • 24. Comparing the Effectiveness of Reasoning Formalisms for Experimental Setup Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 14 / 32
  • 25. Comparing the Effectiveness of Reasoning Formalisms for Experimental Setup Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 15 / 32
  • 26. Comparing the Effectiveness of Reasoning Formalisms for Random Input Generation Partial Models Saadatpanah, Famelis, Gorzny, Robinson, • Meta-model : directed graphs Chechik, Salay • Minimal meta-model Introduction • A few constraints Designer • Most difficult one for solvers Uncertainty Modeling Uncertainty • Randomly decorated with MAVO annotations. Property Checking Process • Parameters are based on real case studies. Verification • Graph density Technologies • Percentage of MAVO annotated elements Experiments • Percentages of M-, S- and V-annotated elements Results Conclusion • 3 Model Size : Small, Medium, Large, X-Large 16 / 32
  • 27. Comparing the Effectiveness of Reasoning Formalisms for Experimental Setup Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 17 / 32
  • 28. Comparing the Effectiveness of Reasoning Formalisms for Relational Encoding I Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay • Relational Algebra used in DBMSs Introduction Designer • Directly translatable into the different formalisms Uncertainty Modeling Uncertainty • Intermediate representation Property Checking • FOL semantics of MAVO Process • Reasoning formalisms Verification Technologies Experiments • Meaningful comparison Results • Most efficient encoding in each formalism : impossible! Conclusion • Solution : common encoding 18 / 32
  • 29. Comparing the Effectiveness of Reasoning Formalisms for Relational Encoding II Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay (Very) high level overview. Introduction • The metamodel encoded Designer as a schema. Uncertainty • Partial model FOL Modeling Uncertainty semantics encoded as Property Checking constraints over the Process schema. Verification Technologies • Creating a concrete Experiments refinement populates the Results database. Conclusion 19 / 32
  • 30. Comparing the Effectiveness of Reasoning Formalisms for Experimental Setup Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 20 / 32
  • 31. Comparing the Effectiveness of Reasoning Formalisms for Translation To Formalisms Partial Models • Alloy Saadatpanah, Famelis, • Relations : Alloy signatures Gorzny, Robinson, • Instances : Atoms Chechik, Salay • MAVO constraints : quantified predicates over signatures • Bound is required Introduction Designer • CSP Uncertainty • Relations : Finite set of Integers Modeling • Instances : Integers Uncertainty • MAVO constraints : cardinality and intersection of sets Property Checking • Bound is required Process • SMT Verification • Relations : Uninterpreted boolean functions Technologies • Instances : Abstract values Experiments • MAVO constraints : Quantified logic over truth table of Results functions Conclusion • ASP • Program rules for both instances and relations • Bound is required 21 / 32
  • 32. Comparing the Effectiveness of Reasoning Formalisms for Experimental Setup Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 22 / 32
  • 33. Comparing the Effectiveness of Reasoning Formalisms for Properties Checked Partial Models Saadatpanah, Famelis, Gorzny, • Inspired by real metamodel constraints. Robinson, Chechik, Salay • No transitive closure, since it is expensive to check. Introduction Designer Uncertainty P1: There exists a node with a self-loop. Modeling Uncertainty P2: All nodes have outgoing edges. Property Checking P3: All nodes have outgoing or incoming edges. Process Verification P4: For all pairs of nodes n1 , n2 there exists at most one edge e Technologies e such that n1 → n2 Experiments Results P5: For every pair of nodes n1 , n2 , n1 = n2 there exist two edges e1 e2 Conclusion e1 , e2 such that n1 → n2 and n2 → n1 . 23 / 32
  • 34. Comparing the Effectiveness of Reasoning Formalisms for Experimental Setup Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 24 / 32
  • 35. Comparing the Effectiveness of Reasoning Formalisms for Experimental Parameters Partial Models Saadatpanah, • Bound (2, 4, 6) Famelis, Gorzny, • Solvers (except SMT) use bound for grounding expressions Robinson, Chechik, Salay to atoms. • How many times can an ’S’-annotated element be Introduction replicated in a refinement. Designer Uncertainty • Repetitions Modeling Uncertainty • 5 times Property Checking • Cutoff time/memory Process • less than 10 minutes Verification Technologies • less than 5 gigabyte Experiments • otherwise timeout Results • What we measure Conclusion • How long does it take for each solver to return an answer • A score out of 1200 • if timeout : zero! 25 / 32
  • 36. Comparing the Effectiveness of Reasoning Formalisms for Findings Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 26 / 32
  • 37. Comparing the Effectiveness of Reasoning Formalisms for Findings Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 26 / 32
  • 38. Comparing the Effectiveness of Reasoning Formalisms for Findings Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 26 / 32
  • 39. Comparing the Effectiveness of Reasoning Formalisms for SMT the champion? Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay • Unaffected by bounds! Introduction • Works at higher level of abstraction Designer (Theory of uninterpreted functions.) Uncertainty Modeling • Unaffected by expensive grounding phase. Uncertainty Property Checking Process Verification Caveat: Technologies Experiments • SMT can theoretically return “I don’t know”. Results • (However: we didn’t observe that.’) Conclusion 27 / 32
  • 40. Comparing the Effectiveness of Reasoning Formalisms for Threats to Validity Partial Models Saadatpanah, Famelis, Gorzny, Randomly generated graphs. Robinson, Chechik, Salay • Tuned the generator with realistic graph properties. Introduction • Values of graph properties from case studies. Designer Uncertainty Modeling Uncertainty Fairness of comparisons. Property • Common encoding to level the field. Checking Process Verification Technologies Choice of specific reasoning engines Experiments Results • When available: winners of competitions. Conclusion • CSP: most convenient input language. 28 / 32
  • 41. Comparing the Effectiveness of Reasoning Formalisms for Summary Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Research Question What is the most efficient formalism for verifying models Introduction Designer containing uncertainty? Uncertainty Modeling Uncertainty Property Checking Process Verification Technologies Experiments Results Conclusion 29 / 32
  • 42. Comparing the Effectiveness of Reasoning Formalisms for Summary Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Designer Uncertainty Modeling Uncertainty Property Checking Process Verification Other results: Technologies • Framework for running experiments (now full tool support). Experiments Results • Random generator for arbitrary type graphs. Conclusion • Relational encoding. • Translations of the RA encoding to different formalisms. 29 / 32
  • 43. Comparing the Effectiveness of Reasoning Formalisms for Future Work Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay • Implement symmetry breaking in the SMT encoding. Introduction Designer Uncertainty Modeling Uncertainty • Experiment with properties that require transitive closure. Property Checking Process Verification Technologies Experiments • Experiment with partial models containing OW. Results Conclusion 30 / 32
  • 45. Comparing the Effectiveness of Reasoning Formalisms for Bibliography I Partial Models Saadatpanah, Famelis, Gorzny, Robinson, Chechik, Salay Introduction Famelis, M., Ben-David, S., Chechik, M., and Salay, R. (2011). “Partial Models: A Position Paper”. Designer In Proceedings of MoDeVVa’11, pages 1–6. Uncertainty Famelis, M., Chechik, M., and Salay, R. (2012). Modeling Uncertainty “Partial Models: Towards Modeling and Reasoning with Uncertainty”. In Proceedings of ICSE’12. Property Checking Salay, R., Famelis, M., and Chechik, M. (2012). Process “Language Independent Refinement using Partial Modeling”. In Proceedings of FASE’12. Verification Technologies Experiments Results Conclusion 32 / 32