Partial Models: Towards Modeling and Reasoning with Uncertainty

1. Partial Models: Towards Modeling and Reasoning with Uncertainty M.Famelis, R.Salay, Partial Models: Towards Modeling and M.Chechik, Reasoning with Uncertainty Introduction Intuition Motivating Example Modeling Michalis Famelis, Rick Salay, and Marsha Chechik Uncertainty Partial Models Semantics University of Toronto Reasoning With Uncertainty June 7, 2012 Property Checking ICSE’12, Zurich, Switzerland Diagnosis Evaluation Experiments Case Study Conclusion 1 / 29

2. Partial Models: Towards Modeling and Intuition: Sudoku Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion Created with GNOME Sudoku 2.32.0. 2 / 29

11. Partial Models: Towards Modeling and Enough About Sudoku Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion Source: Wikimedia, 3 / 29

12. Partial Models: Towards Modeling and Goal: Uncertainty in Software Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Modeling Explicate points of uncertainty Introduction Intuition Correlate points of uncertainty Motivating Example Modeling Uncertainty Partial Models Reasoning Semantics Reasoning Check properties With Give feedback to facilitate diagnosis Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 4 / 29

13. Partial Models: Towards Modeling and Designing a P2P Application Reasoning with Trying to design a P2P client application. Uncertainty M.Famelis, R.Salay, What do I know? M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 5 / 29

14. Partial Models: Towards Modeling and Designing a P2P Application Reasoning with Trying to design a P2P client application. Uncertainty M.Famelis, R.Salay, What do I not know? M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 5 / 29

15. Partial Models: Towards Modeling and Designing a P2P Application Reasoning with Trying to design a P2P client application. Uncertainty M.Famelis, R.Salay, What do I not know? M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 5 / 29

16. Partial Models: Towards Modeling and Designing a P2P Application Reasoning with Trying to design a P2P client application. Uncertainty M.Famelis, R.Salay, How can I explicate my uncertainty and reason in its presence? M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 5 / 29

17. Partial Models: Towards Modeling and Contribution Reasoning with Uncertainty Modeling Uncertainty M.Famelis, R.Salay, M.Chechik, • Encode uncertainty in Partial Models. Introduction • Semantics: sets of conventional models. Intuition Motivating Example Modeling Reasoning in the Presence of Uncertainty Uncertainty Partial Models • Check properties. Semantics Reasoning • Give feedback to facilitate diagnosis. With Uncertainty Property Checking Diagnosis Evaluation of Reasoning Evaluation Experiments • Reasoning with Partial models vs. reasoning with a set of Case Study conventional models Conclusion 6 / 29

18. Partial Models: Towards Modeling and In Paper / Not In Talk Reasoning with Uncertainty M.Famelis, Presentation of these would take too much time: R.Salay, M.Chechik, • Encoding conventional models in logic and back Introduction • Construction algorithm of Partial Models Intuition Motivating • Propositional Normal Form (PNF) Example Modeling • Graphical Normal Form (GNF) Uncertainty Partial Models • Diagnostic cores Semantics Reasoning • “Property-driven” reﬁnement. With Uncertainty • Translation from PNF to GNF and vice versa Property Checking Diagnosis • Evaluation of diagnostic cores and property-driven Evaluation reﬁnement Experiments Case Study • Random generation of experimental inputs Conclusion 7 / 29

20. Partial Models: Towards Modeling and Partial Models Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 9 / 29

26. Partial Models: Towards Modeling and Semantics of Partial Models Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 10 / 29

29. Partial Models: Towards Modeling and Related Ideas Reasoning with Behavioral modeling: Uncertainty • Modal Transition Systems (MTSs) [Larsen’88]. M.Famelis, R.Salay, • Disjunctive MTSs [Larsen’91]. M.Chechik, Introduction Software Product Lines: Intuition Motivating Example • Variability in the metamodel [Morin’09]. Modeling • Featured Transition Systems [Classen’10]. Uncertainty Partial Models Semantics Reasoning Partial Models: With Uncertainty • Language-independent Property Checking not just behavioral models! Diagnosis • May formula: exact encoding Evaluation Experiments thorough reasoning Case Study • Focus on systematic management of uncertainty Conclusion uncertainty-reducing reﬁnement [VOLT’12] transformations [MiSE’12] 11 / 29

31. Partial Models: Towards Modeling and 1) Property Checking Reasoning with Uncertainty Property can be: M.Famelis, • True: holds for all concretizations R.Salay, M.Chechik, • False: holds for none Introduction • Maybe: true for some, false for others Intuition Motivating Example Modeling Uncertainty Partial Models To check a property: Semantics - Encode model and property in propositional logic. Reasoning With Uncertainty - Use SAT solver. Property Checking Diagnosis ΦM ∧ Φp ΦM ∧ ¬Φp Property p SAT SAT Maybe Evaluation Experiments SAT UNSAT True Case Study UNSAT SAT False Conclusion UNSAT UNSAT (model inconsistent) 13 / 29

32. Partial Models: Towards Modeling and Property Checking: Example Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 14 / 29

47. Partial Models: Towards Modeling and 2) Diagnosis Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Feedback: A concretization of the Partial Model for which the Introduction Intuition property does not hold. Motivating Example Modeling Uncertainty Reuse the results of property checking: Partial Models Semantics ΦM ∧ Φp ΦM ∧ ¬Φp Property p Reasoning With SAT SAT Maybe Uncertainty SAT UNSAT True Property UNSAT SAT False Checking Diagnosis UNSAT UNSAT (model inconsistent) Evaluation Experiments Case Study Conclusion 15 / 29

48. Partial Models: Towards Modeling and Diagnosis: Example Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 16 / 29

49. Partial Models: Towards Modeling and Diagnosis: Example Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 16 / 29

51. Partial Models: Towards Modeling and Questions Reasoning with Uncertainty Reasoning with Partial models M.Famelis, R.Salay, vs M.Chechik, Reasoning with a set of conventional models Introduction Intuition Motivating Example Modeling Is there a speedup? Uncertainty Partial Models Semantics How is speedup aﬀected by changing: Reasoning With • model size Uncertainty Property Checking • levels of uncertainty? Diagnosis Evaluation Experiments Case Study Conclusion 18 / 29

52. Partial Models: Towards Modeling and Questions Reasoning with Uncertainty Reasoning with Partial models M.Famelis, R.Salay, vs M.Chechik, Reasoning with a set of conventional models Introduction Intuition Motivating Example Modeling Is there a speedup? Uncertainty Partial Models Semantics How is speedup aﬀected by changing: Reasoning With • model size Uncertainty Property Checking • levels of uncertainty? Diagnosis Evaluation Experiments Case Study To get answers: Conclusion 1) Experiments with random inputs. 2) Real-world case study. 18 / 29

53. Partial Models: Towards Modeling and Experiments Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 19 / 29

62. Partial Models: Towards Modeling and Case Study Reasoning with Uncertainty M.Famelis, Why Case Study? R.Salay, M.Chechik, Triangulate experimental results (randomly inputs) Introduction with observations from a real-world scenario. Intuition Motivating Example Modeling Case Study details: Uncertainty Partial Models • Real-world software project: UMLet. Semantics Reasoning • Real-world bug from UMLet bugzilla. With Uncertainty • Realistic bug ﬁxes. Property Checking Diagnosis • Two properties from literature [V.D.Straeten’03]. Evaluation • 27,261 elements (XL model size) Experiments Case Study • 220 concretizations (XL uncertainty size) Conclusion 20 / 29

63. Partial Models: Towards Modeling and Case Study Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 21 / 29

64. Partial Models: Towards Modeling and Results of Evaluation Reasoning with Uncertainty M.Famelis, R.Salay, Reasoning with Partial models M.Chechik, vs Introduction Reasoning with a set of conventional models Intuition Motivating Example Is there a speedup? Modeling Uncertainty Partial Models – Yes, it is consistently faster than reasoning with the set. Semantics Reasoning With How is speedup aﬀected by changing model size and levels of Uncertainty Property uncertainty? Checking Diagnosis – Speedup decreases with model size. Evaluation Experiments Case Study – Speedup increases with uncertainty. Conclusion – No slowdowns! 22 / 29

65. Partial Models: Towards Modeling and Summary Reasoning with Uncertainty Modeling Uncertainty M.Famelis, R.Salay, M.Chechik, • Encode uncertainty in Partial Models. Introduction • Semantics: sets of conventional models. Intuition Motivating Example Modeling Reasoning in the Presence of Uncertainty Uncertainty Partial Models • Check properties. Semantics Reasoning • Give feedback to facilitate diagnosis. With Uncertainty Property Checking Diagnosis Evaluation of Reasoning Evaluation Experiments • Reasoning with Partial models vs. reasoning with a set of Case Study conventional models Conclusion 23 / 29

66. Partial Models: Towards Modeling and The Big Picture Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 24 / 29

67. Partial Models: Towards Modeling and Next Steps Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion 25 / 29

68. Questions?

69. Partial Models: Towards Modeling and Language Independent! Reasoning with Uncertainty M.Famelis, R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Class Diagram example from [MiSE’12]. Conclusion 27 / 29

70. Partial Models: Towards Modeling and Bibliography I Reasoning with Uncertainty P. Classen, A.and Heymans, P.Y. Schobbens, A. Legay, and J.F. Raskin. “Model Checking Lots of Systems: Efficient Verification of Temporal M.Famelis, Properties in Software Product Lines”. R.Salay, M.Chechik, In Proc. of ICSE’10, pages 335–344, 2010. M. Famelis, Shoham Ben-David, Marsha Chechik, and Rick Salay. Introduction “Partial Models: A Position Paper”. Intuition Motivating In Proc. of MoDeVVa’11, pages 1–6, 2011. Example Modeling M. Famelis, R. Salay, and M. Chechik. Uncertainty “The Semantics of Partial Model Transformations”. Partial Models In Proc. of MiSE’12, 2012. Semantics Reasoning K. G. Larsen and B. Thomsen. With “A Modal Process Logic”. Uncertainty In Proc. of LICS’88, pages 203–210, 1988. Property Checking Diagnosis P. Larsen. Evaluation “The Expressive Power of Implicit Specifications”. Experiments In Proc. of ICALP’91, volume 510 of LNCS, pages 204–216, 1991. Case Study Conclusion M.Famelis, R.Salay, and M. Chechik. “The Semantics of Partial Model Transformations”. In Proc. of MiSE’12, pages 546–560, 2012. 28 / 29

71. Partial Models: Towards Modeling and Bibliography II Reasoning with Uncertainty M.Famelis, B. Morin, G. Perrouin, P. Lahire, O. Barais, G. Vanwormhoudt, and J. M. R.Salay, J´z´quel. e e M.Chechik, “Weaving Variability into Domain Metamodels”. J. Model Driven Engineering Languages and Systems, pages 690–705, 2009. Introduction Intuition R.Salay, M. Chechik, and J.Horkoff. Motivating Example “Managing Requirements Uncertainty with Partial Models”. In Proc. of RE’12, pages 546–560, 2012. Modeling Uncertainty R. Salay, M. Chechik, and J. Gorzny. Partial Models Semantics “Towards a Methodology for Verifying Partial Model Refinements”. Reasoning In Proc. of VOLT’12, 2012. With Uncertainty R. Salay, M. Famelis, and M. Chechik. Property “Language Independent Refinement using Partial Modeling”. Checking Diagnosis In Proc. of FASE’12, 2012. Evaluation R. V. D. Straeten, T. Mens, J. Simmonds, and V. Jonckers. Experiments “Using Description Logic to Maintain Consistency between UML Models”. Case Study In Proc. of UML’03, pages 326–340, 2003. Conclusion 29 / 29

Partial Models: Towards Modeling and Reasoning with Uncertainty

Recommended

Recommended

More Related Content

Similar to Partial Models: Towards Modeling and Reasoning with Uncertainty

Similar to Partial Models: Towards Modeling and Reasoning with Uncertainty (6)

More from Michalis Famelis

More from Michalis Famelis (6)

Recently uploaded

Recently uploaded (20)

Partial Models: Towards Modeling and Reasoning with Uncertainty