Successfully reported this slideshow.

Uncertainty Management With Partial Models

1

Share

1 of 44
1 of 44

More Related Content

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Uncertainty Management With Partial Models

  1. 1. Uncertainty Management With Partial Models M. Famelis Introduction Modeling Uncertainty Uncertainty Management With Partial Models Managing Uncertainty Completed Work Current Work Michalis Famelis Future Work Evalutation University of Toronto Conclusion October 2nd, 2012, MODELS’12 Doctoral Symposium 1 / 33
  2. 2. Uncertainty Management With Partial Models Hi, I’m Michalis Famelis M. Famelis Introduction • 3rd year PhD student. Modeling Uncertainty Managing • ...“a couple years to finish” Uncertainty Completed Work Current Work Future Work Evalutation • Supervisor: Marsha Chechik Conclusion • MSc: Model Management with Relation Types, CS UofT 2010. • Undergrad: Model analysis and transformation. ECE NTUA 2008. 2 / 33
  3. 3. Uncertainty Management With Partial Models Uncertainty in SE M. Famelis Introduction Modeling “The reality of today’s software systems requires us to consider Uncertainty uncertainty as a first-class concern in the design, implementation, Managing Uncertainty and deployment of those systems.” [Garlan, 2010] Completed Work Current Work Future Work Evalutation • Self-adaptive systems Conclusion [Esfahani et al., 2011, Cheng and Garlan, 2007] • Probabilistic systems [Hinton et al., 2006] • Imperfect information in requirements [Noppen et al., 2007] • Requirements clarifications [Knauss et al., 2012] • Risk management [Islam and Houmb, 2010] 3 / 33
  4. 4. Uncertainty Management With Partial Models Design Uncertainty M. Famelis Introduction Uncertainty about design decisions – the contents of a model. Modeling Uncertainty Managing Ultimately, we need to capture and support design uncertainty. Uncertainty Completed Work Current Work Future Work Enable MDSE with design uncertainty: Evalutation • Create representations Conclusion • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 4 / 33
  5. 5. Uncertainty Management With Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram. Introduction Modeling What does the modeler know? Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 5 / 33
  6. 6. Uncertainty Management With Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram. Introduction Modeling What does the modeler not know? Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 5 / 33
  7. 7. Uncertainty Management With Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram. Introduction Modeling What does the modeler not know? Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 5 / 33
  8. 8. Uncertainty Management With Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram. Introduction Modeling What does the modeler not know? Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 5 / 33
  9. 9. Uncertainty Management With Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram. Introduction Modeling What does the modeler not know? Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 5 / 33
  10. 10. Uncertainty Management With Partial Models Dealing with Uncertainty: Options M. Famelis Classical: Introduction Modeling Uncertainty 1 Wait until uncertainty is lifted. Managing Inefficient; under-utilization of resources. Uncertainty Completed Work Current Work Future Work 2 Plan for all possible solutions. Evalutation Intractable: they may be too many. Conclusion 3 Make a decision. Now. Artificial; risk undo or premature commitments. We propose: 4 “Live” with uncertainty, defer resolution until more information becomes available. 6 / 33
  11. 11. Uncertainty Management With Partial Models Design Uncertainty M. Famelis Uncertainty pervasive in MDSE. Introduction Modeling But also: Uncertainty Uncertainty about design decisions – the contents of a model. Managing Uncertainty Completed Work Ultimately, we need to capture and support design uncertainty. Current Work Future Work Evalutation Enable MDSE with design uncertainty: Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 7 / 33
  12. 12. Uncertainty Management With Partial Models Modeling Formalisms to Represent M. Famelis Uncertainty Introduction Modal Modeling Uncertainty Behavioral • Restricted to one model type only. Managing Specification Uncertainty • Only optional elements. Completed Work Current Work e.g.: • Over-approximation of set: Future Work MTSs [Larsen’88] Information loss Evalutation DMTSs [Larsen’90] Conclusion Variability • Different conceptual domain (SPLs). Modeling • Not rich enough. Eg: – No variable meta-attributes. e.g.: (e.g. variable container class) FTSs [Classen’10] – No open world option. TVL [Boucher’10] Clafer [Bak’10] • More heavyweight (maintain an SPL for ephemeral uncertainty?) 8 / 33
  13. 13. Uncertainty Management With Partial Models Uncertainty: a Set of Possible M. Famelis Refinements Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation If we remove all uncertainty, we have a concrete refinement. Conclusion 9 / 33
  14. 14. Uncertainty Management With Partial Models Modeling Uncertainty with Partial M. Famelis Models Introduction Explicating uncertainty in a partial model. Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 10 / 33
  15. 15. Uncertainty Management With Partial Models Modeling Uncertainty with Partial M. Famelis Models Introduction Explicating uncertainty in a partial model. Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion • May: Element is optional. 10 / 33
  16. 16. Uncertainty Management With Partial Models Modeling Uncertainty with Partial M. Famelis Models Introduction Explicating uncertainty in a partial model. Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. 10 / 33
  17. 17. Uncertainty Management With Partial Models Modeling Uncertainty with Partial M. Famelis Models Introduction Explicating uncertainty in a partial model. Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. • Var: Element can be merged with others. 10 / 33
  18. 18. Uncertainty Management With Partial Models Modeling Uncertainty with Partial M. Famelis Models Introduction Explicating uncertainty in a partial model. Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. • Var: Element can be merged with others. • OW: Model is incomplete. 10 / 33
  19. 19. Uncertainty Management With Partial Models Modeling Uncertainty with Partial M. Famelis Models Introduction Explicating uncertainty in a partial model. Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. • Var: Element can be merged with others. • OW: Model is incomplete. 10 / 33
  20. 20. Uncertainty Management With Partial Models “Extended“ Partial Models M. Famelis Additional constraints to correlate points of uncertainty. Introduction e.g. May Model: variant presented in [ICSE’12]: Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 11 / 33
  21. 21. Uncertainty Management With Partial Models Semantics of Partial Models M. Famelis Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 12 / 33
  22. 22. Uncertainty Management With Partial Models Semantics of Partial Models M. Famelis Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 12 / 33
  23. 23. Uncertainty Management With Partial Models Semantics of Partial Models M. Famelis Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion 12 / 33
  24. 24. Uncertainty Management With Partial Models Compared to Related Formalisms M. Famelis Introduction Partial models vs Modal behavioral specification formalisms Modeling Uncertainty • Language/metamodel independent. Managing Uncertainty • Many kinds of partiality. Completed Work Current Work • Exact representation of set. Future Work Evalutation Conclusion Partial models vs Variability modeling formalisms • Specific to uncertainty. (c.f. uncertainty-removing refinement). • Rich formalism. • Ultimately disposable. 13 / 33
  25. 25. Uncertainty Management With Partial Models Status M. Famelis Uncertainty pervasive in MDSE. Introduction Modeling But also: Uncertainty Uncertainty about design decisions – the contents of a model. Managing Uncertainty Completed Work Ultimately, we need to capture and support design uncertainty. Current Work Future Work Evalutation Enable MDSE with design uncertainty: Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 14 / 33
  26. 26. Uncertainty Management With Partial Models Completed Work I M. Famelis Introduction Explicating uncertainty Modeling Uncertainty • May models (M-elements + May formula) [ICSE’12] Managing Uncertainty • MAVO models + FOL semantics [FASE’12] Completed Work Current Work • Relational-algebraic encoding of MAVO [MoDeVVa’12] Future Work Evalutation Conclusion Checking properties • Verifying May models with SAT [ICSE’12] • Verifying MAVO models with Alloy [FASE’12] • Verifying MAV- models with Alloy, CSP, SMT and ASP [MoDeVVa’12] 15 / 33
  27. 27. Uncertainty Management With Partial Models Completed Work II M. Famelis Introduction Modeling Uncertainty-removing refinement Uncertainty • Semantics and verification [FASE’12] Managing Uncertainty • Property-driven refinement [ICSE’12] Completed Work Current Work Future Work Evalutation Conclusion Tooling • Python and MathSAT [ICSE’12] • Alloy [FASE’12] • Translations from annotated Ecore to RA and from RA to SMT, CSP, ASP, Alloy [MoDeVVa’12] 16 / 33
  28. 28. Uncertainty Management With Partial Models Status M. Famelis Uncertainty pervasive in MDSE. Introduction Modeling But also: Uncertainty Uncertainty about design decisions – the contents of a model. Managing Uncertainty Completed Work Ultimately, we need to capture and support design uncertainty. Current Work Future Work Evalutation Enable MDSE with design uncertainty: Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 17 / 33
  29. 29. Uncertainty Management With Partial Models Lifting Transformations [MiSE’12] M. Famelis May models + Graph transformations Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion Q1: How do we transform M directly to N? – Lifted semantics of transformations, using logic. (“transfer predicates”) Q2: Are the concretizations of N exactly the models n1 . . . nk ? – Check equivalence of encodings using SAT. (“correctness criterion”) 18 / 33
  30. 30. Uncertainty Management With Partial Models Transformations: Next Steps M. Famelis Introduction • Compositionally test Correctness Criterion. Modeling – Reuse Alloy-based verification method from [VOLT’12]? Uncertainty Managing Uncertainty Completed Work • Systematically create Transfer Predicates using FOL. Current Work Future Work – Predicates for atomic operations as building blocks. Evalutation – May be hard to simplify resulting formulas. Conclusion • Handle expanding/contracting model vocabularies. – E.g. adding a new element vs. concretizing. • Reuse results from Category Theory? [Ehrig et al., 2006] – Generalize to full MAVO. – What are the right morphisms? 19 / 33
  31. 31. Uncertainty Management With Partial Models Tool support M. Famelis Full MAVO support for Ecore-based model editors. Introduction • As an EMF Profile? [Langer et al., 2011] Modeling Uncertainty • Via RAMification of metamodels? [K¨hne et al., 2009] u Managing – Would solve issues with concrete syntax Uncertainty Completed Work (e.g. when the base model is not well-formed) Current Work Future Work Evalutation Model management support Conclusion • Integration with the Model Management Tool Framework. [Salay et al., 2007] • Inherit macro-management tools for partial models – name-based matching – structural merging – etc... • “Push-button” integration of reasoning. 20 / 33
  32. 32. Uncertainty Management With Partial Models Status M. Famelis Uncertainty pervasive in MDSE. Introduction Modeling But also: Uncertainty Uncertainty about design decisions – the contents of a model. Managing Uncertainty Completed Work Ultimately, we need to capture and support design uncertainty. Current Work Future Work Evalutation Enable MDSE with design uncertainty: Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 21 / 33
  33. 33. Uncertainty Management With Partial Models Guidelines for Encoding M. Famelis Uncertainty Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Evalutation Conclusion “Placement” “Alternatives” “Bottom-up” patterns of uncertainty. • Help guide efficient/correct encoding. • Support for choice of suitable reasoners. 22 / 33
  34. 34. Uncertainty Management With Partial Models Towards a Methodology for Design M. Famelis Uncertainty Introduction Modeling Uncertainty Managing We know how to do reasoning, refinement, transformations Uncertainty under uncertainty. Completed Work Current Work Future Work Evalutation We need to help identify what sorts of reasoning, refinement, Conclusion etc. are necessary in each SE setting. “Top-down” patterns of uncertainty: • How does it appear in SE? • What are its sources? • What are its characteristic properties? • What are refinements/transformations are sensible? 23 / 33
  35. 35. Uncertainty Management With Partial Models A Half Baked Example: M. Famelis “Disagreement Pattern” Introduction Potential sources: Modeling Uncertainty • Conflicting stakeholder requirements. Managing Uncertainty • Merge conflicts after commits. Completed Work Current Work • View-updates. Future Work Evalutation Characteristic properties: Conclusion • Is the conflict reconciliable? • Have we reached agreement? Sensible refinement and transformation checks: • Does a refactoring expose/conceal the source of disagreement? • Is a series of refinements a negotiation? (“are they making progress removing uncertainty?”) 24 / 33
  36. 36. Uncertainty Management With Partial Models Outline M. Famelis Introduction “....enable MDSE with design uncertainty:” Modeling Uncertainty • Create representations Managing Uncertainty • Check properties and give feedback Completed Work Current Work Future Work • Do uncertainty-removing refinement Evalutation • Enable transformations Conclusion • Create tooling support • Create methodological support How do we go about evaluating all this? 25 / 33
  37. 37. Uncertainty Management With Partial Models Evaluation of Individual Aspects M. Famelis Introduction Experiments using randomly generated inputs Modeling • To validate feasibility and Scalability of Property Checking Uncertainty [ICSE’12][MoDeVVa’12] Managing Uncertainty • Experimental driver in MMTF for rapid experiment setups. Completed Work Current Work Future Work Evalutation Case Studies Conclusion • Triangulated results of random experimentation in [ICSE’12]. • Realistic bugfix of real world bug in UMLet Theorem Proving using Alloy • Used to verify partial model refinements in [VOLT’12] • Technique could be adapted for proving transformations 26 / 33
  38. 38. Uncertainty Management With Partial Models Evaluation of the Entire Approach M. Famelis Introduction Modeling Uncertainty Plan for a Power Window Case Study Managing • Comprehensive case study from the Automotive domain. Uncertainty Completed Work • Management of uncertainty throughout the MDE lifecycle. Current Work Future Work Requirements to implementation via transformations. Evalutation Conclusion However: • Is such a case study enough? • Best method for evaluation not clear. • How to evaluate “green-field” research? 27 / 33
  39. 39. Uncertainty Management With Partial Models Summary M. Famelis Uncertainty pervasive in MDSE. Introduction Modeling Not well studied: Uncertainty Uncertainty about design decisions – the contents of a model. Managing Uncertainty Completed Work Ultimately, we need to capture and support design uncertainty. Current Work Future Work Evalutation Enable MDSE with design uncertainty: Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 28 / 33
  40. 40. Questions?
  41. 41. Uncertainty Management With Partial Models Bibliography I M. Famelis Bak, K., Czarnecki, K., and Wasowski, A. (2011). Introduction Feature and meta-models in clafer: Mixed, specialized, and coupled. In SLE’10, pages 102–122. Modeling Uncertainty Boucher, Q., Classen, A., Faber, P., and Heymans, P. (2010). Introducing TVL, a text-based feature modelling language. Managing In VaMoS’10, pages 159–162. Uncertainty Completed Work Cheng, S.-W. and Garlan, D. (2007). Current Work Handling uncertainty in autonomic systems. Future Work In ASE’07. Evalutation Classen, A., Heymans, P., Schobbens, P.-Y., Legay, A., and Raskin, J.-F. (2010). Conclusion Model checking lots of systems: efficient verification of temporal properties in software product lines. In ICSE’10, pages 335–344. Ehrig, H., Ehrig, K., Prange, U., and Taentzer, G. (2006). Fundamentals of Algebraic Graph Transformation. EATCS. Springer. Esfahani, N., Kouroshfar, E., and Malek, S. (2011). Taming uncertainty in self-adaptive software. In ESEC/FSE ’11, pages 234–244. Famelis, M., Ben-David, S., Chechik, M., and Salay, R. (2011). “Partial Models: A Position Paper”. In Proceedings of MoDeVVa’11, pages 1–6. 30 / 33
  42. 42. Uncertainty Management With Partial Models Bibliography II M. Famelis Famelis, M., Chechik, M., and Salay, R. (2012a). Introduction “Partial Models: Towards Modeling and Reasoning with Uncertainty”. In Proceedings of ICSE’12. Modeling Uncertainty Famelis, M., Salay, R., and Chechik, M. (2012b). The semantics of partial model transformations. Managing In MISE at ICSE’12, pages 64 –69. Uncertainty Completed Work Garlan, D. (2010). Current Work Software engineering in an uncertain world. Future Work In FoSER ’10, pages 125–128. Evalutation Hinton, A., Kwiatkowska, M., Norman, G., and Parker, D. (2006). Conclusion Prism: A tool for automatic verification of probabilistic systems. In Hermanns, H. and Palsberg, J., editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 3920 of Lecture Notes in Computer Science, pages 441–444. Islam, S. and Houmb, S. H. (2010). Integrating risk management activities into requirements engineering. In RCIS’10, pages 299–310. Knauss, E., Damian, D., Poo-Caamano, G., and Cleland-Huang, J. (2012). Detecting and classifying patterns of requirements clarifications. In RE’12. K¨hne, T., Mezei, G., Syriani, E., Vangheluwe, H., and Wimmer, M. (2009). u Explicit transformation modeling. In MODELS’09, pages 240–255. 31 / 33
  43. 43. Uncertainty Management With Partial Models Bibliography III M. Famelis Langer, P., Wieland, K., Wimmer, M., and Cabot, J. (2011). From uml profiles to emf profiles and beyond. Introduction In Objects, Models, Components, Patterns, volume 6705, pages 52–67. Modeling Larsen, K. G. and Thomsen, B. (1988). Uncertainty “A Modal Process Logic”. Managing In Proceedings of LICS’88, pages 203–210. Uncertainty Larsen, K. G. and Xinxin, L. (1990). Completed Work Current Work “Equation Solving Using Modal Transition Systems”. Future Work In Proc. of LICS’90, pages 108–117. Evalutation Noppen, J., van den Broek, P., and Akşit, M. (2007). Software development with imperfect information. Conclusion Soft Comput., 12(1):3–28. Saadatpanah, P., Famelis, M., Gorzny, J., Robinson, N., Chechik, M., and Salay, R. (2012). Comparing the effectiveness of reasoning formalisms for partial models. In MoDeVVa’12. Salay, R., Chechik, M., Easterbrook, S., Diskin, Z., McCormick, P., Nejati, S., Sabetzadeh, M., and Viriyakattiyaporn, P. (2007). An eclipse-based tool framework for software model management. In Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange, eclipse ’07, pages 55–59. Salay, R., Chechik, M., and Gorzny, J. (2012a). “Towards a Methodology for Verifying Partial Model Refinements”. In Proceedings of VOLT’12. 32 / 33
  44. 44. Uncertainty Management With Partial Models Bibliography IV M. Famelis Introduction Modeling Uncertainty Managing Uncertainty Completed Work Current Work Future Work Salay, R., Famelis, M., and Chechik, M. (2012b). “Language Independent Refinement using Partial Modeling”. Evalutation In Proceedings of FASE’12. Conclusion 33 / 33

×