Uncertainty Management With Partial Models

610 views
537 views

Published on

MODELS2012 Doctoral Symposium

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
610
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Uncertainty Management With Partial Models

  1. 1. UncertaintyManagementWith Partial Models M. FamelisIntroductionModelingUncertainty Uncertainty Management With Partial ModelsManagingUncertaintyCompleted WorkCurrent Work Michalis FamelisFuture WorkEvalutation University of TorontoConclusion October 2nd, 2012, MODELS’12 Doctoral Symposium 1 / 33
  2. 2. UncertaintyManagementWith Partial Models Hi, I’m Michalis Famelis M. FamelisIntroduction • 3rd year PhD student.ModelingUncertaintyManaging • ...“a couple years to finish”UncertaintyCompleted WorkCurrent WorkFuture WorkEvalutation • Supervisor: Marsha ChechikConclusion • MSc: Model Management with Relation Types, CS UofT 2010. • Undergrad: Model analysis and transformation. ECE NTUA 2008. 2 / 33
  3. 3. UncertaintyManagementWith Partial Models Uncertainty in SE M. FamelisIntroductionModeling “The reality of today’s software systems requires us to considerUncertainty uncertainty as a first-class concern in the design, implementation,ManagingUncertainty and deployment of those systems.” [Garlan, 2010]Completed WorkCurrent WorkFuture WorkEvalutation • Self-adaptive systemsConclusion [Esfahani et al., 2011, Cheng and Garlan, 2007] • Probabilistic systems [Hinton et al., 2006] • Imperfect information in requirements [Noppen et al., 2007] • Requirements clarifications [Knauss et al., 2012] • Risk management [Islam and Houmb, 2010] 3 / 33
  4. 4. UncertaintyManagementWith Partial Models Design Uncertainty M. FamelisIntroduction Uncertainty about design decisions – the contents of a model.ModelingUncertaintyManaging Ultimately, we need to capture and support design uncertainty.UncertaintyCompleted WorkCurrent WorkFuture Work Enable MDSE with design uncertainty:Evalutation • Create representationsConclusion • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 4 / 33
  5. 5. UncertaintyManagementWith Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram.IntroductionModeling What does the modeler know?UncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 5 / 33
  6. 6. UncertaintyManagementWith Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram.IntroductionModeling What does the modeler not know?UncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 5 / 33
  7. 7. UncertaintyManagementWith Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram.IntroductionModeling What does the modeler not know?UncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 5 / 33
  8. 8. UncertaintyManagementWith Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram.IntroductionModeling What does the modeler not know?UncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 5 / 33
  9. 9. UncertaintyManagementWith Partial Models What is Design Uncertainty M. Famelis Example: a simple class diagram.IntroductionModeling What does the modeler not know?UncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 5 / 33
  10. 10. UncertaintyManagementWith Partial Models Dealing with Uncertainty: Options M. Famelis Classical:IntroductionModelingUncertainty 1 Wait until uncertainty is lifted.Managing Inefficient; under-utilization of resources.UncertaintyCompleted WorkCurrent WorkFuture Work 2 Plan for all possible solutions.Evalutation Intractable: they may be too many.Conclusion 3 Make a decision. Now. Artificial; risk undo or premature commitments. We propose: 4 “Live” with uncertainty, defer resolution until more information becomes available. 6 / 33
  11. 11. UncertaintyManagementWith Partial Models Design Uncertainty M. Famelis Uncertainty pervasive in MDSE.IntroductionModeling But also:Uncertainty Uncertainty about design decisions – the contents of a model.ManagingUncertaintyCompleted Work Ultimately, we need to capture and support design uncertainty.Current WorkFuture WorkEvalutation Enable MDSE with design uncertainty:Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 7 / 33
  12. 12. UncertaintyManagementWith Partial Models Modeling Formalisms to Represent M. Famelis UncertaintyIntroduction ModalModelingUncertainty Behavioral • Restricted to one model type only.Managing SpecificationUncertainty • Only optional elements.Completed WorkCurrent Work e.g.: • Over-approximation of set:Future Work MTSs [Larsen’88] Information lossEvalutation DMTSs [Larsen’90]Conclusion Variability • Different conceptual domain (SPLs). Modeling • Not rich enough. Eg: – No variable meta-attributes. e.g.: (e.g. variable container class) FTSs [Classen’10] – No open world option. TVL [Boucher’10] Clafer [Bak’10] • More heavyweight (maintain an SPL for ephemeral uncertainty?) 8 / 33
  13. 13. UncertaintyManagementWith Partial Models Uncertainty: a Set of Possible M. Famelis RefinementsIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutation If we remove all uncertainty, we have a concrete refinement.Conclusion 9 / 33
  14. 14. UncertaintyManagementWith Partial Models Modeling Uncertainty with Partial M. Famelis ModelsIntroduction Explicating uncertainty in a partial model.ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 10 / 33
  15. 15. UncertaintyManagementWith Partial Models Modeling Uncertainty with Partial M. Famelis ModelsIntroduction Explicating uncertainty in a partial model.ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion • May: Element is optional. 10 / 33
  16. 16. UncertaintyManagementWith Partial Models Modeling Uncertainty with Partial M. Famelis ModelsIntroduction Explicating uncertainty in a partial model.ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. 10 / 33
  17. 17. UncertaintyManagementWith Partial Models Modeling Uncertainty with Partial M. Famelis ModelsIntroduction Explicating uncertainty in a partial model.ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. • Var: Element can be merged with others. 10 / 33
  18. 18. UncertaintyManagementWith Partial Models Modeling Uncertainty with Partial M. Famelis ModelsIntroduction Explicating uncertainty in a partial model.ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. • Var: Element can be merged with others. • OW: Model is incomplete. 10 / 33
  19. 19. UncertaintyManagementWith Partial Models Modeling Uncertainty with Partial M. Famelis ModelsIntroduction Explicating uncertainty in a partial model.ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion • May: Element is optional. • Abs: Element can be multiplied to many copies. • Var: Element can be merged with others. • OW: Model is incomplete. 10 / 33
  20. 20. UncertaintyManagementWith Partial Models “Extended“ Partial Models M. Famelis Additional constraints to correlate points of uncertainty.Introduction e.g. May Model: variant presented in [ICSE’12]:ModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 11 / 33
  21. 21. UncertaintyManagementWith Partial Models Semantics of Partial Models M. FamelisIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 12 / 33
  22. 22. UncertaintyManagementWith Partial Models Semantics of Partial Models M. FamelisIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 12 / 33
  23. 23. UncertaintyManagementWith Partial Models Semantics of Partial Models M. FamelisIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion 12 / 33
  24. 24. UncertaintyManagementWith Partial Models Compared to Related Formalisms M. FamelisIntroduction Partial models vs Modal behavioral specification formalismsModelingUncertainty • Language/metamodel independent.ManagingUncertainty • Many kinds of partiality.Completed WorkCurrent Work • Exact representation of set.Future WorkEvalutationConclusion Partial models vs Variability modeling formalisms • Specific to uncertainty. (c.f. uncertainty-removing refinement). • Rich formalism. • Ultimately disposable. 13 / 33
  25. 25. UncertaintyManagementWith Partial Models Status M. Famelis Uncertainty pervasive in MDSE.IntroductionModeling But also:Uncertainty Uncertainty about design decisions – the contents of a model.ManagingUncertaintyCompleted Work Ultimately, we need to capture and support design uncertainty.Current WorkFuture WorkEvalutation Enable MDSE with design uncertainty:Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 14 / 33
  26. 26. UncertaintyManagementWith Partial Models Completed Work I M. FamelisIntroduction Explicating uncertaintyModelingUncertainty • May models (M-elements + May formula) [ICSE’12]ManagingUncertainty • MAVO models + FOL semantics [FASE’12]Completed WorkCurrent Work • Relational-algebraic encoding of MAVO [MoDeVVa’12]Future WorkEvalutationConclusion Checking properties • Verifying May models with SAT [ICSE’12] • Verifying MAVO models with Alloy [FASE’12] • Verifying MAV- models with Alloy, CSP, SMT and ASP [MoDeVVa’12] 15 / 33
  27. 27. UncertaintyManagementWith Partial Models Completed Work II M. FamelisIntroductionModeling Uncertainty-removing refinementUncertainty • Semantics and verification [FASE’12]ManagingUncertainty • Property-driven refinement [ICSE’12]Completed WorkCurrent WorkFuture WorkEvalutationConclusion Tooling • Python and MathSAT [ICSE’12] • Alloy [FASE’12] • Translations from annotated Ecore to RA and from RA to SMT, CSP, ASP, Alloy [MoDeVVa’12] 16 / 33
  28. 28. UncertaintyManagementWith Partial Models Status M. Famelis Uncertainty pervasive in MDSE.IntroductionModeling But also:Uncertainty Uncertainty about design decisions – the contents of a model.ManagingUncertaintyCompleted Work Ultimately, we need to capture and support design uncertainty.Current WorkFuture WorkEvalutation Enable MDSE with design uncertainty:Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 17 / 33
  29. 29. UncertaintyManagementWith Partial Models Lifting Transformations [MiSE’12] M. Famelis May models + Graph transformationsIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion Q1: How do we transform M directly to N? – Lifted semantics of transformations, using logic. (“transfer predicates”) Q2: Are the concretizations of N exactly the models n1 . . . nk ? – Check equivalence of encodings using SAT. (“correctness criterion”) 18 / 33
  30. 30. UncertaintyManagementWith Partial Models Transformations: Next Steps M. FamelisIntroduction • Compositionally test Correctness Criterion.Modeling – Reuse Alloy-based verification method from [VOLT’12]?UncertaintyManagingUncertaintyCompleted Work • Systematically create Transfer Predicates using FOL.Current WorkFuture Work – Predicates for atomic operations as building blocks.Evalutation – May be hard to simplify resulting formulas.Conclusion • Handle expanding/contracting model vocabularies. – E.g. adding a new element vs. concretizing. • Reuse results from Category Theory? [Ehrig et al., 2006] – Generalize to full MAVO. – What are the right morphisms? 19 / 33
  31. 31. UncertaintyManagementWith Partial Models Tool support M. Famelis Full MAVO support for Ecore-based model editors.Introduction • As an EMF Profile? [Langer et al., 2011]ModelingUncertainty • Via RAMification of metamodels? [K¨hne et al., 2009] uManaging – Would solve issues with concrete syntaxUncertaintyCompleted Work (e.g. when the base model is not well-formed)Current WorkFuture WorkEvalutation Model management supportConclusion • Integration with the Model Management Tool Framework. [Salay et al., 2007] • Inherit macro-management tools for partial models – name-based matching – structural merging – etc... • “Push-button” integration of reasoning. 20 / 33
  32. 32. UncertaintyManagementWith Partial Models Status M. Famelis Uncertainty pervasive in MDSE.IntroductionModeling But also:Uncertainty Uncertainty about design decisions – the contents of a model.ManagingUncertaintyCompleted Work Ultimately, we need to capture and support design uncertainty.Current WorkFuture WorkEvalutation Enable MDSE with design uncertainty:Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 21 / 33
  33. 33. UncertaintyManagementWith Partial Models Guidelines for Encoding M. Famelis UncertaintyIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture WorkEvalutationConclusion “Placement” “Alternatives” “Bottom-up” patterns of uncertainty. • Help guide efficient/correct encoding. • Support for choice of suitable reasoners. 22 / 33
  34. 34. UncertaintyManagementWith Partial Models Towards a Methodology for Design M. Famelis UncertaintyIntroductionModelingUncertaintyManaging We know how to do reasoning, refinement, transformationsUncertainty under uncertainty.Completed WorkCurrent WorkFuture WorkEvalutation We need to help identify what sorts of reasoning, refinement,Conclusion etc. are necessary in each SE setting. “Top-down” patterns of uncertainty: • How does it appear in SE? • What are its sources? • What are its characteristic properties? • What are refinements/transformations are sensible? 23 / 33
  35. 35. UncertaintyManagementWith Partial Models A Half Baked Example: M. Famelis “Disagreement Pattern”Introduction Potential sources:ModelingUncertainty • Conflicting stakeholder requirements.ManagingUncertainty • Merge conflicts after commits.Completed WorkCurrent Work • View-updates.Future WorkEvalutation Characteristic properties:Conclusion • Is the conflict reconciliable? • Have we reached agreement? Sensible refinement and transformation checks: • Does a refactoring expose/conceal the source of disagreement? • Is a series of refinements a negotiation? (“are they making progress removing uncertainty?”) 24 / 33
  36. 36. UncertaintyManagementWith Partial Models Outline M. FamelisIntroduction “....enable MDSE with design uncertainty:”ModelingUncertainty • Create representationsManagingUncertainty • Check properties and give feedbackCompleted WorkCurrent WorkFuture Work • Do uncertainty-removing refinementEvalutation • Enable transformationsConclusion • Create tooling support • Create methodological support How do we go about evaluating all this? 25 / 33
  37. 37. UncertaintyManagementWith Partial Models Evaluation of Individual Aspects M. FamelisIntroduction Experiments using randomly generated inputsModeling • To validate feasibility and Scalability of Property CheckingUncertainty [ICSE’12][MoDeVVa’12]ManagingUncertainty • Experimental driver in MMTF for rapid experiment setups.Completed WorkCurrent WorkFuture WorkEvalutation Case StudiesConclusion • Triangulated results of random experimentation in [ICSE’12]. • Realistic bugfix of real world bug in UMLet Theorem Proving using Alloy • Used to verify partial model refinements in [VOLT’12] • Technique could be adapted for proving transformations 26 / 33
  38. 38. UncertaintyManagementWith Partial Models Evaluation of the Entire Approach M. FamelisIntroductionModelingUncertainty Plan for a Power Window Case StudyManaging • Comprehensive case study from the Automotive domain.UncertaintyCompleted Work • Management of uncertainty throughout the MDE lifecycle.Current WorkFuture Work Requirements to implementation via transformations.EvalutationConclusion However: • Is such a case study enough? • Best method for evaluation not clear. • How to evaluate “green-field” research? 27 / 33
  39. 39. UncertaintyManagementWith Partial Models Summary M. Famelis Uncertainty pervasive in MDSE.IntroductionModeling Not well studied:Uncertainty Uncertainty about design decisions – the contents of a model.ManagingUncertaintyCompleted Work Ultimately, we need to capture and support design uncertainty.Current WorkFuture WorkEvalutation Enable MDSE with design uncertainty:Conclusion • Create representations • Check properties and give feedback • Do uncertainty-removing refinement • Enable transformations • Create tooling support • Create methodological support 28 / 33
  40. 40. Questions?
  41. 41. UncertaintyManagementWith Partial Models Bibliography I M. Famelis Bak, K., Czarnecki, K., and Wasowski, A. (2011).Introduction Feature and meta-models in clafer: Mixed, specialized, and coupled. In SLE’10, pages 102–122.ModelingUncertainty Boucher, Q., Classen, A., Faber, P., and Heymans, P. (2010). Introducing TVL, a text-based feature modelling language.Managing In VaMoS’10, pages 159–162.UncertaintyCompleted Work Cheng, S.-W. and Garlan, D. (2007).Current Work Handling uncertainty in autonomic systems.Future Work In ASE’07.Evalutation Classen, A., Heymans, P., Schobbens, P.-Y., Legay, A., and Raskin, J.-F. (2010).Conclusion Model checking lots of systems: efficient verification of temporal properties in software product lines. In ICSE’10, pages 335–344. Ehrig, H., Ehrig, K., Prange, U., and Taentzer, G. (2006). Fundamentals of Algebraic Graph Transformation. EATCS. Springer. Esfahani, N., Kouroshfar, E., and Malek, S. (2011). Taming uncertainty in self-adaptive software. In ESEC/FSE ’11, pages 234–244. Famelis, M., Ben-David, S., Chechik, M., and Salay, R. (2011). “Partial Models: A Position Paper”. In Proceedings of MoDeVVa’11, pages 1–6. 30 / 33
  42. 42. UncertaintyManagementWith Partial Models Bibliography II M. Famelis Famelis, M., Chechik, M., and Salay, R. (2012a).Introduction “Partial Models: Towards Modeling and Reasoning with Uncertainty”. In Proceedings of ICSE’12.ModelingUncertainty Famelis, M., Salay, R., and Chechik, M. (2012b). The semantics of partial model transformations.Managing In MISE at ICSE’12, pages 64 –69.UncertaintyCompleted Work Garlan, D. (2010).Current Work Software engineering in an uncertain world.Future Work In FoSER ’10, pages 125–128.Evalutation Hinton, A., Kwiatkowska, M., Norman, G., and Parker, D. (2006).Conclusion Prism: A tool for automatic verification of probabilistic systems. In Hermanns, H. and Palsberg, J., editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 3920 of Lecture Notes in Computer Science, pages 441–444. Islam, S. and Houmb, S. H. (2010). Integrating risk management activities into requirements engineering. In RCIS’10, pages 299–310. Knauss, E., Damian, D., Poo-Caamano, G., and Cleland-Huang, J. (2012). Detecting and classifying patterns of requirements clarifications. In RE’12. K¨hne, T., Mezei, G., Syriani, E., Vangheluwe, H., and Wimmer, M. (2009). u Explicit transformation modeling. In MODELS’09, pages 240–255. 31 / 33
  43. 43. UncertaintyManagementWith Partial Models Bibliography III M. Famelis Langer, P., Wieland, K., Wimmer, M., and Cabot, J. (2011). From uml profiles to emf profiles and beyond.Introduction In Objects, Models, Components, Patterns, volume 6705, pages 52–67.Modeling Larsen, K. G. and Thomsen, B. (1988).Uncertainty “A Modal Process Logic”.Managing In Proceedings of LICS’88, pages 203–210.Uncertainty Larsen, K. G. and Xinxin, L. (1990).Completed WorkCurrent Work “Equation Solving Using Modal Transition Systems”.Future Work In Proc. of LICS’90, pages 108–117.Evalutation Noppen, J., van den Broek, P., and Akşit, M. (2007). Software development with imperfect information.Conclusion Soft Comput., 12(1):3–28. Saadatpanah, P., Famelis, M., Gorzny, J., Robinson, N., Chechik, M., and Salay, R. (2012). Comparing the effectiveness of reasoning formalisms for partial models. In MoDeVVa’12. Salay, R., Chechik, M., Easterbrook, S., Diskin, Z., McCormick, P., Nejati, S., Sabetzadeh, M., and Viriyakattiyaporn, P. (2007). An eclipse-based tool framework for software model management. In Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange, eclipse ’07, pages 55–59. Salay, R., Chechik, M., and Gorzny, J. (2012a). “Towards a Methodology for Verifying Partial Model Refinements”. In Proceedings of VOLT’12. 32 / 33
  44. 44. UncertaintyManagementWith Partial Models Bibliography IV M. FamelisIntroductionModelingUncertaintyManagingUncertaintyCompleted WorkCurrent WorkFuture Work Salay, R., Famelis, M., and Chechik, M. (2012b). “Language Independent Refinement using Partial Modeling”.Evalutation In Proceedings of FASE’12.Conclusion 33 / 33

×