SlideShare a Scribd company logo
1 of 25
Research Questions for Validation
and Verification in the Context of
Model-Based Engineering
Catherine Dubois
Michalis Famelis
Martin Gogolla
Leonel Nobrega
Ileana Ober
Martina Seidl
Markus Voelter
ENSIIE, Evry, France
University of Toronto,
University of Bremen,
University of Madeira, Funchal,
University of Toulouse,
Johannes Kepler University Linz, Austria
Voelter Ingenieurburo, Heidenheim, Germany
October 1st , 2013
MoDeVVa 2013, Miami, USA
Introduction
Abstraction techniques are one of the promising paths for
the future advances in the field of verification.
– Clarke, Emerson, Sifakis: Turing Lecture 2008
V&V crucial for MBE
– Uncover hidden properties and errors
– Verify transformations
– Ensure quality, etc.
2
About
• This paper is the result of the working group on V & V at
Dagstuhl-Seminar 13182 held in May 2013.
• Dagstuhl: scientific retreat
in western Germany
• Seminar topic:
Meta-Modeling Model-
Based Engineering Tools
• Three break-outs: Informal
Modeling, Compositionality,
Modeling and V&V
3
4
Our goal:
Identify main areas in the synergy between MBE and
V&V where we need to focus research. What is the
status, what are the research questions?
• Two-day
collaborative
brainstorming
workshop.
• Culminated in a
plenary
presentation
…and this report
Disclaimer
• No claim of completeness or exhaustiveness
• Represents the informed opinions of the authors
• May have missed existing answers to some
questions
• Is this the right level of granularity / level of
detail?
5
Thematic Categories
1. Gap between Models and V&V Formalisms
2. Need to Refine Existing Methodologies
3. Design-time vs. Runtime
4. Properties
5. Model Transformations
6. Informal vs. Formal vs. Incomplete Modeling
7. Comparison and Benchmarking
8. Domain-Specific Languages
6
Models and V&V Formalisms (context)
Model Property
Model' Property'
Designer Level
(can be Domain-
Specific)
Verification
formalism level
Verification engine
Verification
feed-back
7
Transformations
and traceability
mechanisms
But behavioral
semantics often leads
to non-bijective
correspondences
between
design time and
runtime artifacts.
V&V tool configuration
Models and V&V Formalisms
(questions)
• How to express properties at the level of models in a way
understandable to clients?
• How to formulate models and properties in a single language
transparent to clients?
• How to report the V&V results and diagnostics in an appropriate
form to clients?
• How to bridge the gap between formally expressed and verified
properties on one side and client attention on the other side?
• Can modeling language extensions help in making explicit the
“needs” of V&V machines?
Refining Existing Methodologies
(context)
• Integrating V&V in development can support early V&V
similar to how debugging is offered by IDEs.
• Generic methodologies identify points where V&V can be
used.
• A Model-based development methodology would need to
allow variations based on application domain, nature of
project, etc.
• Goal: better identify which V&V activities are meaningful at
the various phases of design; take full advantage of V&V
engines.
9
Refining Existing Methodologies
(questions)
• How do we integrate V&V in the overall development
and modeling process?
– On the technical level of tool exchange?
– On the methodological level of using the right technique at
the right time for the right task?
• When are techniques like animation, execution,
symbolic evaluation, testing, simulation, proving or test
case generation used most efficiently during
development?
– For which model and model transformation properties can
they be employed?
10
Design-time vs. Runtime (context)
• Models are specified at design time
• During execution these models are instantiated
• The dynamic nature of structure during execution
makes it difficult to understand and represent
runtime information
• Existing modeling environments offer limited
support for precisely specifying instantiation and
snapshots
11
Design-time vs. Runtime (questions)
 During the V&V phase, how do we obtain an initial model
instantiation?
 How do we obtain large and meaningful instantiations?
 How do we connect design time and runtime artifacts?
 How do we deal with the issue of scalability in the context
of V&V ?
 How do we handle time and space concerns w.r.t. design
time and runtime artifacts?
 How do we automatically or semi-automatically manage
the V&V machine configuration?
12
Properties (context)
• Model and model transformation properties relevant in V&V:
– consistency, reachability, dependence, minimality, conformance, safety,
liveness, deadlock freeness, termination, confluence, correctness
• Confusion caused by:
– Different kinds according to the nature of the model (static/ dynamic),
its level of abstraction, etc.
– Many tools and techniques (potentially complementary)
Main challenge:
What kind of property to verify on which model at what stage with what
kind of technique?
13
Properties (questions)
• What are the benefits and trade-offs between expressing
properties on more abstract modeling levels in contrast to
expressing them on more concrete levels?
• How do we find the right techniques for uncovering static and
dynamic model properties?
• Which techniques are appropriate for uncovering static modeling
language inherent properties, which for static model-specific
properties?
• Which techniques are appropriate for uncovering dynamic generic
properties, which for dynamic model-specific properties?
14
Model Transformations (context)
• Core component of MBE
• Many applications:
– Maintaining inter-model consistency
– Semantics definition of (domain-specific) modelling languages
– …
• Challenge: Verification of model transformations
– Proving correctness, termination, confluence
– What are the differences to “normal” code?
– Is the higher abstraction level beneficial to V&V?
15
Model Transformations (questions)
• What verification techniques are meaningful for verifying
model transformations?
• How do we analyse properties like confluence and
termination?
• How do we analyse correctness of model transformations
w.r.t. a transformation contract?
• How do we infer a transformation contract from a model
transformation?
16
Informal vs. Formal vs. Incomplete
Modeling (context)
• During V&V: switch on or off particular model elements (in class diagrams, e.g.,
multiplicities); configure constraints by negating, deactivating or activating them
(in class diagrams, e.g., class invariants)
• Different types of granularity (a) all model elements may be relaxed (b) only a
manual model element selection can be considered for relaxation (c) a semi-
automatic element selection for relaxation may be offered
• Ultimate vision: sliders on the user interface to gradually go from a strict, formal
model through various intermediate levels to a totally relaxed and informal model;
fewer formal model parts activated means more informality in the model
• Minimal formal frame for test case construction must be preserved: e.g., for class
diagrams of central classes and associations and for state charts of central states
and transitions
• For a completely informal model no formal scenario (no test cases) can be
formulated
17
Informal vs. Formal vs. Incomplete
Modeling (questions)
• How do we leverage informal assumptions found in sketches for
exploratory V&V?
• Are informal sketches close enough to V&V at all?
• What are appropriate relaxation mechanisms for different degrees
of formality?
• How do we handle incomplete or partial models w.r.t. V&V?
• How do we deactivate and activate model units?
• How do we handle the exploration of model properties and
alternatives?
18
Comparison and Benchmarking
(context)
• Benchmarking can boost research:
– Fair comparison of tools
– Impartial benchmark selection which covers the spectrum
of interesting test cases
– Clear documentation of outcomes; reproducibility
– Publicity makes it easy to identify progress, problems
• However, in MBE:
– No common standards
– No community platform for benchmarks
– Not clear what metrics are relevant for measuring
“improvement”
19
Comparison and Benchmarking
(questions)
• How to compare existing V&V tools w.r.t.
functionality, coverage, scalability,
expressiveness, executing system (i.e., for models
at runtime)?
• Which criteria are appropriate for comparison?
• Can we globally compare fairly at all?
– Broad and diverse spectrum of V&V machines:
B, Coq, HOL/Isabelle, SAT, SMT, CSP solvers, Relational logic,
enumerative techniques
20
Domain-Specific Languages
(Context)
• Most verification tools have hard to use input languages,
alien to normal developers.
– Hence, verification tools are often not used.
• MBE approaches become more and more mainstream.
• Models can simplify analysis and verification, because of
the higher degree of domain semantics they express.
• Potential to exploit the two approaches synergistically
– From the high-level models, we can automate the
generation of the input to the verification tools.
21
Domain-Specific Languages
(Questions)
• How to define DSLs close to domain concepts but still allow
generation of meaningful input for V&V tools?
• V&V tools need the specification of properties.
– How to express them at the domain level in a user-friendly way?
– Can the property specifications be integrated with the same
DSL and/or model used for describing the to-be-verified system
without creating self-fulfilling prophecies?
• How to bring V&V feedback back to the domain level and
express it in terms of the DSL-level input?
• Can incremental languages extensions help with making
programs expressed in general-purpose languages more
checkable? For example, the semantics of a specific
extension construct may enable the generation of very rich
inputs to the verification tool.
22
Thematic Categories
1. Gap between Models and
V&V Formalisms
2. Need to Refine Existing
Methodologies
3. Design-time vs. Runtime
4. Properties
5. Model Transformations
6. Informal vs. Formal vs. Incomplete
Modeling
7. Comparison and Benchmarking
8. Domain-Specific Languages
23
Conclusion
• Report-back from working group on MBE and V&V at
Dagstuhl in May 2013.
• Main areas in the synergy of MBE and V&V.
• For each: described status and identified research
questions.
• Our hope:
Spark discussions and debate, help focus
research in MBE and V&V.
24
Questions?
25

More Related Content

Similar to Research Questions for Validation and Verification in the Context of Model-Based Engineering

24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)
24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)
24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)
University of Rennes, INSA Rennes, Inria/IRISA, CNRS
 
W4 ucl@md day2011
W4 ucl@md day2011W4 ucl@md day2011
W4 ucl@md day2011
MDDAY11
 
Discussion Post 1A software process model is a streamlined port.docx
Discussion Post 1A software process model is a streamlined port.docxDiscussion Post 1A software process model is a streamlined port.docx
Discussion Post 1A software process model is a streamlined port.docx
madlynplamondon
 
GSEP - PROCESS AND CHECKPOINT
GSEP - PROCESS AND CHECKPOINTGSEP - PROCESS AND CHECKPOINT
GSEP - PROCESS AND CHECKPOINT
Alex Himmelberg
 

Similar to Research Questions for Validation and Verification in the Context of Model-Based Engineering (20)

24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)
24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)
24 Reasons Why Variability Models Are Not Yet Universal (24RWVMANYU)
 
Unit 2
Unit 2Unit 2
Unit 2
 
Model Based Testing
Model Based TestingModel Based Testing
Model Based Testing
 
Software developer occupational brief
Software developer occupational briefSoftware developer occupational brief
Software developer occupational brief
 
OOSD_UNIT1 (1).pptx
OOSD_UNIT1 (1).pptxOOSD_UNIT1 (1).pptx
OOSD_UNIT1 (1).pptx
 
CSE320 SOFTWARE ENGINEERING Lecture01 (1).ppt
CSE320  SOFTWARE ENGINEERING Lecture01 (1).pptCSE320  SOFTWARE ENGINEERING Lecture01 (1).ppt
CSE320 SOFTWARE ENGINEERING Lecture01 (1).ppt
 
W4 ucl@md day2011
W4 ucl@md day2011W4 ucl@md day2011
W4 ucl@md day2011
 
Modeling should be an independent scientific discipline
Modeling should be an independent scientific disciplineModeling should be an independent scientific discipline
Modeling should be an independent scientific discipline
 
[2015/2016] Software development process
[2015/2016] Software development process[2015/2016] Software development process
[2015/2016] Software development process
 
20121121101127simulation azmi
20121121101127simulation azmi20121121101127simulation azmi
20121121101127simulation azmi
 
Simulation Powerpoint- Lecture Notes
Simulation Powerpoint- Lecture NotesSimulation Powerpoint- Lecture Notes
Simulation Powerpoint- Lecture Notes
 
Discussion Post 1A software process model is a streamlined port.docx
Discussion Post 1A software process model is a streamlined port.docxDiscussion Post 1A software process model is a streamlined port.docx
Discussion Post 1A software process model is a streamlined port.docx
 
SWEBOK Guide Evolution and Its Emerging Areas including Machine Learning Patt...
SWEBOK Guide Evolution and Its Emerging Areas including Machine Learning Patt...SWEBOK Guide Evolution and Its Emerging Areas including Machine Learning Patt...
SWEBOK Guide Evolution and Its Emerging Areas including Machine Learning Patt...
 
010821+presentation+oti.ppt
010821+presentation+oti.ppt010821+presentation+oti.ppt
010821+presentation+oti.ppt
 
Software Life Cycle Modelsrerfgrgreg.pptx
Software Life Cycle Modelsrerfgrgreg.pptxSoftware Life Cycle Modelsrerfgrgreg.pptx
Software Life Cycle Modelsrerfgrgreg.pptx
 
Sanjay
SanjaySanjay
Sanjay
 
ppt2.pptx
ppt2.pptxppt2.pptx
ppt2.pptx
 
GSEP - PROCESS AND CHECKPOINT
GSEP - PROCESS AND CHECKPOINTGSEP - PROCESS AND CHECKPOINT
GSEP - PROCESS AND CHECKPOINT
 
1 sdlc model
1 sdlc model1 sdlc model
1 sdlc model
 
MDA
MDAMDA
MDA
 

More from Michalis Famelis

MAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model UncertaintyMAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model Uncertainty
Michalis Famelis
 
Greece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist eventGreece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist event
Michalis Famelis
 
Partial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with UncertaintyPartial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with Uncertainty
Michalis Famelis
 
The Semantics of Partial Model Transformations
The Semantics of Partial Model TransformationsThe Semantics of Partial Model Transformations
The Semantics of Partial Model Transformations
Michalis Famelis
 

More from Michalis Famelis (9)

Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...
 
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
Photo Exhibition for the Commemoration of the 40th Anniversary of the Polytec...
 
Transformations of Models Containing Uncertainty
Transformations of Models Containing UncertaintyTransformations of Models Containing Uncertainty
Transformations of Models Containing Uncertainty
 
MAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model UncertaintyMAV-Vis: A Notation for Model Uncertainty
MAV-Vis: A Notation for Model Uncertainty
 
Greece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist eventGreece Solidarity Initialive : October 28 antifascist event
Greece Solidarity Initialive : October 28 antifascist event
 
Uncertainty Management With Partial Models
Uncertainty Management With Partial ModelsUncertainty Management With Partial Models
Uncertainty Management With Partial Models
 
Comparing the Effectiveness of Reasoning Formalisms for Partial Models
Comparing the Effectiveness of Reasoning Formalisms for Partial Models Comparing the Effectiveness of Reasoning Formalisms for Partial Models
Comparing the Effectiveness of Reasoning Formalisms for Partial Models
 
Partial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with UncertaintyPartial Models: Towards Modeling and Reasoning with Uncertainty
Partial Models: Towards Modeling and Reasoning with Uncertainty
 
The Semantics of Partial Model Transformations
The Semantics of Partial Model TransformationsThe Semantics of Partial Model Transformations
The Semantics of Partial Model Transformations
 

Recently uploaded

TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Precisely
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 

Recently uploaded (20)

ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 

Research Questions for Validation and Verification in the Context of Model-Based Engineering

  • 1. Research Questions for Validation and Verification in the Context of Model-Based Engineering Catherine Dubois Michalis Famelis Martin Gogolla Leonel Nobrega Ileana Ober Martina Seidl Markus Voelter ENSIIE, Evry, France University of Toronto, University of Bremen, University of Madeira, Funchal, University of Toulouse, Johannes Kepler University Linz, Austria Voelter Ingenieurburo, Heidenheim, Germany October 1st , 2013 MoDeVVa 2013, Miami, USA
  • 2. Introduction Abstraction techniques are one of the promising paths for the future advances in the field of verification. – Clarke, Emerson, Sifakis: Turing Lecture 2008 V&V crucial for MBE – Uncover hidden properties and errors – Verify transformations – Ensure quality, etc. 2
  • 3. About • This paper is the result of the working group on V & V at Dagstuhl-Seminar 13182 held in May 2013. • Dagstuhl: scientific retreat in western Germany • Seminar topic: Meta-Modeling Model- Based Engineering Tools • Three break-outs: Informal Modeling, Compositionality, Modeling and V&V 3
  • 4. 4 Our goal: Identify main areas in the synergy between MBE and V&V where we need to focus research. What is the status, what are the research questions? • Two-day collaborative brainstorming workshop. • Culminated in a plenary presentation …and this report
  • 5. Disclaimer • No claim of completeness or exhaustiveness • Represents the informed opinions of the authors • May have missed existing answers to some questions • Is this the right level of granularity / level of detail? 5
  • 6. Thematic Categories 1. Gap between Models and V&V Formalisms 2. Need to Refine Existing Methodologies 3. Design-time vs. Runtime 4. Properties 5. Model Transformations 6. Informal vs. Formal vs. Incomplete Modeling 7. Comparison and Benchmarking 8. Domain-Specific Languages 6
  • 7. Models and V&V Formalisms (context) Model Property Model' Property' Designer Level (can be Domain- Specific) Verification formalism level Verification engine Verification feed-back 7 Transformations and traceability mechanisms But behavioral semantics often leads to non-bijective correspondences between design time and runtime artifacts. V&V tool configuration
  • 8. Models and V&V Formalisms (questions) • How to express properties at the level of models in a way understandable to clients? • How to formulate models and properties in a single language transparent to clients? • How to report the V&V results and diagnostics in an appropriate form to clients? • How to bridge the gap between formally expressed and verified properties on one side and client attention on the other side? • Can modeling language extensions help in making explicit the “needs” of V&V machines?
  • 9. Refining Existing Methodologies (context) • Integrating V&V in development can support early V&V similar to how debugging is offered by IDEs. • Generic methodologies identify points where V&V can be used. • A Model-based development methodology would need to allow variations based on application domain, nature of project, etc. • Goal: better identify which V&V activities are meaningful at the various phases of design; take full advantage of V&V engines. 9
  • 10. Refining Existing Methodologies (questions) • How do we integrate V&V in the overall development and modeling process? – On the technical level of tool exchange? – On the methodological level of using the right technique at the right time for the right task? • When are techniques like animation, execution, symbolic evaluation, testing, simulation, proving or test case generation used most efficiently during development? – For which model and model transformation properties can they be employed? 10
  • 11. Design-time vs. Runtime (context) • Models are specified at design time • During execution these models are instantiated • The dynamic nature of structure during execution makes it difficult to understand and represent runtime information • Existing modeling environments offer limited support for precisely specifying instantiation and snapshots 11
  • 12. Design-time vs. Runtime (questions)  During the V&V phase, how do we obtain an initial model instantiation?  How do we obtain large and meaningful instantiations?  How do we connect design time and runtime artifacts?  How do we deal with the issue of scalability in the context of V&V ?  How do we handle time and space concerns w.r.t. design time and runtime artifacts?  How do we automatically or semi-automatically manage the V&V machine configuration? 12
  • 13. Properties (context) • Model and model transformation properties relevant in V&V: – consistency, reachability, dependence, minimality, conformance, safety, liveness, deadlock freeness, termination, confluence, correctness • Confusion caused by: – Different kinds according to the nature of the model (static/ dynamic), its level of abstraction, etc. – Many tools and techniques (potentially complementary) Main challenge: What kind of property to verify on which model at what stage with what kind of technique? 13
  • 14. Properties (questions) • What are the benefits and trade-offs between expressing properties on more abstract modeling levels in contrast to expressing them on more concrete levels? • How do we find the right techniques for uncovering static and dynamic model properties? • Which techniques are appropriate for uncovering static modeling language inherent properties, which for static model-specific properties? • Which techniques are appropriate for uncovering dynamic generic properties, which for dynamic model-specific properties? 14
  • 15. Model Transformations (context) • Core component of MBE • Many applications: – Maintaining inter-model consistency – Semantics definition of (domain-specific) modelling languages – … • Challenge: Verification of model transformations – Proving correctness, termination, confluence – What are the differences to “normal” code? – Is the higher abstraction level beneficial to V&V? 15
  • 16. Model Transformations (questions) • What verification techniques are meaningful for verifying model transformations? • How do we analyse properties like confluence and termination? • How do we analyse correctness of model transformations w.r.t. a transformation contract? • How do we infer a transformation contract from a model transformation? 16
  • 17. Informal vs. Formal vs. Incomplete Modeling (context) • During V&V: switch on or off particular model elements (in class diagrams, e.g., multiplicities); configure constraints by negating, deactivating or activating them (in class diagrams, e.g., class invariants) • Different types of granularity (a) all model elements may be relaxed (b) only a manual model element selection can be considered for relaxation (c) a semi- automatic element selection for relaxation may be offered • Ultimate vision: sliders on the user interface to gradually go from a strict, formal model through various intermediate levels to a totally relaxed and informal model; fewer formal model parts activated means more informality in the model • Minimal formal frame for test case construction must be preserved: e.g., for class diagrams of central classes and associations and for state charts of central states and transitions • For a completely informal model no formal scenario (no test cases) can be formulated 17
  • 18. Informal vs. Formal vs. Incomplete Modeling (questions) • How do we leverage informal assumptions found in sketches for exploratory V&V? • Are informal sketches close enough to V&V at all? • What are appropriate relaxation mechanisms for different degrees of formality? • How do we handle incomplete or partial models w.r.t. V&V? • How do we deactivate and activate model units? • How do we handle the exploration of model properties and alternatives? 18
  • 19. Comparison and Benchmarking (context) • Benchmarking can boost research: – Fair comparison of tools – Impartial benchmark selection which covers the spectrum of interesting test cases – Clear documentation of outcomes; reproducibility – Publicity makes it easy to identify progress, problems • However, in MBE: – No common standards – No community platform for benchmarks – Not clear what metrics are relevant for measuring “improvement” 19
  • 20. Comparison and Benchmarking (questions) • How to compare existing V&V tools w.r.t. functionality, coverage, scalability, expressiveness, executing system (i.e., for models at runtime)? • Which criteria are appropriate for comparison? • Can we globally compare fairly at all? – Broad and diverse spectrum of V&V machines: B, Coq, HOL/Isabelle, SAT, SMT, CSP solvers, Relational logic, enumerative techniques 20
  • 21. Domain-Specific Languages (Context) • Most verification tools have hard to use input languages, alien to normal developers. – Hence, verification tools are often not used. • MBE approaches become more and more mainstream. • Models can simplify analysis and verification, because of the higher degree of domain semantics they express. • Potential to exploit the two approaches synergistically – From the high-level models, we can automate the generation of the input to the verification tools. 21
  • 22. Domain-Specific Languages (Questions) • How to define DSLs close to domain concepts but still allow generation of meaningful input for V&V tools? • V&V tools need the specification of properties. – How to express them at the domain level in a user-friendly way? – Can the property specifications be integrated with the same DSL and/or model used for describing the to-be-verified system without creating self-fulfilling prophecies? • How to bring V&V feedback back to the domain level and express it in terms of the DSL-level input? • Can incremental languages extensions help with making programs expressed in general-purpose languages more checkable? For example, the semantics of a specific extension construct may enable the generation of very rich inputs to the verification tool. 22
  • 23. Thematic Categories 1. Gap between Models and V&V Formalisms 2. Need to Refine Existing Methodologies 3. Design-time vs. Runtime 4. Properties 5. Model Transformations 6. Informal vs. Formal vs. Incomplete Modeling 7. Comparison and Benchmarking 8. Domain-Specific Languages 23
  • 24. Conclusion • Report-back from working group on MBE and V&V at Dagstuhl in May 2013. • Main areas in the synergy of MBE and V&V. • For each: described status and identified research questions. • Our hope: Spark discussions and debate, help focus research in MBE and V&V. 24