The document presents a model-based approach called IDBoM for detecting runtime inconsistencies. It aims to reduce manual effort by reusing existing software models. The approach was evaluated using a self-driving car case study. Key results include: (1) IDBoM allows full automation of inconsistency detection steps given connected services and models, (2) usability of model interactions is improved, (3) IDBoM can find inconsistencies involving incorrect method calls and message sequences, and (4) execution time scales linearly with model size. The approach provides a reusable solution for automated runtime inconsistency detection using design models.

1. Business Informatics Group
Institute of Software Technology and Interactive Systems
Vienna University of Technology
Favoritenstraße 9-11/188-3, 1040 Vienna, Austria
phone: +43 (1) 58801-18804 (secretary), fax: +43 (1) 58801-18896
office@big.tuwien.ac.at, www.big.tuwien.ac.at
Model-based Detection of Runtime Inconsistencies
Author: Daniel Lehner
Advisors: Manuel Wimmer, Sabine Wolny

2. Motivation: traditional approach
2
I have to
stop my
car!

3. Motivation: self-driving car
3
Will my
car stop?

4. Challenge
4
• Limited information whether system works as expected
 Static verification: state explosion
 Testing: mostly done before deployment
 Runtime monitoring: a lot of manual effort required for setup
• For my thesis: How to reduce the manual effort for detecting runtime
inconsistencies by reusing already existing software models?

5. Research Questions
5
RQ1 (Automation): Automation potential of reusing existing models for
inconsistency detection.
RQ2 (Usability): Comparative evaluation of the usability of model
interaction features.
RQ3 (Coverage): Defect kinds which can be found by model-based
inconsistency detection.
RQ4 (Scalability): Parameters that influence execution time of model-
based inconsistency detection.

6. 6
Approach: Inconsistency Detection Based on Models (IDBoM)
Design Time
Does actual behavior
match requirements?
UML Class Diagram
(expected structure)
UML Activity Diagram
(expected behavior)
Check for
- inconsistent structure
- inconsistent behavior
ClassB.getY()
Runtime
UML Sequence Diagrams
(runtime traces)
a:ClassAb:ClassB
getX()
ClassA
- getX()
ClassB
- getY()
response

7. 7
Methodology: Design Science [1,2]
[1] Wieringa, Roel J. Design science methodology for information systems and software engineering. Springer, 2014
[2] Hevner, Alan R., et al. "Design science in information systems research." MIS quarterly (2004): 75-105
Research Questions
Requirements
Artefact: IDBoM
Dynamic Checking Service (DynCS)
Service-Oriented Querying and Management of Models (SOMQM)
Evaluation
What is the answer to RQs?
Validation
(Are requirements satisfied by artefact?)
Problem
Static/dynamic Analysis
Mapping table
Demonstration Case
Discussion of results
Related Work from LiteratureComparison of Results
Contribution
Is there some improvement?

8. 8
Architecture of IDBoM

9. • RQ1 (Automation):
• Static analysis (demonstration case)
• RQ2 (Usability)
 Static analysis (comparison framework)
• RQ3 (Coverage)
 Functional testing + static analysis (demonstration case)
• RQ 4 (Scalabilty)
 Dynamic analysis (controlled simulation experiment)
9
Evaluation + applied methods

10. Demonstration case: autonomously driving car
Requirements: automation of the following steps
• Triggering the checking process
• Retrieving required information automation after process is triggered
• Calculating the checking result from retrieved information
• Processing the calculated checking result
Results: automation of the required steps is possible
 Given running and connected services
 Given existing design time models
10
Automation (RQ1)

11. 11
Usability of model interactions (RQ 2) [1]
[1] Shackel, B. (2009). Usability–Context, framework, definition, design and
evaluation. Interacting with computers, 21(5-6), 339-346.
Parameter XMI MDT-UML SOMQM
Required setup for accessing
model content
Prerequisites on programming
language
Capabilities for managing
several models
Abstraction level
Requirements on model input
Documentation
Best satisfaction of parameter
Second best satisfaction of parameter
Worst satisfaction of parameter
Results: Usability of model interactions is improved for all
chosen parameters (RQ2)

12. • Demonstration case: autonomously driving car
• Derive inconsistency cases from CD and AD
• 23 cases identified, involving
 Unallowed method calls
 Incorrect message sequences, with particular regard to
• alternative execution paths
• parallel execution paths
 Inconsistencies between software versions
 Execution time constraints
• Results: IDBoM covers 22 cases
 Execution time constraints not yet supported
12
Coverage (RQ 3)

13. Identified potential parameters
 CD: # Classes, Associations, Operations
 AD: # Actions, Decision Nodes, Parallel Execution Paths
 SD: # Messages
Test data
Test program: check if message sequence is valid with respect to design
model
13
Scalability (RQ 4) – experiment setting (3/3)
[1] Pfleeger, S. L. (1995). Experimental design and analysis in software engineering, part 5:
analyzing the data. ACM SIGSOFT Software Engineering Notes, 20(5), 14-17.

14. • Results: Linear expansion of execution time for all tested parameters
• Runtime is mainly determined by the number of messages in SD
Scalabilty (RQ 4) – Results
14
#Classes in CD
#Actions in AD#Associations in CD
#Operations in CD#Messages in SD

15. 15
 Model-based Detection of Runtime Inconsistencies allows
 End-to-end automation
 Linear development of execution time
 Usability of interacting with models improved by
 Using a service-oriented approach
 Combining querying and management of models
 Automating generation of documentation
 Contribution
 End-to-end solution for model-based inconsistency detection
 Including scalability and automation analysis
 Reusable solution for automated processing of model information
 with improved usability for developers
Conclusion

16. 16
 Impact of chosen parameters for
 Improve absolute execution time values, exploiting
 Parallelization
 Cloud infrastructure
 Extension to an industrial setting
Future Work

17. Model-driven runtime inconsistency detection
Questions?
Find my Implementation on Github
• Inconsistency Checking Framework (Dynamic Checking Service):
https://github.com/derlehner/dyncs
• Model Interaction Framework
(Service-Oriented Management and Querying of Models):
https://github.com/derlehner/somqm
Contact me
daniel.lehner@jku.at
https://www.researchgate.net/profile/Daniel_Lehner
17

18. Business Informatics Group
Institute of Software Technology and Interactive Systems
Vienna University of Technology
Favoritenstraße 9-11/188-3, 1040 Vienna, Austria
phone: +43 (1) 58801-18804 (secretary), fax: +43 (1) 58801-18896
office@big.tuwien.ac.at, www.big.tuwien.ac.at
Daniel

19. Showcase of self-
driving car:
Class Diagram

20. Showcase of self-driving car: Activity Diagram

21. Functionality Example: incorrect method calls

22. Functionality Example: alternative execution paths

23. Functionality Example: parallel execution paths

24. Demonstration