The document presents a model-based approach called IDBoM for detecting runtime inconsistencies. It aims to reduce manual effort by reusing existing software models. The approach was evaluated using a self-driving car case study. Key results include: (1) IDBoM allows full automation of inconsistency detection steps given connected services and models, (2) usability of model interactions is improved, (3) IDBoM can find inconsistencies involving incorrect method calls and message sequences, and (4) execution time scales linearly with model size. The approach provides a reusable solution for automated runtime inconsistency detection using design models.
Unblocking The Main Thread Solving ANRs and Frozen Frames
Model-Driven Runtime Inconsistency Detection
1. Business Informatics Group
Institute of Software Technology and Interactive Systems
Vienna University of Technology
Favoritenstraße 9-11/188-3, 1040 Vienna, Austria
phone: +43 (1) 58801-18804 (secretary), fax: +43 (1) 58801-18896
office@big.tuwien.ac.at, www.big.tuwien.ac.at
Model-based Detection of Runtime Inconsistencies
Author: Daniel Lehner
Advisors: Manuel Wimmer, Sabine Wolny
4. Challenge
4
• Limited information whether system works as expected
Static verification: state explosion
Testing: mostly done before deployment
Runtime monitoring: a lot of manual effort required for setup
• For my thesis: How to reduce the manual effort for detecting runtime
inconsistencies by reusing already existing software models?
5. Research Questions
5
RQ1 (Automation): Automation potential of reusing existing models for
inconsistency detection.
RQ2 (Usability): Comparative evaluation of the usability of model
interaction features.
RQ3 (Coverage): Defect kinds which can be found by model-based
inconsistency detection.
RQ4 (Scalability): Parameters that influence execution time of model-
based inconsistency detection.
6. 6
Approach: Inconsistency Detection Based on Models (IDBoM)
Design Time
Does actual behavior
match requirements?
UML Class Diagram
(expected structure)
UML Activity Diagram
(expected behavior)
Check for
- inconsistent structure
- inconsistent behavior
ClassB.getY()
Runtime
UML Sequence Diagrams
(runtime traces)
a:ClassAb:ClassB
getX()
ClassA
- getX()
ClassB
- getY()
response
7. 7
Methodology: Design Science [1,2]
[1] Wieringa, Roel J. Design science methodology for information systems and software engineering. Springer, 2014
[2] Hevner, Alan R., et al. "Design science in information systems research." MIS quarterly (2004): 75-105
Research Questions
Requirements
Artefact: IDBoM
Dynamic Checking Service (DynCS)
Service-Oriented Querying and Management of Models (SOMQM)
Evaluation
What is the answer to RQs?
Validation
(Are requirements satisfied by artefact?)
Problem
Static/dynamic Analysis
Mapping table
Demonstration Case
Discussion of results
Related Work from LiteratureComparison of Results
Contribution
Is there some improvement?
10. Demonstration case: autonomously driving car
Requirements: automation of the following steps
• Triggering the checking process
• Retrieving required information automation after process is triggered
• Calculating the checking result from retrieved information
• Processing the calculated checking result
Results: automation of the required steps is possible
Given running and connected services
Given existing design time models
10
Automation (RQ1)
11. 11
Usability of model interactions (RQ 2) [1]
[1] Shackel, B. (2009). Usability–Context, framework, definition, design and
evaluation. Interacting with computers, 21(5-6), 339-346.
Parameter XMI MDT-UML SOMQM
Required setup for accessing
model content
Prerequisites on programming
language
Capabilities for managing
several models
Abstraction level
Requirements on model input
Documentation
Best satisfaction of parameter
Second best satisfaction of parameter
Worst satisfaction of parameter
Results: Usability of model interactions is improved for all
chosen parameters (RQ2)
12. • Demonstration case: autonomously driving car
• Derive inconsistency cases from CD and AD
• 23 cases identified, involving
Unallowed method calls
Incorrect message sequences, with particular regard to
• alternative execution paths
• parallel execution paths
Inconsistencies between software versions
Execution time constraints
• Results: IDBoM covers 22 cases
Execution time constraints not yet supported
12
Coverage (RQ 3)
13. Identified potential parameters
CD: # Classes, Associations, Operations
AD: # Actions, Decision Nodes, Parallel Execution Paths
SD: # Messages
Test data
Test program: check if message sequence is valid with respect to design
model
13
Scalability (RQ 4) – experiment setting (3/3)
[1] Pfleeger, S. L. (1995). Experimental design and analysis in software engineering, part 5:
analyzing the data. ACM SIGSOFT Software Engineering Notes, 20(5), 14-17.
14. • Results: Linear expansion of execution time for all tested parameters
• Runtime is mainly determined by the number of messages in SD
Scalabilty (RQ 4) – Results
14
#Classes in CD
#Actions in AD#Associations in CD
#Operations in CD#Messages in SD
15. 15
Model-based Detection of Runtime Inconsistencies allows
End-to-end automation
Linear development of execution time
Usability of interacting with models improved by
Using a service-oriented approach
Combining querying and management of models
Automating generation of documentation
Contribution
End-to-end solution for model-based inconsistency detection
Including scalability and automation analysis
Reusable solution for automated processing of model information
with improved usability for developers
Conclusion
16. 16
Impact of chosen parameters for
Improve absolute execution time values, exploiting
Parallelization
Cloud infrastructure
Extension to an industrial setting
Future Work
17. Model-driven runtime inconsistency detection
Questions?
Find my Implementation on Github
• Inconsistency Checking Framework (Dynamic Checking Service):
https://github.com/derlehner/dyncs
• Model Interaction Framework
(Service-Oriented Management and Querying of Models):
https://github.com/derlehner/somqm
Contact me
daniel.lehner@jku.at
https://www.researchgate.net/profile/Daniel_Lehner
17
18. Business Informatics Group
Institute of Software Technology and Interactive Systems
Vienna University of Technology
Favoritenstraße 9-11/188-3, 1040 Vienna, Austria
phone: +43 (1) 58801-18804 (secretary), fax: +43 (1) 58801-18896
office@big.tuwien.ac.at, www.big.tuwien.ac.at
Daniel