1.
Rapid Performance Modeling
by transforming Use Case Maps
to Palladio Component Models
Christian Vogel (KIT), Heiko Koziolek (ABB), Thomas Goldschmidt (ABB), Erik Burger (KIT)
© ABB Group
Slide 1

2.
Motivation
Performance Modeling at ABB
 Industrial Automation Domain
 Efficient control/data flow
crucial for real-time constraints
 Design sketches on whiteboards
 End-to-end latency as a requirement
 Challenges
 How to reason on
complex control/data flows?
 How to break down global response time
requirements to performance budgets
for software components?
© ABB Group
Slide 2
Sketch
of a simple
webserver

3.
Solution Approach
UCM2PCM
 Palladio Component Model (PCM) allows
consideration and prediction of
performance from the beginning
 But
 Control flow and structure in different
views
 PCM models are difficult to create
for persons from industrial automation
 Thus
 Use Case Maps (UCM) as new
frontend for PCM
 Transformation from UCM to PCM
© ABB Group
Slide 3

4.
Related Work
Model transformations in SW Performance Engineering
 Performance engineering
 C. Smith and L.Williams, Addison-Wesley (2002)
Performance Solutions: a practical guide to creating responsive, scalable software
 D. Menasce, L.W. Dowdy, V. Almeida, Prentice Hall (2004)
Performance by design: computer capacity planning by example
 Survey of approaches for performance predictions
 S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni, IEEE TSE (2004)
Model-based performance prediction in software development: A survey
 H. Koziolek, Elsevier Performance Evaluation (2010)
Performance evaluation of component-based software systems: A survey
 Transformation of UCM into Layered Queuing Networks (LQN)
 D. B. Petriu (2001)
Layered software performance models constructed from use case map specifications
 Use of UML Marte Profile for creating performance models
 M. Woodside (2007)
From annotated software designs (UML SPT/Marte) to model formalisms
© ABB Group
Slide 4

5.
Foundations
Use Case Maps
 Scenario-based modeling
language
 Part of the User Requirements
Notations (URN)
 Intuitive and easy to learn
 Graphical editor exists*
 Allows specification of
performance annotations
© ABB Group
Slide 5
* http://lotos.site.uottawa.ca/ucm/bin/view/ProjetSEG/WebHome

6.
UCM2PCM
Architecture
© ABB Group
Slide 6

7.
UCM2PCM
Performance Annotations added to UCMs
© ABB Group
Slide 7
Loop IterationCount Resource Demands
Component AllocationBranch Probabilities
Scenario Workload

8.
UCM2PCM
Mapping Approach
© ABB Group
Slide 8
UCM Path
Component
Responsibility

9.
UCM2PCM
Mapping Approach
© ABB Group
Slide 9
Return Call
Component Call
All calls

10.
UCM2PCM
Mapping Approach
© ABB Group
Slide 10
SystemRequiredCall
SystemProvidedCall
ParentCall

11.
UCM2PCM
Mapping Approach
© ABB Group
Slide 11
Usage Model

12.
UCM2PCM
Limitations
 Limited Input Assistance
 Performance Annotations need to be added as key-value pairs
 Limited PCM feature support
 Variables & Guarded Branches are not supported
 No Customized ResourceEnvironment supported
 Duplicated SEFFs
 For every call of a function a new SEFF is created
© ABB Group
Slide 12

13.
Evaluation
Goals
 Evaluation of Accuracy
 Apply UCM2PCM to three test systems
 Validate the correctness of the PCM models
resulting from the UCM2PCM transformation
 Measure and check the accuracy of the produced
SimuCom performance results
 Evaluation of Usability
 Tutorial & Assignment for UCM2PCM
 Survey among developers for getting feedback
about the UCM2PCM tool
© ABB Group
Slide 13

14.
Evaluation
Tested Models
 Media Store
 Plain Java web application for storing and retrieving
media files.
 SPECjAppServer
 Industry-standard benchmark, designed to measure the
performance of application servers conforming to the
Java EE 5.0 or later specifications.
 Business Reporting System
 Loosely modeled after a management information
system, formerly analyzed at Carlton University. Users
can retrieve live business data from the system and run
statistical analyses.
© ABB Group
Slide 14

15.
Evaluation
Business Reporting System UCM
© ABB Group
Slide 15

16.
Evaluation
Accuracy: Original vs. transformed UCM model
 Transformation successfully bridged
gap between UCM and PCM
 Simulation result difference to
reference model below 15% and
below 5% in most cases
 Accuracy is seen sufficiently for
early reasoning of different design
alternatives
© ABB Group
Slide 16

17.
Evaluation
Usability: qualitative user survey
 Main Advantages
 Comprehensibility of
model is high, also for
non-experts
 Fast(er) modeling
 Modeling complex
systems is seen critical
with UCM2PCM
 Results are only hints,
without statistical
relevance
© ABB Group
Slide 17

18.
UCM2PCM
Conclusions and Future Work
 Transformation successfully bridges
semantic gap between UCM & PCM
 High result accuracy
 Mainly positive feedback by survey
 Future Work
 Adding Input Assistance
to UCM editor
 Reverse transformation
from PCM to UCM
 Conducting further
case studies & experiments
© ABB Group
Slide 18

19.
© ABB Group
April 25, 2013 | Slide 19