Effecitve uml modeling quality asurance and its economics

Effective UML Modelling
Does Software Modeling Pay ?
- empirical studies in UML -

Dr. Michel R.V. Chaudron, Leiden Institute of Advanced Computer Science
Program director M.Sc. ICT & Business http:/iib.liacs.nl

Leiden University. The university to discover.

Collaborate with us!
Your ideas on software engineering research?

Collaborate with us! chaudron@liacs.nl

- Can we study your project (for free)?

- We would like to study :
- impact on maintenance, requirements
- Communication in off-shored projects
- UML-based test generation
- Compare UML-based with MDA-based development
- Measure risk exposure in iterative sw development

Michel Chaudron, Jeudi des Modeles, 31 March 2011, Nantes Leiden University. The university to discover.

Outline of talk
- Introduction Johan Muskens

- Modeling – State of the Practice
- Styles of modeling Christian Lange

- Quality of models
- Costs of modeling
Ariadi Nugroho
- Benefits of modeling

Werner Heijstek


Introduction Leiden Netherlands

Amsterdam
Leiden

Eindhoven


Research Interest
- Effectiveness of software modelling
- Quality of Modeling
When is a model good (enough)
- Analysis of Non-Functional
Properties of component-based
architectures (model-based)

- What are the pay-offs of investing in
early design/architecture?
Fewer defects? Cheaper maintenance? …

- Software Design


State of the Practice


Styles of Modeling
Sketch

Communication
Recipe for construction


Characteristics of Case Studies

Your project here?

Industrial use of UML diagrams

How UML is used B Dobing, J Parsons - Communications of the ACM, 2006
Class


Statistics about Design Doc’s in general
Typical Word document mixes:
- Text and diagrams
500-800 words / diagram

- Formal UML & inventions
40-50% of diagrams is UML
60-50% of diagrams is not UML

Text is used for:
- Design decisions / Rationale
- Elaboration/explanation
- Linking to context
‘Design’ Tooling must support: models, sketches & text
Uri Dekel, James D. Herbsleb: Notation and representation in collaborative
Michel Chaudron, Jeudi des Modeles, 31 March 2011, Nantes object-oriented design: an observational study. OOPSLA 2007: 261-280

Modelling style and model purpose

Architecture and Design Description. IEEE Software 23(2): 40-46 (2006)
C.F.J.Lange, M.R. V. Chaudron, J.Muskens: In Practice: UML Software
understanding

communicating

managing
complexity
implementation
blueprint

Styles of using UML
– as a sketch – thinking tool/understanding
– for communicating system design
– as a blueprint – guide the implementation work
– as a implementation (MDA) Panel

UML in practice
Sketch

Communication

More effort  More expensive
Recipe for construction


UML for Design:
Suitability of UML - UML is too formal
- Informal information is
Sketch added to diagrams
- designers do use UML
concepts
Communication
UML for Communication: Recipe for construction
- Internationally standardized
UML Blueprint:
meaning seems to work
- Ok
UML for code generation:
- Not formal enough


Quality of UML Models
- Can we distinguish styles by means of some
objective measure of model-quality?


Quality Models
Existing models

Boehm McCall
ISO 9126 Rombach

…

- Decomposition of
characteristics
- Bottom level: metrics
- Differences in
- Relations between characteristics
- Vocabulary Boehm’s Quality Model


How to Define UML Model Quality?

Framework for quality of UML models (Lange & Chaudron, 2005)
The quality attributes of a UML model is determined
by its purpose...


How to measure goodness of UML model?
- Design Heuristics

- Completeness
- Interaction of classes must be described in SD
- Methods of CD must be called in SD
- Consistency
- Messages in SD must correspond to method in class diagram
In many cases it can not be automatically determined
whether a flaw is an incompleteness or an inconsistency
- Correspondence
- Dependencies in the implementation are allowed only if they
occur in the design

Automated Model Checking and Visualization

UML model

Analysis Tool

Quality Metrics/Rules
• Completeness Visualization of
• Consistency model + metrics

Screenshot of the MetricView tool

= violation
= ok

Quality = absence of ‘bad things’
such as omissions, spaghetti

Design Heuristics for UML
– Metrics based on Chidamber & Kemerer OO metrics
Maintainability relates to Coupling, Cohesion, …

Highest coupling = 7 Highest coupling = 4


Distribution of Coupling
250

200
number of classes

150

100

50

0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
coupling

System of several hundred classes

21

MetricView Tool MetaView
to visualize the (hidden)
inter-diagram relations
Example: Which classes
contribute to a certain
use case?

http://www.youtube.com/watch?v=G3HJ_QR9EG4
Supporting task-oriented modeling using interactive UML views, C.F.J. Lange, M. A.M. Wijns, and M. R.V. Chaudron,
Journal of Visual Languages and Computing

Christian Lange 22

MetricView Tool MetricView
The values of metrics are
visualized on class
diagrams using colors
Example: Coupling-
Between-Objects (CBO)


Christian Lange. Leiden University. The university to discover. 23

3D visualization of design measurements


Christian Lange. Leiden University. The university to discover. 24

Industrial Experience with MetricView
as Quality Assurance Tool
- Based on 15+ industrial projects

- If there are weak spots in the design, then these are
indicated as ‘suspects’ by MetricView

- MetricView indicates many weak spots that do not
require improvement according to project architects
(false positives?)
 quality is not applied uniformly across UML models

- The later MetricView is applied, the fewer ‘weak spots’
are removed from the design  process issue


How do developers apply detail to UML models?

Ariadi Nugroho, Michel R. V. Chaudron: A Survey of the Practice of Design -
Code Correspondence amongst Professional Software Engineers. ESEM 2007:
70.0
Disagree
Somewhat Disagree
61%
60.0
Neutral
Somewhat agree
52%
Percentage of the respondents

50.0 Agree

40.0
37%

30.0

20.0

10.0

0.0
Equal details Details proportional More details for Programmers
for all parts to complexity critical parts determine details

Developers apply more detail on critical
and complex parts of a system

How is Level of Detail distributed in a diagram?
Case 1 Case 2

High detail High detail
Low detail Low detail

Robin van den Broek B.Sc. Thesis in CS, 2009, Leiden University

Does the implementation resemble
the design?

design implementation

M.Sc. Thesis, Dennis van Opzeeland TU/e Computer Science, System Architecture and Networking

Model - Source Code Conformance
Does the implementation look like the original design?
 X-Axis
 Metrics of Design
 Y-Axis
 Metrics of
Outlier Implementation

 Points represent
Classes
 Points off the
diagonal indicate
outliers


Does strictness of implementing a model differ

for different modeling constructs?
50.0
Loosely 36%

Somewhat loosely
45.0
Neutral
40.0
Somewhat strict
Strictly
46%
35.0

30.0
31%
25.0

20.0

15.0

10.0

5.0

0.0

Package Dependency Inheritance Class and The order of
structure relations relations method names method calls

Developers favor to implement dependency and
inheritance relations more strictly than other constructs

Which factor most often drives developers to

non-correspondence?

60

Never
50
Sometimes
Often
Very Often 48%
40

30
31%
29%
20

10

0

Meeting Impractical Incomplete Model doesn’t satisfy
deadline model model requirements

Among other factors, developers considered incomplete model
more often drives them to non-correspondence


Findings from State of the Practice
- UML is used in many different ways
- Incompleteness
- Not all parts of system are modelled
- Disproportion
- Different parts of the system are modeled different in level
of quality
- Inconsistency
- Conflicting information in different views of one model

Is this a problem?


Does Quality Matter? Experiment

Effects of Defects in UML Models, C.F.J. Lange, M.R.V. Chaudron, (ICSE 2006)
- Subjects are given UML models
- Ask questions about UML model
- Two types
• Which implementation matches the diagrams?
• How do you interpret these diagrams?
- Answer from the perspective of the developer.

- Multiple-choice test (15 Questions)
- 4 options + 1 option
(“There’s something wrong, I can’t give an answer”)
- + Background questions
(possible confounding factors: training, experience, …)


Example Question

Effects of Defects in UML Models, C.F.J. Lange, M.R.V. Chaudron, (ICSE 2006)
Inconsistency:
Message Name does not correspond to Method Message does not correspond to Method - Control (Q4)

120
80
100

70
Many don’t see the defect 80

64% don’t see the defect
60
60
and just implement 40

50
and just implement 20

40 0
a b c d e(rror)

30

20 ?
10
- Suppose you are developer in this banking software project. It is your task to implement class
0
ATM. Please indicate how you would implement the class ATM given these two UML diagrams?
a b c d e(rror)

A) B) C) D) E)

Ring the
getCardInserted(){ getCardInserted(){ getCardInserted() getCardInserted()
bell!
c.requestPIN(); c.requestPIN(); { { Something
dosomething; dosomething; c.requestPIN(); c.requestPIN(); is
a.open() a.lock() dosomething; dosomething; wrong!
} } a.acknowledge() a.validate()
} }


Does Quality Matter? Case Study
Parts of the design are more detailed than others

attributes

operations

associations


Relation between UML-LoD and Code Quality
Select ‘defects’
in defect DB

Find classes in source component C2 defect
component C2

code that were
requires I2
requires I3I2
requires
uses requires I3
I2.g
uses I2.g
density
uses I3.h

repaired for solving this uses I3.h

defects

O rder
OrderDetail
* 1 -date P ce or
repro ss Parser A er
nalys DC
B reator DF
B iller Inherita R
nce elator DC
B hecker S tis Filte
ta tic r StatisticC lculator
a
-quantity

Find corresponding
-status
-taxS tatus SA A
A T.B T
+calcTax()
+calcS ubTotal()
+calcTotal()
+calcW eight() +calcTotalW eight()
Paym ent 1

-am ount 1

classes in UML models Credit
-num ber
-type
-expDate
Cash
-cashTendered
Check
-nam e
-bankID
+authorized()
+ authorized()

OrderDetail
O rder LoD per
class
* 1 -date P ce
repro ssor P rse
a r A lys
na er DC
B reator DF
B iller Inherita ceR
n elator D C eck
B h er S tisticFilte
ta r StatisticC lc r
a ulato
-quantity

Determine LoD
-status
-taxS tatus SA A
A T.B T
+calcTax()
+calcS ubTotal()
+calcTotal()
+calcW eight() +calcTotalW eight()
Paym ent 1

-am ount 1

for CD and SD -num ber
-type
-expDate
Credit Cash
-cashTendered
Check
-nam e
-bankID
+authorized()
+ authoriz ed()


Relation between Level of Detail
•Does high LoD in sequence diagrams correspond to lower defect
density? and Defect Density

= 1 class

Level of Detail for Sequence Diagrams is significantly
(negatively) correlated with defect density.
More detailed model => fewer defects


Economic Model for Cost of Quality

From: H. Krasner, Cost of Quality, 1998
Cost of SQ = Achievement Cost + Non-conformance Cost


Cost of Modeling
Focus on effort

“Is there a correlation between class-count and the
effort spent in modeling?”

Sources:
‐ Experiment
‐ Set of industrial case studies


Empirical Data
 Experiment
 106 MSc students in (TU/e)
 Organized in 35 teams
 Task: Model Car Navigation System
 Duration: 6 weeks assignment
 Three treatments were applied:
1. No modeling conventions
2. With modeling conventions
3. Tool-supported modeling conventions


Results
 Experiment
40,00

35,00

30,00
Effort (hours)

25,00
Spearman’s Correlation Coefficient
20,00

15,00 Effort (hours)
10,00 Class-count .302
5,00 Sig. (2-tailed) .171
0,00
0 10 20 30 40 50 60 70
Class-count

There is no significant correlation between class-count and
the effort spent in modeling.

 class-count is not a good measure for the effort spent
on modeling?

Measurement of Modeling/Design
in RUP Effort Distribution

modeling designing

Impact of MDD on Effort Distribution
as % of total project effort
construction

constr.
elaboration
transition

transition
elaboration
architecture & code generator
incep.

‘traditional’ Model-driven
(avg. of 17 projects) (1 project)

Cost of Modeling - Summary
- Size is not the issue
- Large variety in cost per
- Developer (expertise)
- project
- Purpose/use of models
- Level of abstraction / target quality of models

- Maintaining model-code correspondence

- Common to all new technology:
- training, licensing, …

The Use of UML and Software Quality

50.0
Reduce 45%
45.0 Somewhat Reduce
Neutral
41%
40.0 Somewhat Improve
Improve

35.0

30.0

25.0

20.0

15.0

10.0

5.0

0.0

Cover Correctness Modularity Testability Understandability
Requirements

A large number of developers indicated the use of UML
improves understandability and modularity

Benefits of Modeling
Req. engineering Doing the right things

Modeling Designing Modularity 

Implementing Defects

Testing

Maintenance

Early insight in system quality
Project Management
Better traceability
Improved shared understanding

Do Models improve understanding?

Eye tracking experiment


Eye tracking UML designs: gazeplots

structure behaviour

Attention is determined by task


Heatmap Person A

structure behaviour

Heatmap Person B

There are large differences in styles of reading

Overview Benefits of Modeling
Improved Improved Improved Project
Quality Productivity Management

Reduced More Efficient
rework Maintenance

Reduced Reduced
Fewer Improved
testing maintenance
Defects Communication
effort effort

“validation” Better Risk
“verification” building the management
building the right system
system right
More
Improved Improved
Accurate
requirements Modularity
Estimating

Better Understanding

Modeling


What is next?
- Domain specific modelling (using UML)
- Domain specific tooling, V&V, generation

- Generation of/from UML
- From requirements
- Test generation based on UML

- Tool integration
- Synchronization between model and code


Concluding Remarks
- UML modeling is being done in a variety of styles
- Modelers ‘economize’ by focusing on complex and critical
parts. QA needs to be risk-aware
- Low hanging fruit: ‘common’ QA techniques should be
applied to models (versioning, reviews, coding-conventions)

- There is a payoff of modeling in:
- improved modularity
- seq.diagrams  code quality
- communication (suspected)

- Future work: maintaining Please send your projects
model-code correspondence to: chaudron@liacs.nl


What is next?
Your ideas on research?

Collaborate with us! chaudron@liacs.nl

- Can we study your project?

- We would like to study :
- impact on maintenance, requirements
- Communication in off-shored projects
- UML-based test generation
- Compare UML-based with MDA-based development
- Measure risk exposure in iterative sw development


How often do design issues lead to problems?
70
Never
Sometimes
60 Often
Very Often

50

40

30

20

10

0
Inconsistency Understandability Inaccuracy Incompleteness

Incompleteness of models was considered most
often to lead to implementation problems

Opinions on Value of Modelling

Value
Agile

MDA “Loose
UML”
Formal Methods

100%
Panel
completeness


Which classes are modeled?

Complexity and Coupling is higher for classes
that are modeled than not modeled

Recommendations
- Apply Quality Assurance to UML modeling
- Use ‘coding conventions’ for UML:
- Naming – correspond to implementation
- Layout
- Use of patterns/stereotypes
- Version Control
- Organize traceability from Use Cases to Sequence
diagrams (tool/conventions)
- Quality of SD’s is important when UML is used as
blueprint for implementation


Maintenance paradox
- Maintainers of software would like better
documentation

- Even if UML models are created during
design, these are used in maintenance.
The more detailed models are, the mode
quickly they become outdated.

Better abstraction in reverse- / round trip-
engineering is needed in the setting of
incremental & iterative development

Conformance
- Between UML model and source code implementation
- It is the degree of similarity between model and implementation
- Reasons for lack of conformance
- Poor model quality (therefore misunderstood in implementation)
- “I know it better”-Attitude of Programmer
- Evolution of implementation during maintenance

- Why is conformance important?
- Ensure similarity between model and implementation
- Model is used for understanding the implementation in maintenance (understanding
the corresponding model is easier than digging through thousands of lines of source
code)

- We developed methods to
- Measure conformance / Identify Deviations
- (Re)construct a mapping between model and implementation elements
- Support in keeping Model and Implementation ‘in sync’

MRV Chaudron
Michel Chaudron, Jeudi des Modeles, 31 March 2011, Nantes Sheet 60

MetricView contributors
- Johan Muskens
- Christian Lange
- Martijn Wijns
- Dennis van Opzeeland
- Ariadi Nugroho
- Werner Heijstek
- Maurice Vermeer
- Alex Telea
- Robin van den Broek

MRV Chaudron
Michel Chaudron, Jeudi des Modeles, 31 March 2011, Nantes Leiden University. The university 61 discover.
Sheet to

Publications
- Journal Publications
- Supporting task-oriented modeling using interactive UML views. PDF
C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Journal of Visual Languages and Computing (Elsevier), Volume 18, Issue 7,
August 2007.
- In Practice: UML Software Architecture and Design Description.
C.F.J. Lange, M.R.V. Chaudron, J. Muskens. IEEE Software, Volume 23, Issue 2, March 2006.
- Conference Publications
- A Visualization Framework for Task-Oriented Modeling using UML.
C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 40th Annual Hawaii International Conference on System
Sciences (HICSS'07). January 2007.
- An Experimental Investigation of UML Modeling Conventions.
C.F.J. Lange, Bart DuBois, M.R.V. Chaudron, Serge Demeyer. Proceedings of the ACM/IEEE International Conference on
Model-driven Engineering Languages and Systems (MoDELS 2006). October 2006.
- Effects of Defects in UML Models - An Experimental Investigation.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006.
(acceptance rate: 9%)
- Improving the Quality of UML Models in Practice.
C.F.J. Lange. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006.
- Visual Exploration of Combined Architectural and Metric Information.
M. Termeer, C.F.J. Lange, A. Telea, M.R.V. Chaudron. Proceedings of VISSOFT 2005, IEEE CS Press. September 2005.
- Combining Metrics Data and the Structure of UML Models using GIS Visualisation Approaches.
C.F.J. Lange, M.R.V. Chaudron. IEEE International Conference on Information Technology 2005. Las Vegas, April 2005.
- An Exploratory Study on the Industrial Use of UML: Improving Control over Design Quality.
C.F.J. Lange, M.R.V. Chaudron. JACQUARD Conference. February 2005.
- Investigations in Applying Metrics to Multi-View Architecture Models.
J. Muskens, M.R.V. Chaudron, C.F.J. Lange. In Proceedings of EUROMICRO’04, Rennes, France, September 2004.
- An Empirical Assessment of Completeness in UML Design.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of the 8th Conference on Empirical Assessment in Software Engineering
(EASE04). May 2004.
- Konsistenz und Vollständigkeit industrieller UML Modelle.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of “Modellierung 2004“, Marburg, Germany, 24-26 March 2004.
Michel Chaudron, Christian Lange, Martijn Wijns discover.
Leiden University. The university to 62

- Workshop Publications
- Towards Task-Oriented Modeling using UML.
C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 10th QAOOSE workshop, co-located with ECOOP2006. July 2006.
- Managing Model Quality in UML-based Software development.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of IEEE Conference on Software Technology and Engineering Practice 2005 (STEP), in the "Workshop on
Software Process Improvement, Quality Assurance and Measurement. September 2005.
- Quantitative Techniques for the Assessment of Correspondence between UML Designs and Implementations.
Dennis J.A. van Opzeeland, Christian F.J. Lange, Michel R.V. Chaudron. Proceedings of the 9th QAOOSE workshop, co-located with ECOOP2005. July 2005.
- An Empirical Investigation in Quantifying Inconsistency and Incompleteness of UML Designs.
C. Lange, M.R.V. Chaudron, J. Muskens, L.J. Somers, H.M. Dortmans. 2nd workshop on consistency problems in UML-based software development, October
20, 2003, in San Francisco, USA as part of the “International Conference on Unified Modeling Language 2003”.

MRV Chaudron
Sheet to

- Journal Papers
Ariadi Nugroho, "Level of detail in UML models and its impact on model comprehension: A controlled experiment",
Information and Software Technology Journal (in Press).
-
Conference Papers
Ariadi Nugroho and Michel R.V. Chaudron, "Evaluating the Impact of UML Modeling on Software Quality: An Industrial
Case Study", ACM/IEEE 12th International Conference on Model Driven Engineering Languages and Systems (MODELS),
Denver, Colorado - USA, 2009.
- Ariadi Nugroho, Bas Flaton, and Michel R.V. Chaudron, "An Empirical Analysis of Level of Detail in UML Models and its
Relation with Defect Density", ACM/IEEE 11th International Conference on Model Driven Engineering Languages and
Systems (MODELS), Toulouse - France, 2008. (received best paper awards). PDF
- Ariadi Nugroho and Michel R.V. Chaudron, "A survey into the Rigor of UML Use and its Perceived Impact on Quality and
Productivity", 2nd International Symposium of Empirical Software Engineering and Measurement (ESEM), Kaiserslautern -
Germany, 2008. PDF
- Ariadi Nugroho and Michel R.V. Chaudron, "A Survey of the Practice of Design - Code Correspondence amongst
Professional Software Engineers", 1st International Symposium on Empirical Software Engineering & Measurement
(ESEM), Madrid, 2007. PDF
- Workshop Papers
Ariadi Nugroho and Christian F.J. Lange, "On the Relation between Class-count and Modeling Effort", Model Size Metrics
Workshop, co-located with MODELS Conference, Nashville, Tennessee, 2007. (best paper). PDF
- Chapters in Books
Ariadi Nugroho and Michel R.V. Chaudron, “Managing the Quality of UML Models in Practice”, in Model-Driven Software
Development: Integrating Quality Assurance, Idea Group Inc., New York, 2008. More info

MRV Chaudron
Sheet to

- W. Heijstek and Michel R. V. Chaudron (2010) The Impact of Model Driven Development on the Software Architecture
Process, 36th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2010) Lille, France
bibtex

W. Heijstek, Michel R. V. Chaudron, Libing Qiu and Christian C. Schouten (2010) A Comparison of Industrial Process
Descriptions for Global Custom Software Development, 5th International Conference on Global Software Engineering
(ICGSE 2010) Princeton, New Jersey, USA - Pages 277-284
bibtex

W. Heijstek and Michel R. V. Chaudron (2009) Empirical Investigations of Model Size, Complexity and Effort in Large Scale,
Distributed Model Driven Development Processes - A Case Study, 35th Euromicro Conference on Software Engineering and
Advanced Applications (SEAA 2009) Patras, Greece
bibtex

W. Heijstek and Michel R. V. Chaudron (2008) Evaluating RUP Software Development Processes Through Visualization of
Effort Distribution, 34th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2008) Parma,
Italy
bibtex

W. Heijstek and Michel R. V. Chaudron (2008) Exploring Effort Distribution in RUP Projects 2nd International Symposium on
Software Engineering and Measurement (ESEM 2008) Kaiserslautern, Germany
bibtex

W. Heijstek and Michel R. V. Chaudron (2007) Effort distribution in model-based development. 2nd Workshop on Model
Size Metrics (MSM 2007)
bibtex

MRV Chaudron
Sheet to

Effecitve uml modeling quality asurance and its economics

More Related Content

Viewers also liked

Similar to Effecitve uml modeling quality asurance and its economics

Recently uploaded

Effecitve uml modeling quality asurance and its economics