This document analyzes how different contexts affect the distribution of software maintainability metrics. It examines six contexts: age, number of changes, life span, number of downloads, application domain, and programming language.
The study finds that 51% of metrics are impacted by all six contexts. At least 72% of metrics are impacted by a single context. It analyzes 39 maintainability metrics across complexity, coupling, cohesion, abstraction, encapsulation, and documentation. For each metric category, it identifies the contexts that have a significant impact.
The key contexts that influence many metrics are application domain and programming language. The results provide guidance on which contexts should be considered when benchmarking software maintainability metrics.
Measuring Program Comprehension: A Large-Scale Field Study with Professionals
Icsm2013 contexts (1)
1. How does context affect the
distribution of software
maintainability metrics?
Feng Zhang, Audris Mockus, Ying Zou, Foutse Khomh,
and Ahmed E. Hassan
6. 6
What are the Contexts of Software?
Age
(AG)
Number of Changes
(NC)
Life Span
(LS)
Number of Downloads
(ND)
Application Domain
(AD)
Programming Language
(PL)
13. 13
RQ1. Analysis Methods (cont’)
For example
C Java PascalC++ C#
Metric
1
Metric
1
Metric
1
Metric
1
Metric
1
Metric
n
Metric
n
Metric
n
Metric
n
Metric
n
KruskalWallis test
KruskalWallis test
14. 14
Complexity (8/14 metrics) Abstraction (1/5 metrics)
Coupling (5/8 metrics)
Cohesion (2/4 metrics)
Encapsulation (1/4 metrics)
Documentation (3/4 metrics)
YES!! the Contexts Matter!
51 % of metrics are
impacted by all Six
Contexts
15. 15
and Among the Six Contexts …
at least 72 % of
metrics are impacted
by a Single Context
16. 16
Does it mean ALL six contexts
should be considered all the time?
19. 19
RQ2. Analysis Methods (cont’)
C Java PascalC++ C#
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i Mann-Whitney U test
Metric
i
20. 20
RQ2. Analysis Methods (cont’)
0.147 0.330 0.474Cliff’s delta
14.7% 33.0% 47.4%% of non-overlap
Small Medium LargeCohen’s standard
0.20 0.50 0.80Cohen’s d