Measuring Program Comprehension: A Large-Scale Field Study with Professionals
Icsm2013 contexts (1)
1. How does context affect the
distribution of software
maintainability metrics?
Feng Zhang, Audris Mockus, Ying Zou, Foutse Khomh,
and Ahmed E. Hassan
6. 6
What are the Contexts of Software?
Age
(AG)
Number of Changes
(NC)
Life Span
(LS)
Number of Downloads
(ND)
Application Domain
(AD)
Programming Language
(PL)
13. 13
RQ1. Analysis Methods (cont’)
For example
C Java PascalC++ C#
Metric
1
Metric
1
Metric
1
Metric
1
Metric
1
Metric
n
Metric
n
Metric
n
Metric
n
Metric
n
KruskalWallis test
KruskalWallis test
14. 14
Complexity (8/14 metrics) Abstraction (1/5 metrics)
Coupling (5/8 metrics)
Cohesion (2/4 metrics)
Encapsulation (1/4 metrics)
Documentation (3/4 metrics)
YES!! the Contexts Matter!
51 % of metrics are
impacted by all Six
Contexts
15. 15
and Among the Six Contexts …
at least 72 % of
metrics are impacted
by a Single Context
16. 16
Does it mean ALL six contexts
should be considered all the time?
19. 19
RQ2. Analysis Methods (cont’)
C Java PascalC++ C#
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i
Metric
i Mann-Whitney U test
Metric
i Mann-Whitney U test
Metric
i
20. 20
RQ2. Analysis Methods (cont’)
0.147 0.330 0.474Cliff’s delta
14.7% 33.0% 47.4%% of non-overlap
Small Medium LargeCohen’s standard
0.20 0.50 0.80Cohen’s d