Design-patterns are recognized as a means to improve software maintenance by furnishing an explicit specification of class and object interactions and their underlying intent [Gamma et al. 1995]. Only a few empirical investigations have been conducted to assess whether the kind of documentation for design-patterns implemented in source code affects its comprehensibility. To investigate this aspect, we conducted a family of four controlled experiments with 88 participants having different experience (i.e., professionals and Bachelor, Master, and PhD students). In each experiment, the participants were divided into three groups and were asked to comprehend a non-trivial chunk of an open-source software system. Depending on the group, each participant was, or was not, provided with graphical or textual representations of the design-patterns implemented within the source code. We graphically documented design-pattern instances with UML class diagrams. Textually documented instances are directly reported in source code as comments. Our results indicate that documenting design-pattern instances yields an improvement in correctness of understanding of source code for those participants with an adequate level for experience. Presentation made at UNIBZ on Dec 18, 2014, within the course on Requirements and Design of the European Master in Software Engineering

1. Documenting design pattern instances:
A Family of experiments on
Source code comprehensibility
Gabriella Dodero (uniBZ)
Joint work with G.Scanniello (uniBAS), C.Gravino, M.Risi,
G.Tortora (uniSA)
RDSS, December 2014

2. Summary:
1. A research question
2. A family of experiments
3. Our results

3. A research question
● Consider a software maintenance task, typically done by
someone who is NOT an author of such code.
● The system under maintenance contains several
patterns
● Is there a difference in such a task, if code is
DOCUMENTED showing use of pattern instances?
● That is, not documenting patterns w.r.t. documenting
wherever patterns have been used
● Is there a difference between ways of documenting
patterns?
● That is, textual w.r.t. graphical documentation

4. A family of experiments-1
● Make an experiment about software maintenance, i.e.
give a set of „software comprehension tasks“
● Repeat the experiment with different populations
(developers with different experience)
● What is measured:
– Correctness of understanding
– Time
– Confidence
– Perceived difficulty

5. A family of experiments - 2
Choose a piece of
software
● A chunk of JHotDraw v5.1
(1326 LOC, non trivial!)
Choose a notation for
documenting patterns
● Textual descriptions reported
in the code as comments
● Graphic documentation like
class diagrams

6. A family of experiments - 3
● „Family“ means replicating an original experiment
under controlled conditions
● In this case: using populations with different
experience as developers
Participants Where How many
Practitioners Companies 25
2nd year Msc U.Salerno 25
3rd year Bsc U.Basilicata 23
2-3 yr PhD unibz 15

7. A family of experiments - 4
● In each experiment,
● 1/3 of participants were given textual documentation,
● 1/3 were given graphical documentation, and
● 1/3 had no documentation on pattern instances
● Hypotheses:
(1) There is no difference in Correctness nor Time
between people having documentation and not having
documentation
(2)There is no difference between people with different
experience

8. A family of experiments - 5
● What was asked: 14 questions like this one:
● A post assessment questionnaire on the experience
● Typical duration was 2-3 hours

9. Results - 1
● Hypothesis (1) is rejected for Correctness of
understanding, by professionals and PhD
students only
– Time is not affected by documentation
● But there is no significant difference between
those who had textual documentation or
graphical documentation
● Hypothesis (2) is rejected for Experience, with
respect to Correctness of Understanding and
Time

10. Results - 2
● Correctness of understanding is significantly greater when
pattern instances are correctly identified
● 30% improvement on Correctness of understanding
● Correctness of understanding is greater also for Msc
students, when patterns are documented, but this is not
statistically significant
● Experienced participants (professionals, PhD) benefit
more than non-experienced participants (BSc, Msc
students) from the documentation on patterns
● Experienced participants with textually documented
patterns were the fastest to do the task

11. Results - 3
● Lessons for practitioners:
● Documenting instances improves code
comprehensibility, for maintainers with experience
● Reading such documentation does not slow down
comprehension tasks
● Comprehensibility improves when pattern instances are
correctly identified, when developers have at least a
Bsc degree
● Lessons for researchers:
● Bsc or Msc students may be too inexperienced
compared to professionals, to gather reliable results
by performing experiments on them!!!

12. Thank You!
Questions?
Gdodero @ unibz.it