High-quality code comments assist developers in program comprehension and maintenance tasks, but ensuring their quality is a difficult task. Quality is a multi-dimensional concept, therefore it requires a multi-perspective analysis. We analyze code comments from three perspectives, (i) what academic support is there to assess comment quality, (ii) what are developer commenting practices across languages, and (iii) what are developer concerns regarding code comments.

1. Pooja Rani
PhD Defense
Supervisors:
Prof. Dr. Oscar Nierstrasz
Dr. Sebastiano Panichella
31 January 2022
Assessing Comment Quality in
Object-Oriented Languages
class Browser:
"""Controls
the state of a
browser."""
def name():
.....

2. Roadmap
developer commenting practices
across languages
P2
developer concerns
about comments
P3
Introduction
Our analyses
Challenges
Conclusions
Future work
P1 academic support

3. 3
Code comments
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}

4. 4
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Code comments

5. 5
Java
class comment
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Code comments

6. 6
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Code comments
Trustworthy form of documentation
- McMillan et al. 2010

7. 7
High-quality comments support developers in various activities
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Code comments
Trustworthy form of documentation
- McMillan et al. 2010
- Dekel et al. 2009

8. /**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
8
Does this
comment contain any
example?
Code comments

9. /**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
9
Does this
comment contain any
example?
Code comments

10. 10
How to ensure comment quality?
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
} Does this
comment contain any
example?
Is this a high-
quality comment?

11. 11
How to ensure comment quality?
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
} Does this
comment contain any
example?
Is this a high-
quality comment?
correct?
consistent?
adequate?
Other quality attributes?

12. 12
Challenges
No standard definition of comment quality
No strict syntax and style conventions
Lack of quality assessment tools

13. 13
Challenges
No standard definition of comment quality
No strict syntax and style conventions
Lack of quality assessment tools

14. 14
Challenges
No standard definition of comment quality
No strict syntax and style conventions
Lack of quality assessment tools

15. 15
All these make quality assessment a non-trivial problem
Challenges
No standard definition of comment quality
No strict syntax and style conventions
Lack of quality assessment tools

16. 16
Increasing multi-language environments

17. 17
Increasing multi-language environments

18. 18
Increasing multi-language environments
- Tomassetti et al. 2014
97% of open-source projects used two or more programming languages

19. 19
Gillies et al. 2011
Quality is a multi-dimensional concept

20. 20
Gillies et al. 2011
It requires a multi-perspective view

21. 21
P1
academic support
developer concerns
about comments
P3
developer
commenting practices
across languages P2
We de
fi
ne three perspectives

22. 22
P1
academic support
developer concerns
about comments
P3
developer
commenting practices
fi
ne three perspectives

23. 23
P1
academic support
developer concerns
about comments
P3
developer
commenting practices
fi
ne three perspectives

24. 24
How do researchers measure comment quality?

25. 25
Previous work on software documentation
Zhi et al. 2014
All kinds of software
documentation
Studies of 1971-2010
Only 10% of the studies focused on code comments

26. Systematic literature review
10 years timeline (2010-2020)
195 software engineering venues
332 proceedings
48 relevant papers
26
How do researchers measure comment quality?

27. 27
Analyzed dimensions
method comments, class comments
Comment types

28. 28
consistency, completeness
Quality attributes
method comments, class comments
Comment types
Analyzed dimensions

29. 29
consistency, completeness
Quality attributes
Techniques
heuristic-based, machine learning-based
Comment types
method comments, class comments
Analyzed dimensions

30. 30
Comment types

31. 31
Comment types
52% of the studies

32. 32
Comment types
48% of the studies

33. 33
Comment types
The studies focus on specific types of comments but class comments

34. 34
Comment types
% of studies

35. % of studies
35
Comment types
88% of the studies focus on Java

36. 36
Comment types
% of studies

37. 37
Comment types
% of studies

38. 38
Quality attributes
21 quality attributes

39. 39
Quality attributes
Consistency
Completeness
Accuracy
Readability
Up-to-date-ness
Content relevance
Maintainability
Spelling and grammar
Quality attributes
21 quality attributes

40. 40
Quality attributes
Some quality attributes are frequently considered
Consistency
Completeness
Accuracy
Readability
Up-to-date-ness
Content relevance
Maintainability
Spelling and grammar
Quality attributes

41. 41
Quality attributes Consistency
Completeness
Accuracy
Readability
Up-to-date-ness
Content relevance
Maintainability
Spelling and grammar
Conciseness
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Documentation technoogy
Internatioalization
Author-related
Quality attributes
Some quality attributes are frequently considered

42. 42
Quality attributes
Some quality attributes are rarely considered
Consistency
Completeness
Accuracy
Readability
Up-to-date-ness
Content relevance
Maintainability
Spelling and grammar
Conciseness
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Documentation technoogy
Internatioalization
Author-related
Quality attributes

43. 43
Techniques Consistency
Completeness
Accuracy
Readability
Up-to-date-ness
Content relevance
Maintainability
Spelling and grammar
Conciseness
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Documentation technoogy
Internatioalization
Author-related
Quality attributes

44. 44
Techniques
Manual assessment
Consistency
ML-based
Heuristic-based
Completeness
Accuracy
Metric-based
Static analysis
NLP
Readability
Up-to-date-ness
Content relevance
DNN-based
Maintainability
Spelling and grammar
Conciseness
First order logic
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Empirically
fNIRS
Documentation technoogy
Internatioalization
Author-related
Techniques
Quality attributes

45. 45
Techniques
Manual assessment
Consistency
ML-based
Heuristic-based
Completeness
Accuracy
Metric-based
Static analysis
NLP
Readability
Up-to-date-ness
Content relevance
DNN-based
Maintainability
Spelling and grammar
Conciseness
First order logic
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Empirically
fNIRS
Documentation technoogy
Internatioalization
Author-related
Techniques
Quality attributes

46. Manual assessment
Consistency
ML-based
Heuristic-based
Completeness
Accuracy
Metric-based
Static analysis
NLP
Readability
Up-to-date-ness
Content relevance
DNN-based
Maintainability
Spelling and grammar
Conciseness
First order logic
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Empirically
fNIRS
Documentation technoogy
Internatioalization
Author-related
Techniques
Quality attributes
46
Techniques
Manual assessment is still the most
frequent technique to measure
quality attributes

47. Manual assessment
Consistency
ML-based
Heuristic-based
Completeness
Accuracy
Metric-based
Static analysis
NLP
Readability
Up-to-date-ness
Content relevance
DNN-based
Maintainability
Spelling and grammar
Conciseness
First order logic
Usability
Correctness
Traceability
Accessibility
Coherence
Format
Information organization
Understandability
Empirically
fNIRS
Documentation technoogy
Internatioalization
Author-related
Techniques
Quality attributes
47
Techniques
Deep learning-based techniques have
not been extensively explored for
comment analysis

48. 48
Take-home messages
Class comments provide an overview of a program,
but are not analyzed enough.
Studies focus mainly on Java.
Manual assessment is still the most frequent technique to measure
various quality attributes.

49. 49
Take-home messages
Class comments provide an overview of a program, but are not
analyzed enough.
Studies focus mainly on Java.
Manual assessment is still the most frequent technique to measure
various quality attributes.

50. 50
Take-home messages
Class comments provide an overview of a program, but are not
analyzed enough.
Studies focus mainly on Java.
Manual assessment is still the most frequent
technique to measure various quality attributes.

51. 51
P. Rani, A. Blasi, N. Stulova, S. Panichella, A. Gorla, and O.
Nierstrasz. A Decade of comment quality assessment: A
systematic literature review, Journal of Systems & Software,
2021

52. 52
We de
fi
ne three perspectives
P1
academic support
developer concerns
about comments
P3
developer
commenting practices

53. fi
ne three perspectives
P1
academic support
developer concerns
about comments
P3
developer
commenting practices

54. fi
ne three perspectives
P1
academic support
developer concerns
about comments
P3
developer
commenting practices
across languages P2

55. 55
P1
academic support
developer concerns
about comments
P3
developer
commenting practices
across languages P2
P2: class commenting practices

56. Do they follow the coding style guidelines?
56
What information developers write in class
comments across languages?

57. Do they follow the coding style guidelines?
57
What information developers write in class
comments across languages?

58. 58
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Information types in comments

59. 59
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Usage
Information types in comments

60. 60
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Usage
Pointer
Information types in comments

61. 61
Comment taxonomies in Java and Python
Zhang et al., 2018
Pascarella et al., 2017
Java Python

62. 62
Comment taxonomies in Java and Python
Zhang et al., 2018
Pascarella et al., 2017
The taxonomies do not focus specifically on class comments
Java Python

63. 63
Smalltalk class comments
Intent

64. 64
Smalltalk class comments
Example

65. 65
Smalltalk class comments
Pharo 7 core
Implementation
details

66. 66
Smalltalk class comments
Implementation
details
Pharo 7 core
Intent
Example
Unlike Java and Python, there is no comment taxonomy

67. 67
Information types in Smalltalk
Pharo 7 core
Smalltalk
6,324 class comments
363 sample comments

68. 68
Information types in Smalltalk
Pharo 7 core
Smalltalk
6,324 class comments
363 sample comments
1
1
2
3
4
3
5
4
8
10
11
21
29
34
33
43
40
51
57
75
134
231
256
0 50 100 150 200 250 300
License
TODO comments
Links
Coding guidelines
Extensions
Other
Naming conventions
Observations
Subclasses explanation
Recommendations
Discourse
ReferencesOtherResources
Dependencies
Contracts
Key messages
Instance variables
Warnings
Implementation points
Examples
Class references
Collaborators
Responsibility
Intent
# comments
23
Categories

69. 69
Information types in Smalltalk
1
1
2
3
4
3
5
4
8
10
11
21
29
34
33
43
40
51
57
75
134
231
256
0 50 100 150 200 250 300
License
TODO comments
Links
Coding guidelines
Extensions
Other
Naming conventions
Observations
Subclasses explanation
Recommendations
Discourse
ReferencesOtherResources
Dependencies
Contracts
Key messages
Instance variables
Warnings
Implementation points
Examples
Class references
Collaborators
Responsibility
Intent
# comments
23
Categories
High-level overview

70. 70
Information types in Smalltalk
1
1
2
3
4
3
5
4
8
10
11
21
29
34
33
43
40
51
57
75
134
231
256
0 50 100 150 200 250 300
License
TODO comments
Links
Coding guidelines
Extensions
Other
Naming conventions
Observations
Subclasses explanation
Recommendations
Discourse
ReferencesOtherResources
Dependencies
Contracts
Key messages
Instance variables
Warnings
Implementation points
Examples
Class references
Collaborators
Responsibility
Intent
# comments
23
Categories
Conversation
exchanged
High-level overview

71. 71
We mapped
fi
rst the existing taxonomies of
Java, Python, and Smalltalk and then
adapted them to class comments.

72. 20 open-source projects
Java, Python, Smalltalk
37,446 class comments
1,066 sample comments
72
Information types across languages

73. 73
Deprecation
Expand
Pointer
Rationale
Summary
Usage
Class reference
Collaborator
Examples
Intent
Key implementation point
Key message
Responsibility
Development notes
Expand
Links
Summary
Usage
Java Smalltalk Python
Information types across languages

74. 74
Deprecation
Expand
Pointer
Rationale
Summary
Usage Class reference
Collaborator
Example
Instance variable
Intent
Key implementation point
Key message
ReferenceOtherResource
Responsibility
Warning
Development notes
Expand
Links
Parameters
Summary
Usage
Version
Java Smalltalk Python
Information types across languages
Developers write similar kinds of information in class comments across languages

75. 75
Deprecation
Expand
Pointer
Rationale
Summary
Usage Class reference
Collaborator
Example
Instance variable
Intent
Key implementation point
Key message
ReferenceOtherResource
Responsibility
Warning
Development notes
Expand
Links
Parameters
Summary
Usage
Version
Java Smalltalk Python
Information types across languages
There are also other language-specific information types

76. 76
Automatic identi
fi
cation of information types
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}

77. 77
Automatic identi
fi
cation of information types
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Recurrent natural language patterns exist in various information types

78. 78
Automatic identi
fi
cation of information types
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Represents [something]
[verb]s [noun]
Recurrent natural language patterns exist in various information types

79. 79
Automatic identi
fi
cation of information types
Summary
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Pointer
Represents [something]
[verb]s [noun]
Recurrent natural language patterns exist in various information types

80. 80
Automatic identi
fi
cation of information types
Summary Represents [something]
[verb]s [noun]
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Pointer Sees [something]
Recurrent natural language patterns exist in various information types

81. 81
Automatic identi
fi
cation of information types
Summary Represents [something]
[verb]s [noun]
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Pointer Sees [something]
How to extract such patterns?

82. • To automatically identify textual
patterns in informal software
documents, intention mining can
be used.
82
Intention mining in informal software documents
Di Sorbo et al., 2015
Di Sorbo et. al., Development Emails Content Analyzer: Intention Mining in Developer Discussions (T). ASE 2015: 12-23

83. • To automatically identify textual
patterns in informal software
documents, intention mining can
be used.
• Di Sorbo et al., developed a tool,
NEON, to detect natural language
patterns.
83
Intention mining in informal software documents
Di Sorbo et al., 2015
Di Sorbo et. al., An NLP-based Tool for Software Artifact Analysis. ICSME 2015

84. 84
Automatic identification of information types
Supervised ML
algorithms
Ground truth: 1,066
classified comments
Techniques
Textual Analysis (TA)
Features
1) 2) 3)
Learning phase Evaluation
4)
TA Features
NLP Rule Features
J48
Naive Bayes
Random Forest,
Natural Language
Processing (NLP)
Projects CCTM
CCTM
Features: recurrent NL
patterns + text features

85. 85
Automatic identification of information types
Supervised ML
algorithms
Ground truth: 1,066
classified comments
Techniques
Textual Analysis (TA)
Features
1) 2) 3)
Learning phase Evaluation
4)
TA Features
NLP Rule Features
J48
Naive Bayes
Random Forest,
Natural Language
Processing (NLP)
Projects CCTM
CCTM
Features: recurrent NL
patterns + text features

86. 86
Results
Random Forest technique classifies comments better
0
0.2
0.4
0.6
0.8
1
Summary
Expand
Ownership
Pointer
Usage
Deprecation
Rationale
Accuracy
Top categories in Java comments
NaiveBayes J48 RandomForest
Summary
Usage
Expand
DevelopmentNotes
Parameters
Top categories in Python comments
NaiveBayes J48 RandomForest
Responsibility
Intent
Collaborators
Examples
ClassReferences
KeyMessage
ImplementationPoint
Top categories in Smalltalk comments
NaiveBayes J48 RandomForest
Top categories in Java comments Top categories in Python comments Top categories in Smalltalk comments
S
u
m
m
a
r
y
E
x
p
a
n
d
O
w
n
e
r
s
h
i
p
P
o
i
n
t
e
r
U
s
a
g
e
D
e
p
r
e
c
a
t
i
o
n
R
a
t
i
o
n
a
l
e
P
a
r
a
m
e
t
e
r
s
R
e
s
p
o
n
s
i
b
i
l
i
t
y
I
n
t
e
n
t
C
o
l
l
a
b
o
r
a
t
o
r
s
E
x
a
m
p
l
e
s
C
l
a
s
s
R
e
f
e
r
e
n
c
e
K
e
y
M
e
s
s
a
g
e
I
m
p
l
e
m
e
n
t
a
t
i
o
n
P
o
i
n
t
S
u
m
m
a
r
y
U
s
a
g
e
E
x
p
a
n
d
D
e
v
e
l
o
p
m
e
n
t
N
o
t
e
s

87. 87
Class comments contain high-level design to low-
level implementation details.
Using recurrent NL patterns as features improves the classi
fi
cation.
Some information types need more advanced techniques to
accurately identify.
Take-home messages

88. 88
Class comments contain high-level design to low-level
implementation details.
Using recurrent NL patterns as features improves
the classi
fi
cation.
Some information types need more advanced techniques to
accurately identify.
Take-home messages

89. 89
Class comments contain high-level design to low-level
implementation details.
Using recurrent NL patterns as features improves the classi
fi
cation.
Some information types need more advanced
techniques to accurately identify.
Take-home messages

90. 90
P. Rani, S. Panichella, M. Leuenberger, M. Ghafari, and O.
Nierstrasz. What do class comments tell us? An
investigation of comment evolution and practices in Pharo
Smalltalk, Empirical Software Engineering, 2021
Replication Package on GitHub
https://github.com/poojaruhal/CommentAnalysisInPharo

91. 91
P. Rani, S. Panichella, M. Leuenberger, A. Sorbo, and O.
Nierstrasz. How to identify class comment types? A multi-
language approach for class comment classi
fi
cation, Journal
of Systems & Software, 2021
Replication Package on GitHub
https://github.com/poojaruhal/RP-class-comment-classification.

92. Do they follow the coding style guidelines?
92
What information developers write in class
comments across languages?

93. 93
Coding style guidlines
Each community has its own guidelines
Apache
Oracle
Google Numpy
PEP8/257

94. Each community has its own guidelines
Apache
Oracle
Google Numpy
PEP8/257
94
Coding style guidlines
First sentence should summarise the class
Use phrases instead of complete sentences
The @deprecated description should tell
what to use as a replacement

95. 95
Coding style guidlines
94 rules 13 rules 29 rules 222 rules 76 rules
Comment related rules are hard to locate
First sentence should summarise the class
Use phrases instead of complete sentences
The @deprecated description should tell
what to use as a replacement
Each community has its own guidelines
Apache
Oracle
Google
PEP8/257
Numpy

96. 96
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Do developers follow comment conventions?
First sentence should summarise the class
Use phrases instead of complete sentences
The @deprecated description should tell
what to use as a replacement

97. First sentence should summarise the class
97
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Do developers follow comment conventions?
Followed

98. First sentence should summarise the class
Use phrases instead of complete sentences
98
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Do developers follow comment conventions?
Not Followed

99. First sentence should summarise the class
Use phrases instead of complete sentences
The @deprecated description should tell
what to use as a replacement
99
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Do developers follow comment conventions?
The rule is not applicable

100. 20 open-source projects
Java, Python, Smalltalk
1,245 class comment conventions
1,066 sample class comments
100
Do developers follow the style guidelines?

101. 101
38%
37%
Python
Java
Followed rules
Do developers follow comment conventions?

102. 102
31%
38%
37%
30%
7%
2%
40%
55%
60%
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Do developers follow comment conventions?

103. 103
31%
38%
37%
30%
7%
2%
40%
55%
60%
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Do developers follow comment conventions?

104. 104
31%
38%
37%
30%
7%
2%
40%
55%
60%
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Do developers follow comment conventions?

105. 105
31%
38%
37%
30%
7%
2%
40%
55%
60%
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Developers do not always adopt conventions (not applicable rules)
Java developers violate rules less often than Python and Smalltalk
Do developers follow comment conventions?

106. 106
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules
Do developers follow comment conventions?

107. 107
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules
Do developers follow comment conventions?

108. 108
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules
Do developers follow comment conventions?

109. 109
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
36%
32%
33%
31%
26%
25%
26%
7%
2%
10%
3%
6%
10%
12%
3%
3%
2%
2%
3%
2%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules Not followed rules
Do developers follow comment conventions?

110. 110
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
36%
32%
33%
31%
26%
25%
26%
7%
2%
10%
3%
6%
10%
12%
3%
3%
2%
2%
3%
2%
26%
40%
37%
46%
42%
45%
41%
44%
83%
43%
86%
40%
42%
50%
70%
65%
73%
74%
35%
45%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Do developers follow comment conventions?

111. 111
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
36%
32%
33%
31%
26%
25%
26%
7%
2%
10%
3%
6%
10%
12%
3%
3%
2%
2%
3%
2%
26%
40%
37%
46%
42%
45%
41%
44%
83%
43%
86%
40%
42%
50%
70%
65%
73%
74%
35%
45%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Do developers follow comment conventions?

112. 112
38%
28%
29%
23%
32%
30%
33%
49%
16%
47%
11%
54%
49%
38%
27%
32%
25%
24%
63%
53%
36%
32%
33%
31%
26%
25%
26%
7%
2%
10%
3%
6%
10%
12%
3%
3%
2%
2%
3%
2%
26%
40%
37%
46%
42%
45%
41%
44%
83%
43%
86%
40%
42%
50%
70%
65%
73%
74%
35%
45%
Pharo 1
Pharo 2
Pharo 3
Pharo 4
Pharo 5
Pharo 6
Pharo 7
Django
iPython
Mailpile
Pandas
Pipenv
Pytorch
Requests
Eclipse
Vaadin
Spark
Hadoop
Guava
Guice
Smalltalk
Python
Java
Followed rules Not followed rules Not applicable rules
Do developers follow comment conventions?

113. 113
P. Rani, S. Abukar, N. Stulova, A. Bergel, and O. Nierstrasz. Do
comments follow commenting conventions? A case study
in Java and Python, In Proceedings of 21st International
Working Conference on Source Code Analysis and
Manipulation (SCAM), 2021
Replication Package on Zenodo
https://doi.org/10.5281/zenodo.5296443
Video on YouTube
https://youtu.be/mX_9XxQTSxQ

114. 114
We de
fi
ne three perspectives
P1
academic support
developer concerns
about comments
P3
developer
commenting practices
across languages P2

115. 115
We de
fi
ne three perspectives
P1
academic support
developer concerns
about comments
P3
developer
commenting practices

117. Availability of multiple style guidelines
Lack of tools to verify all aspects of comments

118. 118
Developer concerns about comments
Various syntax and structure conventions
Availability of multiple style guidelines
Lack of tools to verify all aspects of comments

119. 119
Developer concerns about comments
Various syntax and structure conventions
Availability of multiple style guidelines
Lack of tools to verify all aspects of comments

120. 120
Developer concerns about comments
Various syntax and structure conventions
Availability of multiple style guidelines
Lack of tools to verify all aspects of comments
Confusion among developers on what convention to
adopt and when

121. 121
Ask questions on various online platforms
Ask questions
Quora
Stack Over
fl
ow

122. Stack Over
fl
ow, Quora
23,631 comment related posts
1,400 sample posts
Automated analysis (LDA topic modelling)
Manual analysis
122
Developer concerns about comments

123. 123
LDA Topic modelling
# Topic Name
1 Syntax and Format
2 IDEs & Editors
3 R Documenta
ti
on
4 Code Conven
ti
ons
5 Developing frameworks for thread commen
ti
ng
10 topics

124. 124
LDA Topic modelling
# Topic Name
1 Syntax and Format
2 IDEs & Editors
3 R Documenta
ti
on
4 Code Conven
ti
ons
5 Developing frameworks for thread commen
ti
ng
6 Open-source so
ft
ware
7 Documenta
ti
on genera
ti
on
8 Thread comments in web-sites
9 Naming conven
ti
ons & data types
10 Seeking documenta
ti
on & learning language
Expected topics like Documentation Generation or Syntax and Format were
successfully identi
fi
ed by LDA.

125. 125
LDA Topic modelling
Comment term is used in various contexts and environments
# Topic Name
1 Syntax and Format
2 IDEs & Editors
3 R Documenta
ti
on
4 Code Conven
ti
ons
5 Developing frameworks for thread commen
ti
ng
6 Open-source so
ft
ware
7 Documenta
ti
on genera
ti
on
8 Thread comments in web-sites
9 Naming conven
ti
ons & data types
10 Seeking documenta
ti
on & learning language

126. 126
Manual analysis
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns

127. 127

128. 128
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns

129. 129
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns

130. 130
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns

131. 131
Developer concerns about comments
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns

132. 132
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns

133. 133
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns
how to write/implement comments

134. 134
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns
Is it possible to do this?

135. 135
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types

136. 136
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
Developers ask frequently “how to write comments?”

137. 137
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
Followed by “is it possible” questions

138. 138
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types

139. 139
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
Stack Overflow Quora

140. 140
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
Stack Overflow Quora
Developers prefer different sources to know different aspects of comments

141. 141
Developer concerns about comments
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
% of question types
0% 10% 20% 30% 40%
Error
Opinion
Implementation problem
Limitation and possibility
Background information
Best practice
Implementation strategy
Types
of
questions
Stack Overflow Quora
Developers prefer different sources to know different aspects of comments

142. 142
Syntax and Format

143. 143
Syntax and Format
Developers want to improve Function documentation

144. 144
Features asked
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
Adding information in comments

145. 145
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns
Visualizing comments or part of comments
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
% of posts asking for a feature
Feautres
developer
ask
Features asked

146. 146
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns
Detecting inconsistent, incomplete, or missing comments
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
Features asked

147. 147
T
y
p
e
s
o
f
q
u
e
s
t
i
o
n
s
Recommendation
I
n
f
o
r
m
a
t
i
o
n
n
e
e
d
s
Comment
concerns
0 % 5 % 10 % 15 % 20 % 25 %
Show inherited methods comments
Detect Todo comments
Detect undocumented code files
Color scheme
Highlight comments
Synchronize code comment changes
Ask for documentation
Multi-lingual comments
Add comments
Render documentation
Other
Show comments
Add media
Generate documentation
Add more information
% of posts asking for a feature
Feautres
developer
ask
Features asked

148. 148
Take-home messages
Prefer different sources to know different aspects.
Confusion about how to write comments and use various comment
tools.
Interest in embedding various information in comments but lack
conventions and tools to do it automatically.

149. 149
Take-home messages
Prefer different sources to know different aspects.
Confusion about how to write comments and use
various comment tools.
Interest in embedding various information in comments but lack
conventions and tools to do it automatically.

150. 150
Take-home messages
Prefer different sources to know different aspects.
Confusion about how to write comments and use various comment
tools.
Interest in embedding various information in
comments but lack conventions and tools to do it
automatically.

151. 151
P. Rani, M. Birrer, S Panichella, M Ghafari, and O Nierstrasz.
What do developers discuss about code comments?, In
Proceedings of 21st International Working Conference on
Source Code Analysis and Manipulation (SCAM), 2021
Replication Package on Zenodo
https://doi.org/10.5281/zenodo.5044270
Video on
https://youtu.be/EUQINZ38ziU

152. 152
P. Rani, M. Birrer, S. Panichella, and O. Nierstrasz. Makar: A
Framework for Multi-source Studies based on Unstructured
Data, In proceedings of IEEE International Conference on
Software Analysis, Evolution and Reengineering (SANER), 2021
Replication Package on Zenodo
https://doi.org/10.5281/zenodo.4434822
Video on
https://youtu.be/Yqj1b4Bv-58
Hosted on
https://github.com/maethub/makar

153. 153
Conclusions
P1
academic support
Studies focus on Java
21 quality attributes for comments
Many attributes are assessed manually
developer commenting
practices
across languages P2
16 different types of information in class comments
Recurrent patterns help in identifying information
Developers do not adopt various conventions
developer concerns about
comments
P3
Add more information to comments
Visualize comments
Detect inconsistent, incomplete, missing comments

154. 154
Future work
Assessing speci
fi
c required information from comments

155. 155
/**
* A class representing a window on the screen.
*
* For example:
* <pre>
* Window win = new Window(parent);
* win.show();
* </pre>
*
* @author Sami Shaio
* @version 1.13, 06/08/06
* @see java.awt.BaseWindow
* @see java.awt.Button
*/
class Window extends BaseWindow {
…
}
Assessing specific required information
Task Design Implementation Maintenance
Summary
Usage
Pointer
Information Quality attributes
Accessibility
Readability
Author-related
Completeness
Consistency
Correctness
Audience User Developer

156. 156
Future work
Assessing speci
fi
c required information from comments
Visualizing documentation effort

157. 157
Visualizing documentation effort
Visualizing a project hierarchy for documentation effort
a
b c
d e f g h

158. 158
Visualizing documentation effort
Visualizing a project hierarchy for documentation effort
a root class with comments
a method without comment
a
b c
d e f g h
a
b c
d e f g h

159. 159
Future work
Assessing speci
fi
c required information from comments
Visualizing documentation effort
Analyzing support of documentation tools to comments

160. 160
Future work
Assessing speci
fi
c required information from comments
Visualizing documentation effort
Analyzing support of documentation tools to comments
Speculative Analysis of comment quality

161. 161
Future work
Assessing speci
fi
c required information from comments
Visualizing commenting effort
Analyzing support of documentation tools to comments
Speculative Analysis of comment quality

162. 162
Summary

163. –Pooja Rani
Thank you
163

164. Backup slides