Toward Understanding How Developers Recognize Features in Source Code from Descriptions
1. Toward Understanding
How Developers Recognize
Features in Source Code
from Descriptions
Shinpei Hayashi†,Takashi Ishio‡,
Hiroshi Kazato*, andTsuyoshi Oshima**
†Tokyo Institute of Technology ‡Osaka University
*NTT DATA CORPORATION **NTT Software Innovation Center
1
2. Feature Location (FL)
A first step of software maintenance
– Which modules/functions/methods implement a
feature?
– How they interact?
Feature location process (for Java)
– A feature description A list of relevant methods
– Search keywords to find seed methods
– Explore the seed methods and their neighbors
Many automated FL techniques proposed
2
3. Feature Boundary
Our aim:To clarify the boundary of features
– Or:To specify features by description, etc.
Why?
– For analysts:Writing a clear change specification for
another implementer
– For researchers: Building a better FL benchmark
4
feature
(description)
set of modules
4. FL Benchmark
A basic infrastructure for comparing/evaluating
FL techniques
Content: Input of FL and corresponding output
– Source code
– A set of features
• Name of feature
• Natural language description explaining the feature
– Oracle: location of the feature
• e.g., Set of modules (methods in Java)
5
5. Usage of FL Benchmark
6
Benchmark
FL techniqueFL user/
researcher
Source
code
Feature name/
description
Dynamic trace
(for dynamic FL)
Input
Output
Set of modules
(ordered/unordered)
Set of modules
(Oracle)
Compare
Precision/Recall
effectiveness measure,
etc.
6. Current FL Benchmark
Based on change history and issue tracking
– Assumes that modules modified when adding the
feature will implement the feature
– Associating features and modules by analyzing change
history- and issue-repositories
7
Change
historySVN
ITS
Release ofVer. 2.0
Extracting
source code
#123: Feature XXX
It would be nice if XXX...
r456: Implementing #123.
class Foo {
void foo() {
bar();
+ baz();
} ...
Location of Feature XXX: { Foo.foo(), ... }
Features:
• XXX
• ...
7. 10
Gaps
(1) Feature description vs. Feature
– The description from ITS do not always describe
only the feature introduced by the revision.
(2) Feature vs. Change history
– Change history do not always record only the
changes related the the target feature.
Developer FeatureFeature
description
SVN
Change
history
read
(1)
(2)
8. Research Question
Does a clearer description of a feature enable
developers to identify the same set of modules?
Developer Source
code
Feature
description
Developer Source
code
Clearer Feature
description
Output
Set of modules
Output
Set of modules
?
11
9. Original Description
The description in the benchmark
–Extracted from ITS
12
As suggested in the [http://www.mucommander.
com/forums/viewtopic.php?f=2&t=938 forums]:
adding a “Skip all” button when an error occurs in
a multiple file move / copy operation would be a
nice feature to have.
muCommander Feature #231:
“Skip all” for errors that occur during a file transfer operation
10. Refined Description
2 paragraphs for explaining base/core separately
muCommander has a feature that copies/moves
files selected by a user.When a user tries to
execute a copy, muCommander shows a dialog to
specify a destination directory. Pushing the Copy
button in the dialog starts a copy process. If an
error occurred during the copying of a file, then an
error dialog shows a message and asks the user to
skip the file, retry to copy the file, or cancel the
copy process.
The new feature is “Skip All.”The dialog to specify a
destination directory has a new check box with the
caption is “Skip errors.” If a user checked the box,
then muCommander automatically skips a file if an
error occurred, without showing a dialog.The error
message dialog also has a new button “Skip all.” If
the button is pushed, then muCommander shows
no error dialog in further errors, as “Skip errors” is
checked.
Base behavior:
Explanation of basic
behavior which exists
before implementing the
target feature
Core behavior:
Explanation of the target
feature based on the
recorded change
13
11. Controlled Experiment
Task: Read a description and vote for modules
2 rounds
– One for Feature 1 w/ {original, refined} description
– One for Feature 2 w/ {original, refined} description
Accumulates the votes of all subjects
Subject
Feature
description
Input
Source
code
Output
Votes
This module is
- relevant.
- useful (for understanding).
- irrelevant.
14
12. Subjects and Dataset
Subjects
– 18 subjects (Students in Osaka U. + in Tokyo Tech.)
Dataset: using Dit et al.’s Benchmark
– Feature 1: jEdit #1747300
– Feature 2: muCommander #231
Obtained treatments
– 9 of Feature 1 w/ original description
– 9 of Feature 1 w/ refined description
– 9 of Feature 2 w/ original description
– 9 of Feature 2 w/ refined description
15
15. More vs. Less votes
Differences of # votes to modules in goldset
– Voted by many subjects: actually relevant?
– Voted by less subjects: actually irrelevant?
Unsuitableness in the benchmark?
18
e.addFoldStyleChooser()
Gutter.Gutter(TextArea)
Gutter.paintLine(Graphics2D,...)
TriangleFoldPainter.paintFoldEnd(Gutter,...)
TriangleFoldPainter.paintFoldStart(Gutter,...)
GutterOptionPane._init()
GutterOptionPane._save()
TriangleFoldPainter.paintFoldMiddle(Gutter,...)
EditPane.propertiesChanged()
Gutter.setFoldPainter(FoldPainter)
StandaloneTextArea.StandaloneTextArea(IPropertyManager)
ShapedFoldPainter.paintFoldStart(Gutter,...)
ShapedFoldPainter.paintFoldEnd(Gutter,...)
FoldPainter.paintFoldStart(Gutter,...)
FoldPainter.paintFoldEnd(Gutter,...)
FoldPainter.paintFoldMiddle(Gutter,...)
JEditTextArea.getFoldPainterName()
JEditTextArea.getFoldPainter()
TextArea.getFoldPainter()
ServiceManager.getServiceNames(String)
AbstractOptionPane.save()
DummyFoldHandler.DummyFoldHandler()
ExplicitFoldHandler.ExplicitFoldHandler()
IndentFoldHandler.IndentFoldHandler()
JEditBuffer.propertiesChanged()
MouseHandler.mousePressed(MouseEvent)
JEditTextArea.JEditTextArea(View)
DefaultFoldHandlerProvider.DefaultFoldHandlerProvider()
ExplicitFoldHandler.getFoldLevel(JEditBuffer,...)
ServiceManager.getService(String,...)
Gutter.updateBorder()
CircleFoldPainter.CircleFoldPainter()
Gutter.setBorder(Border)
SquareFoldPainter.SquareFoldPainter()
StandaloneTextArea.initPainter()
TextAreaPainter.setFoldLineStyle(SyntaxStyle[])
TriangleFoldPainter.TriangleFoldPainter()
CircleFoldPainter.paintFoldShape(Graphics2D,...)
SquareFoldPainter.paintFoldShape(Graphics2D,...)
ShapedFoldPainter.paintFoldMiddle(Gutter,...)
Gutter.paintComponent(Graphics)
MouseHandler.getFoldEndOffset(int)
DefaultFoldHandlerProvider.addFoldHandler(FoldHandler)
StandaloneTextArea.initGutter()
ShapedFoldPainter.paintFoldShape(Graphics2D,...)
JEditBuffer.JEditBuffer()
DisplayManager.expandFolds(int)
Gutter.setGutterEnabled(boolean)
Included in
the goldset
Not included in
the goldset
PreferencesDialog.setCommitButtonsEnabled(boolean)
GeneralPreferencesDialog.setCommitButtonsEnabled(boolean)
PreferencesDialog.initUI()
GeneralPreferencesDialog.componentChanged(PrefComponent)
PreferencesDialog.commit()
ThemeEditorDialog.componentChanged(PrefComponent)
PrefTextField.addDialogListener(PreferencesDialog)
PrefComboBox.addDialogListener(PreferencesDialog)
GeneralPreferencesDialog.GeneralPreferencesDialog()
PrefCheckBox.addDialogListener(PreferencesDialog)
PrefRadioButton.addDialogListener(PreferencesDialog)
ColorButton.setCurrentColor(Color,...)
FoldersPanel.FoldersPanel(PreferencesDialog)
GeneralPanel.GeneralPanel(PreferencesDialog)
MailPanel.MailPanel(PreferencesDialog)
MiscPanel.MiscPanel(PreferencesDialog)
ThemeFontChooserListener.stateChanged(ChangeEvent)
GeneralPreferencesDialog.commit()
PreferencesDialog.actionPerformed(ActionEvent)
PreferencesDialog.PreferencesDialog(Frame,...)
PrefEncodingSelectBox.addDialogListener(PreferencesDialog)
PreferencesDialog.PreferencesDialog(Dialog,...)
ThemeEditorDialog.commit()
PreferencesDialog.componentChanged(PrefComponent)
ThemeEditorDialog.ThemeEditorDialog(Dialog,...)
ShowPreferencesAction.performAction()
PreferencesDialog.checkCommit()
PrefTable.addDialogListener(PreferencesDialog)
AbstractButton.setEnabled(boolean)
ThemeEditorDialog.checkCommit()
ThemeData.isIdentical(ThemeData)
AppearancePanel.AppearancePanel(PreferencesDialog)
CustomizeDialog.initUI()
TransferDestinationDialog.TransferDestinationDialog(MainFrame,...)
ShortcutsPanel.ShortcutsPanel(PreferencesDialog)
CommandBarDialog.createCustomizationPanel()
CustomizeDialog.actionPerformed(ActionEvent)
LicenseDialog.createLicensePanel()
PrefComponent.hasChanged()
QuitDialog.QuitDialog(MainFrame)
InitialSetupDialog.InitialSetupDialog(Frame)
AboutDialog.createCreditsPanel()
Included in
the goldset
Not included in
the goldset useful (original)
relevant (original)
relevant (refined)
useful (refined)
16 votes 1 vote
Finding 1:
16. Original vs. Refined
More subjects voted some modules in goldset
using refined description
Refined description contributed for
some modules?
19
e.addFoldStyleChooser()
Gutter.Gutter(TextArea)
Gutter.paintLine(Graphics2D,...)
TriangleFoldPainter.paintFoldEnd(Gutter,...)
TriangleFoldPainter.paintFoldStart(Gutter,...)
GutterOptionPane._init()
GutterOptionPane._save()
TriangleFoldPainter.paintFoldMiddle(Gutter,...)
EditPane.propertiesChanged()
Gutter.setFoldPainter(FoldPainter)
StandaloneTextArea.StandaloneTextArea(IPropertyManager)
ShapedFoldPainter.paintFoldStart(Gutter,...)
ShapedFoldPainter.paintFoldEnd(Gutter,...)
FoldPainter.paintFoldStart(Gutter,...)
FoldPainter.paintFoldEnd(Gutter,...)
FoldPainter.paintFoldMiddle(Gutter,...)
JEditTextArea.getFoldPainterName()
JEditTextArea.getFoldPainter()
TextArea.getFoldPainter()
ServiceManager.getServiceNames(String)
AbstractOptionPane.save()
DummyFoldHandler.DummyFoldHandler()
ExplicitFoldHandler.ExplicitFoldHandler()
IndentFoldHandler.IndentFoldHandler()
JEditBuffer.propertiesChanged()
MouseHandler.mousePressed(MouseEvent)
JEditTextArea.JEditTextArea(View)
DefaultFoldHandlerProvider.DefaultFoldHandlerProvider()
ExplicitFoldHandler.getFoldLevel(JEditBuffer,...)
ServiceManager.getService(String,...)
Gutter.updateBorder()
CircleFoldPainter.CircleFoldPainter()
Gutter.setBorder(Border)
SquareFoldPainter.SquareFoldPainter()
StandaloneTextArea.initPainter()
TextAreaPainter.setFoldLineStyle(SyntaxStyle[])
TriangleFoldPainter.TriangleFoldPainter()
CircleFoldPainter.paintFoldShape(Graphics2D,...)
SquareFoldPainter.paintFoldShape(Graphics2D,...)
ShapedFoldPainter.paintFoldMiddle(Gutter,...)
Gutter.paintComponent(Graphics)
MouseHandler.getFoldEndOffset(int)
DefaultFoldHandlerProvider.addFoldHandler(FoldHandler)
StandaloneTextArea.initGutter()
ShapedFoldPainter.paintFoldShape(Graphics2D,...)
JEditBuffer.JEditBuffer()
DisplayManager.expandFolds(int)
Gutter.setGutterEnabled(boolean)
Included in
the goldset
Not included in
the goldset
PreferencesDialog.setCommitButtonsEnabled(boolean)
GeneralPreferencesDialog.setCommitButtonsEnabled(boolean)
PreferencesDialog.initUI()
GeneralPreferencesDialog.componentChanged(PrefComponent)
PreferencesDialog.commit()
ThemeEditorDialog.componentChanged(PrefComponent)
PrefTextField.addDialogListener(PreferencesDialog)
PrefComboBox.addDialogListener(PreferencesDialog)
GeneralPreferencesDialog.GeneralPreferencesDialog()
PrefCheckBox.addDialogListener(PreferencesDialog)
PrefRadioButton.addDialogListener(PreferencesDialog)
ColorButton.setCurrentColor(Color,...)
FoldersPanel.FoldersPanel(PreferencesDialog)
GeneralPanel.GeneralPanel(PreferencesDialog)
MailPanel.MailPanel(PreferencesDialog)
MiscPanel.MiscPanel(PreferencesDialog)
ThemeFontChooserListener.stateChanged(ChangeEvent)
GeneralPreferencesDialog.commit()
PreferencesDialog.actionPerformed(ActionEvent)
PreferencesDialog.PreferencesDialog(Frame,...)
PrefEncodingSelectBox.addDialogListener(PreferencesDialog)
PreferencesDialog.PreferencesDialog(Dialog,...)
ThemeEditorDialog.commit()
PreferencesDialog.componentChanged(PrefComponent)
ThemeEditorDialog.ThemeEditorDialog(Dialog,...)
ShowPreferencesAction.performAction()
PreferencesDialog.checkCommit()
PrefTable.addDialogListener(PreferencesDialog)
AbstractButton.setEnabled(boolean)
ThemeEditorDialog.checkCommit()
ThemeData.isIdentical(ThemeData)
AppearancePanel.AppearancePanel(PreferencesDialog)
CustomizeDialog.initUI()
TransferDestinationDialog.TransferDestinationDialog(MainFrame,...)
ShortcutsPanel.ShortcutsPanel(PreferencesDialog)
CommandBarDialog.createCustomizationPanel()
CustomizeDialog.actionPerformed(ActionEvent)
LicenseDialog.createLicensePanel()
PrefComponent.hasChanged()
QuitDialog.QuitDialog(MainFrame)
InitialSetupDialog.InitialSetupDialog(Frame)
AboutDialog.createCreditsPanel()
Included in
the goldset
Not included in
the goldset useful (original)
relevant (original)
relevant (refined)
useful (refined)
Finding 2:
17. Votes and Goldset
Many votes to some modules out of the goldset
– Including the ones mostly using refined description
Actually relevant?
20
e.addFoldStyleChooser()
Gutter.Gutter(TextArea)
Gutter.paintLine(Graphics2D,...)
TriangleFoldPainter.paintFoldEnd(Gutter,...)
TriangleFoldPainter.paintFoldStart(Gutter,...)
GutterOptionPane._init()
GutterOptionPane._save()
TriangleFoldPainter.paintFoldMiddle(Gutter,...)
EditPane.propertiesChanged()
Gutter.setFoldPainter(FoldPainter)
StandaloneTextArea.StandaloneTextArea(IPropertyManager)
ShapedFoldPainter.paintFoldStart(Gutter,...)
ShapedFoldPainter.paintFoldEnd(Gutter,...)
FoldPainter.paintFoldStart(Gutter,...)
FoldPainter.paintFoldEnd(Gutter,...)
FoldPainter.paintFoldMiddle(Gutter,...)
JEditTextArea.getFoldPainterName()
JEditTextArea.getFoldPainter()
TextArea.getFoldPainter()
ServiceManager.getServiceNames(String)
AbstractOptionPane.save()
DummyFoldHandler.DummyFoldHandler()
ExplicitFoldHandler.ExplicitFoldHandler()
IndentFoldHandler.IndentFoldHandler()
JEditBuffer.propertiesChanged()
MouseHandler.mousePressed(MouseEvent)
JEditTextArea.JEditTextArea(View)
DefaultFoldHandlerProvider.DefaultFoldHandlerProvider()
ExplicitFoldHandler.getFoldLevel(JEditBuffer,...)
ServiceManager.getService(String,...)
Gutter.updateBorder()
CircleFoldPainter.CircleFoldPainter()
Gutter.setBorder(Border)
SquareFoldPainter.SquareFoldPainter()
StandaloneTextArea.initPainter()
TextAreaPainter.setFoldLineStyle(SyntaxStyle[])
TriangleFoldPainter.TriangleFoldPainter()
CircleFoldPainter.paintFoldShape(Graphics2D,...)
SquareFoldPainter.paintFoldShape(Graphics2D,...)
ShapedFoldPainter.paintFoldMiddle(Gutter,...)
Gutter.paintComponent(Graphics)
MouseHandler.getFoldEndOffset(int)
DefaultFoldHandlerProvider.addFoldHandler(FoldHandler)
StandaloneTextArea.initGutter()
ShapedFoldPainter.paintFoldShape(Graphics2D,...)
JEditBuffer.JEditBuffer()
DisplayManager.expandFolds(int)
Gutter.setGutterEnabled(boolean)
Included in
the goldset
Not included in
the goldset
PreferencesDialog.setCommitButtonsEnabled(boolean)
GeneralPreferencesDialog.setCommitButtonsEnabled(boolean)
PreferencesDialog.initUI()
GeneralPreferencesDialog.componentChanged(PrefComponent)
PreferencesDialog.commit()
ThemeEditorDialog.componentChanged(PrefComponent)
PrefTextField.addDialogListener(PreferencesDialog)
PrefComboBox.addDialogListener(PreferencesDialog)
GeneralPreferencesDialog.GeneralPreferencesDialog()
PrefCheckBox.addDialogListener(PreferencesDialog)
PrefRadioButton.addDialogListener(PreferencesDialog)
ColorButton.setCurrentColor(Color,...)
FoldersPanel.FoldersPanel(PreferencesDialog)
GeneralPanel.GeneralPanel(PreferencesDialog)
MailPanel.MailPanel(PreferencesDialog)
MiscPanel.MiscPanel(PreferencesDialog)
ThemeFontChooserListener.stateChanged(ChangeEvent)
GeneralPreferencesDialog.commit()
PreferencesDialog.actionPerformed(ActionEvent)
PreferencesDialog.PreferencesDialog(Frame,...)
PrefEncodingSelectBox.addDialogListener(PreferencesDialog)
PreferencesDialog.PreferencesDialog(Dialog,...)
ThemeEditorDialog.commit()
PreferencesDialog.componentChanged(PrefComponent)
ThemeEditorDialog.ThemeEditorDialog(Dialog,...)
ShowPreferencesAction.performAction()
PreferencesDialog.checkCommit()
PrefTable.addDialogListener(PreferencesDialog)
AbstractButton.setEnabled(boolean)
ThemeEditorDialog.checkCommit()
ThemeData.isIdentical(ThemeData)
AppearancePanel.AppearancePanel(PreferencesDialog)
CustomizeDialog.initUI()
TransferDestinationDialog.TransferDestinationDialog(MainFrame,...)
ShortcutsPanel.ShortcutsPanel(PreferencesDialog)
CommandBarDialog.createCustomizationPanel()
CustomizeDialog.actionPerformed(ActionEvent)
LicenseDialog.createLicensePanel()
PrefComponent.hasChanged()
QuitDialog.QuitDialog(MainFrame)
InitialSetupDialog.InitialSetupDialog(Frame)
AboutDialog.createCreditsPanel()
Included in
the goldset
Not included in
the goldset useful (original)
relevant (original)
relevant (refined)
useful (refined)
Finding 3:
18. Further Directions
More investigations about descriptions
Fuzzy benchmark
– Likelihood values are added based on votes
Support for writing clearer descriptions
22
• PreferenecsDialog.initUI():0.7 (7 of 10 developers answered relevant)
...
List of
modules
fragile against
changes
Abstract
description
insufficient to
recognize
?