This document discusses code recommenders in the Eclipse IDE. It describes how the recommenders work by extracting patterns from code examples, building models of framework usage, and generating recommendations for code completion and call chains based on these models and other developers' common usage. Examples are provided of intelligent code completion and call chain completion recommendations. The goal is to leverage the wisdom of the software engineering crowds to help developers be more productive.

1. Eclipse Code
Recommenders
IDE
2.0:
Leveraging
the
Wisdom
of
the
So:ware
Engineering
Crowds eim
nh
tw:
@MarcelBruch an
g+:
+Marcel
Bruch up
M
ro
r
G
se
a
U
Jav

2. 2

3. Eclipse
Code
Recommenders
All Frameworks Code
Code
Eclipse Code
Developers Who Called “new Text()” Method Also Called
setText() addListener() setLayout() setFont()
89% 53% 99% 35%
What Do Developers Ultimately Override After Extending This Class?
100 % overwrote
PreferencePage.createContents()
96 % overwrote
PreferencePage.performOk()
5 % overwrote
PreferencePage.performCancel()
3

4. “Other
developers
frequently
bought
used
the
following
methods…”
INTELLIGENT
CODE
COMPLETION

5. Code
Comple+on…
@Override
protected Control createDialogArea(final Composite parent) {
Composite container = (Composite) super.createDialogArea(parent);
swtTextWidget = new Text(container, SWT.BORDER);
swtTextWidget.|
return container;
}
What
does
the
developer
need,
i.e.,
which
methods
should
the
code
compleKon
present
to
the
user?
5

7. All
164
Methods
of
Text
?

8. JBuSon?
381
Methods.

9. Code
Comple+on…
@Override
protected Control createDialogArea(final Composite parent) {
final Composite container = (Composite) super.createDialogArea(parent);
swtTextWidget = new Text(container, SWT.BORDER);
swtTextWidget.|
return container;
}
What
does
the
developer
need,
i.e.,
which
methods
should
the
code
compleKon
present
to
the
user?
8

10. Intelligent
Code
Comple+on
…
or
just
the
three
missing
ones?
9

11. How
it
works
-­‐
in
a
nutshell
10

12. How
it
works
-­‐
in
a
nutshell
Framework
10

13. How
it
works
-­‐
in
a
nutshell
Framework
Example
Example
...
ApplicaKon
1 ApplicaKon
n
10

14. How
it
works
-­‐
in
a
nutshell
Framework
Example
Example
...
ApplicaKon
1 ApplicaKon
n
extract
facts
about
how
the
framework
is
reused
«extends:A» «extends:A»
«overrides:A.b» ... «overrides:A.d»
«calls:B.c» «calls:B.c»
… …
10

15. How
it
works
-­‐
in
a
nutshell
Framework
Example
Example
...
ApplicaKon
1 ApplicaKon
n
extract
facts
about
how
the
framework
is
reused
«extends:A» «extends:A»
«overrides:A.b» ... «overrides:A.d»
«calls:B.c» «calls:B.c»
… …
10

16. How
it
works
-­‐
in
a
nutshell
Framework
If
you
extend
A,
Example
Example
you
should
... call
B.c
ApplicaKon
1 ApplicaKon
n
extract
facts
about
how
the
framework
is
reused
«extends:A» «extends:A»
«overrides:A.b» ... «overrides:A.d»
«calls:B.c» «calls:B.c»
… …
10

17. From
code
to
models...
()
ts
en ()
nt Ok
Co rm
te fo () t() t() t()
ea er t> ex on ex
cr .p ni tT tF tT
e. ge <i se se ge
ag Pa t. t. t. t.
class Example extends Page { :P n: ex ex ex ex ..
Text t; in i T T T T .
@Override 1 0 1 1 0 0 …
void createContents() {
t = new Text();
t.setText(..);
..
} 0 1 0 0 0 1 …
} … … … … … … …
11

18. From
code
to
models...
()
ts
en ()
nt Ok
Co rm
te fo () t() t() t()
ea er t> ex on ex
cr .p ni tT tF tT
e. ge <i se se ge
ag Pa t. t. t. t.
class Example extends Page { :P n: ex ex ex ex ..
Text t; in i T T T T .
@Override 1 0 1 1 0 0 …
void performOk() {
t.getText();
..
} 0 1 0 0 0 1 …
} … … … … … … …
12

19. From
models
to
recommenda+ons...
()
ts
en ()
nt Ok
Co rm
class MyPage extends Page { te fo () t() t() t()
ea er t> ex on ex
Text t; cr .p ni tT tF tT
@Override e. ge <i se se ge
ag Pa t. t. t. t.
void createContents() { :P n: ex ex ex ex ..
t = new Text(); in i T T T T .
t.|<^space> 1 0 1 ? ? ? …
}
}
13

20. From
models
to
recommenda+ons...
()
ts
en ()
nt Ok
Co rm
te fo () t() t() t()
ea er t> ex on ex
cr .p ni tT tF tT
e. ge <i se se ge
ag Pa t. t. t. t.
:P n: ex ex ex ex ..
in i T T T T .
1 0 1 1 1 0 …
1 0 1 0 0 0 …
1 0 1 1 0 0 …
0 1 0 0 0 1 …
… … … … … … …
1 0 1 ? ? ? …
14

21. From
models
to
recommenda+ons...
()
ts
en ()
nt Ok
Co rm
te fo () t() t() t()
ea er t> ex on ex
cr .p ni tT tF tT
e. ge <i se se ge
ag Pa t. t. t. t.
:P n: ex ex ex ex ..
in i T T T T .
1 0 1 1 1 0 …
1 0 1 0 0 0 …
1 0 1 1 0 0 …
0 1 0 0 0 1 …
… … … … … … …
1 0 1 ⅔
? ? ? …
14

22. From
models
to
recommenda+ons...
()
ts
en ()
nt Ok
Co rm
te fo () t() t() t()
ea er t> ex on ex
cr .p ni tT tF tT
e. ge <i se se ge
ag Pa t. t. t. t.
:P n: ex ex ex ex ..
in i T T T T .
1 0 1 1 1 0 …
1 0 1 0 0 0 …
1 0 1 1 0 0 …
0 1 0 0 0 1 …
… … … … … … …
1 0 1 ⅔
? ⅓
? ? …
14

23. From
models
to
recommenda+ons...
()
ts
en ()
nt Ok
Co rm
te fo () t() t() t()
ea er t> ex on ex
cr .p ni tT tF tT
e. ge <i se se ge
ag Pa t. t. t. t.
:P n: ex ex ex ex ..
in i T T T T .
1 0 1 1 1 0 …
1 0 1 0 0 0 …
1 0 1 1 0 0 …
0 1 0 0 0 1 …
… … … … … … …
1 0 1 ⅔
? ⅓
? 0
? …
14

24. “Welcome
to
the
forest...”
CALL-­‐CHAIN
COMPLETION

25. How
to
obtain
an
instance
of...
public class MyView extends ViewPart {
public void updateMessage(final String newMessage) {
// How do I get an instance of IStatusLineManager?
final IStatusLineManager manager = |
manager.setMessage(newMessage);
}
There is no trivial solution anymore.
Again, what should code completion return?

26. What
we
need...
!"#$%&'(
5#(!"#$*"(#67
public void updateMessage(final String newMessage) {
// How do I get an instance of
)!"#$*"(#
final IStatusLineManager manager =
this
.getViewSite() 5#(+,-./8&'167
.getActionBars()
.getStatusLineManager();
manager.setMessage(newMessage); )+,-./0&'1
}
5#(*(&(213"/#4&/&5#'67
)*(&(213"/#4&/&5#'
17

27. 01/1' %1"-(
2B&.#)&(& ,#."1-
2B&.# )#*"+# 01BF15"(#
,#+(&-./# D#-7
201-('"67(1' E"5(#-#'
201-C.7'&91-3/#B#-( 234(#-5"1- E&A17(
?=#// 2D3
)"5F/&A
?+'1//:&'
01-('1/
23>"(1'%&'(
2D#-7D&-&.#'
2?#'*"+#E1+&(1'
!"#$%&'(
28>&F(&6/#
2?#/#+91-
2;1'<6#-+=;"->1$
2G&*".&91-H"5(1'A
2;1'<6#-+=%&'( 2;1'<"-.?#(
2;1'<6#-+=%&'(?"(# 2@#A:"->"-.?#'*"+# 234(#-5"1-I'&+<#'
2;1'<6#-+=%&.# 2!"#$,#J#'#-+#
2!"#$?"(# 2?#/#+91-%'1*">#' 2!"#$%&'(
23>"(1',#J#'#-+#
2;1'<6#-+=;"->1$ 2%&'(?#'*"+#
2;1'<6#-+=
2?#/#+91-?#'*"+#
2?#'*"+#E1+&(1'
2D#-7D&-&.#'
28+91-:&'5
2?(&(75E"-#D&-&.#'
28+91-
2I11/:&'D&-&.#'

28. However,
someEmes
you
get...
Here,
implicit
user
feedbacks
(“clicks”)
will
help
to
find
the
nt...
right
(read
“the
most
commonly
used”)
paths. me
e lop
ev
In
d

29. Tools
Outline
20

30. Tools
Outline
Intelligent
Code
CompleEon
20

31. Tools
Outline
Intelligent
Code
CompleEon Smart
Bug
DetecEon
What
have
I
missed?
20

32. Tools
Outline
Intelligent
Code
CompleEon Smart
Bug
DetecEon
What
have
I
missed?
Extended
DocumentaEon
20

33. Tools
Outline
Intelligent
Code
CompleEon Smart
Bug
DetecEon
What
have
I
missed?
Extended
DocumentaEon Code
Example
Recommender
How
do
I
get
an
instance
of…?
20

34. Tools
Outline
Intelligent
Code
CompleEon Smart
Bug
DetecEon
What
have
I
missed?
Stacktrace
Search
Engine
Extended
DocumentaEon Code
Example
Recommender
What
caused
that
stacktrace?
How
do
I
get
an
instance
of…?
20

35. Tools
Outline
Intelligent
Code
CompleEon Smart
Bug
DetecEon
What
have
I
missed?
Stacktrace
Search
Engine
Extended
DocumentaEon Code
Example
Recommender
What
caused
that
stacktrace?
How
do
I
get
an
instance
of…?
20

36. Tools
Outline
Intelligent
Code
CompleEon Smart
Bug
DetecEon
What
have
I
missed?
Stacktrace
Search
Engine
Extended
DocumentaEon Code
Example
Recommender
x=
What
caused
that
stacktrace?
How
do
I
get
an
instance
of…?
20

37. “The
good
ones
in
the
crock,
the
bad
ones
in…
the
problem
view.”
SMART
BUG
DETECTION
–
DRIVEN
BY
REAL
USAGES

38. What’s
wrong
with
this
code?
“Hello!”
“id”
22

39. At
run+me
you
get…
23

40. ...and
a
lengthy
stacktrace
org.eclipse.core.runEme.AsserEonFailedExcepEon:
null
argument:
at
org.eclipse.core.runEme.Assert.isNotNull(Assert.java:85)
at
org.eclipse.core.runEme.Assert.isNotNull(Assert.java:73)
at
org.eclipse.jface.wizard.Wizard.createPageControls(Wizard.java:174)
at
org.eclipse.jface.wizard.WizardDialog.createPageControls(WizardDialog.java:734)
at
org.eclipse.jface.wizard.WizardDialog.setWizard(WizardDialog.java:1162)
at
org.eclipse.jface.wizard.WizardDialog.updateForPage(WizardDialog.java:1221)
at
org.eclipse.jface.wizard.WizardDialog.access$4(WizardDialog.java:1218)
at
org.eclipse.jface.wizard.WizardDialog$6.run(WizardDialog.java:1207)
at
org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
at
org.eclipse.jface.wizard.WizardDialog.showPage(WizardDialog.java:1205)
at
org.eclipse.ui.internal.dialogs.ImportExportPage.treeDoubleClicked(ImportExportPage.java)
at
org.eclipse.ui.internal.dialogs.ImportPage$2.doubleClick(ImportPage.java:72)
at
org.eclipse.jface.viewers.StructuredViewer$1.run(StructuredViewer.java:824)
at
org.eclipse.core.runEme.SafeRunner.run(SafeRunner.java:42)
at
org.eclipse.ui.internal.JFaceUEl$1.run(JFaceUEl.java:49)
at
org.eclipse.jface.uEl.SafeRunnable.run(SafeRunnable.java:175)
at
org.eclipse.jface.viewers.StructuredViewer.fireDoubleClick(StructuredViewer.java:822)
at
org.eclipse.jface.viewers.AbstractTreeViewer.handleDoubleSelect(AbstractTreeViewer.java)
at
org.eclipse.jface.viewers.StructuredViewer$4.widgetDefaultSelected(StructuredViewer.java)
... 24

41. So what have we missed?
25

42. If
all
of
your
colleagues
do
it
–
why
don’t
you?
26

43. If
all
of
your
colleagues
do
it
–
why
don’t
you?
26

44. How
to
detect
such
wrong
uses?
27

45. How
to
detect
such
wrong
uses?
27

46. How
to
detect
such
wrong
uses?
27

47. How
to
detect
such
wrong
uses?
27

48. How
to
detect
such
wrong
uses?
That’s
“strange”
27

49. How
to
detect
such
wrong
uses?
28

50. Just
one
formula…
0 .5 .9 1
x = {observed method calls on a variable}
E(x) = {exact similar usages for variables of type x in the code base}
A(x) = {almost similar usages, i.e., x+1 method call}
29

51. Missing
Calls
Detector
in
Eclipse
.
t..
m en
elop
dev
In
30

52. Applied
on
Eclipse
3.5...
31

53. “Developers
who
bought
overwrote
this
method
typically
also
overwrote...”
USAGE-­‐DRIVEN
JAVADOC

54. In
good
documenta+on
we
trust...
/**
* Rollsback the transaction if any and clears different lists to
* start with an empty resource again.
* Note that the super.doUnload is not called because that clears
* the list resulting in all kinds of undesirable inverseremoves.
*/
@Override
protected void doUnload() {
super.doUnload();
}
33

55. In
good
documenta+on
we
trust...
/**
* Rollsback the transaction if any and clears different lists to
* start with an empty resource again.
* Note that the super.doUnload is not called because that clears
* the list resulting in all kinds of undesirable inverseremoves.
*/
@Override
protected void doUnload() {
super.doUnload();
}
33

56. In
good
documenta+on
we
trust...
/**
* Rollsback the transaction if any and clears different lists to
* start with an empty resource again.
* Note that the super.doUnload is not called because that clears
* the list resulting in all kinds of undesirable inverseremoves.
*/
@Override
protected void doUnload() {
super.doUnload();
}
No?
Are
you
sure?
33

57. In
good documenta+on
we
trust...
good
/**
* Rollsback the transaction if any and clears different lists to
* start with an empty resource again.
* Note that the super.doUnload is not called because that clears
* the list resulting in all kinds of undesirable inverseremoves.
*/
@Override
protected void doUnload() {
super.doUnload();
}
No?
Are
you
sure?
33

58. Genera+ng
high-­‐quality
API
documenta+on
34

59. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

60. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

61. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

62. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

63. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

64. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

65. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
34

66. Genera+ng
high-­‐quality API
documenta+on
high-­‐quality
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
35

67. Genera+ng
high-­‐quality
API
documenta+on
/**
* The number of questions.
*/
private int numberOfQuestions;
/**
* Sets the number of questions.
*
* @param numberOfQuestions the number of questions
* @throws IllegalArgumentException the illegal argument exception
*/
public void setNumberOfQuestions(int numberOfQuestions)
throws IllegalArgumentException {
if (numberOfQuestions < 0) {
throw new IllegalArgumentException("numberOfQuestions < 0");
}
this.numberOfQuestions = numberOfQuestions;
}
35

68. What
people
say
about
JAutodoc...
36

69. What
people
say
about
JAutodoc...
User: Anonymous Rating: 9 2009-08-02 11:32:37
Wow exactly what I needed!
36

70. What
people
say
about
JAutodoc...
User: Anonymous Rating: 9 2009-08-02 11:32:37
Wow exactly what I needed!
User: Anonymous Rating: 9 2009-02-13 19:58:32
Thank you... this plugin rocks!
36

71. What
people
say
about
JAutodoc...
User: Anonymous Rating: 9 2009-08-02 11:32:37
Wow exactly what I needed!
User: Anonymous Rating: 9 2009-02-13 19:58:32
Thank you... this plugin rocks!
User: Anonymous Rating: 9 2009-02-13 19:58:32
Works perfectly. Smarter than I expected!
36

72. What
people
say
about
JAutodoc...
User: Anonymous Rating: 9 2009-08-02 11:32:37
Wow exactly what I needed!
User: Anonymous Rating: 9 2009-02-13 19:58:32
Thank you... this plugin rocks!
User: Anonymous Rating: 9 2009-02-13 19:58:32
Works perfectly. Smarter than I expected!
WTH?
What
did
you
expect?!
36

73. How
Recommenders
is
coming
to
rescue...
37

74. Some
facts
about
the
documentaEon
of
overridable
methods
Overridable 2.074
Overridden 623
0 700 1400 2100
38

75. What
clients
do
with
your
API...
Overridden 623
Documented * 155
0 325 650
*
documented
as
overridable
by
using
phrases
like
“clients
may/should/must
override/extend
this
method” 39

76. What
clients
do
with
your
API...
Overridden 623
Documented * 155 468
0 325 650
*
documented
as
overridable
by
using
phrases
like
“clients
may/should/must
override/extend
this
method” 39

77. What
documenta+on
would
you
expect
when
subclassing
Dialog?
package org.eclipse.recommenders.examples.demo;
import org.eclipse.jface.dialogs.Dialog;
import org.eclipse.jface.window.IShellProvider;
public class MyDialog extends Dialog {
protected MyDialog(final IShellProvider parentShell) {
super(parentShell);
}
}
40

78. What
Javadoc
gives
you...
org.eclipse.jface.dialogs.Dialog
41

79. What
Javadoc
gives
you...
org.eclipse.jface.dialogs.Dialog
A
dialog
is
a
specialized
window
used
for
narrow-­‐focused
communicaEon
with
the
user.
41

80. What
Javadoc
gives
you...
org.eclipse.jface.dialogs.Dialog
A
dialog
is
a
specialized
window
used
for
narrow-­‐focused
communicaEon
with
the
user.
Dialogs
are
usually
modal.
Consequently,
it
is
generally
bad
pracEce
to
open
a
dialog
without
a
parent.
A
modal
dialog
without
a
parent
is
not
prevented
from
disappearing
behind
the
applicaEon's
other
windows,
making
it
very
confusing
for
the
user.
If
there
is
more
than
one
modal
dialog
is
open
the
second
one
should
be
parented
off
of
the
shell
of
the
first
one
otherwise
it
is
possible
that
the
OS
will
cus
to
the
first
dialog
potenEally
blocking
the
UI.
41

81. What
Javadoc
gives
you...
org.eclipse.jface.dialogs.Dialog
A
dialog
is
a
specialized
window
used
for
narrow-­‐focused
communicaEon
with
the
user.
Dialogs
are
usually
modal.
Consequently,
it
is
generally
bad
pracEce
to
open
a
dialog
without
a
parent.
A
modal
dialog
without
a
parent
is
not
prevented
from
But what are the
disappearing
behind
the
applicaEon's
other
windows,
making
it
very
confusing
for
the
user.
hot-spots of Dialog?
If
there
is
more
than
one
modal
dialog
is
open
the
second
one
should
be
parented
off
of
the
shell
of
the
first
one
otherwise
it
is
possible
that
the
OS
will
cus
to
the
first
dialog
potenEally
blocking
the
UI.
41

82. What
code
comple+on
offers...
42

83. What
code
comple+on
offers...
Which of the 56 methods
should we override?
42

84. Recommending
method
overrides...
public class MyDialog extends Dialog {
43

85. Recommending
Self-­‐Calls
public class MyWizard extends Wizard {
! @Override
public void addPages() {
};
44

86. Mining
for
Subclassing
pa`erns
public class MyViewerSorter extends ViewerSorter {
45

87. Mining
example
code
snippets
public class MyDialog extends Dialog {
@Override
protected Control createDialogArea(Composite parent) {
46

88. Sca`ered
Resources
47

89. Extended
Documenta+on
Plaaorm
+ Machine Learning
48

90. Extended
Documenta+on
Plaaorm
+ Machine Learning
48

91. Extended
Documenta+on
49

92. How
it
works
-­‐
in
a
nutshell
50

93. “Why
is
Google
Codesearch
not
‘google
for
code
search’?
”
CODE-­‐SEARCH
ENGINES
LIFT
OFF

94. How
many
request
per
minutes
do
code
search
engines
have?
52

95. 20
Number
for
koders.com
in
2009
according
to
“Analyzing
and
mining
a
code
search
engine
usage
log”
h`p://dx.doi.org/10.1007/s10664-­‐010-­‐9144-­‐6 53

96. Naviga+ng
the
API
jungle…
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
54

97. Naviga+ng
the
API
jungle…
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
How
to
solve
String
-­‐>
ASTParser
-­‐>
CompilaKonUnit?
54

98. When
asking
Google
code
search...
55

99. When
asking
Google
code
search...
55

100. When
asking
Google
code
search...
55

101. When
asking
Google
code
search...
55

102. Why?
56

103. Why?
Text-based
retrieval!
56

104. Why?
Text-based
retrieval!
Inexpressive query
language!
56

105. Why?
Text-based
retrieval!
Inexpressive query
language!
Ignores prior
knowledge!
56

106. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

107. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

108. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
Query
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

109. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
Query
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

110. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
Query
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

111. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
Query
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

112. Example
code
search
engine
public class MyJavaEditor extends EditorPart {
private CompilationUnit createCompilationUnit(String source) {
// how can I create and configure an ASTParser?
ASTParser parser = null;
parser.setSource(source.toCharArray());
Query
// ASTParser --> CompilationUnit: how?
CompilationUnit cu = null;
return cu;
}
57

113. Sample
search
query
Extracted from MyJavaEditor.createCompilationUnit():
--- data ---
extends: org.eclipse.ui.EditorPart
uses: org.eclipse.jdt.core.CompilationUnit
uses: org.eclipse.jdt.core.ASTParser
calls: org.eclipse.jdt.core.ASTParser.setSource(char[])
--- requested ---
def: org.eclipse.jdt.core.CompilationUnit (def=null)
def: org.eclipse.jdt.core.ASTParser (def=null)
.
t..
m en
elop
dev
In
58

114. “Why
is
Google
Codesearch
not
‘google
for
code
search’?
”
LEVERAGING
USER
FEEDBACK
TO
IMPROVE
CODE
SEARCH
ENGINES

115. Leveraging
user
feedback
60

116. Leveraging
user
feedback
60

117. Refine
the
ranking
based
on
feedback
61

118. Refine
the
ranking
based
on
feedback
61

119. Learning
how
to
rank
62

120. Ranking
the
examples
63

121. Ranking
the
examples
document
63

122. Ranking
the
examples
document
query
63

123. Ranking
the
examples
document
query feature-­‐score
i
[0..1]
63

124. Ranking
the
examples
document
query weight
feature-­‐score
i
(-­‐∞..+∞) [0..1]
63

125. Upda+ng
the
feature
weights...
64

126. Upda+ng
the
feature
weights...
64

127. Upda+ng
the
feature
weights...
10.000
∗
64

128. Benefits
of
automated
tweaking
#
kendall’s
ι
#
queries
baseline
click-­‐through
65

129. Interac+ons
diagram
Client Server
1
Developer «triggers»
2 Search Invocation
3 Code Snippet Ranking
Incomplete
Source Code Query Ranking Search
Function Index
4 Code Snippet Presentation Code «updates»
Examples
5 Feedback Processing
Code Code
Summarization Highlighter
Weights Learning
Clickthrough
Cli kth h Function
data
66

130. Applica+on
scenarios
for
your
company
Open
Source
Repositories
Company
Repositories
67

131. Applica+on
scenarios
for
your
company
Open
Source
Repositories
Company
Repositories
67

132. Applica+on
scenarios
for
your
company
Open
Source
Repositories
Company
Repositories
67

133. Applica+on
scenarios
for
your
company
Open
Source
Repositories
Company
Repositories
67

134. Applica+on
scenarios
for
your
company
Open
Source
Repositories
Company
Repositories
Filesystem
67

135. “People
that
bought
had
this
NullPointerExcepKon,
made
this
mistake:...”
WHAT
CAUSED
THAT
STACKTRACE?

136. So
far
we
have…
69

137. So
far
we
have…
Intelligent
Code
CompleEon
69

138. So
far
we
have…
Intelligent
Code
CompleEon Extended
DocumentaEon
69

139. So
far
we
have…
Intelligent
Code
CompleEon Extended
DocumentaEon
Code
Example
Recommender
How
do
I
get
an
instance
of…?
69

140. So
far
we
have…
Intelligent
Code
CompleEon Extended
DocumentaEon
Smart
Bug
DetecEon Code
Example
Recommender
What
have
I
missed? How
do
I
get
an
instance
of…?
69

141. But
what
if
you
get
this?
70

142. Debugging
–
the
old
way
71

143. Debugging
–
the
old
way
71

144. Debugging
–
the
old
way
72

145. But
more
ojen
we
get…
73

146. How
can
we
fix
that?
Don’t
stacktraces
share
some
commonali+es?
74

147. How
can
we
fix
that?
Don’t
stacktraces
share
some
commonali+es?
org.eclipse.swt.SWTException: Invalid thread access
at org.eclipse.swt.SWT.error(SWT.java:3884)
at org.eclipse.swt.SWT.error(SWT.java:3799)
at org.eclipse.swt.SWT.error(SWT.java:3770)
at org.eclipse.swt.widgets.Widget.error(Widget.java:463)
at org.eclipse.swt.widgets.Widget.checkWidget(Widget.java:355)
at org.eclipse.swt.widgets.ProgressBar.setSelection(ProgressBar.java:317)
at org.eclipselabs.cr.stacktraces.demo.ThreadDemo$1.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
74

148. How
can
we
fix
that?
Don’t
stacktraces
share
some
commonali+es?
excepEon
type
org.eclipse.swt.SWTException: Invalid thread access
at org.eclipse.swt.SWT.error(SWT.java:3884)
at org.eclipse.swt.SWT.error(SWT.java:3799)
at org.eclipse.swt.SWT.error(SWT.java:3770)
at org.eclipse.swt.widgets.Widget.error(Widget.java:463)
at org.eclipse.swt.widgets.Widget.checkWidget(Widget.java:355)
at org.eclipse.swt.widgets.ProgressBar.setSelection(ProgressBar.java:317)
at org.eclipselabs.cr.stacktraces.demo.ThreadDemo$1.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
74

149. How
can
we
fix
that?
Don’t
stacktraces
share
some
commonali+es?
excepEon
type message
org.eclipse.swt.SWTException: Invalid thread access
at org.eclipse.swt.SWT.error(SWT.java:3884)
at org.eclipse.swt.SWT.error(SWT.java:3799)
at org.eclipse.swt.SWT.error(SWT.java:3770)
at org.eclipse.swt.widgets.Widget.error(Widget.java:463)
at org.eclipse.swt.widgets.Widget.checkWidget(Widget.java:355)
at org.eclipse.swt.widgets.ProgressBar.setSelection(ProgressBar.java:317)
at org.eclipselabs.cr.stacktraces.demo.ThreadDemo$1.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
74

150. How
can
we
fix
that?
Don’t
stacktraces
share
some
commonali+es?
excepEon
type message
org.eclipse.swt.SWTException: Invalid thread access
at org.eclipse.swt.SWT.error(SWT.java:3884)
at org.eclipse.swt.SWT.error(SWT.java:3799)
at org.eclipse.swt.SWT.error(SWT.java:3770)
at org.eclipse.swt.widgets.Widget.error(Widget.java:463)
at org.eclipse.swt.widgets.Widget.checkWidget(Widget.java:355)
at org.eclipse.swt.widgets.ProgressBar.setSelection(ProgressBar.java:317)
at org.eclipselabs.cr.stacktraces.demo.ThreadDemo$1.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
stackframes
74

151. How
can
we
fix
that?
Don’t
stacktraces
share
some
commonali+es?
excepEon
type message
org.eclipse.swt.SWTException: Invalid thread access
at org.eclipse.swt.SWT.error(SWT.java:3884)
at org.eclipse.swt.SWT.error(SWT.java:3799)
at org.eclipse.swt.SWT.error(SWT.java:3770)
at org.eclipse.swt.widgets.Widget.error(Widget.java:463)
at org.eclipse.swt.widgets.Widget.checkWidget(Widget.java:355)
at org.eclipse.swt.widgets.ProgressBar.setSelection(ProgressBar.java:317)
at org.eclipselabs.cr.stacktraces.demo.ThreadDemo$1.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
stackframes other
informaEon
74

152. How
about
stacktraces.org?
75

153. Knowledge
sources
Forums
Eclipse
htp://stacktraces.org
Webplasorm Mailing
Lists
76

154. Applica+on
scenarios
for
your
company
Community-­‐
driven
(forums,
mailinglists,...)
Company
Knowledgebase
77

155. “Templates
meet
Social
-­‐
reloaded!”
SNIPMATCH

156. Eclipse
Java
Editor
Templates

157. Eclipse
Java
Editor
Templates
42x
Java
Templates.

158. Eclipse
Java
Editor
Templates
42x
Java
Templates.
35x
SWT
Templates.

159. Eclipse
Java
Editor
Templates
42x
Java
Templates.
35x
SWT
Templates.
7
Years.

160. The
rise
of
code
snippet
repositories
and
many
more...

161. Searching
snippets
in
your
IDE...

162. “IDE
2.0?
Leveraging
the
wisdom
of
the
(Eclipse)
community…”
ECLIPSE
CODE
RECOMMENDERS

163. From
IDE
1.0
to
IDE
2.0
83

164. From
IDE
1.0
to
IDE
2.0
83

165. Build
directly
from
version
control
-­‐
or
p2/m2
repository...
P2
84

166. Leveraging
your
IDE’s
build
environment
85

167. Leveraging
your
IDE’s
build
environment
project
environment
85

168. Leveraging
your
IDE’s
build
environment
context
project
environment
85

169. Leveraging
your
IDE’s
build
environment
context
object
usage
project
environment
85

170. Leveraging
your
IDE’s
build
environment
context
object
usage
project selected
environment proposals
85

171. Leveraging
your
IDE’s
build
environment
context
object
usage
project selected
environment proposals
click
feedback
85

172. Leveraging
your
IDE’s
build
environment
context
object
usage
project selected
environment proposals
click
feedback
Stacktraces 85

173. Crowd-­‐sourcing...
86

174. “Yours
is
ours
and
mine
is
mine...”
BUT
HOW
ABOUT
PRIVACY?

175. Usage
data
sharing
example
import com.mycompany.*;
import org.eclipse.*;
public class MyCompanyDialog extends Dialog implements ICompanyService {
MyCompanyController controller = new MyCompanyController();
@Override
protected Control createDialogArea(Composite parent) {
Composite container = (Composite) super.createDialogArea(parent);
Button sync = new Button(container, SWT.CHECK);
sync.addSelectionListener(new SelectionAdapter() {
@Override
public void widgetSelected(final SelectionEvent e) {
if (sync.getSelection()) {
controller.synchronize();
// lot of more things done with controller
}
}
});
return container;
}
@Override
public void myCompanyServiceMethod() {
controller.setup();
}
} 88

176. Usage
data
sharing
example
import com.mycompany.*;
import org.eclipse.*;
public class MyCompanyDialog extends Dialog implements ICompanyService {
MyCompanyController controller = new MyCompanyController();
@Override
protected Control createDialogArea(Composite parent) {
Composite container = (Composite) super.createDialogArea(parent);
Button sync = new Button(container, SWT.CHECK);
sync.addSelectionListener(new SelectionAdapter() {
@Override
public void widgetSelected(final SelectionEvent e) {
if (sync.getSelection()) {
controller.synchronize();
// lot of more things done with controller
}
}
});
return container;
}
@Override
public void myCompanyServiceMethod() {
controller.setup();
}
} 88

177. Informa+on
cleared
by
privacy
selngs
import com.mycompany.*;
import org.eclipse.*;
public class MyCompanyDialog extends Dialog implements ICompanyService {
MyCompanyController controller = new MyCompanyController();
@Override
protected Control createDialogArea(Composite parent) {
Composite container = (Composite) super.createDialogArea(parent);
Button sync = new Button(container, SWT.CHECK);
sync.addSelectionListener(new SelectionAdapter() {
@Override
public void widgetSelected(final SelectionEvent e) {
if (sync.getSelection()) {
controller.synchronize();
// lot of more things done with controller
}
}
});
return container;
}
@Override
public void myCompanyServiceMethod() {
controller.setup();
}
} 89

178. Effec+ve
usage
data
shared
import com.mycompany.*;
import org.eclipse.*;
public class MyCompanyDialog extends Dialog implements ICompanyService {
MyCompanyController controller = new MyCompanyController();
@Override
protected Control createDialogArea(Composite parent) {
Composite container = (Composite) super.createDialogArea(parent);
Button sync = new Button(container, SWT.CHECK);
sync.addSelectionListener(new SelectionAdapter() {
@Override
public void widgetSelected(final SelectionEvent e) {
if (sync.getSelection()) {
controller.synchronize();
}
}
});
return container;
}
@Override
public void myCompanyServiceMethod() {
controller.setup();
}
} 90

179. Effec+ve
usage
data
shared
import com.mycompany.*;
import org.eclipse.*;
public class MyCompanyDialog extends Dialog implements ICompanyService {
MyCompanyController controller = new MyCompanyController();
@Override
protected Control createDialogArea(Composite parent) {
Composite container = (Composite) super.createDialogArea(parent);
Button sync = new Button(container, SWT.CHECK);
sync.addSelectionListener(new SelectionAdapter() {
@Override
public void widgetSelected(final SelectionEvent e) {
if (sync.getSelection()) {
controller.synchronize();
}
Only
the
informaKon
how
you
}
});
return container;
}
used
Eclipse
is
shared.
@Override
public void myCompanyServiceMethod() {
controller.setup();
}
} 90

180. “Support
your
own
framework.
Live
walk-­‐through...”
RECOMMENDERS
FOR
ANDROID

181. TextViews
LinearLayout
ListAcEvity
92

182. “Support
your
own
framework.
Live
walk-­‐through...”
TEAM
SERVER
FOR
ANDROID

183. Code
Recommenders
team
server
server
PowerBook G4
devel
#1
.
PowerBook G4 PowerBook G4
t..
devel
#2 devel
#3
m en
elop
dev
In

184. Demo
setup
PowerBook G4
developer
ws
PowerBook G4
examples
ws
Eclipse
“blue” Eclipse
“red”

185. Summary
Intelligent
Code
CompleEon Usage-­‐Driven
Javadocs
Stacktrace
Search
Engine
Smart
Bug
DetecEon Code
Example
Recommender
x=
What
caused
that
stacktrace?
What
have
I
missed? How
do
I
get
an
instance
of…?
96

186. A
developer’s
day...
Switching
Applications
Edit Code
Searching Code
Testing
Navigate
Dependencies
Reading API Doc
Comprehend Task
Handling
Comprehending Interruptions
Code
Source:
Eclipse
Mylyn
Developer
Techtalk
2011,
IEEE
TSE,
Vol.
32,
No.
12,
2006 97

187. Where
does
IDE
2.0
improve
the
process?
Edit Code
Testing
Comprehend Task
Handling
Interruptions
…
98

188. Contributors
so
far…
Jan
Kassens,
Peter
Schroeder,
Daniel
Glöckner,
Maik
Görtz,
Johannes
Lerch,
Johannes
Born,
Mohsen
Parisay,
Andreas
Sewe,
SebasKan
Ahlfeld,
SebasKan
Kasten,
Daniel
Staesche,
David
Kalnischkies,
SebasKan
Wörner,
Boyan
Yurukov,
Jan
Stolzenburg,
Nico
Wombacher,
Dirk
Kröhan,
Florianrian
Jakob,
Julius
Rückert,
Steffen
Remus,
Christopher
Mann,
Stefan
Henss,
Andreas
Kaluza,
Nikolay
Shindov,
Michael
Novotny,
KrisKn
Arand,
Sinem
Emeröz,
Michael
Kutschke,
SebasKan
Proksch,
Tomasz
Kalbarczyk,
Marko
MarKn,
Sheip
Dargutev,
David
Schuld,
Jens
Krause,
KrisKjan
Madunic,
Daniel
Brandtner,
Roman
GeSo,
ChrisKan
Kilb,
Johannes
Kastl,
Dennis
Sänger,
Annie
Liu,
Markus
Migenda,
Tjark
Vandommele,
SebasKan
Denel,
Florian
Nöll,
Gary
Fritz,
Dennis
Siebert,
Peter
Sinzig,
Laura
Altmüller,
Paul
Schatygin,
Jan-­‐Michael
Heller,
Minh
Hoang
Nguyen,
Sascha
Nordquist,
Paul
Emmanuel
Faidherbe,
...
99

189. Q
&
A
100

190. Q
&
A
Intelligent
Code
CompleEon Usage-­‐Driven
Javadocs
Stacktrace
Search
Engine
Smart
Bug
DetecEon Code
Example
Recommender
What
caused
that
stacktrace?
What
have
I
missed? How
do
I
get
an
instance
of…?
101

191. It’s
there.
htp://eclipse.org/recommenders/
Follow
me
on
twiter:
@MarcelBruch
google+:
+Marcel
Bruch 102