1. A Lightweight Approach
to Uncover Technical
Information in
Unstructured Data
Nicolas Bettenburg, Bram Adams,Ahmed E. Hassan
Queen’s University, Kingston, ON
Michel Smidt
University of Bremen, Germany
1

2. 2

3. The code after "if callback.isAcceleratorInUse
(SWT.ALT | character ) )" inside Eclipse's
MenuManager.java removes the mnemonic, but it seems
like Eclipse should be checking isAcceleratorInUse.
2

4. “Our developers have a severe problem.”
3

5. “Our developers have a severe problem.”
[ Our_PRP$
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
NLP
3

6. “Our developers have a severe problem.”
[ Our_PRP$
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
pronoun, possessive
NLP
3

7. “Our developers have a severe problem.”
[ Our_PRP$
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
pronoun, possessive
noun, common, plural
NLP
3

8. “Our developers have a severe problem.”
[ Our_PRP$
developers_NNS ])
<: have_VBP :>
([ a_DT
severe_JJ
problem_NN ])._.
pronoun, possessive
noun, common, plural
verb, present tense
determiner
adjective, ordinal
noun, common, singular
NLP
3

9. Structured Text
4

10. NLP?
can’t deal with
the source code parts
PARSERS?
can’t deal with
the natural language parts
5

11. Past Solutions
infoZilla
Bettenburg et al. - MSR’08
•Based on heuristics
•Stack Traces
•Code snippets at block level
•Patches
6

12. Past Solutions
Miler
Bacchelli et al. - ICSE’10,
ICPC’10
•Based on heuristics
•Classifies lines as source code
•Classifies documents as containing code
•Finds class names
7

13. Past Solutions
Miler
Bacchelli et al. - ICPC’10
infoZilla
Bettenburg et al. - MSR’08
•only specific kinds of technical information!
•new heuristics to extend (complex, error prone)
•for some kind of technical information, infeasible
8

14. More information:
The code after "if (callback.isAcceleratorInUse(SWT.ALT | character))" inside
Eclipse's MenuManager.java removes the mnemonic, but it seems like Eclipse
should be checking "isAcceleratorInUse" only for top level menumanagers like
File,Edit,...,Help, etc. :
/* (non-Javadoc)
* @see org.eclipse.jface.action.IContributionItem#update(java.lang.String)
*/
public void update(String property) {
IContributionItem items[] = getItems();
for (int i = 0; i < items.length; i++) {
items[i].update(property);
}
[...]
}
Any status on this bug?
I'd consider any contributions for M6 (API) or M7 (non-API) [...]
A 3.5 fix would be to make that behaviour optional in MenuManager with API and
off by default early in 3.5, and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions contributed MenuManagers turn it on
(if I can find MenuManagers in the correct place).
9

15. More information:
The code after "if (callback.isAcceleratorInUse(SWT.ALT | character))" inside
Eclipse's MenuManager.java removes the mnemonic, but it seems like Eclipse
should be checking "isAcceleratorInUse" only for top level menumanagers like
File,Edit,...,Help, etc. :
/* (non-Javadoc)
* @see org.eclipse.jface.action.IContributionItem#update(java.lang.String)
*/
public void update(String property) {
IContributionItem items[] = getItems();
for (int i = 0; i < items.length; i++) {
items[i].update(property);
}
[...]
}
Any status on this bug?
I'd consider any contributions for M6 (API) or M7 (non-API) [...]
A 3.5 fix would be to make that behaviour optional in MenuManager with API and
off by default early in 3.5, and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions contributed MenuManagers turn it on
(if I can find MenuManagers in the correct place).
9

16. More information:
The code after "if (callback.isAcceleratorInUse(SWT.ALT | character))" inside
Eclipse's MenuManager.java removes the mnemonic, but it seems like Eclipse
should be checking "isAcceleratorInUse" only for top level menumanagers like
File,Edit,...,Help, etc. :
/* (non-Javadoc)
* @see org.eclipse.jface.action.IContributionItem#update(java.lang.String)
*/
public void update(String property) {
IContributionItem items[] = getItems();
for (int i = 0; i < items.length; i++) {
items[i].update(property);
}
[...]
}
Any status on this bug?
I'd consider any contributions for M6 (API) or M7 (non-API) [...]
A 3.5 fix would be to make that behaviour optional in MenuManager with API and
off by default early in 3.5, and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions contributed MenuManagers turn it on
(if I can find MenuManagers in the correct place).
Challenge!
9

17. Spelling and
Grammar Checkers
... are really good at finding “what’s not right“ in
natural language text!
10

18. A 3.5 fix would be to make taht behaviour optional in
MenuManager with API and off by default early in 3.5,
and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions
contributed MenuManagers turn it on (if I can find
MenuManagers in the corect place).
11

19. A 3.5 fix would be to make taht behaviour optional in
MenuManager with API and off by default early in 3.5,
and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions
contributed MenuManagers turn it on (if I can find
MenuManagers in the corect place).
12

20. A 3.5 fix would be to make taht behaviour optional in
MenuManager with API and off by default early in 3.5,
and to have the WorkbenchActionBuilder contributed
MenuManagers and actionSets/editorActions
contributed MenuManagers turn it on (if I can find
MenuManagers in the corect place).
Actual Spelling Mistakes!
13

21. Add Heuristics
14

22. Add Heuristics
H1: Camel Case
camelCase, CamelCase, CamelCASE, ...
14

23. Add Heuristics
H1: Camel Case
camelCase, CamelCase, CamelCASE, ...
H2: Programming Language Keywords
printf, fork, fi, ...
14

24. Add Heuristics
H1: Camel Case
camelCase, CamelCase, CamelCASE, ...
H2: Programming Language Keywords
printf, fork, fi, ...
H3: Special Characters
tree();
14

25. Evaluation
Manually annotated 20 complete Bug Reports and
Discussions from ECLIPSE.
Manually annotated 20 complete Email Discussions from
POSTGRESQL developers Mailing List.
15

26. 1
2
3
Annotation GUI
16

27. Precision / Recall
Precision(Si) =
T PSi
T PSi
+F PSi
Recall(Si) =
T PSi
T PSi
+F NSi
TP = We annotated and tool annotated
FP = Tool annotated, we did not
FN = We annotated, tool did not
17

28. Results
Spellchecker Precision Recall
JOrtho 88.01% 64.31%
Jazzy 84.16% 68.30%
Hunspell 86.40% 68.34%
18

29. Results
Spellchecker Precision Recall
JOrtho 88.01% 64.31%
Jazzy 84.16% 68.30%
Hunspell 86.40% 68.34%
18

30. Results
Spellchecker Precision Recall
JOrtho 88.01% 64.31%
Jazzy 84.16% 68.30%
Hunspell 86.40% 68.34%
Hunspell used by OpenOffice and
Mozilla Suite
18

31. Comparison
Line-wise classification of source code
1 Launch the plugin as part of Eclipse IDE 3. Press Alt+H to
2 bring down the Help menu (to go along with our example in #1)
3
4 BUG: Notice "Software Updates" is missing its mnemonic.
5
6 public void update(String property) {
7 IContributionItem items[] = getItems();
8 for (int i = 0; i < items.length; i++) {
9 items[i].update(property);
10 }
11 }
12
13 Any status on this bug?
19

32. Comparison
Line-wise classification of source code
1 Launch the plugin as part of Eclipse IDE 3. Press Alt+H to
2 bring down the Help menu (to go along with our example in #1)
3
4 BUG: Notice "Software Updates" is missing its mnemonic.
5
6 public void update(String property) {
7 IContributionItem items[] = getItems();
8 for (int i = 0; i < items.length; i++) {
9 items[i].update(property);
10 }
11 }
12
13 Any status on this bug?
20

33. Comparison
Precision Recall
Our approach 89.27% 86.46%
State-of-the-Art 66.13% 69.37%
Line-wise classification of source code
21

34. Comparison
Precision Recall
Our approach 89.27% 86.46%
State-of-the-Art 66.13% 69.37%
Line-wise classification of source code
21

35. Summary
22

36. Summary
22

37. Summary
22

38. Summary
22

39. Summary
22

40. mozilla
paulc
zhangchunlin
kbrosnan
sdwilsh
samuel.sidler+old
hasham8888
myles7897
deletesoftware
abillings
eddy_nigg
jmjeffery
sgautherie.bz
john.p.baker
l10n
adelfino
jo.hermans
jruderman
nightstalkerz
alice0775
hskupin
mmortal03
tchung
marcia
me.at.work
fittysix
steve.england
cbook
tonglebeak
ctalbert
VYV03354
ehsan
alex
nrthomas
aarobertxtr
smichaud
shaver
johnjbartonmanujsabarwal
jdaggett
matt
bzbarsky
dtownsend
davemgarrett
info
stephen.donner
elmar.ludwig sdaugherty
mak77
jdarmochwal
polidobj
vseerror
twalker
dietrich
mconnor
beltzner
steffen.wilberg
mano
highmind63
ria.klaassen
robert.bugzilla
edilee
kliu
faaborg
marco.zehe
sylvain.pasche
bugzilla
rotis
uliss
cl-bugs-new2
anselm.meyer
timwi
RainerStroebel
tomer
gavin.sharp
jbecerra
johnath
kev
martijn.martijn
cwwmozilla
longsonr
m-wada
zenikodveditz
matspal
philringnalda
zurtex
bomfog
cjcypoi02
corevette
masayuki
reed
phiw
timeless
matti
mh+mozilla
dao
klaas1988
sziadeh mark.finkle
23

41. mozilla
paulc
zhangchunlin
kbrosnan
sdwilsh
samuel.sidler+old
hasham8888
myles7897
deletesoftware
abillings
eddy_nigg
jmjeffery
sgautherie.bz
john.p.baker
l10n
adelfino
jo.hermans
jruderman
nightstalkerz
alice0775
hskupin
mmortal03
tchung
marcia
me.at.work
fittysix
steve.england
cbook
tonglebeak
ctalbert
VYV03354
ehsan
alex
nrthomas
aarobertxtr
smichaud
shaver
johnjbartonmanujsabarwal
jdaggett
matt
bzbarsky
dtownsend
davemgarrett
info
stephen.donner
elmar.ludwig sdaugherty
mak77
jdarmochwal
polidobj
vseerror
twalker
dietrich
mconnor
beltzner
steffen.wilberg
mano
highmind63
ria.klaassen
robert.bugzilla
edilee
kliu
faaborg
marco.zehe
sylvain.pasche
bugzilla
rotis
uliss
cl-bugs-new2
anselm.meyer
timwi
RainerStroebel
tomer
gavin.sharp
jbecerra
johnath
kev
martijn.martijn
cwwmozilla
longsonr
m-wada
zenikodveditz
matspal
philringnalda
zurtex
bomfog
cjcypoi02
corevette
masayuki
reed
phiw
timeless
matti
mh+mozilla
dao
klaas1988
sziadeh mark.finkle
UI
JavaScript
Engine
XML Parser
Internet Explorer
23

42. 24