Slides used for the talk at VL/HCC 2008.

1. NextGenerationof Bug
TrackingSystems
Towards the
Sascha Just
R. Premraj T. Zimmermann

2. Tell the Developers ...
2

3. What can be improved in current
BugTracking Systems?
3

4. Ask the Community
4

5. Comments
Developer survey
Which of the following items have you previously used when fixing bugs?
Which three items helped you the most?
Which of the following problems have you encountered when fixing bugs?
Which three problems caused you most delay in fixing bugs?
5

6. Comments
Reporter survey
What information have you previously provided in bug reports?
What of the above information was most difficult to provide?
In your opinion, what information is most relevant to developers?
6

7. Comments
Reporter survey
What information have you previously provided in bug reports?
What of the above information was most difficult to provide?
In your opinion, what information is most relevant to developers?
6
Comments
Developer survey
Which of the following items have you previously used when fixing bugs?
Which three items helped you the most?
Which of the following problems have you encountered when fixing bugs?
Which three problems caused you most delay in fixing bugs?
5
N. Bettenburg, S. Just, A. Schröter, C. Weiß, R. Premraj,
T. Zimmermann. What makes a good bug report?
In FSE ’08: Proceedings of the Symposium on Foundations of
Software Engineering, Atlanta, USA. ACM 2008 (to appear).
7

8. Comments
Reporter survey
What information have you previously provided in bug reports?
What of the above information was most difficult to provide?
In your opinion, what information is most relevant to developers?
6
Comments
Developer survey
Which of the following items have you previously used when fixing bugs?
Which three items helped you the most?
Which of the following problems have you encountered when fixing bugs?
Which three problems caused you most delay in fixing bugs?
5
Comments from Developers
Comments from Reporters
Total Comments
48
127
175
8

9. Card Sort
Convert comments
into cards
Another frustrating issue with bug reporting
is insensitivity to language issues. I've
bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
y opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
international projects, sites should develop
standard procedures for dealing with bugs filed
different languages. For example, there
be triage volunteers for the different
languages who could help with the translation
interacting with the reporter. I don't think
developers realize that there are people
me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
9

10. Card Sort
Convert comments
into cards
Another frustrating issue with bug reporting
is insensitivity to language issues. I've
bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
y opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
international projects, sites should develop
standard procedures for dealing with bugs filed
different languages. For example, there
be triage volunteers for the different
languages who could help with the translation
interacting with the reporter. I don't think
developers realize that there are people
me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
237 cards
9

11. Organize cards
into themes
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Card Sort
Convert comments
into cards
Another frustrating issue with bug reporting
is insensitivity to language issues. I've
bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
y opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
international projects, sites should develop
standard procedures for dealing with bugs filed
different languages. For example, there
be triage volunteers for the different
languages who could help with the translation
interacting with the reporter. I don't think
developers realize that there are people
me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
237 cards
9

12. Organize cards
into themes
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Card Sort
Convert comments
into cards
Another frustrating issue with bug reporting
is insensitivity to language issues. I've
bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
y opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
international projects, sites should develop
standard procedures for dealing with bugs filed
different languages. For example, there
be triage volunteers for the different
languages who could help with the translation
interacting with the reporter. I don't think
developers realize that there are people
me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
looking at).
Language Support
Analyze cards
within themes
Another frustrating issue with bug reporting
sites is insensitivity to language issues. I've
seen bugs immediately closed because they
weren't filed in English, without even asking or
waiting for someone to translate it into English.
In my opinion, it's insulting to say that a bug
report is worthless unless it's filed in English.
On international projects, sites should develop
standard procedures for dealing with bugs filed
in different languages. For example, there
could be triage volunteers for the different
languages who could help with the translation
and interacting with the reporter. I don't think
that developers realize that there are people
(like me) who enjoy doing this sort of thing. Bug
reporting sites should be inclusive, not
snobbish and insulting (like, if you can't write in
English, certainly your bug report isn't worth
Language Support
Another frustratin
sites is insensitivit
seen bugs immediatel
weren't filed in
waiting for someone to translate it into Engli
237 cards
9

13. CardSort
Themes from
Tool
Centric
Process
Centric
Information
Centric
User
Centric
10

14. CardSort
Themes from
Tool
Centric
Process
Centric
Information
Centric
User
Centric
11

15. In OSS, there is a big gap with the knowledge level
of bug reporters. [...]
Firefox Reporter
“
”
[...] they just don’t understand what we’re asking for
in bug reports and why! Firefox Developer
“
”
12
User SupportUser
Centric

16. 13

17. Provide differentuser interfaces for
different levelsof users.
14

18. 15

19. 15
User Level: Advanced

20. 15
User Level: AdvancedNovice

21. Give cuesto inexperienced users what
information to provide and how.
16

22. 17

23. 17

24. 18

25. Well known reporters usually get more
consideration than unknown reporters [...]
Eclipse Developer
“
”
[...] even if a“well-known”reporter reports a bug which is pretty
vague, he will get more attention than another reporter.
Firefox Developer
“
”
19
Reporter ReputationUser
Centric

26. Integrate reputationto identify
experienced reporters.
20

27. 21

28. 21

29. Duplicates [...] often add useful information. That this information
is filed under a new report is not ideal thought.
Apache Developer
“
”
It is frustrating to have spent lots of time making an exceptionally
detailed bug report to only have it marked as a duplicate.
Eclipse Reporter
“
”
Duplicates andTool SupportTool
Centric
22

30. Provide tool support to consolidate
information into a single report.
23

31. Bug #103484
Stacktraces
lisa
163677.txt
john
024434.txt
Add more information...
24
Steps to reproduce
lisa
Repr_on_mac
john
STR_on_WinXP
Improve Data Presentation

32. Bug #98123
Steps to reproduce
joe
tool_crashes
john
STR_on_WinXP
Patches
m
Crash_fix.patch
pe
Rebuild_menu.pa...
Bug #103484
Stacktraces
lisa
163677.txt
john
024434.txt
Add more information...
24
Merge all data. Discard changes.
Steps to reproduce
lisa
Repr_on_mac
john
STR_on_WinXP
Improve Data Presentation

33. Bug #98123
PatchesSteps to reproduce
joe
tool_crashes
john
STR_on_WinXP
m
Crash_fix.patch
pe
Rebuild_menu.pa...
Bug #103484
Stacktraces
lisa
163677.txt
john
024434.txt
Steps to reproduce
lisa
Repr_on_mac
Add more information...Patches
Improve Data Presentation
john
STR_on_WinXP
mark
Crash_fix.patch
peter
Rebuild_menu.pa...
Merge all data. Discard changes.
25

35. ThankYou.