What makes a good bug report?
Upcoming SlideShare
Loading in...5
×
 

What makes a good bug report?

on

  • 2,886 views

Presented at the International Symposium on Foundations of Software Engineering \'08, Atlanta.

Presented at the International Symposium on Foundations of Software Engineering \'08, Atlanta.

Statistics

Views

Total Views
2,886
Views on SlideShare
2,704
Embed Views
182

Actions

Likes
4
Downloads
115
Comments
1

11 Embeds 182

http://thomas-zimmermann.com 83
http://www.cs.vu.nl 61
http://www.slideshare.net 12
http://itspice.net 11
http://www.linkedin.com 5
http://bugstrack.blogspot.com 4
http://www.bugstrack.blogspot.com 2
http://74.125.79.132 1
http://74.125.155.132 1
http://www.gamedevportal.com 1
https://www.linkedin.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Greetings,

    Read crash course on Reporting Bugs at http://tuppad.com/blog/wp-content/uploads/2010/08/Crash-course-on-Reporting-Bugs-in-Software-Testing.pdf & please share this link with testers you know through e-mail or chat or any medium.

    Thanks!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

What makes a good bug report? What makes a good bug report? Presentation Transcript

  • What makes a good bug report? Nicolas Bettenburg Sascha Just Adrian Schröter Saarland University Saarland University Saarland University Cathrin Weiss Rahul Premraj Thomas Zimmermann Saarland University Saarland University Saarland University 1
  • What makes a good bug report? Nicolas Bettenburg Sascha Just Adrian Schröter Queen’s University Saarland University University of Victoria Cathrin Weiss Rahul Premraj Thomas Zimmermann University of Zurich Free Uni. Amsterdam University of Calgary
  • What makes a good bug report? Nicolas Bettenburg Sascha Just Adrian Schröter Queen’s University Saarland University University of Victoria Cathrin Weiss Rahul Premraj Thomas Zimmermann University of Zurich Free Uni. Amsterdam Microsoft Research
  • Bug 31021 Summary: Tree - Selection listener stops default expansion Product: [Eclipse] Platform Reporter: Veronika Irvine <veronika_irvine@ca.ibm.com> Component: SWT Assignee: Steve Northover <steve_northover@ca.ibm.com> Status: RESOLVED WONTFIX QA Contact: Severity: normal Priority: P3 Version: 2.1 Target Milestone: --- Hardware: PC OS: Windows XP Whiteboard: Opened: 2003-02-05 14:52 - Description: 0400 I20030205 Run the following example. Double click on a tree item and notice that it does not expand. Comment out the Selection listener and now double click on any tree item and notice that it expands. public static void main(String[] args) { Display display = new Display(); Shell shell = new Shell(display); shell.setLayout(new FillLayout()); Tree tree = new Tree(shell, SWT.BORDER); for (int i = 0; i < 4; i++) { TreeItem item = new TreeItem(tree, SWT.NONE); item.setText(quot;item quot; + i); for (int j = 0; j < 4; j++) { TreeItem subItem = new TreeItem(item, SWT.NONE); subItem.setText(quot;item quot; + j); for (int k = 0; k < 4; k++) { TreeItem subsubItem = new TreeItem(subItem, SWT.NONE); subsubItem.setText(quot;item quot; + k); } } } tree.addSelectionListener(new SelectionAdapter() {}); shell.setSize(200, 200); shell.open(); while (!shell.isDisposed()) { if (!display.readAndDispatch()) display.sleep(); } display.dispose(); } ------- Comment #1 From Steve Northover 2004-10-07 19:39:16 -0400 ------- This behavior can't be changed now because application code now relies on it. Sorry. 4
  • Home | New | Search | Find | Reports | Requests | New Account | Log In | Terms of Use Bug 31021 Summary: Tree - Selection listener stops default expansion Product: [Eclipse] Platform Reporter: Veronika Irvine <veronika_irvine@ca.ibm.com> Component: SWT Assignee: Steve Northover <steve_northover@ca.ibm.com> Status: RESOLVED WONTFIX QA Contact: Severity: normal Priority: P3 Version: 2.1 Target Milestone: --- Hardware: PC OS: Windows XP Whiteboard: Opened: 2003-02-05 14:52 - Description: 0400 I20030205 Run the following example. Double click on a tree item and notice that it does not expand. Comment out the Selection listener and now double click on any tree item and notice that it expands. public static void main(String[] args) { Display display = new Display(); Shell shell = new Shell(display); shell.setLayout(new FillLayout()); Tree tree = new Tree(shell, SWT.BORDER); for (int i = 0; i < 4; i++) { TreeItem item = new TreeItem(tree, SWT.NONE); item.setText(quot;item quot; + i); for (int j = 0; j < 4; j++) { TreeItem subItem = new TreeItem(item, SWT.NONE); subItem.setText(quot;item quot; + j); for (int k = 0; k < 4; k++) { TreeItem subsubItem = new TreeItem(subItem, SWT.NONE); subsubItem.setText(quot;item quot; + k); } } } tree.addSelectionListener(new SelectionAdapter() {}); shell.setSize(200, 200); shell.open(); while (!shell.isDisposed()) { if (!display.readAndDispatch()) display.sleep(); } display.dispose(); } ------- Comment #1 From Steve Northover 2004-10-07 19:39:16 -0400 ------- This behavior can't be changed now because application code now relies on it. Sorry. 4
  • Whiteboard: Bug 31021 Summary: Tree - Selection listener stops default expansion Opened: 2003-02-05 14:52 - Description: Product: [Eclipse] Platform Reporter: Veronika Irvine <veronika_irvine@ca.ibm.com> 0400 Component: SWT Assignee: Steve Northover <steve_northover@ca.ibm.com> Status: RESOLVED WONTFIX I20030205 QA Contact: Severity: normal Priority: P3 Run the following example. Double click on a tree item and notice that it does Version: 2.1 not expand. Target Milestone: --- Hardware: PC Comment out the Selection listener and now double click on any tree item and OS: Windows XP notice that it expands. Whiteboard: public static void main(String[] 2003-02-05 14:52 - Opened: args) { Description: Display display = new Display(); 0400 I20030205 Shell shell = new Shell(display); shell.setLayout(new FillLayout()); Run the following example. Double click on a tree item and notice that it does not expand. Tree tree = new Tree(shell, SWT.BORDER); for (int i = 0; i < 4; i++) { Comment out the Selection listener and now double click on any tree item and notice that it expands. TreeItem item = new TreeItem(tree, SWT.NONE); public static void main(String[] args) { item.setText(quot;item quot; + i); Display display = new Display(); for (int j = 0; j < 4; j++) { Shell shell = new Shell(display); shell.setLayout(new FillLayout()); TreeItem subItem = new TreeItem(item, SWT.NONE); Tree tree = new Tree(shell, SWT.BORDER); subItem.setText(quot;item quot; + j); for (int i = 0; i < 4; i++) { TreeItem item = new TreeItem(tree, SWT.NONE); for (int k = 0; k < 4; k++) { item.setText(quot;item quot; + i); for (int j = 0; j < 4; j++) { TreeItem subsubItem = new TreeItem(subItem, SWT.NONE); subItem = new TreeItem(item, SWT.NONE); TreeItem subItem.setText(quot;item quot; + j); for (int k = 0; k < 4; k++) { subsubItem.setText(quot;item quot; + k); TreeItem subsubItem = } new TreeItem(subItem, SWT.NONE); } subsubItem.setText(quot;item quot; + k); } } } } tree.addSelectionListener(new SelectionAdapter() {}); tree.addSelectionListener(new SelectionAdapter() {}); shell.setSize(200, 200); shell.setSize(200, 200); shell.open(); shell.open(); while (!shell.isDisposed()) { while (!shell.isDisposed()) { if (!display.readAndDispatch()) display.sleep(); if (!display.readAndDispatch()) } display.dispose(); display.sleep(); } } display.dispose(); ------- Comment #1 From Steve Northover 2004-10-07 19:39:16 -0400 ------- } This behavior can't be changed now because application code now relies on it. Sorry. ------- Comment #1 From Steve Northover 2004-10-07 19:39:16 -0400 ------- 4
  • INVALID Severity: normal Priority: P3 CC: norbert.ploett@siemens.com Bug 175222 Version: 3.1.1 Summary: Target Milestone: using CDT. Product: Create a new plugin [Tools] CDT --- Reporter: Sabeesh C.S. <sabeesh.cs@ditro.com> Component: cdt-build Assignee: cdt-build-inbox@eclipse.org <cdt-build- Hardware: PC inbox@eclipse.org> Status: RESOLVED QA Contact: OS: INVALID Linux Severity: normal Whiteboard: Priority: P3 CC: norbert.ploett@siemens.com Version: 3.1.1 Target Milestone: --- Hardware: PC OS: Linux Opened: 2007-02-22 23:35 - Description: Whiteboard: 0400 Opened: 2007-02-22 23:35 - Description: 0400 I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please... CDT. After that I wand to impliment it in my programe. If it possible?. Any ------- Comment #1 From Norbert Plött 2007-02-23 01:55:42 -0400 ------- one can help me please... Saabeesh, the way I understand it you want to create a plug-in and do the code implementation in C/C++ using the CDT. Eclipse supports only Java implementations. In theory you could probably call into your C/C++ library via ------- Comment #1 From Norbert Plött 2007-02-23 01:55:42 -0400 ------- the Java native interface but this is probably not what you want to do. In any case you would need to create at least a Java wrapper around your Saabeesh, implementation. Do I read you right? the way I understand it you want to create a plug-in and do the code ------- Comment #2 From Chris Recoskie 2007-02-23 08:14:04 -0400 ------- I implementation described here... this question the on the don't see a bug or enhancement in C/C++ using belongs CDT. Eclipse supports only Java newsgroup. implementations. In theory you could probably call into your C/C++ library via the Java native interface but this is probably not what you want to do. In any case you would need to create at least a Java wrapper around your implementation. 5
  • 1 2 Survey Cuezilla 6
  • 1 2 Survey Cuezilla 6
  • Survey the users 872 developers 1,354 reporters with at least 50 bugs who have reported assigned to them. at least 30 bugs. 7
  • Survey the users 872 developers 1,354 reporters 156 responses (19%) 310 responses (26%) 8
  • Survey’s aim 9
  • Survey’s aim Learn developers’ problems and information needs—and what reporters provide. 9
  • Problems faced You were given wrong There were errors in The reporter used Others product name code examples bad grammar duplicates component name steps to reproduce unstructured text spam operating system test cases prose text incomplete info. observed behaviour stack traces too long text viruses/worms expected behaviour no spell check 10
  • Problems faced Question 1 Which of the following problems have your encountered when fixing bugs? (select as many as you wish) You were given wrong There were errors in The reporter used Others product name code examples bad grammar duplicates component name steps to reproduce unstructured text spam operating system test cases prose text incomplete info. observed behaviour stack traces too long text viruses/worms expected behaviour no spell check 10
  • Problems faced Question 1 Which of the following problems have your encountered when fixing bugs? (select as many as you wish) Question 2 Which three caused you the most delay in fixing bugs? (select at most three items) You were given wrong There were errors in The reporter used Others product name code examples bad grammar duplicates component name steps to reproduce unstructured text spam operating system test cases prose text incomplete info. observed behaviour stack traces too long text viruses/worms expected behaviour no spell check 10
  • Responses Most delay caused Encountered when fixing a bug 11
  • Responses Most delay caused wrong system information viruses/spam Encountered when fixing a bug 11
  • Responses Most delay caused bad grammar errors in test cases duplicates wrong system information viruses/spam Encountered when fixing a bug 11
  • Responses Most delay caused wrong observed behaviour wrong expected behaviour wrong product information bad grammar errors in test cases duplicates wrong system information viruses/spam Encountered when fixing a bug 11
  • Responses incomplete information Most delay caused erroneous steps to reproduce wrong observed behaviour wrong expected behaviour wrong product information bad grammar errors in test cases duplicates wrong system information viruses/spam Encountered when fixing a bug 11
  • Information needs product hardware observed behaviour screen shots component operating system expected behaviour code examples version summary steps to reproduce error reports severity build information stack traces test cases 12
  • Information needs Question 1 3 Which of the following items have you previously used when fixing bugs? (select as many as you wish) product hardware observed behaviour screen shots component operating system expected behaviour code examples version summary steps to reproduce error reports severity build information stack traces test cases 12
  • Information needs Question 1 3 Which of the following items have you previously used when fixing bugs? (select as many as you wish) Question 4 Which three items helped you the most? (select at most three items) product hardware observed behaviour screen shots component operating system expected behaviour code examples version summary steps to reproduce error reports severity build information stack traces test cases 12
  • Responses Helped most 13 Used when fixing a bug
  • Responses Helped most version hardware severity 13 Used when fixing a bug
  • Responses Helped most screenshots observed behavior test cases/examples expected behavior version hardware severity 13 Used when fixing a bug
  • Responses Helped most steps to reproduce stack traces screenshots observed behavior test cases/examples expected behavior version hardware severity 13 Used when fixing a bug
  • Questions to reporters Same options as for developers product hardware observed behaviour screen shots component operating system expected behaviour code examples version summary steps to reproduce error reports severity build information stack traces test cases 14
  • Questions to reporters Question 1 What information have you previously provided in bug reports? (select as many as you wish) Same options as for developers product hardware observed behaviour screen shots component operating system expected behaviour code examples version summary steps to reproduce error reports severity build information stack traces test cases 14
  • Questions to reporters Question 1 What information have you previously provided in bug reports? (select as many as you wish) Question 2 What of the above information was most difficult to provide? (select at most three items) Question 3 In your opinion, what information is most relevant to developers? (select at most three items) 15
  • Rated most helpful by developers steps to reproduce steps to reproduce steps to reproduce observed behaviour stack traces test cases expected behaviour test cases observed behaviour product observed behaviour stack traces version screenshots expected behaviour operating system expected behaviour version summary code examples code examples component summary error reports severity version build information build information error 16 reports summary
  • Provided by Rated most helpful most reporters by developers steps to reproduce steps to reproduce steps to reproduce observed behaviour stack traces test cases expected behaviour test cases observed behaviour product observed behaviour stack traces version screenshots expected behaviour operating system expected behaviour version summary code examples code examples component summary error reports severity version build information build information error 16 reports summary
  • Provided by Rated most helpful Rated most helpful most reporters by developers by reporters steps to reproduce steps to reproduce steps to reproduce observed behaviour stack traces test cases expected behaviour test cases observed behaviour product observed behaviour stack traces version screenshots expected behaviour operating system expected behaviour version summary code examples code examples component summary error reports severity version build information build information error 16 reports summary
  • Provided by Rated most helpful Rated helpful most reporters by developers by most reporters steps to reproduce steps to reproduce steps to reproduce observed behaviour stack traces test cases expected behaviour test cases observed behaviour product observed behaviour stack traces version screenshots expected behaviour operating system expected behaviour version summary code examples code examples component summary error reports severity version build information build information error 17 reports summary
  • Provided by Rated most helpful Rated helpful most reporters by developers by most reporters steps to reproduce steps to reproduce steps to reproduce observed behaviour stack traces test cases expected behaviour test cases observed behaviour productReporters need betterstack traces observed behaviour version tool assistance. expected behaviour screenshots operating system expected behaviour version summary code examples code examples component summary error reports severity version build information build information error 17 reports summary
  • Provided by Rated most helpful Rated helpful most reporters by developers by most reporters steps to reproduce steps to reproduce steps to reproduce observed behaviour stack traces test cases expected behaviour test cases observed behaviour product observed behaviour stack traces version screenshots expected behaviour operating system expected behaviour version summary code examples code examples component summary error reports severity version build information build information error 18 reports summary
  • 1 2 Survey Cuezilla 19
  • Assist reporters 20
  • Assist reporters 20
  • Assist reporters 20
  • Rating bug reports Create a new plugin using CDT. I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please... 21
  • Rating bug reports Create a new plugin using CDT. I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please... Poor Average Good 21
  • Rating bug reports Total reports voted: 289 Total votes received: 1,186 Create a new plugin using CDT. I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please... Poor Average Good 21
  • CUE Bug 31021 Summary: Tree - Selection listener stops default expansion Product: [Eclipse] Platform Reporter: Veronika Irvine <veronika_irvine@ca.ibm.com> Component: SWT Assignee: Steve Northover <steve_northover@ca.ibm.com> Status: RESOLVED WONTFIX QA Contact: ZILLA Severity: normal Priority: P3 Version: 2.1 Target Milestone: --- Hardware: PC OS: Windows XP Whiteboard: Opened: 2003-02-05 14:52 - Description: 0400 I20030205 Run the following example. Double click on a tree item and notice that it does not expand. Comment out the Selection listener and now double click on any tree item and notice that it expands. public static void main(String[] args) { Display display = new Display(); Shell shell = new Shell(display); shell.setLayout(new FillLayout()); Tree tree = new Tree(shell, SWT.BORDER); for (int i = 0; i < 4; i++) { TreeItem item = new TreeItem(tree, SWT.NONE); item.setText(quot;item quot; + i); for (int j = 0; j < 4; j++) { TreeItem subItem = new TreeItem(item, SWT.NONE); subItem.setText(quot;item quot; + j); for (int k = 0; k < 4; k++) { TreeItem subsubItem = new TreeItem(subItem, SWT.NONE); subsubItem.setText(quot;item quot; + k); } } } tree.addSelectionListener(new SelectionAdapter() {}); shell.setSize(200, 200); shell.open(); while (!shell.isDisposed()) { if (!display.readAndDispatch()) display.sleep(); } display.dispose(); } ------- Comment #1 From Steve Northover 2004-10-07 19:39:16 -0400 ------- This behavior can't be changed now because application code now relies on it. Sorry. 22
  • CUE Bug 31021 Summary: Tree - Selection listener stops default expansion Product: [Eclipse] Platform Reporter: Veronika Irvine <veronika_irvine@ca.ibm.com> Component: SWT Assignee: Steve Northover <steve_northover@ca.ibm.com> Status: RESOLVED WONTFIX QA Contact: ZILLA Severity: normal Priority: P3 Version: 2.1 Target Milestone: --- Hardware: PC OS: Windows XP Whiteboard: Opened: 2003-02-05 14:52 - Description: 0400 I20030205 Run the following example. Double click on a tree item and notice that it does not expand. Comment out the Selection listener and now double click on any tree item and notice that it expands. public static void main(String[] args) { Display display = new Display(); Shell shell = new Shell(display); shell.setLayout(new FillLayout()); Tree tree = new Tree(shell, SWT.BORDER); for (int i = 0; i < 4; i++) { TreeItem item = new TreeItem(tree, SWT.NONE); item.setText(quot;item quot; + i); for (int j = 0; j < 4; j++) { TreeItem subItem = new TreeItem(item, SWT.NONE); subItem.setText(quot;item quot; + j); for (int k = 0; k < 4; k++) { TreeItem subsubItem = new TreeItem(subItem, SWT.NONE); subsubItem.setText(quot;item quot; + k); } } } tree.addSelectionListener(new SelectionAdapter() {}); shell.setSize(200, 200); shell.open(); while (!shell.isDisposed()) { if (!display.readAndDispatch()) display.sleep(); } display.dispose(); } ------- Comment #1 From Steve Northover 2004-10-07 19:39:16 -0400 ------- This behavior can't be changed now because application code now relies on it. Sorry. 22
  • detects Itemisations Keywords CUE Code samples Stack traces ZILLA Patches Screenshots computes Readability 23
  • Build and evaluate Cuezilla build CUE ZILLA 24
  • Build and evaluate Cuezilla build CUE ZILLA evaluate CUE ZILLA 24
  • Evaluation Setup Within projects 25
  • Evaluation Setup Within projects 48% agreement 91% off by one 25
  • Make recommendations CUE ZILLA + Stack trace 26
  • Make recommendations CUE ZILLA + Stack trace 26
  • Incentives to reporters 27
  • Incentives to reporters • Bug reports containing stack traces get fixed sooner. • Bug reports that are easier to read have lower lifetimes. • Code samples in your bug report increase chances of it getting fixed. 28
  • Duplicates “ Duplicates often add useful information. That this information is filed under a new report is not ideal. Apache Developer ” 29
  • Future of bug tracking lies in thinking outside the box! 30
  • Summary Provided by Rated most helpful Rated helpful most reporters by developers by most reporters Responses Helped most steps to reproduce steps to reproduce steps to reproduce steps to reproduce stack traces observed behaviour stack traces test cases expected behaviour test cases observed behaviour screenshots product observed behaviour stack traces observed behavior version screenshots expected behaviour test cases/examples operating system expected behaviour version expected behavior summary code examples code examples component summary error reports version severity version build information hardware severity build information error reports summary 14 Used when fixing a bug Assist reporters Future of bug tracking lies in thinking outside the box! 29 31