Software debugging is tedious and time consuming. To reduce the manual effort needed for debugging, researchers have proposed a considerable number of techniques to automate the process of fault localization; in particular, techniques based on information retrieval (IR) have drawn increased attention in recent years. Although reportedly effective, these techniques have some potential limitations that may affect their performance. First, their effectiveness is likely to depend heavily on the quality of the bug reports; unfortunately, high-quality bug reports that contain rich information are not always available. Second, these techniques have not been evaluated through studies that involve actual developers, which is less than ideal, as purely analytical evaluations can hardly show the actual usefulness of debugging techniques. The goal of this work is to evaluate the usefulness of IR-based techniques in real-world scenarios. Our investigation shows that bug reports do not always contain rich information, and that low-quality bug reports can considerably affect the effectiveness of these techniques. Our research also shows, through a user study, that high-quality bug reports benefit developers just as much as they benefit IR-based techniques. In fact, the information provided by IR-based techniques when operating on high-quality reports is only helpful to developers in a limited number of cases. And even in these cases, such information only helps developers get to the faulty file quickly, but does not help them in their most time consuming task: understanding and fixing the bug within that file.
Transferring Software Testing and Analytics Tools to PracticeTao Xie
Keynote Talk in the Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques (TAIC PART 2016) http://www2016.taicpart.org/
A Large-Scale Empirical Comparison of Static and DynamicTest Case Prioritizat...Kevin Moran
The large body of existing research in Test Case Prioritization (TCP) techniques, can be broadly classified into two categories: dynamic techniques (that rely on run-time execution information) and static techniques (that operate directly on source and test code). Absent from this current body of work is a comprehensive study aimed at understanding and evaluating the static approaches and comparing them to dynamic approaches on a large set of projects.
In this work, we perform the first extensive study aimed at empirically evaluating four static TCP techniques comparing them with state-of-research dynamic TCP techniques at different test-case granularities (e.g., method and class level) in terms of effectiveness, efficiency and similarity of faults detected. This study was performed on 30 real-word Java programs encompassing 431 KLoC. In terms of effectiveness, we find that the static call-graph based technique outperforms the other static techniques at test-class level, but the topic-model-based technique performs better at test-method level. In terms of efficiency, the static call-graph based technique is also the most efficient when compared to other static techniques. When examining the similarity of faults detected for the four static techniques compared to the four dynamic ones, we find that on average, the faults uncovered by these two groups of techniques are quite dissimilar, with the top 10% of test cases agreeing on only ~ 25% - 30% of detected faults. This prompts further research into the severity/importance of faults uncovered by these techniques, and into the potential for combining static and dynamic information for more effective approaches.
Transferring Software Testing and Analytics Tools to PracticeTao Xie
Keynote Talk in the Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques (TAIC PART 2016) http://www2016.taicpart.org/
A Large-Scale Empirical Comparison of Static and DynamicTest Case Prioritizat...Kevin Moran
The large body of existing research in Test Case Prioritization (TCP) techniques, can be broadly classified into two categories: dynamic techniques (that rely on run-time execution information) and static techniques (that operate directly on source and test code). Absent from this current body of work is a comprehensive study aimed at understanding and evaluating the static approaches and comparing them to dynamic approaches on a large set of projects.
In this work, we perform the first extensive study aimed at empirically evaluating four static TCP techniques comparing them with state-of-research dynamic TCP techniques at different test-case granularities (e.g., method and class level) in terms of effectiveness, efficiency and similarity of faults detected. This study was performed on 30 real-word Java programs encompassing 431 KLoC. In terms of effectiveness, we find that the static call-graph based technique outperforms the other static techniques at test-class level, but the topic-model-based technique performs better at test-method level. In terms of efficiency, the static call-graph based technique is also the most efficient when compared to other static techniques. When examining the similarity of faults detected for the four static techniques compared to the four dynamic ones, we find that on average, the faults uncovered by these two groups of techniques are quite dissimilar, with the top 10% of test cases agreeing on only ~ 25% - 30% of detected faults. This prompts further research into the severity/importance of faults uncovered by these techniques, and into the potential for combining static and dynamic information for more effective approaches.
Query Performance Prediction by Means of Intent-Aware Metrics in Systematic ...Giorgio Di Nunzio
London Information Retrieval Meetup September 2021 @ DESIRED 2021
Giorgio Maria Di Nunzio, Department of Information Engineering, University of Padova 15/09/2021
Intelligent Interactive Information Access (IIIA) Hub http://iiia.dei.unipd.it
During the past ten years, static analysis tools have become a vital part of software development for many organizations. However, the question arises, “Can we quantify the benefits of static analysis?” William Oliver presents the results of a Lawrence Livermore National Laboratory study that first measured the cost of finding software defects using formal testing on a system without static analysis; then, they integrated a static analysis tool into the process and, over a period of time, recalculated the cost of finding software defects. Join William as he shares the results of their study and discusses the value and benefits of static testing. Learn how commercial and open source analysis tools can perform sophisticated source code analysis over large code bases. Take back proof that employing static analysis can not only reduce the time and cost of finding defects and their subsequent debugging but ultimately can reduce the number of defects making their way into your releases.
Jogging While Driving, and Other Software Engineering Research Problems (invi...David Rosenblum
invited talk presented for the Distinguished Lecturer Series of the Department of Computer Science at the University of Illinois at Chicago, 10 April 2014
During the 24th Dutch Testing Day organized by TU Delft, Software Engineering Professor Arie van Deursen gave this talk about The European STAMP project on testing and Continuous Integration.
Rob Gallo shares with you information on the Pecha Kucha method of presenting information. The Pecha Kucha method can be used for business meetings, training seminars, academic events, and virtually all public speaking events.
Field Failure Reproduction Using Symbolic Execution and Genetic ProgrammingAlex Orso
Talk at the 30th CREST Open Workshop on Search Based Software Testing (SBST) and Dynamic Symbolic Execution (DSE) -- http://crest.cs.ucl.ac.uk/cow/30/
University College London, January 2014
Video available at http://youtu.be/i4T2g-mdJ-U
A recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects showed that the ability to recreate field failures is considered of fundamental importance when investigating bug reports. Unfortunately, the information typically contained in a bug report, such as memory dumps or call stacks, is usually insufficient for recreating the problem. Even more advanced approaches for gathering field data and help in-house debugging tend to collect either too little information, and be ineffective, or too much information, and be inefficient. This talk presents two techniques that address these issues: BugRedux and SBFR. Both techniques aim to provide support for in-house debugging of field failures by synthesizing, using execution data collected in the field, executions that mimic the observed field failures. BugRedux relies on symbolic execution, whereas SBFR leverages dynamic programming. The talk discusses the two techniques' complementary strengths and weaknesses.
IOSR Journal of Economics and Finance (IOSR-JEF) discourages theoretical articles that are limited to axiomatics or that discuss minor variations of familiar models. Similarly, IOSR-JEF has little interest in empirical papers that do not explain the model's theoretical foundations or that exhausts themselves in applying a new or established technique (such as cointegration) to another data set without providing very good reasons why this research is important.
Query Performance Prediction by Means of Intent-Aware Metrics in Systematic ...Giorgio Di Nunzio
London Information Retrieval Meetup September 2021 @ DESIRED 2021
Giorgio Maria Di Nunzio, Department of Information Engineering, University of Padova 15/09/2021
Intelligent Interactive Information Access (IIIA) Hub http://iiia.dei.unipd.it
During the past ten years, static analysis tools have become a vital part of software development for many organizations. However, the question arises, “Can we quantify the benefits of static analysis?” William Oliver presents the results of a Lawrence Livermore National Laboratory study that first measured the cost of finding software defects using formal testing on a system without static analysis; then, they integrated a static analysis tool into the process and, over a period of time, recalculated the cost of finding software defects. Join William as he shares the results of their study and discusses the value and benefits of static testing. Learn how commercial and open source analysis tools can perform sophisticated source code analysis over large code bases. Take back proof that employing static analysis can not only reduce the time and cost of finding defects and their subsequent debugging but ultimately can reduce the number of defects making their way into your releases.
Jogging While Driving, and Other Software Engineering Research Problems (invi...David Rosenblum
invited talk presented for the Distinguished Lecturer Series of the Department of Computer Science at the University of Illinois at Chicago, 10 April 2014
During the 24th Dutch Testing Day organized by TU Delft, Software Engineering Professor Arie van Deursen gave this talk about The European STAMP project on testing and Continuous Integration.
Rob Gallo shares with you information on the Pecha Kucha method of presenting information. The Pecha Kucha method can be used for business meetings, training seminars, academic events, and virtually all public speaking events.
Field Failure Reproduction Using Symbolic Execution and Genetic ProgrammingAlex Orso
Talk at the 30th CREST Open Workshop on Search Based Software Testing (SBST) and Dynamic Symbolic Execution (DSE) -- http://crest.cs.ucl.ac.uk/cow/30/
University College London, January 2014
Video available at http://youtu.be/i4T2g-mdJ-U
A recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects showed that the ability to recreate field failures is considered of fundamental importance when investigating bug reports. Unfortunately, the information typically contained in a bug report, such as memory dumps or call stacks, is usually insufficient for recreating the problem. Even more advanced approaches for gathering field data and help in-house debugging tend to collect either too little information, and be ineffective, or too much information, and be inefficient. This talk presents two techniques that address these issues: BugRedux and SBFR. Both techniques aim to provide support for in-house debugging of field failures by synthesizing, using execution data collected in the field, executions that mimic the observed field failures. BugRedux relies on symbolic execution, whereas SBFR leverages dynamic programming. The talk discusses the two techniques' complementary strengths and weaknesses.
IOSR Journal of Economics and Finance (IOSR-JEF) discourages theoretical articles that are limited to axiomatics or that discuss minor variations of familiar models. Similarly, IOSR-JEF has little interest in empirical papers that do not explain the model's theoretical foundations or that exhausts themselves in applying a new or established technique (such as cointegration) to another data set without providing very good reasons why this research is important.
Software engineering practitioners often spend significant amount of time and effort to debug. To help practitioners perform this crucial task, hundreds of papers have proposed various fault localization techniques. Fault localization helps practitioners to find the location of a defect given its symptoms (e.g., program failures). These localization techniques have pinpointed the locations of bugs of various systems of diverse sizes, with varying degrees of success, and for various usage scenarios. Unfortunately, it is unclear whether practitioners appreciate this line of research. To fill this gap, we performed an empirical study by surveying 386 practitioners from more than 30 countries across 5 continents about their expectations of research in fault localization. In particular, we investigated a number of factors that impact practitioners’ willingness to adopt a fault localization technique. We then compared what practitioners need and the current state-of-research by performing a literature review of papers on fault localization techniques published in ICSE, FSE, ESEC-FSE, ISSTA, TSE, and TOSEM in the last 5 years (2011-2015). From this comparison, we highlight the directions where researchers need to put effort to develop fault localization techniques that matter to practitioners.
Practical Tools for Measurement Systems AnalysisGabor Szabo, CQE
Practical Tools for Measurement Systems Analysis presented at the American Statistical Association's Orange County and Long Beach Chapter quarterly meeting
Software testing tools are evolving. More testing frameworks are emerging through the open source community and commercial vendors. In addition, we’re starting to see the rise of machine-learning (ML) and artificial intelligence (AI) in testing solutions.
Given this evolution, it is important to map the tools that match both the practitioners’ skills and their testing types. When referring to the testing practitioners, we mainly look at three different personas:
-The business tester
-The software developer in test (SDET)
-The software developer
These practitioners are tasked with creating, maintaining, and executing unit tests, build acceptance tests, integration, regression, and other nonfunctional tests.
In this webinar led by Perfecto’s Chief Evangelist, Eran Kinsbruner, you will learn the following:
-How should testing types be dispersed among the three personas and throughout the DevOps pipeline?
-What tools should each of these three personas use for the creation and execution of tests?
-What are the key benefits to continuous testing when mapped correctly?
Slides from Software Testing Techniques course offered at Kansas State University in Spring'16 and Spring'17. Entire course material can be found at https://github.com/rvprasad/software-testing-course.
Imagine you’re a test manager starting a new assignment. On the first day of work, you’re presented with a list of metrics you are to report. Soon, you realize that most of the metrics are not really connected to what should be measured. Or, consider the situation where you’re told that there is no value collecting metrics because “we’re agile.” In either situation, what would be your next step be? Join Pablo Garcia as he shares his experience with the dangers of poor metrics. Believing that some metrics can have value in helping testing be effective and efficient, Pablo shares his favorite metrics including a couple of crazy ones―requirements coverage, defect detection percentage, faults in production, and cost per bug. Each is discussed, evaluating what it really measures, when to use it, and how to present it to send the correct message. Take back a toolbox of testing metrics that will make your testing role easier.
Problem Statement:One of the common concerns from the customers is that how to effectively optimize the testing given the
multiple integration points in a distributed/composite system environments, which does expose at least the below
pain points:
1. Avoid Exhausted testing
2. Meet all the boundary conditions
3. Limited time to execute 100% test execution
4. Include all the critical business functions
5. Efficient Regression Testing
and the list goes on...
Resolution: The solution is detailed in the attachment and have effectively implemented in various client places.
VIP Workshop: Effective Habits of Development TeamsPaul Schreiber
There’s so many tools out there, but what are the best for managing development teams? Paul Schreiber, Developer at FiveThirtyEight, will walk through best practices and tools for workflow, automation and testing, along with good practices for managing development teams.
Chapter 10 Testing and Quality Assurance1Unders.docxketurahhazelhurst
Chapter 10:
Testing and Quality
Assurance
1
Understand quality & basic techniques for software verification and validation.
Analyze basics of software testing and testing techniques.
Discuss the concept of “inspection” process.
Objectives
2
Quality assurance (QA): activities designed
to measure and improve quality in a product— and process.
Quality control (QC): activities designed to validate and verify the quality of the product through detecting faults and “fixing” the defects.
Need good techniques, process, tools,
and team.
Testing Introduction
similar
3
Two traditional definitions:
Conforms to requirements.
Fit to use.
Verification: checking software conforms to
its requirements (did the software evolve
from the requirements properly; does the software “work”?)
Is the system correct?
Validation: checking software meets user requirements (fit to use)
Are we building the correct system?
What Is “Quality”?
4
Testing: executing program in a controlled environment and “verifying/validating” output.
Inspections and reviews.
Formal methods (proving software correct).
Static analysis detects “error-prone conditions.”
Some “Error-Detection” Techniques (finding errors)
5
Error: a mistake made by a programmer or software engineer that caused the fault, which in turn may cause a failure
Fault (defect, bug): condition that may cause a failure in the system
Failure (problem): inability of system to perform a function according to its spec due to some fault
Fault or failure/problem severity (based on consequences)
Fault or failure/problem priority (based on importance of developing a fix, which is in turn based
on severity)
Faults and Failures
6
Activity performed for:
Evaluating product quality
Improving products by identifying defects and having them fixed prior to software release
Dynamic (running-program) verification of program’s behavior on a finite set of test cases selected from execution domain.
Testing can NOT prove product works 100%—even though we use testing to demonstrate that parts of the software works.
Testing
Not always
done!
7
Who tests?
Programmers
Testers/Req. Analyst
Users
What is tested?
Unit code testing
Functional code testing
Integration/system testing
User interface testing
Testing (cont.)
Why test?
Acceptance (customer)
Conformance (std, laws, etc.)
Configuration (user vs. dev.)
Performance, stress, security, etc.
How (test cases designed)?
Intuition
Specification based (black box)
Code based (white box)
Existing cases (regression)
8
Progression of Testing
Equivalence Class Partitioning
Divide the input into several groups, deemed “equivalent” for purposes of finding errors.
Pick one “representative” for each class used for testing.
Equivalence classes determined by req./design specifications and some intuition
Example: pick “larger” of
two integers and . . .
Lessen duplication.
Complete coverage.
10
Suppose we have n distinct functional requirements.
Su ...
Multi-source connectivity as the driver of solar wind variability in the heli...Sérgio Sacani
The ambient solar wind that flls the heliosphere originates from multiple
sources in the solar corona and is highly structured. It is often described
as high-speed, relatively homogeneous, plasma streams from coronal
holes and slow-speed, highly variable, streams whose source regions are
under debate. A key goal of ESA/NASA’s Solar Orbiter mission is to identify
solar wind sources and understand what drives the complexity seen in the
heliosphere. By combining magnetic feld modelling and spectroscopic
techniques with high-resolution observations and measurements, we show
that the solar wind variability detected in situ by Solar Orbiter in March
2022 is driven by spatio-temporal changes in the magnetic connectivity to
multiple sources in the solar atmosphere. The magnetic feld footpoints
connected to the spacecraft moved from the boundaries of a coronal hole
to one active region (12961) and then across to another region (12957). This
is refected in the in situ measurements, which show the transition from fast
to highly Alfvénic then to slow solar wind that is disrupted by the arrival of
a coronal mass ejection. Our results describe solar wind variability at 0.5 au
but are applicable to near-Earth observatories.
Nutraceutical market, scope and growth: Herbal drug technologyLokesh Patil
As consumer awareness of health and wellness rises, the nutraceutical market—which includes goods like functional meals, drinks, and dietary supplements that provide health advantages beyond basic nutrition—is growing significantly. As healthcare expenses rise, the population ages, and people want natural and preventative health solutions more and more, this industry is increasing quickly. Further driving market expansion are product formulation innovations and the use of cutting-edge technology for customized nutrition. With its worldwide reach, the nutraceutical industry is expected to keep growing and provide significant chances for research and investment in a number of categories, including vitamins, minerals, probiotics, and herbal supplements.
Cancer cell metabolism: special Reference to Lactate PathwayAADYARAJPANDEY1
Normal Cell Metabolism:
Cellular respiration describes the series of steps that cells use to break down sugar and other chemicals to get the energy we need to function.
Energy is stored in the bonds of glucose and when glucose is broken down, much of that energy is released.
Cell utilize energy in the form of ATP.
The first step of respiration is called glycolysis. In a series of steps, glycolysis breaks glucose into two smaller molecules - a chemical called pyruvate. A small amount of ATP is formed during this process.
Most healthy cells continue the breakdown in a second process, called the Kreb's cycle. The Kreb's cycle allows cells to “burn” the pyruvates made in glycolysis to get more ATP.
The last step in the breakdown of glucose is called oxidative phosphorylation (Ox-Phos).
It takes place in specialized cell structures called mitochondria. This process produces a large amount of ATP. Importantly, cells need oxygen to complete oxidative phosphorylation.
If a cell completes only glycolysis, only 2 molecules of ATP are made per glucose. However, if the cell completes the entire respiration process (glycolysis - Kreb's - oxidative phosphorylation), about 36 molecules of ATP are created, giving it much more energy to use.
IN CANCER CELL:
Unlike healthy cells that "burn" the entire molecule of sugar to capture a large amount of energy as ATP, cancer cells are wasteful.
Cancer cells only partially break down sugar molecules. They overuse the first step of respiration, glycolysis. They frequently do not complete the second step, oxidative phosphorylation.
This results in only 2 molecules of ATP per each glucose molecule instead of the 36 or so ATPs healthy cells gain. As a result, cancer cells need to use a lot more sugar molecules to get enough energy to survive.
Unlike healthy cells that "burn" the entire molecule of sugar to capture a large amount of energy as ATP, cancer cells are wasteful.
Cancer cells only partially break down sugar molecules. They overuse the first step of respiration, glycolysis. They frequently do not complete the second step, oxidative phosphorylation.
This results in only 2 molecules of ATP per each glucose molecule instead of the 36 or so ATPs healthy cells gain. As a result, cancer cells need to use a lot more sugar molecules to get enough energy to survive.
introduction to WARBERG PHENOMENA:
WARBURG EFFECT Usually, cancer cells are highly glycolytic (glucose addiction) and take up more glucose than do normal cells from outside.
Otto Heinrich Warburg (; 8 October 1883 – 1 August 1970) In 1931 was awarded the Nobel Prize in Physiology for his "discovery of the nature and mode of action of the respiratory enzyme.
WARNBURG EFFECT : cancer cells under aerobic (well-oxygenated) conditions to metabolize glucose to lactate (aerobic glycolysis) is known as the Warburg effect. Warburg made the observation that tumor slices consume glucose and secrete lactate at a higher rate than normal tissues.
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...University of Maribor
Slides from:
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Track: Artificial Intelligence
https://www.etran.rs/2024/en/home-english/
This pdf is about the Schizophrenia.
For more details visit on YouTube; @SELF-EXPLANATORY;
https://www.youtube.com/channel/UCAiarMZDNhe1A3Rnpr_WkzA/videos
Thanks...!
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
The ASGCT Annual Meeting was packed with exciting progress in the field advan...
Evaluating the Usefulness of IR-Based Fault LocalizationTechniques
1. Evaluating the Usefulness
of IR-Based Fault
Localization Techniques
Qianqian Wang* Chris Parnin** Alessandro Orso*
* Georgia Institute of Technology, USA
** North Carolina State University, USA
6. IR-Based FL Techniques
• How do they work?
• Rank source files based on their lexical similarity to bug
reports
• How well do they work?
Top 1 Top 5 Top 10
Percentage 35% 58% 69%
7. Source code file: CTabFolder.java
public class CTabFolder extends Composite {
// tooltip
int [] toolTipEvents = new int[] {SWT.MouseExit,
SWT.MouseHover, SWT.MouseMove,
SWT.MouseDown, SWT.DragDetect};
Listener toolTipListener;
…
/ * Returns <code>true</code> if the CTabFolder
only displys the selected tab
* and <code>false</code> if the CTabFolder
displays multiple tabs.
*/
…void onMouseHover(Event event) {
showToolTip(event.x, event.y);
}
void onDispose() {
inDispose = true;
hideToolTip();
…
}
}
Understanding IR-based FL Techniques
Bug ID: 90018
Summary: Native tooltips left around on
CTabFolder.
Description: Hover over the PartStack
CTabFolder inside eclipse until some native
tooltip is displayed. For example, the
maximize button. When the tooltip appears,
change perspectives using the keybinding.
the CTabFolder gets hidden, but its tooltip
is permanently displayed and never goes
away. Even if that CTabFolder is disposed
(I'm assuming) when the perspective is
closed.
--------------------------------------------------------------------------
8. Source code file: CTabFolder.java
public class CTabFolder extends Composite {
// tooltip
int [] toolTipEvents = new int[] {SWT.MouseExit,
SWT.MouseHover, SWT.MouseMove,
SWT.MouseDown, SWT.DragDetect};
Listener toolTipListener;
…
/ * Returns <code>true</code> if the CTabFolder
only displys the selected tab
* and <code>false</code> if the CTabFolder
displays multiple tabs.
*/
…void onMouseHover(Event event) {
showToolTip(event.x, event.y);
}
void onDispose() {
inDispose = true;
hideToolTip();
…
}
}
Understanding IR-based FL Techniques
Bug ID: 90018
Summary: Native tooltips left around on
CTabFolder.
Description: Hover over the PartStack
CTabFolder inside eclipse until some native
tooltip is displayed. For example, the
maximize button. When the tooltip appears,
change perspectives using the keybinding.
the CTabFolder gets hidden, but its tooltip
is permanently displayed and never goes
away. Even if that CTabFolder is disposed
(I'm assuming) when the perspective is
closed.
--------------------------------------------------------------------------
9. • Does the presence of technical information affect the fault
localization results?
• How often do bug reports contain such information?
• Is such information enough for developers to find the faulty
files easily?
Assessing IR-based FL Techniques
14. Q1: Results
Program
entity
Stack
trace
Test case
Results √ X X
• Does bug report information affect fault
localization results?
√ Statistically significant difference: p < 0.05
X No statistically significant difference: p >= 0.05
15. Q1: Results
Program
entity
Stack
trace
Test case
Results √ X X
• Does bug report information affect fault
localization results?
√ Statistically significant difference: p < 0.05
X No statistically significant difference: p >= 0.05
16. Q1: Results
Program
entity
Stack
trace
Test case
Results √ X X
• Does bug report information affect fault
localization results?
√ Statistically significant difference: p < 0.05
X No statistically significant difference: p >= 0.05
Bug report characteristics affect IR-based
fault localization results
17. • How often bug reports contain technical
information?
• Select 10,000 bug reports from SWT Bugzilla
• Check presence of technical information:
• Stack traces
• Test cases
• Program entity names
Q2: Method
18. • How often bug reports contain technical
information?
Q2: Results
Stack traces Test cases
Program
entity
Percentage 10% 3% 32%
19. • How often bug reports contain technical
information?
Q2: Results
Stack traces Test cases
Program
entity
Percentage 10% 3% 32%
20. • How often bug reports contain technical
information?
Q2: Results
Stack traces Test cases
Program
entity
Percentage 10% 3% 32%
The majority bug reports do not contain
enough information
21. Additional finding
“Optimistic” Evaluation Approach
• Assumption: Changed files = faulty files
• Reality:
• 40% bugs contain multiple changed files
• Not all changed files contain bugs
• Best-ranked files may not be faulty
22. Additional finding
“Optimistic” Evaluation Approach
• Assumption: Changed files = faulty files
• Reality:
• 40% bugs contain multiple changed files
• Not all changed files contain bugs
• Best-ranked files may not be faulty
Results of existing studies might be
worse than what reported
23. • Q3: Does bug report information affect
developers’ performance?
• Q4: Do IR-based techniques help developers’
performance?
User Study
24. Experiment Protocol: Setup
Participants:
70 developers
Graduate Students
Software subject:
• Eclipse SWT
• 2 bugs for each developer
Task: find and fix the bug
Tools:
• Eclipse plug-in
• Integrating ranked lists
• Logging
…"
1)"
2)"
3)"
4)"
✔
✔
✔
28. Q3: Results
Time used to find
the faulty file
Time used to locate
the bug
Debugging score
√ X √
Compared the performance of 2 groups:
1. without tool, good bug reports
2. without tool, bad bug reports
√ Statistically significant difference: p < 0.05
X No statistically significant difference: p >= 0.05
29. Q3: Results
Time used to find
the faulty file
Time used to locate
the bug
Debugging score
√ X √
Compared the performance of 2 groups:
1. without tool, good bug reports
2. without tool, bad bug reports
√ Statistically significant difference: p < 0.05
X No statistically significant difference: p >= 0.05
30. Q3: Results
Time used to find
the faulty file
Time used to locate
the bug
Debugging score
√ X √
Compared the performance of 2 groups:
1. without tool, good bug reports
2. without tool, bad bug reports
Good bug reports (i.e., with entity names) allow
developers to shorten the time to find the
faulty file and help them find better fixes
√ Statistically significant difference: p < 0.05
X No statistically significant difference: p >= 0.05
31. Q4: Results
Condition
Debugging
score
Time to find
the file
Time to locate
the bug
X X X
X √ X
X X X
X X X
Compared the performance of 2 groups under 4 conditions:
1. without tool, {good|bad} bug reports, {good|bad} ranked list
2. with tool, {good|bad} bug reports, {good|bad} ranked list
32. Q4: Results
Condition
Debugging
score
Time to find
the file
Time to locate
the bug
X X X
X √ X
X X X
X X X
Compared the performance of 2 groups under 4 conditions:
1. without tool, {good|bad} bug reports, {good|bad} ranked list
2. with tool, {good|bad} bug reports, {good|bad} ranked list
Good ranked
list
Bad ranked
list
Good bug
report
Bad bug
report
X Not statist, sign.
√ Statist. significant
33. Q4: Results
Condition
Debugging
score
Time to find
the file
Time to locate
the bug
X X X
X √ X
X X X
X X X
Compared the performance of 2 groups under 4 conditions:
1. without tool, {good|bad} bug reports, {good|bad} ranked list
2. with tool, {good|bad} bug reports, {good|bad} ranked list
Good ranked
list
Bad ranked
list
Good bug
report
Bad bug
report
X Not statist, sign.
√ Statist. significant
34. Q4: Results
Condition
Debugging
score
Time to find
the file
Time to locate
the bug
X X X
X √ X
X X X
X X X
Compared the performance of 2 groups under 4 conditions:
1. without tool, {good|bad} bug reports, {good|bad} ranked list
2. with tool, {good|bad} bug reports, {good|bad} ranked list
Good ranked
list
Bad ranked
list
Good bug
report
Bad bug
report
X Not statist, sign.
√ Statist. significant
Only perfect ranked lists help when users
can not get enough hints from bug reports
35. Q4: Results
Condition
Debugging
score
Time to find
the file
Time to locate
the bug
X X X
X √ X
X X X
X X X
Compared the performance of 2 groups under 4 conditions:
1. without tool, {good|bad} bug reports, {good|bad} ranked list
2. with tool, {good|bad} bug reports, {good|bad} ranked list
Good ranked
list
Bad ranked
list
Good bug
report
Bad bug
report
X Not statist, sign.
√ Statist. significant
Only perfect ranked lists help when users
can not get enough hints from bug reports
The tool only helps find the faulty file, but
developers spend much more time locating
the bug in the faulty file than finding such file
36. Additional Observations
• Developers used program entity names in the
bug report as search keywords.
• Ranked lists generated by IR-based techniques
affected users’ debugging behavior
• Gave a starting point
• Gave them confidence
37. Summary
• Studied the practical usefulness of IR-based FL techniques
• Performed both an analytical study and a user study
• Main findings
• Bug report characteristics affect IR-based fault localization results
• Results of existing studies might be worse than what reported
• The majority of bug reports do not contain enough information
• “Good” bug reports allow developers to shorten the time to find the
faulty file and help them find better fixes
• Only perfect ranked lists help when users can not get enough hints from
bug reports
• The tool only helps find the faulty file, but developers spend much more
time locating the bug in the faulty file than finding such file
38. Implications
• Better bug reports are needed
• Automated debugging techniques should focus
on improving results for bug reports with little
information
• Automated debugging techniques should
provide finer-grained information and context
• More user studies and realistic evaluations are
needed