Delta debugging is a technique for isolating the cause of failures in programs. It works by systematically removing parts of changes made between a version where the program worked and a version where it fails. The algorithm efficiently determines the minimal set of changes responsible for the failure. The technique was able to isolate a single change causing a failure in GDB from 178,000 changed lines within a few hours. Automating debugging through techniques like delta debugging can help developers quickly find and fix bugs.
21. Vom Fachbereich f¨ r Mathematik und Informatik
u
der Technischen Universit¨ t Braunschweig
a
genehmigte Dissertation
zur Erlangung des Grades eines
Doktor-Ingenieurs (Dr.-Ing.)
Andreas Zeller
Configuration Management with Version Sets
A Unified Software Versioning Model
and its Applications
1. April 1997
1. Referent: Prof. Dr. Gregor Snelting
2. Referent: Prof. Dr. Walter F. Tichy
Eingereicht am: 1. November 1996
22.
23. Bug Reports
From: Brian Kahne <bkahne@ibmoto.com>
To: DDD Bug Report Address <bug-ddd@gnu.org>
Subject: Problem with DDD and GDB 4.17
When using DDD with GDB 4.16, the run command
correctly uses any prior command-line arguments, or
the value of "set args". However, when I switched to
GDB 4.17, this no longer worked: If I entered a run
command in the console window, the prior
command-line options would be lost. [...]
44. Bug Reports
From: Brian Kahne <bkahne@ibmoto.com>
To: DDD Bug Report Address <bug-ddd@gnu.org>
Subject: Problem with DDD and GDB 4.17
When using DDD with GDB 4.16, the run command
correctly uses any prior command-line arguments, or
the value of "set args". However, when I switched to
GDB 4.17, this no longer worked: If I entered a run
command in the console window, the prior
command-line options would be lost. [...]
55. Bisection
✔ ✔ ✔✘ ✘
Yesterday Today
Failure Cause
56. What was Changed
$ diff -r gdb-4.16 gdb-4.17
diff -r gdb-4.16/COPYING gdb-4.17/COPYING
5c5
< 675 Mass Ave, Cambridge, MA 02139, USA
---
> 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
282c282
< Appendix: How to Apply These Terms to Your New Programs
---
> How to Apply These Terms to Your New Programs
…and so on for 178,200 lines (8,721 locations)
93. Isolating Changes
Delta Debugging Log
100000
GDB with ddmin algorithm
10000 ... with dd algorithm
... plus scope information
Changes left
1000
100
10
1
0 50 100 150 200 250 300
Tests executed
• Result after 98 tests (= 1 hour)
94. The Failure Cause
diff -r gdb-4.16/gdb/infcmd.c gdb-4.17/gdb/infcmd.c
1239c1278
< "Set arguments to give program being debugged when it is
started.n
---
> "Set argument list to give program being debugged when
it is started.n
• Documentation becomes GDB output
• DDD expects Arguments,
but GDB outputs Argument list
95. Andreas Zeller · TU Braunschweig
Gestern lief mein Programm.
Heute nicht mehr. Warum?
Fehlersuche mit Delta Debugging
Andreas Zeller
Mittagsseminar, TU Braunschweig, 16. M¨rz 1998
a
0
96. Yesterday, my program worked.
Today, it does not. Why?
Andreas Zeller
Universit¨ t Passau
a
Lehrstuhl f¨ r Software-Systeme
u
Innstraße 33, D-94032 Passau, Germany
zeller@acm.org
Abstract. Imagine some program and a number of changes. If none of these
changes is applied (“yesterday”), the program works. If all changes are applied
(“today”), the program does not work. Which change is responsible for the fail-
ure? We present an efficient algorithm that determines the minimal set of failure-
inducing changes. Our delta debugging prototype tracked down a single failure-
inducing change from 178,000 changed GDB lines within a few hours.
1 A True Story
The GDB people have done it again. The new release 4.17 of the GNU debugger [6]
149. Contracts
set_hour (h: INTEGER) is
-- Set the hour from `h'
require
sane_h: 0 <= h and h <= 23
ensure
hour_set: hour = h
minute_unchanged: minutes = old minutes
second_unchanged: seconds = old seconds
160. Scienti c Method
Problem Report
Code
Hypothesis Prediction
Run
More Runs
161. Scienti c Method
Problem Report
Code
Hypothesis Prediction Experiment
Run
More Runs
162. Scienti c Method
Problem Report
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
More Runs
163. Scienti c Method
Problem Report Hypothesis is supported:
re ne hypothesis
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
More Runs
164. Scienti c Method
Problem Report Hypothesis is supported:
re ne hypothesis
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
Hypothesis is rejected:
More Runs create new hypothesis
165. Scienti c Method
Problem Report Hypothesis is supported:
re ne hypothesis
Code
Observation
Hypothesis Prediction Experiment + Conclusion
Run
Hypothesis is rejected:
More Runs create new hypothesis
Diagnosis
169. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment
Observation
Conclusion
170. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment Observe a[0] at Line 37.
Observation
Conclusion
171. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment Observe a[0] at Line 37.
Observation a[0] = 0 holds as predicted.
Conclusion
172. An Explicit Process
Hypothesis The execution causes a[0] = 0
Prediction At Line 37, a[0] = 0 should hold.
Experiment Observe a[0] at Line 37.
Observation a[0] = 0 holds as predicted.
Conclusion Hypothesis is confirmed.
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You're never gonna keep me down
(so serious as a radical these days)
And this is me in 1997
I was not singing at all at the time &#x2013;&#xA0;I was doing my PhD&#x2026;
with this man, my advisor Gregor Snelting.
we would usually find a cool formalism first, and then look at applications.
Say: Girard&#x2019;s linear logic
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
personal perspective: look at the code, pizza, clock, etc.
we had slicing, algorithmic debugging &#x2013;&#xA0;that&#x2019;s it!
Sometimes life gives you a blank sheet of paper, and it&#x2019;s up to you to fill it.
In my case, it was: bugs bugs bugs
In my case, it was: bugs bugs bugs
In my case, it was: bugs bugs bugs
In my case, it was: bugs bugs bugs
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
First idea: bisection.
But: bisection doesn&#x2019;t cut it!
But how do you get to this minimal difference? I tried quite a number of approaches, including genetic programming and search-based methods such as simulated annealing. Eventually, I had to come up with my own.
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
&#x2026; plus a few slides on how it works, etc., plus more applications
1998 talk
Pic of 1999 paper. Note that I submitted this 1 year after the first talk. That&#x2019;s because I spent all this time simplifying it.
That&#x2019;s because I spent one year making it as simple as possible, but no simpler. And this worked quite well.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
a few &#x201C;simple&#x201D; ideas have obtained great traction
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
applications on input, program state
Today, delta debugging is a core part of git, mercurial, and other version control tools. (which is how I contributed to version control after all)
entry on delta debugging, s/w architecture, more
entry on delta debugging, s/w architecture, more
So, that&#x2019;s what we have right now: tools tools tools
So, that&#x2019;s what we have right now: tools tools tools
So, that&#x2019;s what we have right now: tools tools tools
what do we still need to do?
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations
Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
We lack good benchmarks.
http://www.jjchandler.com/tombstone/download.php
We cannot tell bugs from features.
And by the way, this implies that interactive debuggers are the wrong tools.
Again, remember the long nights.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
So maybe it&#x2019;s not bugs bugs bugs
So maybe it&#x2019;s not bugs bugs bugs
So maybe it&#x2019;s not bugs bugs bugs
or tools tools tools
or tools tools tools
or tools tools tools
but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;
let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;
let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;