• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Debugging Debugging
 

Debugging Debugging

on

  • 4,082 views

Talk recording available! See ...

Talk recording available! See
http://www.st.cs.uni-saarland.de/zeller/Debugging-Debugging/

"Yesterday, my program worked. Today, it does not. Why?" This paper, originally published at ESEC/FSE 1999, introduced the concept of delta debugging, a simple, yet very effective method for automated debugging.

In 2009, this paper received the ACM SIGSOFT impact award as the most influential software engineering publication of 1999. In his keynote at ESEC/FSE in Amsterdam on August 27, 2009, Andreas Zeller reviews the state of automated debugging in the past and the future. He shares stories about how delta debugging came to be, how it can be hard to be simple, what programmers really need, and what research should do (and should not do) to explore and cater to these needs.

Talk recording available! See
http://www.st.cs.uni-saarland.de/zeller/Debugging-Debugging/

Statistics

Views

Total Views
4,082
Views on SlideShare
4,054
Embed Views
28

Actions

Likes
5
Downloads
45
Comments
0

3 Embeds 28

http://www.slideshare.net 26
http://www.linkedin.com 1
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You&apos;re never gonna keep me down <br /> (so serious as a radical these days)
  • Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You&apos;re never gonna keep me down <br /> (so serious as a radical these days)
  • Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You&apos;re never gonna keep me down <br /> (so serious as a radical these days)
  • Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You&apos;re never gonna keep me down <br /> (so serious as a radical these days)
  • Elton John - like a Candle in the Wind * Los del Rio - A la tuhuelpa legria macarena Eeeh, macarena * Spice Girls - If you wanna be my lover, you gotta get with my friends * Chumbawamba - I get knocked down but I get up again, You&apos;re never gonna keep me down <br /> (so serious as a radical these days)
  • And this is me in 1997 <br /> I was not singing at all at the time &#x2013;&#xA0;I was doing my PhD&#x2026;
  • with this man, my advisor Gregor Snelting.
  • we would usually find a cool formalism first, and then look at applications. <br /> Say: Girard&#x2019;s linear logic
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • personal perspective: look at the code, pizza, clock, etc.
  • we had slicing, algorithmic debugging &#x2013;&#xA0;that&#x2019;s it!
  • Sometimes life gives you a blank sheet of paper, and it&#x2019;s up to you to fill it.
  • In my case, it was: bugs bugs bugs
  • In my case, it was: bugs bugs bugs
  • In my case, it was: bugs bugs bugs
  • In my case, it was: bugs bugs bugs
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • First idea: bisection.
  • But: bisection doesn&#x2019;t cut it!
  • But how do you get to this minimal difference? I tried quite a number of approaches, including genetic programming and search-based methods such as simulated annealing. Eventually, I had to come up with my own.
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • &#x2026; plus a few slides on how it works, etc., plus more applications
  • 1998 talk
  • Pic of 1999 paper. Note that I submitted this 1 year after the first talk. That&#x2019;s because I spent all this time simplifying it.
  • That&#x2019;s because I spent one year making it as simple as possible, but no simpler. And this worked quite well.
  • to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
  • to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
  • to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
  • to me, it brought a tenured position, and all the &#x2022; riches and &#x2022; privileges that &#x2022;&#xA0;followed.
  • a few &#x201C;simple&#x201D; ideas have obtained great traction
  • applications on input, program state
  • applications on input, program state
  • applications on input, program state
  • applications on input, program state
  • applications on input, program state
  • applications on input, program state
  • applications on input, program state
  • Today, delta debugging is a core part of git, mercurial, and other version control tools. (which is how I contributed to version control after all)
  • entry on delta debugging, s/w architecture, more
  • entry on delta debugging, s/w architecture, more
  • So, that&#x2019;s what we have right now: tools tools tools
  • So, that&#x2019;s what we have right now: tools tools tools
  • So, that&#x2019;s what we have right now: tools tools tools
  • what do we still need to do?
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • Traditional approach: Defect localization &#x2013; Produce a list of possible defect locations <br /> Our approach: Automatic fixes &#x2013; Produce a set of valid fixes
  • We lack good benchmarks.
  • http://www.jjchandler.com/tombstone/download.php
  • We cannot tell bugs from features.
  • And by the way, this implies that interactive debuggers are the wrong tools.
  • Again, remember the long nights.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • There is a systematic way to do this. Delta debugging is its implementation; but you can (and you should!) do this manually, too.
  • Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
  • Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
  • Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
  • Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
  • Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
  • Keep a log: We&#x2019;re systematic, and we&#x2019;re explicit, again, all through our process.
  • So maybe it&#x2019;s not bugs bugs bugs
  • So maybe it&#x2019;s not bugs bugs bugs
  • So maybe it&#x2019;s not bugs bugs bugs
  • or tools tools tools
  • or tools tools tools
  • or tools tools tools
  • but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
  • but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
  • but developers developers developers (and their process - their debugging process). That&#x2019;s what we need to study, and that&#x2019;s what we need to fix.
  • let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;
  • let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;
  • let me state that again: their process - their debugging process, but also their development process. That&#x2019;s what we need to study &#x2013;

Debugging Debugging Debugging Debugging Presentation Transcript

  • Debugging Debugging Andreas Zeller
  • Debugging
  • Debugging Tracking
  • Debugging Tracking Simplifying
  • Debugging Tracking Simplifying Automating
  • Debugging Tracking Simplifying Automating Fixing
  • Tracking Simplifying Debugging Automating Fixing
  • Tracking Debugging
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • 1997
  • Gregor Snelting
  • Vom Fachbereich f¨ r Mathematik und Informatik u der Technischen Universit¨ t Braunschweig a genehmigte Dissertation zur Erlangung des Grades eines Doktor-Ingenieurs (Dr.-Ing.) Andreas Zeller Configuration Management with Version Sets A Unified Software Versioning Model and its Applications 1. April 1997 1. Referent: Prof. Dr. Gregor Snelting 2. Referent: Prof. Dr. Walter F. Tichy Eingereicht am: 1. November 1996
  • Bug Reports From: Brian Kahne <bkahne@ibmoto.com> To: DDD Bug Report Address <bug-ddd@gnu.org> Subject: Problem with DDD and GDB 4.17 When using DDD with GDB 4.16, the run command correctly uses any prior command-line arguments, or the value of "set args". However, when I switched to GDB 4.17, this no longer worked: If I entered a run command in the console window, the prior command-line options would be lost. [...]
  • Henry Lieberman
  • Simplifying Debugging
  • Bug Reports From: Brian Kahne <bkahne@ibmoto.com> To: DDD Bug Report Address <bug-ddd@gnu.org> Subject: Problem with DDD and GDB 4.17 When using DDD with GDB 4.16, the run command correctly uses any prior command-line arguments, or the value of "set args". However, when I switched to GDB 4.17, this no longer worked: If I entered a run command in the console window, the prior command-line options would be lost. [...]
  • Bisection
  • Bisection
  • Bisection ✔ Yesterday
  • Bisection ✔ ✘ Yesterday Today
  • Bisection ✔ ✘ Yesterday Today
  • Bisection ✔ ✔ ✘ Yesterday Today
  • Bisection ✔ ✔ ✘ Yesterday Today
  • Bisection ✔ ✔ ✘ ✘ Yesterday Today
  • Bisection ✔ ✔ ✘ ✘ Yesterday Today
  • Bisection ✔ ✔ ✔✘ ✘ Yesterday Today
  • Bisection ✔ ✔ ✔✘ ✘ Yesterday Today Failure Cause
  • What was Changed $ diff -r gdb-4.16 gdb-4.17 diff -r gdb-4.16/COPYING gdb-4.17/COPYING 5c5 < 675 Mass Ave, Cambridge, MA 02139, USA --- > 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 282c282 < Appendix: How to Apply These Terms to Your New Programs --- > How to Apply These Terms to Your New Programs …and so on for 178,200 lines (8,721 locations)
  • Causality Today Program works Program fails Yesterday Cause
  • David Lewis 1941–2001
  • Causality Today Program works Program fails Yesterday Cause
  • Actual Causes “The” cause (actual cause) is a minimal difference Program works Program fails Actual cause
  • Today
  • Today ✘
  • Today ✘ Yesterday
  • Today ✘ ✔ Yesterday
  • Today ✘ ✔ Yesterday
  • Today ✘ ✔ ✔ Yesterday
  • Today ✘ ✔ ✔ Yesterday
  • Today ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ? ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ? ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✘ ✔ ✔ Yesterday
  • Today ✘ ✘ ✘ … ✘ ✔ … ✔ ✔ Yesterday
  • Today ✘ ✘ ✘ … ✘ ✔ … ✔ ✔ Yesterday
  • Today ✘ ✘ ✘ … ✘ ✔ … Failure Cause ✔ ✔ Yesterday
  • Today test(c✘ ) = ✘ ✘ ✘ ✘ … ∆ = c✘ c✔ ✘ ✔ … Failure Cause test(c✔ ✔✔ )= ✔ Yesterday
  • test(c✘ ) = ✘ test(c✔ ) = ✔ ∆ = c✘ c✔
  • test(c✘ ) = ✘ test(c✔ ) = ✔ ∆ = c✘ c✔ dd(c✔ , c✘ ) = dd (c✔ , c✘ , 2)
  • test(c✘ ) = ✘ test(c✔ ) = ✔ ∆ = c✘ c✔ dd(c✔ , c✘ ) = dd (c✔ , c✘ , 2) dd (c✔ , c✘ , n) =  (c✔ , c✘ )   if |∆| = 1  dd (c ∆i , c , 2) if ∃i ∈ {1..n} · test(c✘ ∆i ) = ✔  ✘ ✘    dd (c , c ∪ ∆ , 2) if ∃i ∈ {1..n} · test(c✔ ∪ ∆i ) = ✘     ✔ ✔ i dd c✔ ∪ ∆i , c✘ , max(n − 1, 2) else if ∃i ∈ {1..n} · test(c✔ ∪ ∆i ) = ✔  dd c , c ∆ , max(n − 1, 2) else if ∃i ∈ {1..n} · test(c✘ ∆i ) = ✘     ✔ ✘ i  dd c , c , min(2n, |∆|)  else if n < |∆| (“increase granularity”) ✔ ✘    (c , c ) otherwise  ✔ ✘
  • Delta Debugging dd(c✔ , c✘ ) = dd (c✔ , c✘ , 2) dd (c✔ , c✘ , n) =  (c✔ , c✘ )   if |∆| = 1  dd (c ∆i , c , 2) if ∃i ∈ {1..n} · test(c✘ ∆i ) = ✔  ✘ ✘    dd (c , c ∪ ∆ , 2) if ∃i ∈ {1..n} · test(c✔ ∪ ∆i ) = ✘     ✔ ✔ i dd c✔ ∪ ∆i , c✘ , max(n − 1, 2) else if ∃i ∈ {1..n} · test(c✔ ∪ ∆i ) = ✔  dd c , c ∆ , max(n − 1, 2) else if ∃i ∈ {1..n} · test(c✘ ∆i ) = ✘     ✔ ✘ i  dd c , c , min(2n, |∆|)  else if n < |∆| (“increase granularity”) ✔ ✘    (c , c ) otherwise  ✔ ✘
  • def dd(c_pass, c_fail): n = 2 while 1: delta = listminus(c_fail, c_pass) deltas = split(delta, n); offset = 0; j = 0 while j < n: i = (j + offset) % n next_c_pass = listunion(c_pass, deltas[i]) next_c_fail = listminus(c_fail, deltas[i]) if test(next_c_fail) == FAIL and n == 2: c_fail = next_c_fail; n = 2; offset = 0; break elif test(next_c_fail) == PASS: c_pass = next_c_fail; n = 2; offset = 0; break elif test(next_c_pass) == FAIL: c_fail = next_c_pass; n = 2; offset = 0; break elif test(next_c_fail) == FAIL: c_fail = next_c_fail; n = max(n - 1, 2); offset = i; break elif test(next_c_pass) == PASS: c_pass = next_c_pass; n = max(n - 1, 2); offset = i; break else: j = j + 1 if j >= n: if n >= len(delta): return (delta, c_pass, c_fail) else: n = min(len(delta), n * 2)
  • Isolating Changes
  • Isolating Changes Delta Debugging Log 100000 GDB with ddmin algorithm 10000 ... with dd algorithm ... plus scope information Changes left 1000 100 10 1 0 50 100 150 200 250 300 Tests executed • Result after 98 tests (= 1 hour)
  • The Failure Cause diff -r gdb-4.16/gdb/infcmd.c gdb-4.17/gdb/infcmd.c 1239c1278 < "Set arguments to give program being debugged when it is started.n --- > "Set argument list to give program being debugged when it is started.n • Documentation becomes GDB output • DDD expects Arguments, but GDB outputs Argument list
  • Andreas Zeller · TU Braunschweig Gestern lief mein Programm. Heute nicht mehr. Warum? Fehlersuche mit Delta Debugging Andreas Zeller Mittagsseminar, TU Braunschweig, 16. M¨rz 1998 a 0
  • Yesterday, my program worked. Today, it does not. Why? Andreas Zeller Universit¨ t Passau a Lehrstuhl f¨ r Software-Systeme u Innstraße 33, D-94032 Passau, Germany zeller@acm.org Abstract. Imagine some program and a number of changes. If none of these changes is applied (“yesterday”), the program works. If all changes are applied (“today”), the program does not work. Which change is responsible for the fail- ure? We present an efficient algorithm that determines the minimal set of failure- inducing changes. Our delta debugging prototype tracked down a single failure- inducing change from 178,000 changed GDB lines within a few hours. 1 A True Story The GDB people have done it again. The new release 4.17 of the GNU debugger [6]
  • Automating Debugging
  • VALUE="Mac System 7">Mac System 7<OPTION VALUE="Mac System 7.5">Mac System 7.5<OPTION VALUE="Mac System 7.6.1">Mac System 7.6.1<OPTION VALUE="Mac System 8.0">Mac System 8.0<OPTION VALUE="Mac System Delta Debugging 8.5">Mac System 8.5<OPTION VALUE="Mac System 8.6">Mac System 8.6<OPTION VALUE="Mac System 9.x">Mac System 9.x<OPTION VALUE="MacOS X">MacOS X<OPTION VALUE="Linux">Linux<OPTION VALUE="BSDI">BSDI<OPTION VALUE="FreeBSD">FreeBSD<OPTION VALUE="NetBSD">NetBSD<OPTION VALUE="OpenBSD">OpenBSD<OPTION VALUE="AIX">AIX<OPTION VALUE="BeOS">BeOS<OPTION VALUE="HP-UX">HP-UX<OPTION VALUE="IRIX">IRIX<OPTION VALUE="Neutrino">Neutrino<OPTION VALUE="OpenVMS">OpenVMS<OPTION VALUE="OS/2">OS/2<OPTION VALUE="OSF/ 1">OSF/1<OPTION VALUE="Solaris">Solaris<OPTION VALUE="SunOS">SunOS<OPTION VALUE="other">other</SELECT> </td> <td align=left valign=top> <SELECT NAME="priority" MULTIPLE SIZE=7> > <OPTION VALUE="--">--<OPTION VALUE="P1">P1<OPTION VALUE="P2">P2<OPTION VALUE="P3">P3<OPTION VALUE="P4">P4<OPTION VALUE="P5">P5</SELECT> </td> <td align=left valign=top <SELECT NAME="bug_severity" MULTIPLE SIZE=7> <OPTION VALUE="blocker">blocker<OPTION
  • VALUE="Mac System 7">Mac System 7<OPTION VALUE="Mac System 7.5">Mac System 7.5<OPTION VALUE="Mac System 7.6.1">Mac System 7.6.1<OPTION VALUE="Mac System 8.0">Mac System 8.0<OPTION VALUE="Mac System 8.5">Mac System 8.5<OPTION VALUE="Mac System 8.6">Mac System 8.6<OPTION VALUE="Mac System 9.x">Mac System 9.x<OPTION VALUE="MacOS X">MacOS X<OPTION VALUE="Linux">Linux<OPTION VALUE="BSDI">BSDI<OPTION VALUE="FreeBSD">FreeBSD<OPTION VALUE="NetBSD">NetBSD<OPTION VALUE="OpenBSD">OpenBSD<OPTION VALUE="AIX">AIX<OPTION VALUE="BeOS">BeOS<OPTION VALUE="HP-UX">HP-UX<OPTION VALUE="IRIX">IRIX<OPTION VALUE="Neutrino">Neutrino<OPTION VALUE="OpenVMS">OpenVMS<OPTION VALUE="OS/2">OS/2<OPTION VALUE="OSF/ 1">OSF/1<OPTION VALUE="Solaris">Solaris<OPTION VALUE="SunOS">SunOS<OPTION VALUE="other">other</SELECT> </td> <td align=left valign=top> <SELECT NAME="priority" MULTIPLE SIZE=7> > <OPTION VALUE="--">--<OPTION VALUE="P1">P1<OPTION VALUE="P2">P2<OPTION VALUE="P3">P3<OPTION VALUE="P4">P4<OPTION VALUE="P5">P5</SELECT> </td> <td align=left valign=top <SELECT NAME="bug_severity" MULTIPLE SIZE=7> <OPTION VALUE="blocker">blocker<OPTION
  • <SELECT>
  • Failure Cause
  • Ralf Hildebrandt Holger Cleve Martin Burger
  • Statistical Debugging
  • Jim Jones Ben Liblit
  • Omniscient Debugging
  • Bil Lewis
  • Dynamic Slicing
  • Andy Ko
  • Bug Mining
  • Tom Zimmermann Harald Gall Nachi Nagappan
  • Automatic Fixing
  • Westley Weimer
  • WINNER OF JOLT PRODUCTIVITY AWARD ANDREAS ZELLER WHY PROGRAMS FAIL A GUIDE TO SYSTEMATIC DEBUGGING SECOND EDITION
  • Fixing Debugging
  • Rob DeLine Gina Venolia
  • Contracts set_hour (h: INTEGER) is -- Set the hour from `h' require sane_h: 0 <= h and h <= 23 ensure hour_set: hour = h minute_unchanged: minutes = old minutes second_unchanged: seconds = old seconds
  • Bertrand Meyer Gary Leavens
  • Scienti c Method
  • Scienti c Method Hypothesis
  • Scienti c Method Code Hypothesis
  • Scienti c Method Problem Report Code Hypothesis
  • Scienti c Method Problem Report Code Hypothesis Run
  • Scienti c Method Problem Report Code Hypothesis Run More Runs
  • Scienti c Method Problem Report Code Hypothesis Prediction Run More Runs
  • Scienti c Method Problem Report Code Hypothesis Prediction Experiment Run More Runs
  • Scienti c Method Problem Report Code Observation Hypothesis Prediction Experiment + Conclusion Run More Runs
  • Scienti c Method Problem Report Hypothesis is supported: re ne hypothesis Code Observation Hypothesis Prediction Experiment + Conclusion Run More Runs
  • Scienti c Method Problem Report Hypothesis is supported: re ne hypothesis Code Observation Hypothesis Prediction Experiment + Conclusion Run Hypothesis is rejected: More Runs create new hypothesis
  • Scienti c Method Problem Report Hypothesis is supported: re ne hypothesis Code Observation Hypothesis Prediction Experiment + Conclusion Run Hypothesis is rejected: More Runs create new hypothesis Diagnosis
  • An Explicit Process
  • An Explicit Process Hypothesis Prediction Experiment Observation Conclusion
  • An Explicit Process Hypothesis The execution causes a[0] = 0 Prediction Experiment Observation Conclusion
  • An Explicit Process Hypothesis The execution causes a[0] = 0 Prediction At Line 37, a[0] = 0 should hold. Experiment Observation Conclusion
  • An Explicit Process Hypothesis The execution causes a[0] = 0 Prediction At Line 37, a[0] = 0 should hold. Experiment Observe a[0] at Line 37. Observation Conclusion
  • An Explicit Process Hypothesis The execution causes a[0] = 0 Prediction At Line 37, a[0] = 0 should hold. Experiment Observe a[0] at Line 37. Observation a[0] = 0 holds as predicted. Conclusion
  • An Explicit Process Hypothesis The execution causes a[0] = 0 Prediction At Line 37, a[0] = 0 should hold. Experiment Observe a[0] at Line 37. Observation a[0] = 0 holds as predicted. Conclusion Hypothesis is confirmed.
  • Tracking Debugging Simplifying Debugging Automating Debugging Fixing Debugging