Debugging is an inevitable activity in most software projects, often difficult and more time-consuming than expected, giving it the nickname the “dirty little secret of computer science.” Surprisingly, we have little knowledge on how software engineers debug software problems in the real world, whether they use dedicated debugging tools, and how knowledgeable they are about debugging. This study aims to shed light on these aspects by following a mixed-methods research approach. We conduct an online survey capturing how 176 developers reflect on debugging. We augment this subjective survey data with objective observations on how 458 developers use the debugger included in their integrated development environments (IDEs) by instrumenting the popular ECLIPSE and INTELLIJ IDEs with the purpose-built plugin WATCHDOG 2.0. To clarify the insights and discrepancies observed in the previous steps, we followed up by conducting interviews with debugging experts and regular debugging users. Our results indicate that IDE-provided debuggers are not used as often as expected, as “printf debugging” remains a feasible choice for many programmers. Furthermore, both knowledge and use of advanced debugging features are low. These results call to strengthen hands-on debugging experience in computer science curricula and have already refined the implementation of modern IDE debuggers.

1. What do these nerds have in common?
Linus Torvalds does
not use debuggers.
Thanks, Daniel Lemire! https://lemire.me/blog/2016/06/21/i-do-not-use-a-debugger/
Robert C. Martin
thinks that
debuggers are a
wasteful timesink.
Guido van Rossum uses
printf for 90% of his
debugging.
They all dislike debugging.

2. On the Dichotomy of Debugging
Behavior Among Programmers
Moritz Beller, Niels Spruit, Andy Zaidman Diomidis Spinellis

3. What do we know about debugging?

4. “We have anecdotes from people using our debugger here and there or
looking over their shoulders, but nothing substantial.”
Eclipse Debug Lead
“
What do we know about debugging?

5. Research Questions
RQ1 What do developers know about debugging and how do
they reflect on it?
RQ2 How do developers debug in their IDEs?
RQ3 How do individual debugger users and experts interpret our
findings from RQ1 and RQ2?

6. Study Design
176 survey respondents, ⅓
Eclipse and ⅓ IntelliJ. 84% Java
programmers.
458 WatchDog participants, ⅓ Eclipse and ⅓ IntelliJ.
100% Java programmers. Over 2 months, recorded 10.3
IDE work years. Heterogeneous mix of private,
open-source, and commercial projects.
7 interviews with 9 experts in the
field including developers of IDE
debuggers.

7. RQ1 What do developers know about
debugging?

8. RQ1 What do developers know about
debugging?
of developers use the IDE debugger.
Only 8.5% explicitly do not.81%
of developers examine log files and print statements.
72%
Testing is integral part of the debugging process.
Experience has no impact on self-reported debugging knowledge.

9. RQ1.13 “Printf debugging is still the best
invention in debugging.”
People saying that printfs are better never learned how to use a
debugger.“
100% Agree.
“
printf is travelling by foot, a GUI debugger is travelling [by] plane. You can go to more
places by foot, but you can only go that far.“

10. RQ1.13 “Printf debugging is still the best
invention in debugging.”

11. RQ2 How do developers debug in their
IDEs?
Measured with 2.0
www.testroots.org

13. RQ2 How do developers debug in their
IDEs?
In 9% of IDE sessions, developers run a debugger.
On average, debugging consumes < 14% of in-IDE time.
88% of debugging sessions are < 10 minutes.

14. RQ2 How do developers debug in their
IDEs?
of debugging sessions start after reading (52%) or modifying (19%)
code. Only 0.5% after running a test.71%
Smaller classes are debugged more than larger classes. There is
no correlation between the size of the class and its debugging
duration.
Developers who spend more time executing tests are likely to
debug more. Developers who read or modify tests longer do not
debug less (ρ = -0.08).
ρ =
0.58
ρ =
-0.75

15. RQ2 How do developers debug in their
IDEs?
Breakpoint type #
Method 4,229
Class prepare 99
4 Others 216
Σ 4,544

16. RQ2 How do developers debug in their
IDEs?
‘Advanced’ debugging features are seldom used.
Event type #
Add breakpoint 4,544
Remove breakpoint 4,362
Define watch 343
Modify variable value 4
Hit count 0
Step over 19,543
Step into 3,480
Step out 351
5 others 543
Σ 33,170

17. Consolidation with experts
Finding RQ1 RQ2 RQ3
6. Developers are divided on printf debugging. RQ1 ~ ~
3. Advanced debugging features are seldom used. RQ1 RQ2 ✔
4. Debuggers are difficult to use. RQ1 ✔ ✔
5. There is a lack of knowledge on how to use them. RQ1 ✔ ✔
1. Debugging is a fast-paced act. ✘ RQ2 ✔
2. Debugging and testing are tightly intertwined. RQ1 ✘ ~

18. What can we do?
As educators, integrate debugging into courses. Teach debugging hands-on,
not only abstractly.
As debugger developers, care about usability first and number of features later.
- Back-in-time debuggers
- Live REPL
- Domain-specific debuggers
- Many small improvements can lead to a big improvement

19. What did we do?

20. “Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it.”
Brian Kernighan
“