Successfully reported this slideshow.

CHASE 2012 - An Exploratory Study of Backtracking Strategies Used by Developers



1 of 23
1 of 23

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

CHASE 2012 - An Exploratory Study of Backtracking Strategies Used by Developers

  1. 1. An Exploratory Study of Backtracking Strategies Used by Developers YoungSeok Yoon ( Institute for Software Research (ISR) Brad A. Myers ( Human-Computer Interaction Institute (HCII) CHASE 2012 School of Computer Science June 2nd, 2012. Carnegie Mellon University
  2. 2. What is Backtracking? • When developers write source code, they often need to step back when something goes wrong • We define backtracking as: going back to an earlier state of at least some part of the code, either by removing inserted code or by restoring removed code CHASE 2012 2
  3. 3. Examples of Backtracking Correcting minor mistakes Tuning parameters Trying out methods while learning an unfamiliar API Experimenting with different classes (e.g. UI layout classes) CHASE 2012 3
  4. 4. Existing Support for Backtracking CHASE 2012 4
  5. 5. Existing Support for Backtracking Conventional Undo Command • Only works for the most recent changes • Have to undo all changes in order • Undone changes are thrown away after making a new edit CHASE 2012 5
  6. 6. Existing Support for Backtracking Version Control • The desired version needs to be committed in advance • Unstable code should not be committed to the repository • Inappropriate when wanted and unwanted code is mixed CHASE 2012 6
  7. 7. Existing Support for Backtracking Manual Edit / Commenting Out • Have to manually find all the relevant code • Redo the work by hand CHASE 2012 7
  8. 8. Research Question As a first step towards supporting more robust backtracking tools… How can we better understand developers' backtracking behaviors? CHASE 2012 8
  9. 9. STUDY #1. LAB STUDY CHASE 2012 9
  10. 10. Study Settings • 12 participants, all MS/PhD students at CMU • 2 hrs, THINK ALOUD • Code base: Paint program [Ko 2005][Fogarty 2005] CHASE 2012 10
  11. 11. Tasks • Two sets of features each done in two different ways, ▪ intended to lead participants to backtrack Feature 1 (F1) Feature 2 (F2) “Thickness Control” “X, Y coordinates indicator” F1-1 F2-1 F1-2 F2-2 CHASE 2012 11
  12. 12. Commenting Out • 7 out of 12 participants preferred to comment out rather than to delete code • 3 main reasons for commenting out Reuse Good Bad later example example e.g. activating one leave the code as a remind themselves alternative at a time good example that the code was not good CHASE 2012 12
  13. 13. Problems while Backtracking • Difficulties in finding relevant code fragments that have to be reverted when: ▪ there are two or more similar looking code fragments ▪ relevant code fragments are scattered across multiple locations • Forgetting to remove temporary code ▪ e.g., Debug output statements such as: • System.out.println • JOptionPane.showMessageDialog CHASE 2012 13
  14. 14. Backtracking as Restoring Code • 2 participants accidentally deleted some code • One participant failed to restore the code from memory after spending about 10 mins. • The other succeeded using the undo command because he knew: ▪ what the code looked like ▪ that it was deleted very recently CHASE 2012 14
  15. 15. Backtracking as Restoring Code • When trying to restore deleted code, participants often remembered some features about the desired code ▪ the original location of the code ▪ the surrounding code ▪ the names of some code elements (variable, method, …) ▪ what the code looked like • Participants also reproduced the same code fragments from memory repeatedly ▪ e.g. complex formatted string looking like “(X, Y) = ” + x + “, ” + y + “)” CHASE 2012 15
  16. 16. STUDY #2. ONLINE SURVEY CHASE 2012 16
  17. 17. Demographics Female • 48 professional 8% programmers Male • 13 yrs exp. on average 92% • recruited from online developer communities CHASE 2012 17
  18. 18. Flexibility of their work Highly specified before I The results that my code start developing achieves The architecture of the code Specified, but some itself opportunity to negotiate changes Which APIs/libraries are used Somewhat flexible, within broad constraints Which elements of the API are used Highly flexible The details of the implementation code Completely unspecified; I can do whatever I want 0% 20% 40% 60% 80% 100% CHASE 2012 18
  19. 19. How often do they have to backtrack? Tuning parameters All the time Figuring out how to use an API correctly Frequently Fixing code just added, because it is not working Sometimes Trying out various user interface designs Rarely Trying to find an appropriate algorithm Never 0% 20% 40% 60% 80% 100% CHASE 2012 19
  20. 20. Backtracking Strategies • For each backtracking situation, only a few strategies were primarily used ▪ e.g. “select & overtype” when tuning parameters ▪ This hints at how to detect a certain backtracking situation • Other strategies in the open-ended responses ▪ using Boolean variables to temporarily turn on/off code ▪ writing a small code snippet (outside the main project) ▪ moving parameters out of the code ▪ writing unit tests to see how the API works CHASE 2012 20
  21. 21. Future Work • Conducting an extended version of this study ▪ A field study is ongoing, using our Eclipse logging tool called FLUORITE [Yoon 2011] ▪ If you are interested in participating, please let me know! • Designing new backtracking tools, based on the needs identified in these studies CHASE 2012 21
  22. 22. Conclusion • Backtracking happens a lot while coding • The backtracking strategies used by developers are still manual and error-prone • Better backtracking tools would help developers write code more correctly and effectively CHASE 2012 22
  23. 23. Questions? Contact Info: YoungSeok Yoon E-mail: Web: FLUORITE logging tool can be found at: Thanks for funding from: CHASE 2012 23