Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How do software engineers understand code changes?

Presented at FSE2012 http://www.sigsoft.org/fse20/

  • Be the first to comment

  • Be the first to like this

How do software engineers understand code changes?

  1. 1. How Do Software Engineers Understand Code Changes? An Exploratory Study in Industry Yida Tao (HKUST), Yingong Dang (MSRA), Tao Xie (NCSU) Dongmei Zhang (MSRA), Sunghun Kim (HKUST)
  2. 2. 2
  3. 3. > if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup() “Why this change here? This is the only one that doesn’t seem to make sense for me…” 3
  4. 4. > if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup() “Why this change here? This is the only one that doesn’t seem to make sense for me…” 4 >+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service; “What is this used for, I can’t spot it in use anywhere and every component and service seems to have it set to false.”
  5. 5. > if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup() “Why this change here? This is the only one that doesn’t seem to make sense for me…” 5 >+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service; “What is this used for, I can’t spot it in use anywhere and every component and service seems to have it set to false.” > browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html “These files are missing from this patch, aren’t they?”
  6. 6. > if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup() “Why this change here? This is the only one that doesn’t seem to make sense for me…” 6 >+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service; “What is this used for, I can’t spot it in use anywhere and every component and service seems to have it set to false.” > browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html “These files are missing from this patch, aren’t they?” >+ for (var i = aURL.length – 1; i >= 1; i--) { >+ var chPrev = aURL.charAt(i – 1) ; >+ var ch = aURL.charAt(i) ; “I’m not sure why you walk this char by char, javascript has awesome string methods… ”
  7. 7. > if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup() “Why this change here? This is the only one that doesn’t seem to make sense for me…” 7 >+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service; “What is this used for, I can’t spot it in use anywhere and every component and service seems to have it set to false.” > browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html “These files are missing from this patch, aren’t they?” >+ for (var i = aURL.length – 1; i >= 1; i--) { >+ var chPrev = aURL.charAt(i – 1) ; >+ var ch = aURL.charAt(i) ; “I’m not sure why you walk this char by char, javascript has awesome string methods… ”
  8. 8. > if (hudRef && hud) { > if (hudRef.consolePanel) { > + hudRef.consolePanel.hidePopup() “Why this change here? This is the only one that doesn’t seem to make sense for me…” 8 >+ struct CIDEntry >+ { >+ const nsCID* cid; >+ bool service; “What is this used for, I can’t spot it in use anywhere and every component and service seems to have it set to false.” > browser_hide_removing.js >+ browser_imageReload.js >+ image_Reload.html “These files are missing from this patch, aren’t they?” >+ for (var i = aURL.length – 1; i >= 1; i--) { >+ var chPrev = aURL.charAt(i – 1) ; >+ var ch = aURL.charAt(i) ; “I’m not sure why you walk this char by char, javascript has awesome string methods… ” “…” “…” “…”
  9. 9. “How do software engineers understand code changes?” 9
  10. 10. Research Questions RQ1: How frequent is code change understanding practiced and in which development tasks it is required? RQ2: What are engineers’ information needs and difficulty for understanding code changes? RQ3: How to improve the effectiveness and efficiency of the practices in understanding code changes? 10
  11. 11. Study Methodology Literature Review 11 •Potential information needs Questionnaire Design •Investigate RQ1, RQ2 Pilot Interview •Question is relevant & clear
  12. 12. Study Methodology Literature Review 12 •Potential information needs Questionnaire Design •Investigate RQ1, RQ2 Pilot Interview •Question is relevant & clear Online Survey • 16% response rate (180 respondents) Follow-up Interview •Investigate RQ3 Analysis •Answering RQs
  13. 13. Survey Participants 13 Role Distribution Product Team Dev 55% Test 31% PM 14%  OS  Desktop App  Web App  Mobile App  Service  Others
  14. 14. RQ1 • Frequency ? 14 • Development tasks ? RQ2 • Information needs ? • Difficulty ? RQ3 • Improvement ? RQs
  15. 15. RQ1: Frequency of Understanding Code Changes 15 How often do you need to understand code changes? o Several times each hour o About once an hour o Several times each day o About once a day o Several times each week o About once a week o Rarely o Never
  16. 16. RQ1: Frequency of Understanding Code Changes 16 50 45 40 35 30 25 20 15 10 5 0 Absolute # of responses Dev Test PM
  17. 17. RQ1: Tasks Requiring Code Change Understanding “Select the top three tasks that most often require you to understand code changes” 17 [Design/Planning] Refactoring [Implementation] Developing new feature [Implementation] Fixing bug [Integration] Resolving merge conflict [Verification] Reviewing others code changes [Verification] Reviewing my own code changes [Verification] Writing & updating test cases Other, please specify
  18. 18. RQ1: Tasks Requiring Code Change Understanding 18 0% 15% 30% 45% 60% 75% 121 100 89 73 48 34 30 Reviewing others' changes Fixing bug Developing new feature Reviewing my own changes Writing/updating test cases Refactoring Resolving merge conflict Percentage of participants who select the task
  19. 19. RQ1 • Frequently practiced 19 • Major development tasks RQ2 • Information needs ? • Difficulty ? RQ3 • Improvement ? Answers to RQs
  20. 20. Potential Information Needs Literature review (code-change analysis and management) 20 180 articles in 10 SE venues over the past decade
  21. 21. Potential Information Needs Literature review (code-change analysis and management) 21 180 articles in 10 SE venues over the past decade Reasoning & assessing the change • Completeness • Clones • Design • … Exploring the change’s context & impact •Risk •Consistency •Tests •... Evaluating the change history •Change-proneness •Defect-proneness
  22. 22. Survey Questions “Rate the importance & difficulty of each information need (formulated as question) in a change understanding task” Very Important Important Somewhat Important Not Important 22 3 2 1 0
  23. 23. Survey Questions “Rate the importance & difficulty of each information need (formulated as question) in a change understanding task” Very Important Important Somewhat Important Not Important 23 Very Difficult Difficult Relatively Easy Straightfor -ward 3 2 1 0
  24. 24. Survey Questions “Rate the importance & difficulty of each information need (formulated as question) in a change understanding task” 24 Does this change introduce code clones? Does this change break any code elsewhere? Which tests should be run to verify this change? Is this changed location a hotspot for past fixes? …… Very Important Important Somewhat Important Not Important Very Difficult Difficult Relatively Easy Straightfor -ward 3 2 1 0
  25. 25. RQ2: Information Needs 25 3 2 1 0 0 1 2 3 Difficulty of acquiring the information Importance
  26. 26. RQ2: Information Needs 26 3 2 1 0 Consistency Risk 0 1 2 3 Difficulty of acquiring the information Importance Completeness Design
  27. 27. RQ2: Information Needs 27 3 2 1 0 Consistency Risk 0 1 2 3 Difficulty of acquiring the information Importance Completeness Design Rationale
  28. 28. RQ2: Information Needs 28 3 2 1 0 Consistency Risk Defect-proneness 0 1 2 3 Difficulty of acquiring the information Importance Completeness Design Rationale Change-proneness
  29. 29. RQ1 • Frequently practiced 29 • Major development tasks RQ2 • Risk & Quality are important but difficult to know RQ3 • Improvement ? Answers to RQs
  30. 30. RQ3: Interview Items 30 3 2 1 0 Defect-proneness 0 1 2 3 Difficulty of acquiring the information Importance Risk Change-proneness Rationale
  31. 31. Determining a Change’s Risk 31 3 2 1 0 0 1 2 3 Difficulty of acquiring the information Importance Risk
  32. 32. Current Practice on Determining a Change’s Risk 32 Manual Code Review •Error-prone •Cross-components •Unclear interface •Hidden assumptions •… Unit & Regression Testing •Time consuming •Depends on how thorough the tests are •…
  33. 33. Support Determining a Change’s Risk Manual code review 33 •Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change
  34. 34. Support Determining a Change’s Risk Manual code review 34 •Navigation in diff: using code analysis tools (e.g., go to definition, find all references, caller/callee tree) on the code change Diff “…miss a level of understanding object Code Analysis relationships” Navigation in diff
  35. 35. Support Determining a Change’s Risk Testing 35 •which code must be retested as it is dependent upon the change? •who owns testing that dependency? •which tests must be run? “An ‘Intelli-sense’ for updating these (affected) tests would be nice as well.”
  36. 36. Discussion 36 3 2 1 0 Defect-proneness 0 1 2 3 Difficulty of acquiring the information Importance Rationale Change-proneness
  37. 37. Discussion 37 Why is understanding the rationale of a change easy? • Availability & Quality of commit message • “It’s entirely up to the dev making the change as to how hard or easy it is for someone else to figure out why the change was made.” Why are historical metrics not that important? • Developers • Here and now • Short-term issue • Own Knowledge • Testers & PMs • Historical metrics might be good to reflect bugginess and complexity of a specific area
  38. 38. Other Information Needs “In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?” 38
  39. 39. Other Information Needs “In addition to the information needs listed above, what else would you ask when you try to understand a code change? How difficult is it for you to answer?” 39 “Can this change be broken into smaller discreet changes?”
  40. 40. Composite Code Change 40
  41. 41. Understanding a Composite Code Change 41
  42. 42. Decomposing a Composite Code Change 42
  43. 43. RQ1 • Frequently practiced 43 • Major development tasks RQ2 • Risk & Quality are important but difficult to know RQ3 • Determining a change’s risk • Decomposing a composite change Answers to RQs
  44. 44. Summary Evidence Understanding code changes is a fundamental practice that happens frequently in major development tasks 44
  45. 45. Summary Evidence Understanding code changes is a fundamental practice that happens frequently in major development tasks Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change 45
  46. 46. Summary Evidence Understanding code changes is a fundamental practice that happens frequently in major development tasks Challenges Determining a change’s risk Assessing a changes’ consistency, completeness Understanding composite change Practical Needs Navigation in diff Change decomposition Available & informative commit message 46
  47. 47. Acknowledgment All participants of survey / interview Miryung Kim, Robin Moeur, Thomas Zimmermann, Jacek Czerwonka, and Kathryn McKinley 47

    Be the first to comment

    Login to see the comments

Presented at FSE2012 http://www.sigsoft.org/fse20/

Views

Total views

738

On Slideshare

0

From embeds

0

Number of embeds

3

Actions

Downloads

8

Shares

0

Comments

0

Likes

0

×