Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Impact of Tool Support in Patch Construction

62 views

Published on

Anil Koyuncu, Tegawendé F. Bissyandé, Dongsun Kim, Jacques Klein, Martin Monperrus, and Yves Le Traon, “Impact of Tool Support in Patch Construction,” in Proceedings of the 26th International Symposium on Software Testing and Analysis (ISSTA 2017), Santa Barbara, California, United States, July 10-14, 2017.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Impact of Tool Support in Patch Construction

  1. 1. Impact of Tool Support in Patch Construction Anil Koyuncu, Tegawendé F. Bissyandé, Dongsun Kim, Jacques Klein, Martin Monperrus and Yves Le Traon 1
  2. 2. Motivation 2
  3. 3. Traditional Patch Construction 3 Detection Localization Generation
  4. 4. Thanks to… 4 Test Automation Automated Bug/Fault Localization Program Repair Manual Patching Fully Automated Patching Static/DynamicAnalysis
  5. 5. Really? 5 They use? Or don’t use? Adopted by whom? Patch à Accepted? Stable?
  6. 6. 1991 2017 … Long Life Software PProblem Solution Single problem per patch 7 ~ 15 millions LOC Rich code repository Subject --- Linux Kernel Project
  7. 7. Subject --- Data Sources 8 Bug Reports Change History Developer Discussion
  8. 8. Patch Construction Processes + + Process H (Human) Process DLH (Detection Localization Human) Process HMG (Human Match Generation) 10 •Fully Manual •Automated Localization (static/dynamic analysis) •Manual Patch Generation •Manual Design of PatchTemplate •Automatic Application
  9. 9. H patches • Identification based on direct link to Bugzilla IDs 11
  10. 10. DLH Patches • Detection based on <tool> names + 12
  11. 11. DLH Patches – Tools 13 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Author Dates 20 40 60 80 100#ofPatches checkpatch Sparse Linux Driver Verification Project Smatch Coverity cppcheck Strace Syzkaller
  12. 12. HMG Patches + • mentioning “coccinelle” or “semantic patch” 14
  13. 13. HMG Patches – Coccinelle SmPL (Semantic Patch Language) for specifying desired matches and transformations in C code. 15 Patch derived from the SmPLSmPL
  14. 14. H Patches DLH Patches HMG Patches Linux 2.6.12 (June 2005) -Linux 4.8 (October 2016) 616,291 commits 5758 commits 729 commits 4050 commits Dataset 16
  15. 15. Temporal Distribution 17 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Author Dates 500 1000 #ofPatches H patches DLH Patches HMG patches
  16. 16. arch drivers fs include kernel net sound staging others kernel subsystem (code directory) 0 10 20 30 40 50 60 70 80 90 100 Percentages H patches DLH patches HMG patches Spatial Distribution 18
  17. 17. RQ2 - Who is using? Research Questions RQ1 - Community Reaction RQ3 – Impact on Stability RQ4 - Kind of Bugs 19
  18. 18. RQ1 - Community Reaction • Delays in integrating commits • Gaps between proposed and integrated patches 20 RQ1 - Community Reaction
  19. 19. RQ1 - Commit Acceptance Delay • Finding: Integration ofTool-supported patches are slower than traditional H patches. 21 Submission date acceptance date
  20. 20. RQ1- LKML mentioning HMG (Coccinelle) • Finding:The gap is closing, patches appear to be accepted. 0 500 1000 1500 2000 2500 3000 2008 2009 2010 2011 2012 2013 2014 2015 2016 # of reference or commits Patch Submitted in LKML Accepted Commit Developer / Maintainer Reply 22 nt. Per- of DLH are ap- shown entied ed in a to the hat end, he gaps munity ge sug- n need ainline he criti- ggested correlate this frequency on a monthly basis with the corresponding statistics on accepted DLH patches related to the specic tools. 0 100 200 300 400 500 600 #ofreferenceorcommits Patch Submitted in LKML Accepted Commit Developer / Maintainer Reply (a) Data on checkpatch-related (DLH) patches. 0.5 0.6 0.7 0.8 0.9 1 1.1 0 20 40 60 80 % timeline Gap Slope (Linear regression) (b) Evolution of the Gap. 0 50 100 150 200 250 300 350 400 #ofreferenceorcommits Patch Submitted in LKML Accepted Commit Developer / Maintainer Reply (c) Data on coccinelle-related (HMG) patches. -6 -5 -4 -3 -2 -1 0 1 0 20 40 60 80 % timeline Gap Slope (linear regression) (d) Evolution of the Gap. Figure 7: # of Patches submitted / discussed / accepted. We have crawled all emails archived in the Linux Kernel Mailing List (LKML) using Scrapy14. We use heuristics to dierentiate mes-
  21. 21. RQ2 - Profile of Patch Authors • Specialty • Commitment 23
  22. 22. RQ2 - Specialty 24 • Finding: HMG Patches are often generated by less specialized developers. Speciality is dened as a metric for characterizing the extent to which a developer is focused on a specic subsystem. We compute it as the percentage of patches, among all her/his patches, which a developer contributes to a specic subsystem. Thus, speciality is measured with respect to each Linux code directory. We then draw, in Figure 8, the distributions of speciality metric values of developers for the dierent types of patches: e.g., for an automated patch applied to a le in a subsystem, we consider the commit author speciality w.r.t that subsystem. % of Speciality H Patches DLH Patches HMG Patches Figure 8: Speciality of developers Vs. Patch types. H patches are mostly provided by specialized developers. This may imply that the developers focus on implementing specic func- tionalities over time. Similarly, DLH patches appear to be mostly Focus on Specific modules Contribute to all modules
  23. 23. RQ2 - Commitment of developers • Finding: Patch application tools (HMG) enable developers to remain committed to the code base. 25 # days between first patch and last patch #patches integrated into Linux Commitment
  24. 24. to roll back changes, it is common to revert commit e.g., Commit message: revert hash 26 RQ3 - Stability of Patches
  25. 25. 0 50 100 % of patches reverted H patches DLH patches HMG patches Ledleagueinwins 2.81 0.27 0.32 RQ3 - Stability of Patches 27 • Finding: Tool-supported patches are generally stable.
  26. 26. RQ3 - Stability of Patches • Finding: Issues on fix patterns appear to be discovered quickly, however bug detection tools need long time. 28
  27. 27. RQ4 - Kind of Bugs • Spread of buggy code ~ Locality of the patches • Complexity of the bugs ~ Change operations 29
  28. 28. RQ4 - Bug Locality 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% H pathes DLH Patches HMG Patches % of patches 1 file 2 files 3 files 4 files 5+ files 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% H pathes DLH Patches HMG Patches % of patches 1 hunk 2 hunks 3 hunks 4 hunks 5+ hunks 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% H pathes DLH Patches HMG Patches % of patches 1 line 2 lines 3 lines 4 lines 5+ lines More Local Several Hunks Several Lines 30
  29. 29. Change operations - Gumtree If/Unary:del Ident/GenericString:mv If/FunCall:add Ident/GenericString:mv 31 • AST DiffTool to identify change operations
  30. 30. RQ4 - Change operations in patches • Finding: Several change operations 0 10 20 30 40 50 60 70 Ident/GenericString GenericList/Left Compound/ExprStatement Program/Declaration Compound/If % of patches HMG Patches upd mv add del 0 10 20 30 40 50 60 70 80 Program/CppTop Ident/GenericString Program/Declaration Compound/ExprStatement Compound/If % of patches H Patches upd mv add del 0 5 10 15 20 25 30 35 40 45 Ident/GenericString Compound/ExprStatement Left/Constant Compound/If If/Compound % of patches DLH Patches upd mv add del 32
  31. 31. Take-aways 33 (1) Tools are gradually adopted. (2) HMG patches leverage micro-clones. (1) DLH HMG patches need more time to be accepted. Perhaps due to less-severity. (2) HMG patch acceptance has been fast.
  32. 32. Take-aways 34 (1) DLH HMG patches can also change several lines. (2) HMG patches change several files due to APIs. (1) More opportunities à HMG patches leverage redundancy. (2) Need to target more complex defects.
  33. 33. Really? 6 They use? Or don’t use? Adopted by whom? Patch à Accepted? Stable? Subject --- Data Sources 10 Bug Reports Change History Developer Discussion Patch Types + + Process H Process DLH Process HMG 13 •Fully Manual •Automated Localization (static/dynamic analysis) •Manual Patch Generation •Manual Design of Patch Template •Automatic Application RQ2 - Commitment of developers • Finding: Patch application tools enable developers to remain committed to contributing patches to the code base. 31 = # patches integrated into Linux * # days between first patch and last patch

×