Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Bug Triagewith Bug Tossing GraphsGaeul Jeong, Sunghun Kim and Thomas Zimmermann                               August 26, 2...
Bug triageBu                                                    Bu     g                                                  ...
Problem    More than                More than300 reports per day       1,000 developers       Bu            g             ...
ConsequencesMozilla                      297,999 bugs                      116,890 bugs                                   ...
Bug tossing                Tossing (reassign)Bu                                             Bu     g                      ...
Consequences                       %!!                       $"!!"#$%&#(%)*+,-%.#                       $!!               ...
Two Challenges• Finding appropriate developer is hard• Wrong assignment(bug tossing) make the  fixing process slow down    ...
Our solution• Learning from previous tossing history• To help   • Assist manual bug triage   • Reducing tossing paths   • ...
Bug tossing path• Extracting bug tossing information• Identify more appropriate developers using  tossing relationship    ...
Learning from tossing paths                                     Path tokens                                  A→D (2), A→E(...
Learning from tossing paths                           Path tokens Generated graph                        A→D (2), A→E(1)  ...
Learning from tossing paths                           Path tokens Generated graph                        A→D (2), A→E(1)  ...
Learning from tossing paths                           Path tokens Generated graph                        A→D (2), A→E(1)  ...
Learning from tossing paths                                   Path tokens   Generated graph                               ...
Tossing graph from Eclipse            12
Manual process with cliques             13
Developer structure         14
Developer feedback• “Very neat stuff! The clustering was correct  for the team”• “This would be useful for both integrator...
Path reduction• Reducing tossing paths by graph search • Original path       A        B         C       D • Recommended pa...
Greedy search              for optimal path   • Start from a given node (A)   • Visit heavy neighboring nodes first   F    ...
Reduced paths                             $!"#$%&#$(()**%+,((-#+,.                             #                          ...
Our solution• Learning from previous tossing history• To help   • Assist manual bug triage   • Reducing tossing paths   • ...
Automatic bug triage                 Bu               Re                 po   g                   rt               Machine...
Automatic bug triage                         Bu                       Re                         po   g                   ...
Automatic bug triage                         Bu                       Re                         po   g                   ...
Experiment• Integrating bug tossing information • existing approaches          P1    P2                   P3      P = [ 1P...
Experiment• Integrating bug tossing information •  existing approaches ....... First 3 : [p1, p2, p3 ]        P = [ 1Pp2 ,...
Eclipse triage resultFirst 2First 3                                                          Naïve + Tossing              ...
Eclipse triage resultFirst 2First 3                                                          Naïve + Tossing              ...
Mozilla triage resultFirst 2First 3                                                       Naïve + Tossing                 ...
Mozilla triage resultFirst 2First 3                                                       Naïve + Tossing                 ...
Summary• Tossing graphs help to • Assist manual bug triage • Reduce tossing paths • Assign bug automatically              ...
Thank youGaeul Jeong, Sunghun Kim and Thomas Zimmermann       Joint meeting of the European Software Engineering Conferenc...
Experiment•   They use the first 165,385 Eclipse bugs for    training•   The machine learners predict developers for    bug...
ML+program   ML algorithm   selection        ML only                 improvement                                          ...
Upcoming SlideShare
Loading in …5
×

BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)

1,795 views

Published on

Gaeul presented it at ESEC/FSE 2009

Published in: Technology, News & Politics
  • Be the first to comment

BugTriage with Bug Tossing Graphs (ESEC/FSE 2009)

  1. 1. Bug Triagewith Bug Tossing GraphsGaeul Jeong, Sunghun Kim and Thomas Zimmermann August 26, 2009 Joint meeting of the European Software Engineering Conference (ESEC) and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE), Armsterdam, The Netherlands
  2. 2. Bug triageBu Bu g g X FixedNew Assigned Resolved The life cycle of a bug report (A. Zeller) 2
  3. 3. Problem More than More than300 reports per day 1,000 developers Bu g ? Bu g X Fixed New Assigned Resolved 3
  4. 4. ConsequencesMozilla 297,999 bugs 116,890 bugs First First Reported Assignment Action 26.1 days 161.1 days (a) All Bugs pse 131,958 bugs 49,431 bugsEcli Reported 144,102 bugs First Action 88,706 bugs First Assignment 7.1 days First 38.1 days First Reported Assignment Action 16.7 days 23.6 days (b) Verified Bugs (a) All Bugs 18,498 bugs 15,045 bugs First First Reported Assignment Action 5.2 days 4 19.3 days
  5. 5. Bug tossing Tossing (reassign)Bu Bu g g X FixedNew Assigned Resolved The life cycle of a bug report 5
  6. 6. Consequences %!! $"!!"#$%&#(%)*+,-%.# $!! #"! +,-.//0 #!! 12/.345 "! ! # $ % & " ( ) * #! -,**/0&1+#2* 6
  7. 7. Two Challenges• Finding appropriate developer is hard• Wrong assignment(bug tossing) make the fixing process slow down 7
  8. 8. Our solution• Learning from previous tossing history• To help • Assist manual bug triage • Reducing tossing paths • Assign bug automatically 8
  9. 9. Bug tossing path• Extracting bug tossing information• Identify more appropriate developers using tossing relationship 9
  10. 10. Learning from tossing paths Path tokens A→D (2), A→E(1)Extracted path information B→D(1)A B C DA C D E C→D(2), C→E(1)C E A F D D→E(1) E→D(1) F→D(1) 10
  11. 11. Learning from tossing paths Path tokens Generated graph A→D (2), A→E(1) B→D(1) C→D(2), C→E(1) D→E(1) E→D(1) F→D(1) 11
  12. 12. Learning from tossing paths Path tokens Generated graph A→D (2), A→E(1) A B→D(1) C→D(2), C→E(1) D→E(1) E→D(1) F→D(1) 11
  13. 13. Learning from tossing paths Path tokens Generated graph A→D (2), A→E(1) A B→D(1) 67% C→D(2), C→E(1) D D→E(1) E→D(1) F→D(1) 11
  14. 14. Learning from tossing paths Path tokens Generated graph A→D (2), A→E(1) F A B→D(1)100% 67% 33% C→D(2), C→E(1) D E 100%100% D→E(1) 67% 33% C E→D(1) B F→D(1) 11
  15. 15. Tossing graph from Eclipse 12
  16. 16. Manual process with cliques 13
  17. 17. Developer structure 14
  18. 18. Developer feedback• “Very neat stuff! The clustering was correct for the team”• “This would be useful for both integrators and managers wanting to understand the life cycle of bugs” 15
  19. 19. Path reduction• Reducing tossing paths by graph search • Original path A B C D • Recommended path? A ? Search from A 16
  20. 20. Greedy search for optimal path • Start from a given node (A) • Visit heavy neighboring nodes first F Original : A B C D A100% 67% 33% Our path : A D D E 100% 17
  21. 21. Reduced paths $!"#$%&#$(()**%+,((-#+,. # +,-.//0 " 12/.345 ! " # $ % & ( ) !* !! !" /"%,%+0-(()**%+,(-#+,. 18
  22. 22. Our solution• Learning from previous tossing history• To help • Assist manual bug triage • Reducing tossing paths • Assign bug automatically 19
  23. 23. Automatic bug triage Bu Re po g rt Machine Machine learner Who should fix this bug? (J. Anvik) 20
  24. 24. Automatic bug triage Bu Re po g rt New! Bu Re po g rt Machine Machine learner Who should fix this bug? (J. Anvik) 20
  25. 25. Automatic bug triage Bu Re po g rt New! Bu Re po g rt Predict Machine Machine learner developers Who should fix this bug? (J. Anvik) 20
  26. 26. Experiment• Integrating bug tossing information • existing approaches P1 P2 P3 P = [ 1Pp2 , ..., 1 ,np]2 , ..., pn } {p , = {p p } • our approach T1 T2 T3 = = , t t 2 t t2 , ..., t } RPRP [{p1{p1 , p1 , p2 , ..., pn ,pn , ]tn } 21
  27. 27. Experiment• Integrating bug tossing information • existing approaches ....... First 3 : [p1, p2, p3 ] P = [ 1Pp2 , ..., 1 ,np]2 , ..., pn } {p , = {p p } • our approach .................. First 3 : [p1, t1, p2] = = , t t 2 t t2 , ..., t } RPRP [{p1{p1 , p1 , p2 , ..., pn ,pn , ]tn } 22
  28. 28. Eclipse triage resultFirst 2First 3 Naïve + Tossing NaïveFirst 4First 5 37.500 45.625 53.750 61.875 70.000 (%) 23
  29. 29. Eclipse triage resultFirst 2First 3 Naïve + Tossing NaïveFirst 4First 5 37.500 45.625 53.750 61.875 70.000 (%) 24
  30. 30. Mozilla triage resultFirst 2First 3 Naïve + Tossing NaïveFirst 4First 5 25.00 38.75 52.50 66.25 80.00 (%) 25
  31. 31. Mozilla triage resultFirst 2First 3 Naïve + Tossing NaïveFirst 4First 5 25.00 38.75 52.50 66.25 80.00 (%) 26
  32. 32. Summary• Tossing graphs help to • Assist manual bug triage • Reduce tossing paths • Assign bug automatically 27
  33. 33. Thank youGaeul Jeong, Sunghun Kim and Thomas Zimmermann Joint meeting of the European Software Engineering Conference (ESEC)and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE), Armsterdam, The Netherlands
  34. 34. Experiment• They use the first 165,385 Eclipse bugs for training• The machine learners predict developers for bugs in a testing set, Eclipse bug from 165,397 to 211,822 0 165,397 211,822 Training set Testing set 29
  35. 35. ML+program ML algorithm selection ML only improvement tossing graph first2 43.7 44.71 1.01 first3 49.87 53.15 3.27 Naive Bayes first4 56.42 59.95 3.53 first5 60.71 63.48 2.77Eclipse first2 57.91 58.29 0.38 Bayesian first3 66.71 68.47 1.76 Network first4 69.47 71.48 2.01 first5 75.88 77.14 1.26 first2 33.41 56.39 15.84 first3 45.39 63.82 12.81 Naive Bayes first4 52.94 69.51 12.05 first5 59.35 72.92 11.29Mozilla first2 40.02 55.85 22.98 Bayesian first3 50.25 63.05 18.43 Network first4 55.4 67.45 16.57 first5 59.53 70.82 13.58 30

×