Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software Visualization - Promises & Perils

1,430 views

Published on

An invited talk I gave at WBVS 2011(The 1st Brazilian Workshop on Software Visualization), in Sao Paulo, Brazil on Sep 27 2011

Published in: Technology, Education
  • Be the first to comment

Software Visualization - Promises & Perils

  1. 1. Software VisualizationPromises & Perils Michele Lanza REVEAL @ Faculty of Informatics University of Lugano, Switzerland
  2. 2. The use of the crafts of typography, graphic design, animation, and cinematography with modern human- computer interaction and computergraphics technology to facilitate both thehuman understanding and effective use of computer software. John Stasko et al. Software Visualization Programming as a Multimedia Experience MIT University Press, 1998
  3. 3. algorithm visualizationsoftware visualization program visualization
  4. 4. the visualization of t h e h i g h e r- l e v e l algorithm visualization abstractions which describe softwaresoftware visualization program visualization
  5. 5. the visualization of t h e h i g h e r- l e v e l algorithm visualization abstractions which describe softwaresoftware visualization the visualization of the actual program program visualization code or data structures in static or dynamic form
  6. 6. program visualization
  7. 7. static code/data visualizationprogram visualization dynamic code/data visualization
  8. 8. what issoftware? Soft...
  9. 9. Software: Theintangible objects inside the computer. where?
  10. 10. Programming is a kind of writing. Gerald M. Weinberg Psychology of Computer Programming Dorset House, 1971
  11. 11. programming ~ writing
  12. 12. programming ~ writingsoftware ~ text
  13. 13. Deciduous shrub, glabrous or nearly so, withweak, trailing sub-glaucous, often purple-tinted stems, either decumbent and forminglow bushes 50 - 100 cm high, or climbingover other shrubs, rarely more erectand reaching 2 m. Prickles hooked,all +- equal. L’flets 2-3 pairs, 1-3.5cm, ovate or ovate-elliptic, simply,rarely double serrate, glabrous onboth sides or pubescent on theveins (rarely all over) beneath,rather thin; petiole usuallywith some stalked glands;stipules narrow, auriclesstraight. Flowers 1-6, white3-5 cm diam.; pedicels 2-4 cmwith stalked glands, rarelysmooth; buds short... etc. etc.
  14. 14. Deciduous shrub, glabrous or nearly so, withweak, trailing sub-glaucous, often purple-tinted stems, either decumbent and forminglow bushes 50 - 100 cm high, or climbingover other shrubs, rarely more erectand reaching 2 m. Prickles hooked,all +- equal. L’flets 2-3 pairs, 1-3.5cm, ovate or ovate-elliptic, simply,rarely double serrate, glabrous onboth sides or pubescent on theveins (rarely all over) beneath,rather thin; petiole usuallywith some stalked glands;stipules narrow, auriclesstraight. Flowers 1-6, white3-5 cm diam.; pedicels 2-4 cmwith stalked glands, rarelysmooth; buds short... etc. etc.
  15. 15. It would be a great mistake to suppose that the dominant role of visualobservation has now been overthrown by more ‘analytical’ methods. John Ziman Reliable Knowledge An Exploration of the Grounds for Belief in Science Cambridge University Press, 1978
  16. 16. old habits die hard
  17. 17. /***************************************************************************/ if(p<3&y==E)H=y^16; /* shift capt.sqr. H if e.p.*//* micro-Max, */ t=b[H];if(t&k|p<3&!(r&7)!=!t)break; /* capt. own, bad pawn mode *//* A chess program smaller than 2KB (of non-blank source), by H.G. Muller */ i=99*w[t&7]; /* value of capt. piece t *//***************************************************************************/ if(i<0||E-S&&b[E]&&y-E<2&E-y<2)m=I; /* K capt. or bad castling *//* version 3.2 (2000 characters) features: */ if(m>=l)goto C; /* abort on fail high *//* - recursive negamax search */ if(h=d-(y!=z)) /* remaining depth(-recapt.)*//* - quiescence search with recaptures */ {v=p<6?b[x+8]-b[y+8]:0; /* center positional pts. *//* - recapture extensions */ b[G]=b[H]=b[x]=0;b[y]=u&31; /* do move, strip virgin-bit*//* - (internal) iterative deepening */ if(!(G&M)){b[F]=k+6;v+=30;} /* castling: put R & score *//* - best-move-first sorting */ if(p<3) /* pawns: *//* - a hash table storing score and best move */ {v-=9*(((x-2)&M||b[x-2]!=u)+ /* structure, undefended *//* - full FIDE rules (expt minor ptomotion) and move-legality checking */ ((x+2)&M||b[x+2]!=u)-1); /* squares plus bias */ if(y+r+1&S){b[y]|=7;i+=C;} /* promote p to Q, add score*/#define F(I,S,N) for(I=S;I<N;I++) }#define W(A) while(A) v=-D(24-k,-l-(l>e),m>q?-m:-q,-e-v-i, /* recursive eval. of reply */#define K(A,B) *(int*)(T+A+(B&8)+S*(B&7)) J+J(0),Z+J(8)+G-S,F,y,h); /* J,Z: hash keys */#define J(A) K(y+A,b[y])-K(x+A,u)-K(H+A,t) v-=v>e; /* delayed-gain penalty */ if(z==9) /* called as move-legality */#define U 16777224 {if(v!=-I&x==K&y==L) /* checker: if move found */struct _ {int K,V;char X,Y,D;} A[U]; /* hash table, 16M+8 entries*/ {Q=-e-i;O=F;return l;} /* & not in check, signal */ v=m; /* (prevent fail-lows on */int V=112,M=136,S=128,I=8e4,C=799,Q,N,i; /* V=0x70=rank mask, M=0x88 */ } /* K-capt. replies) */ b[G]=k+38;b[F]=b[y]=0;b[x]=u;b[H]=t; /* undo move,G can be dummy */char O,K,L, if(Y&8){m=v;Y&=~8;goto A;} /* best=1st done,redo normal*/w[]={0,1,1,3,-1,3,5,9}, /* relative piece values */ if(v>m){m=v;X=x;Y=y|S&G;} /* update max, mark with S */o[]={-16,-15,-17,0,1,16,0,1,16,15,17,0,14,18,31,33,0, /* step-vector lists */ } /* if non castling */ 7,-1,11,6,8,3,6, /* 1st dir. in o[] per piece*/ t+=p<5; /* fake capt. for nonsliding*/ 6,3,5,7,4,5,3,6}, /* initial piece setup */ if(p<3&6*k+(y&V)==S /* pawn on 3rd/6th, or */b[129], /* board: half of 16x8+dummy*/ ||(u&~24)==36&j==7&& /* virgin K moving sideways,*/T[1035], /* hash translation table */ G&M&&b[G=(x|7)-(r>>1&7)]&32 /* 1st, virgin R in corner G*/ &&!(b[G^1]|b[G^2]) /* 2 empty sqrs. next to R */n[]=".?+nkbrq?*?NKBRQ"; /* piece symbols on printout*/ ){F=y;t--;} /* unfake capt., enable e.p.*/ }W(!t); /* if not capt. continue ray*/D(k,q,l,e,J,Z,E,z,n) /* recursive minimax search, k=moving side, n=depth*/ }}}W((x=x+9&~M)-B); /* next sqr. of board, wrap */int k,q,l,e,J,Z,E,z,n; /* (q,l)=window, e=current eval. score, E=e.p. sqr.*/ C:if(m>I/4|m<-I/4)d=99; /* mate is indep. of depth */{ /* e=score, z=prev.dest; J,Z=hashkeys; return score*/ m=m+I?m:-D(24-k,-I,I,0,J,Z,S,S,1)/2; /* best loses K: (stale)mate*/ int j,r,m,v,d,h,i=9,F,G; if(!a->K|(a->X&M)!=M|a->D<=d) /* if new/better type/depth:*/ char t,p,u,x,y,X,Y,H,B; {a->K=Z;a->V=m;a->D=d;A->K=0; /* store in hash,dummy stays*/ struct _*a=A; a->X=X|8*(m>q)|S*(m<l);a->Y=Y; /* empty, type (limit/exact)*/ /* lookup pos. in hash table*/ } /* encoded in X S,8 bits */ j=(k*E^J)&U-9; /* try 8 consec. locations */ /*if(z==8)printf("%2d ply, %9d searched, %6d by (%2x,%2x)n",d-1,N,m,X,Y&0x77);*/ W((h=A[++j].K)&&h-Z&&--i); /* first empty or match */ } a+=i?j:0; /* dummy A[0] if miss & full*/ if(z&8){K=X;L=Y&~M;} if(a->K) /* hit: pos. is in hash tab */ return m; {d=a->D;v=a->V;X=a->X; /* examine stored data */ } if(d>=n) /* if depth sufficient: */ {if(v>=l|X&S&&v<=q|X&8)return v; /* use if window compatible */ main() d=n-1; /* or use as iter. start */ { }X&=~M;Y=a->Y; /* with best-move hint */ int j,k=8,*p,c[9]; Y=d?Y:0; /* dont try best at d=0 */ }else d=X=Y=0; /* start iter., no best yet */ F(i,0,8) N++; /* node count (for timing) */ {b[i]=(b[i+V]=o[i+24]+40)+8;b[i+16]=18;b[i+96]=9; /* initial board setup*/ W(d++<n|z==8&N<1e7&d<98) /* iterative deepening loop */ F(j,0,8)b[16*j+i+8]=(i-4)*(i-4)+(j-3.5)*(j-3.5); /* center-pts table */ {x=B=X; /* start scan at prev. best */ } /*(in unused half b[])*/ Y|=8&Y>>4; /* request try noncastl. 1st*/ F(i,M,1035)T[i]=random()>>9; m=d>1?-I:e; /* unconsidered:static eval */ do{u=b[x]; /* scan board looking for */ W(1) /* play loop */ if(u&k) /* own piece (inefficient!)*/ {F(i,0,121)printf(" %c",i&8&&(i+=7)?10:n[b[i]&15]); /* print board */ {r=p=u&7; /* p = piece type (set r>0) */ p=c;W((*p++=getchar())>10); /* read input line */ j=o[p+16]; /* first step vector f.piece*/ N=0; W(r=p>2&r<0?-r:-o[++j]) /* loop over directions o[] */ if(*c-10){K=c[0]-16*c[1]+C;L=c[2]-16*c[3]+C;}else /* parse entered move */ {A: /* resume normal after best */ D(k,-I,I,Q,1,1,O,8,0); /* or think up one */ y=x;F=G=S; /* (x,y)=move, (F,G)=castl.R*/ F(i,0,U)A[i].K=0; /* clear hash table */ do{H=y+=r; /* y traverses ray */ if(D(k,-I,I,Q,1,1,O,9,2)==I)k^=24; /* check legality & do*/ if(Y&8)H=y=Y&~M; /* sneak in prev. best move */ } if(y&M)break; /* board edge hit */ }
  18. 18. free your mind
  19. 19. Software Visualization The use of the crafts of typography, graphic design, animation, and cinematography with modern human- computer interaction and computergraphics technology to facilitate both thehuman understanding and effective use of computer software. Lanza, 2010
  20. 20. (software)visualization myths
  21. 21. a picture is wortha thousand words
  22. 22. n ga picture is worth ro words wa thousand
  23. 23. UML is softwarevisualization
  24. 24. gUML is software n r o wvisualization
  25. 25. visual programming issoftware visualization
  26. 26. n gvisual programming is r o wsoftware visualization
  27. 27. static visualizationsoftware visualization dynamic visualization
  28. 28. ng ro ngw static visualization ro wsoftware visualization ng o dynamic wr visualization
  29. 29. about software ofsoftware software visualization
  30. 30. aboutsoftware software of software information visualization visualization
  31. 31. William Playfair 1759 - 1823
  32. 32. Jacques BertinSemiology of Graphics,1967
  33. 33. visualization ~ story
  34. 34. It was the twenty-eight of November. Animmense confused mass of men, horses,vehicles besieged the narrow entrances tothe bridges and began to flow over them.
  35. 35. It was the twenty-eight of November. Animmense confused mass of men, horses,vehicles besieged the narrow entrances tothe bridges and began to flow over them.Those in front, pushed by the weight of those behind werecrushed, trampled on, or forced into the ice-filled water of theBerezina.The confusion was so great that when Napoleon himself wishedto cross, it was necessary to use force to clear a passage.Some there were who, determined to pass at all costs, cut ahorrible way for themselves with their swords. Others openedan even crueler road for their carriages, driving them pitilesslythrough the helpless crowd, crushing men and women, in theirodious greed sacrificing their companions in misery.
  36. 36. seeing ~ understanding
  37. 37. We acquire more information through vision than throughall the other senses combined. Colin WareInformation Visualization Perception for Design Morgan Kaufmann, 2004
  38. 38. Sensation Perception (Physical Process) (Cognitive Process)Stimulus Sensory Organ Perceptual Organ Brain
  39. 39. The EyeA camera with a maximumresolution of 625 dpi in the fovealzone of the retina.
  40. 40. The EyeA camera with a maximumresolution of 625 dpi in the fovealzone of the retina.The retina is coated with rods (fordim light) and cones (for brightlight and colors).
  41. 41. The EyeA camera with a maximumresolution of 625 dpi in the fovealzone of the retina.The retina is coated with rods (fordim light) and cones (for brightlight and colors).Three types of cones, specializedon the blue, green, and red rangeof the color spectrum.
  42. 42. Breaking NewsWe do not see with the eyes
  43. 43. we see withthe brain
  44. 44. iconicmemory
  45. 45. short-termmemory
  46. 46. long-termmemory
  47. 47. Pre-attentiveprocessing
  48. 48. Category Form Color Spatial Motion Motion Orientation Hue 2D position Flicker Line length Intensity Direction Line width SizeAttribute Shape Curvature Added marks Enclosure
  49. 49. Orientation
  50. 50. Line Length
  51. 51. Line Width
  52. 52. 2D Position / Proximity
  53. 53. Enclosure
  54. 54. Shape / Similarity
  55. 55. Connectedness
  56. 56. Categorical vs Quantitative Encoding of Pre-attentive Attributes
  57. 57. Type Attribute Quantitative? Orientation No Line Length Yes Line Width Partially Size Partially Form Shape No Curvature No Added Marks No Enclosure No Hue No Color Intensity PartiallyPosition 2D Position Yes
  58. 58. about software ofsoftware software visualization
  59. 59. Stephen Few Edward TufteShow Me the Numbers The Visual Display of Quantitative InformationAnalytics Press, 2004 Morgan Kaufmann, 2004
  60. 60. about software ofsoftware visualization software
  61. 61. Computer graphics ✔
  62. 62. Computer graphics ✔Gestalt principles ✔
  63. 63. simple..but useless
  64. 64. Computer graphics ✔Gestalt principles ✔Semantics ✘
  65. 65. InterludeMetrics
  66. 66. Interlude IISoftwareAnalysis
  67. 67. WCRE 1999, TSE 2003
  68. 68. visualization ~ interaction
  69. 69. visualization ~ interactioninteraction ~ tool
  70. 70. visualization ~ interactioninteraction ~ tooltool ~ cool
  71. 71. CodeCrawler (1998 - 2005)
  72. 72. X-Ray Eclipse Plug-in xray.inf.usi.ch
  73. 73. the class blueprint
  74. 74. Initialize Interface Internal Accessor Attribute
  75. 75. Initialize Interface Internal Accessor Attribute
  76. 76. RegularInitialize Interface Internal Accessor Attribute Overriding Extending Abstract Constant Delegating Setter Getter OOPSLA 2001, TSE 2005
  77. 77. RegularOverridingExtending Abstract ConstantDelegating Setter Getter
  78. 78. RegularOverridingExtending Abstract ConstantDelegating Setter Getter
  79. 79. RegularOverridingExtending Abstract ConstantDelegating Setter Getter
  80. 80. OK! But, is this useful?
  81. 81. the unbearablelightness of beingvisual
  82. 82. Solution 1: Anecdotal Evidence
  83. 83. Solution 2: Pattern Language
  84. 84. Software Systems as Cities
  85. 85. City Metaphor VISSOFT 2007
  86. 86. City Metaphor class buildingpackage district VISSOFT 2007
  87. 87. City Metaphor class buildingpackage district VISSOFT 2007
  88. 88. City Metaphor class buildingpackage district nesting level color VISSOFT 2007
  89. 89. City Metaphor number of methods (NOM) height number of attributes (NOA) base size number of lines of code (LOC) color class buildingpackage district nesting level color VISSOFT 2007
  90. 90. Program Comprehension ArgoUML LOC 136,325 ICPC 2007
  91. 91. Program Comprehension
  92. 92. FacadeMDRImplNOA 3 skyscraperNOM 349LOC 3,413
  93. 93. CPPParserNOA 85 office buildingNOM 204LOC 9,111
  94. 94. JavaTokenTypesNOA 173 parking lotNOM 0LOC 0
  95. 95. house PropPanelEvent NOA 2 NOM 3 LOC 37
  96. 96. Program Comprehension ICPC 2007
  97. 97. Design Quality Assessmentdisharmony mapArgoUML classes brain class 8 god class 30 god + brain 6 data class 17 unaffected 1,715 SoftVis 2008
  98. 98. System Evolution Analysistime traveling WCRE 2008
  99. 99. System Evolution Analysis time ArgoUMLtime traveling 8 major releases 6 years WCRE 2008
  100. 100. System Evolution Analysis time ArgoUMLtime traveling 8 major releases 6 years WCRE 2008
  101. 101. http://codecity.inf.usi.ch implemented in Smalltalk ICSE 2008 tool demo
  102. 102. Is ituseful ?
  103. 103. A Controlled Experiment
  104. 104. Design
  105. 105. technical report 2010State of the art?
  106. 106. Design desiderata 1 Avoid comparing using a technique against not using it. 2 Involve participants from the industry. 3 Provide a not-so-short tutorial of the experimental tool to the participants. 4 Avoid, whenever possible, giving the tutorial right before the experiment. 5 Use the tutorial to cover both the research behind the approach and the tool. 6 Find a set of relevant tasks. 7 Choose real object systems that are relevant for the tasks. 8 Include more than one object system in the design. 9 Provide the same data to all participants.10 Limit the amount of time allowed for solving each task.11 Provide all the details needed to make the experiment replicable.12 Report results on individual tasks.13 Include tasks on which the expected result is not always to the advantage of the tool being evaluated.14 Take into account the possible wide range of experience level of the participants.
  107. 107. Design desiderata 1 Avoid comparing using a technique against not using it. 2 Involve participants from the industry. 3 Provide a not-so-short tutorial of the experimental tool to the participants. 4 Avoid, whenever possible, giving the tutorial right before the experiment. 5 Use the tutorial to cover both the research behind the approach and the tool. 6 Find a set of relevant tasks. 7 Choose real object systems that are relevant for the tasks. 8 Include more than one object system in the design. 9 Provide the same data to all participants.10 Limit the amount of time allowed for solving each task.11 Provide all the details needed to make the experiment replicable.12 Report results on individual tasks.13 Include tasks on which the expected result is not always to the advantage of the tool being evaluated.14 Take into account the possible wide range of experience level of the participants.
  108. 108. Finding a baseline1. program comprehension2. design quality assessment3. system evolution analysis
  109. 109. Finding a baseline1. program comprehension2. design quality assessment3. system evolution analysis
  110. 110. Finding a baseline1. program comprehension2. design quality assessment3. system evolution analysis
  111. 111. Finding a baseline1. program comprehension2. design quality assessment
  112. 112. Tasks
  113. 113. Tasks program comprehension 6A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.
  114. 114. Tasks program comprehension 6A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.B1.1 Identify the package with the highest percentage of god classes.B1.2 Identify the god class with the largest number of methods. Identify the dominant (affecting the highest number of classes) class-levelB2.1 design problem.B2.2 Write an overview of the class-level design problems in the system. design quality assessment 4
  115. 115. Tasks program comprehension 6 5A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.B1.1 Identify the package with the highest percentage of god classes.B1.2 Identify the god class with the largest number of methods. Identify the dominant (affecting the highest number of classes) class-levelB2.1 design problem.B2.2 Write an overview of the class-level design problems in the system. design quality assessment 4
  116. 116. Tasks quantitative 9 8A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.B1.1 Identify the package with the highest percentage of god classes.B1.2 Identify the god class with the largest number of methods. Identify the dominant (affecting the highest number of classes) class-levelB2.1 design problem.B2.2 Write an overview of the class-level design problems in the system. qualitative 1
  117. 117. Main research questions
  118. 118. Main research questions 1 Does the use of CodeCity increase the correctness of the solutions to program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?
  119. 119. Main research questions 1 Does the use of CodeCity increase the correctness of the solutions to program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size? 2 Does the use of CodeCity reduce the time needed to solve program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?
  120. 120. Variables of the experiment
  121. 121. Variables of the experiment correctnessdependent completion time CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advancedcontrolled academia background industry
  122. 122. Variables of the experiment correctnessdependent completion time CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advancedcontrolled academia background industry
  123. 123. Variables of the experiment correctnessdependent completion time CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advancedcontrolled academia background industry
  124. 124. Variables of the experiment correctnessdependent FindBugs completion time 1,320 classes 93,310 LOC CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advanced Azureuscontrolled academia 4,656 classes background industry 454,387 LOC
  125. 125. Variables of the experiment correctnessdependent completion time CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advancedcontrolled academia background industry
  126. 126. Variables of the experiment correctnessdependent completion time CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advancedcontrolled academia background industry
  127. 127. The experiment’s design between-subjects randomized-block
  128. 128. The experiment’s design between-subjects randomized-block CodeCity T1 large Tool T2 medium Size Ecl+Excl T3 large T4 medium
  129. 129. The experiment’s designbackground academia industryexperience beginner advanced beginner advanced B1 B2 B3 B4 between-subjects randomized-block CodeCity T1 large Tool T2 medium Size Ecl+Excl T3 large T4 medium
  130. 130. The experiment’s designbackground academia industryexperience beginner advanced beginner advanced B1 B2 B3 B4 between-subjects randomized-block CodeCity T1 large Tool T2 medium Size Ecl+Excl T3 large T4 medium
  131. 131. Execution
  132. 132. Experimental runs
  133. 133. Experimental runs day 1 timetraining session(1 hour)
  134. 134. Experimental runs day 1 timetraining session(1 hour) e1experiment session(2 hours) c1
  135. 135. Experimental runs day 1 day 2 timetraining session(1 hour) e1 e2experiment session(2 hours) c1 c2
  136. 136. Experimental runs day 1 day 2 day 3 timetraining session(1 hour) e1 e2 e3experiment session(2 hours) c1 c2
  137. 137. Experimental runs day 1 day 2 day 3 day 4 timetraining session(1 hour) e1 e2 e3experiment session(2 hours) c1 c2 c4
  138. 138. Testing the waters 2009 November December 18 24 25 2 9Lugano 1 3 1 1 1 1 1
  139. 139. Timeline of the experiment 2009 2010 November December January February ... April 18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14Lugano 1 3 1 1 1 1 1 1 1 1 3 Bologna 2 1 6 1 1 1 1 Antwerp 5 6Bern 4 1 6
  140. 140. Timeline of the experiment 2009 2010 November December January February ... April 18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14Lugano 1 3 1 1 1 1 1 1 1 1 3 remote sessions Bologna 2 1 6 1 1 1 1 Antwerp 5 6 remote sessionBern 4 1 6
  141. 141. Treatments and subjects academia industry beginner advanced advanced large 2 2 6 10 CodeCity medium 3 2 7 12 large 2 3 3 8 Ecl+Excl medium 2 5 4 11 9 12 20 41
  142. 142. Collecting raw data
  143. 143. Collecting raw data solution
  144. 144. Collecting raw data completion time
  145. 145. Controlling time
  146. 146. Controlling timecommon time
  147. 147. Controlling timecommon timeinfo on subjectsName (Task): Remaining time
  148. 148. Assessing correctness 1 T2: Findbugs, analyzed with CodeCity A3: Impact Analysis B1.2 A1 Multiple locations. The god class containing the largest number of methods in the system is There are 40/41 [0.5pts] classes class MainFrame [0.8pts] Dispersed. [1pt] defined in the following 3 packages [1/6pts for each]: defined in package edu.umd.cs.findbugs.gui2 [0.1pts] which contains 119 [0.1pts] methods. • edu.umd.cs.findbugs A2.1 • edu.umd.cs.findbugs.bcel B2.1 Localized [0.5pts] in package edu.umd.cs.findbugs.detect [0.5pts]. • edu.umd.cs.findbugs.detect The dominant class-level design problem is DataClass [0.5pts] A2.2 A4.1 which affects a number of 67 [0.5pts] classes. Dispersed The 3 classes with the highest number of methods are [ 1 pts each correctly placed and 1 pts each misplaced]: 3 6 in the following (max. 5) packages [0.2pts for each]: 1. class AbstractFrameModelingVisitor • edu.umd.cs.findbugs defined in package edu.umd.cs.findbugs.ba contains 195 methods; • edu.umd.cs.findbugs.anttask 2. class MainFrame • edu.umd.cs.findbugs.ba defined in package edu.umd.cs.findbugs.gui2 contains 119 methods; • edu.umd.cs.findbugs.ba.deref 3. class BugInstance • edu.umd.cs.findbugs.ba.jsr305 defined in package edu.umd.cs.findbugs • edu.umd.cs.findbugs.ba.npe contains 118 methods or • edu.umd.cs.findbugs.ba.vna class TypeFrameModelingVisitor defined in package edu.umd.cs.findbugs.ba.type • edu.umd.cs.findbugs.bcel contains 118 methods. • edu.umd.cs.findbugs.classfile A4.2 • edu.umd.cs.findbugs.classfile.analysis The 3 classes with the highest average number of lines of code per method are [ 1 pts each correctly placed and 1 pts each • edu.umd.cs.findbugs.classfile.engine 3 6 misplaced]: • edu.umd.cs.findbugs.classfile.impl 1. class DefaultNullnessAnnotations • edu.umd.cs.findbugs.cloud defined in package edu.umd.cs.findbugs.ba has an average of 124 lines of code per method; • edu.umd.cs.findbugs.cloud.db 2. class DBCloud.PopulateBugs • edu.umd.cs.findbugs.detect defined in package edu.umd.cs.findbugs.cloud.db has an average of 114.5 lines of code per method; • edu.umd.cs.findbugs.gui 3. class BytecodeScanner • edu.umd.cs.findbugs.gui2 defined in package edu.umd.cs.findbugs.ba • edu.umd.cs.findbugs.jaif has an average of 80.75 lines of code per method. • edu.umd.cs.findbugs.model B1.1 • edu.umd.cs.findbugs.visitclass oracles The package with the highest percentage of god classes in the system is • edu.umd.cs.findbugs.workflow edu.umd.cs.findbugs.ba.deref [0.8pts] which contains 1 [0.1pts] god classes out of a total of 3 [0.1pts] classes.
  149. 149. Assessing correctnessblinding 1 T2: Findbugs, analyzed with CodeCity A3: Impact Analysis B1.2 A1 Multiple locations. The god class containing the largest number of methods in the system is There are 40/41 [0.5pts] classes class MainFrame [0.8pts] Dispersed. [1pt] defined in the following 3 packages [1/6pts for each]: defined in package edu.umd.cs.findbugs.gui2 [0.1pts] which contains 119 [0.1pts] methods. • edu.umd.cs.findbugs A2.1 • edu.umd.cs.findbugs.bcel B2.1 Localized [0.5pts] in package edu.umd.cs.findbugs.detect [0.5pts]. • edu.umd.cs.findbugs.detect The dominant class-level design problem is DataClass [0.5pts] A2.2 A4.1 which affects a number of 67 [0.5pts] classes. Dispersed The 3 classes with the highest number of methods are [ 1 pts each correctly placed and 1 pts each misplaced]: 3 6 in the following (max. 5) packages [0.2pts for each]: 1. class AbstractFrameModelingVisitor • edu.umd.cs.findbugs defined in package edu.umd.cs.findbugs.ba contains 195 methods; • edu.umd.cs.findbugs.anttask 2. class MainFrame • edu.umd.cs.findbugs.ba defined in package edu.umd.cs.findbugs.gui2 contains 119 methods; • edu.umd.cs.findbugs.ba.deref 3. class BugInstance • edu.umd.cs.findbugs.ba.jsr305 defined in package edu.umd.cs.findbugs • edu.umd.cs.findbugs.ba.npe contains 118 methods or • edu.umd.cs.findbugs.ba.vna class TypeFrameModelingVisitor defined in package edu.umd.cs.findbugs.ba.type • edu.umd.cs.findbugs.bcel contains 118 methods. • edu.umd.cs.findbugs.classfile A4.2 • edu.umd.cs.findbugs.classfile.analysis The 3 classes with the highest average number of lines of code per method are [ 1 pts each correctly placed and 1 pts each • edu.umd.cs.findbugs.classfile.engine 3 6 misplaced]: • edu.umd.cs.findbugs.classfile.impl 1. class DefaultNullnessAnnotations • edu.umd.cs.findbugs.cloud defined in package edu.umd.cs.findbugs.ba has an average of 124 lines of code per method; • edu.umd.cs.findbugs.cloud.db 2. class DBCloud.PopulateBugs • edu.umd.cs.findbugs.detect defined in package edu.umd.cs.findbugs.cloud.db has an average of 114.5 lines of code per method; • edu.umd.cs.findbugs.gui 3. class BytecodeScanner • edu.umd.cs.findbugs.gui2 defined in package edu.umd.cs.findbugs.ba • edu.umd.cs.findbugs.jaif has an average of 80.75 lines of code per method. • edu.umd.cs.findbugs.model B1.1 • edu.umd.cs.findbugs.visitclass oracles The package with the highest percentage of god classes in the system is • edu.umd.cs.findbugs.workflow edu.umd.cs.findbugs.ba.deref [0.8pts] which contains 1 [0.1pts] god classes out of a total of 3 [0.1pts] classes.
  150. 150. Results
  151. 151. Statistical test two-way analysis of variance (ANOVA) 95% confidence interval
  152. 152. Correctness Ecl+Excl CodeCityDoes the use of CodeCity increase the 8correctness of the solutions to programcomprehension tasks, compared to non- 7visual exploration tools, regardless of the 6object system size? 5 4 3 2 1 0 medium large
  153. 153. Correctness Ecl+Excl CodeCityDoes the use of CodeCity increase the 8correctness of the solutions to programcomprehension tasks, compared to non- 7visual exploration tools, regardless of the 6object system size? 5 24.26% 4 3 2 more correct with CodeCity 1 large effect size (d=0.89) 0 medium large
  154. 154. Correctness Ecl+Excl CodeCityDoes the use of CodeCity increase the 8correctness of the solutions to programcomprehension tasks, compared to non- 7visual exploration tools, regardless of theobject system size? 6 24.26% 5 4 more correct with CodeCity 3 large effect size (d=0.89) 2 medium large
  155. 155. Completion time Ecl+Excl CodeCityDoes the use of CodeCity reduce the time 60needed to solve program comprehensiontasks, compared to non-visual exploration 50tools, regardless of the object system size? 40 30 20 10 0 medium large
  156. 156. Completion time Ecl+Excl CodeCityDoes the use of CodeCity reduce the time 60needed to solve program comprehensiontasks, compared to non-visual exploration 50tools, regardless of the object system size? 40 12.01% 30 20 less time with CodeCity 10 moderate effect size (d=0.63) 0 medium large
  157. 157. Completion time Ecl+Excl CodeCityDoes the use of CodeCity reduce the time 60needed to solve program comprehensiontasks, compared to non-visual explorationtools, regardless of the object system size? 50 12.01% 40 30 less time with CodeCity moderate effect size (d=0.63) 20 medium large
  158. 158. after the first roundCodeCityvsEcl+Excl+24% correctness-12% completion time

×