This document discusses program understanding and how program analysis can help programmers understand code. It describes various program analysis techniques like call graph construction, program slicing, concept analysis, change impact analysis, and metrics that can help with understanding. However, it also notes limitations in how these analyses integrate with a programmer's workflow and that the goal should be just-in-time understanding for specific tasks rather than complete understanding. It advocates using visualizations to present analysis results in a compact way and argues understanding is needed to implement features, fix bugs, or improve performance when changes are needed.

1. Program Understanding:What Programmers Really WantEinar W. HøstMay 5th 2011

2. DescriptionIntroductionHow Analysis can helpUnderstandingUnderstanding Program UnderstandingAgendaAgendaEvaluationExamplesConsiderLimitationsofCurrentToolsLook at somewell-known Analyses

3. Program AnalysisProgram UnderstandingProgram Analysis

4. Why is program understanding important?

5. It’s what programmers do!

6. “To program is to understand” - Kristen Nygaard

7. Why is program understanding hard?

8. Size.

9. Complexity.

10. Heterogenity.

11. Side-effects.

12. The concept assignment problem.

13. Intangibility.

14. =>

15. Disorientation.

16. “Few are the programmers who can explain their code well enough so that reading it is not incredibly frustrating.” - JefRaskin

17. How can program analysis help?

18. Program Understanding?$s.=($f=$ARGV[$_%2])x(substr$s,$_,1or$f)for0..500;

19. Program Understanding!$s.=($f=$ARGV[$_%2])x(substr$s,$_,1or$f)for0..500;Through themagic lens of a Tool

20. What do programmers want?

21. ”Helpme in findingwhatneeds to be changed...

22. ...make thechange...

23. ...and getthe !$&?/% out!”

24. Just-in-time program understanding.

25. Complete understanding isNot realistic

26. Not cost-effective

27. Not necessaryUnderstand the program well enough toimplement a new feature

28. Understand the program well enough tofix a bug

29. Understand the program well enough toimproveperformance

30. Without change, there is no need for understanding!

31. Program understanding Understanding execution behavior

32. Understanding control flow

33. Understanding data flow

34. Understanding dependencies

35. Understanding side-effects

36. Understanding change impact

37. Reasoning about designat the code level

38. Program understanding Understanding what is relevantat the task level

39. Comprehension strategies

40. Comprehension strategiesReconstructknowledgeaboutthe program domain and mapping it to thesourcecode.Top-down

41. Comprehension strategiesReadcodestatements and mentallygroupthesestatementsintohigherlevelabstractions.Bottom-up

42. Comprehension strategiesReadcode in detail,followcontrolflow,gain global understanding.Systematic

43. Comprehension strategiesScancode, looking for cluesindicatingrelevance to thetask at hand.Opportunistic

44. Comprehension strategiesconjecturequestionsearchreadInquiries

45. Comprehension strategiesUnderstanding is formed by switchingbetweendifferentstrategies as needed.Integrated

46. Tools for program understanding

47. AnalysisPresentation

48. Analysis toolsCallgraphs

49. Program slicing

50. Feature location

51. Effectsofchange

52. Software metricsPresentation toolsVisualisations

53. Context-awarenessWhat is a program?

54. print”This is not a program.”

55. print(”This is not a program.”)

56. The program astext

57. The program asbinary

58. The program asversioned

59. The program asprocess

60. The program ecosystem Source code / Compiled binary

61. Execution environment (runtime)

62. Test suite

63. Version control system

64. Issue tracking system

65. Integrated development environment

66. The programmer

67. External sourcesWhat questions can we ask about a program?

68. Behavior-centric What does this piece of code do?

69. What happens if I change this?

70. Where is this code used?

71. Which parts of the program are affected by change?Requires program code

72. Evolution-centric Who wrote this piece of code?

73. Who can help me understand this?

74. Why was this feature implemented this way?

75. What are the most bug-prone parts of the program?

76. Where do changes happen most often?Requires program history

77. Interaction-centric How was this piece of code written?

78. Whichtasksare hard to accomplish?

79. Which parts of the program are hard to change? Requires coding-session history

80. What is program analysis?

81. Traditional program analysisAll you need is codeAll you need is codeAll you need is code, codeCode is all you need

82. StaticDynamic

83. StaticAll possible executions

84. StaticSound+Conservative

85. StaticTrades precision for soundness

86. Sample executionsDynamic

87. Efficient+PreciseDynamic

88. Trades completenessforefficiencyDynamic

89. StaticSynergyDynamic

90. Program analysis for program understanding

91. Call graph construction

92. Call graph constructionShow calling relationshipsbetween parts of a program.Essential idea

93. Call graph constructionfhmaingSample graph

94. Call graph constructionThe relationdescribingexactlythosecallsmade from oneentity to another in anypossibleexecutionofthe program.The ideal call graph

95. Call graph constructionMake the program seem less fragmented by showingthecontrolflow links between program parts.Benefit

96. Call graph constructionThe graphbecomestoo large and complexto be comprehensible.Limitation

97. Call graph construction”The sightofgcc'scallgraphfrightened my students so muchthattheyrequesteda differentproject” - ArunLakhotiaLimitation

99. Program slicing

100. Program slicingFind a reduced program exhibitingthe same behaviorofinterest as the full program.Essential idea

101. Program slicingThe reduced program is called a program slice.

102. Program slicingC = < x, V >x : a statement in program PV : a subsetof variables in PSlicing criterion

103. Criterion: <12, z>1begin2read(x,y)3total := 0.04sum := 0.05if x <= 16 then sum := y7 else begin8read(z)9 total := x*y10 end11write(total, sum)12 end.1 begin2read(x,y)5 if x <= 16 then7 else8read(z)12 end.

104. Criterion: <12, x>1begin2read(x,y)3total := 0.04sum := 0.05if x <= 16 then sum := y7 else begin8read(z)9 total := x*y10 end11write(total, sum)12 end.1 begin2read(x,y)12 end.

105. Program slicingWhatwouldthe program look like ifweassume an initial statesatisfyingC ?Forward conditioning

106. Program slicingDeletesstatementsthatwill not be executed given the initial state.Forward conditioning

107. Program slicingWhatwouldthe program look like ifweassume an eventual statesatisfyingC ?Backward conditioning

108. Program slicingDeletesstatementswhichcannot lead to the eventual state.Backward conditioning

109. Program slicingThe reduced programis smaller.Benefit

110. Program slicingThe reduced programlooks foreign.Limitation

111. Program slicingHard to integrateintotheprogrammer’sworkflow.Limitation

112. Concept analysis

113. Concept analysisIdentifygroupingsofobjectsthat have commonattributes.Essential idea

114. Concept analysisC = (O, A, R)O Set of objectsA Set of attributesR Relation R⊆O ×AFormal context

115. Concept analysisσ(O) = {a∈A⎮∀o∈O : (o, a)∈R}O Set of objectsA Set of attributesR Relation R⊆O ×ACommon attributes

116. Concept analysisτ(A) = {o∈O⎮∀a∈A : (o, a)∈R}O Set of objectsA Set of attributesR Relation R⊆O ×ACommon objects

117. Concept analysisA pair (O, A) is a conceptif A = σ(O) and O = τ(A)Definition of concept

118. Concept analysisO=> SubprogramsA=> FeaturesFeature location

119. Concept analysis(s, f) ⊆ Rif subprogram s is invoked when feature f is invokedFeature location

120. Concept analysisIdentify parts ofthe program relevant for a feature.Benefit

121. Concept analysisRecoveryofcomponents and generationofhigh-levelarchitectureviews.Benefit

122. Concept analysisImperfecthigh-leveldescriptions have limited application for concretetasks.Limitation

123. Concept analysisRequireseffort from the programmer, withunclearbenefits.Limitation

124. Concept analysisHard to integrateintotheprogrammer’sworkflow.Limitation

125. Change impact analysis

126. Change impact analysisIdentifythepotentialconsequencesof a program change.Essential idea

127. Change impact analysisEstimatewhat must be modified to accomplish a changeofbehavior.Essential idea

128. Change impact analysisSubtyping and dynamicdispatchmeansthatchangeimpactcan be non-local and unexpected.Challenges in OO languages

129. Change impact analysisAtomic changes

130. Change impact analysisProvide a boundaryaroundtheeffectsof a program edit.Benefit

131. Change impact analysisProvideconfidencethat an editdoes not have unexpectedeffectsoutsidetheboundary.Benefit

132. Change impact analysisLimited by theprecisionofthechangeimpactanalysis.Limitation

133. Change impact analysisStill needassurancethatnounexpectedeffectsoccur in theimpacted part ofthe program.Limitation

134. Change impact analysisHard to integrateintotheprogrammer’sworkflow.Limitation

135. Program metrics

136. Program metricsQuantifyaspectsofthe program presumed to be relevant.Essential idea

137. Program metrics Lines of code

138. Depth of inheritance tree

139. Internal cohesion

140. Coupling to other elements

141. Cyclomatic complexity

142. Halstead complexity

143. ...Code-centric metrics

144. Program metrics Test coverage

145. Bug density

146. Change rate

147. ...Other metrics

148. Program metricsAnswer questions regardingprogram quality.Benefit

149. Program metricsIdentify potential problemareas in the program.Benefit

150. Program metricsMetrics are indirectindicators of quality.Limitation

151. Program metricsTask-generating, not task-solving.Limitation

152. Software visualisation

153. Software visualisationAvoidoverwhelmingthe programmer by compressinginformation and usinggraphics.Goal

154. Table lens

155. Table lensZoom outthetable layout and display cells as pixel bars scaled and colored by data valuesIdea

156. Table lensExample

157. Table lensPresent a compactviewof software metrics for program elements.Application

158. Treemap

159. TreemapDisplay hierarchical data as nestedrectangles.Idea

160. TreemapExample

161. TreemapComparemetricvalues for various program elements.Application

162. Polymetric view

163. Polymetric viewCombineseveralmetrics for an entityinto a single view, by relating a visualaspect to eachmetric.Idea

164. Polymetric viewExample

165. Polymetric viewPresent a compactviewof software metrics for program elements.Application

166. Hierarchical edge bundles

167. Hierarchical edge bundlesBundle association data withstructure data in a radial view.Idea

168. Hierarchical edge bundlesExample

169. Hierarchical edge bundlesCompactviewofassociationsbetween program elements.Application

170. What about task-orientedprogram understanding?

171. The paradox of software visualisation

172. Programmers usevisualisation all the time whendiscussing programs informally.

173. Computers aregood at presenting informationgraphically.

174. ...and yet...

175. Software visualisation tools are rarely used for everyday software development and understanding.

176. Why?

177. HypothesisResearch is outoftouch withreality!

178. ClaimSoftware visualisationis a genericsolutionlooking for problems.

179. ClaimProgrammers needspecificsolutions to specific problems.

180. RemedyAn easy-to-usetoolthat lets the programmer definethe problem thatneedsvisualisation.

181. Task-aware environments

182. Task-aware environmentsAdaptinterface to reflectthetaskthe programmer is currentlyworking on.Idea

183. Task-aware environmentsIntroducesthenotionofdevelopmentsessions.Details

184. Task-aware environmentsLinks developmentsessions to tasks.Details

185. Task-aware environmentsSessionscan be thrownaway or saved for later.Details

186. Code bubbles

187. DescriptionIntroductionHow Analysis can helpUnderstandingUnderstanding Program UnderstandingAgendaSummaryEvaluationExamplesConsiderLimitationsofCurrentToolsLook at somewell-known Analyses