Software Measurement: Lecture 1. Measures and Metrics
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Software Measurement: Lecture 1. Measures and Metrics

  • 14,241 views
Uploaded on

Materials of the lecture on metrics and measures held by Programeter leadership during the Software Economics course at Tartu University: courses.cs.ut.ee/2010/se

Materials of the lecture on metrics and measures held by Programeter leadership during the Software Economics course at Tartu University: courses.cs.ut.ee/2010/se

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
14,241
On Slideshare
14,239
From Embeds
2
Number of Embeds
1

Actions

Shares
Downloads
470
Comments
2
Likes
8

Embeds 2

http://www.programeter.com 2

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Software Measurement Software Economics 2010
  • 2. Anton Litvinenko
    • Co-founder and CTO at
      • Metrics tracking kit for software development
      • 3. Key competence: software measurement and metrics
    • 9 years of software development at Programeter, Mobi, and MicroLink
    • 4. MSc in computer science at Tartu University
  • 5. Agenda
    • What is a software metric?
  • 6. Are software metrics “ good ” or “ bad ”?
  • 7. What is a “ measure ”?
  • 8. What is a “measure”? “ Way of associating a number with some attribute of a physical object ” height -> meters temperature -> degrees Celsius
  • 9. What is “measure”? One-to-one mapping between physical objects and formal objects (e.g numbers)
  • 10. Same Stuff Formally
    • Relational System – tuple consisting of
      • Set of objects ( e.g. collections of apples )
      • 11. Relations on these objects ( e.g. more, equal )
      • 12. Binary operations on these objects ( e.g. combine, put together )
    • What would be the corresponding formal relational system?
  • 13. Relationships and Operations
    • Apples:
      • Steve Jobs has 7 apples
      • 14. Steve Ballmer has 4 apples
    • Jobs has more apples
    • 15. Jobs and Ballmer can cooperate and put their apples together to have a larger pile
  • 16. We defined a “ complete transition ” from “ real world ” into “ formal world ”
  • 17. Same Stuff Formally... again
    • Let A be a relational system of physical objects (e.g. apples)
    • 18. B be a relational system of formal objects ( e.g. numbers)
    • 19. m be a measure from A to B then
    • 20. Tuple A , B and m is a scale if
      • Relations from A equivalent to relations from B
      • 21. For each operator in A there is a corresponding operator in B
  • 22. Why is this important?
  • 23. What Can You Say?
  • 24. Why is this important?
    • Software design:
      • 10 modules with complexity 20 – 30 range
      • 25. 20 modules with complexity 10 – 30 range
    • Which one is less complex?
    • 26. We don't have intuition for such cases
  • 27. Intelligence Barrier
  • 28. Example: Temperature
    • Facts :
      • Steve : today is 40 ºF, yesterday was 80ºF
      • 29. Anton : today is 4ºC, yesterday was 27ºC
    • Statements :
      • Steve : Yesterday was warmer than today
      • 30. Anton : Yesterday was warmer than today
  • 31. Example: Temperature
    • Facts :
      • Steve : today is 40 ºF, yesterday was 80ºF
      • 32. Anton : today is 4ºC, yesterday was 27ºC
    • Statements :
      • Steve : Yesterday was 2x times warmer
      Is this a meaningful statement about temperature?
  • 33. Statement is meaningful when it gives same result on all similar scales
  • 34. Scales are similar when there is a transformation from one scale to another that retains all defined relations and operations
  • 35. Nominal Scale
    • Giving “ names ” to objects
      • Equality
    • Gender
      • Any naming is similar to any other
    • Numbers on t-shirts of football players
      • Any unique numbering is similar to any other
  • 36. View from 3000 feet :) Nominal Scales Gender T-shirt Numbering
  • 37. Ordinal Scale
    • Giving “ names ” in particular order
      • More .... than ....
      • 38. Middle element – median
    • Rating of tennis players
      • Similar: any other rating that retains the order
  • 39. All Ordinal Scales Are Nominal Nominal Ordinal Gender T-shirt Numbering Top 100 Grading
  • 40. Interval Scale
    • Assigning numbers so that interval is also meaningful
      • Both median and arithmetic mean
      • 41. Similar – reachable via positive linear transformation : t(x) = ax + b
    • Temperature in Celsius scale
      • Similar: Fahrenheit scale
  • 42. Interval Scales Are Ordinal Nominal Ordinal Interval Gender T-shirt Numbering Top 100 Grading Temperature
  • 43. Ratio Scale
    • Ratio of two measures is meaningful
      • All statistical measures
      • 44. Similar – reachable via positive linear transformation in form of t(x) = ax
    • Length, height, ...
      • Similar: Imperial units
  • 45. Ratio Scales Are Interval Nominal Ordinal Interval Ratio Gender T-shirt Numbering Top 100 Grading Temperature Length Height
  • 46. Absolute Scale
    • Only one way of measuring objects !
      • Similar – identity transformation: t(x) = x
    • Counting:
      • My team has 5 members
      • 47. My software is 25 lines of code
  • 48. Absolute Scales Are Ratio Nominal Ordinal Interval Ratio Absolute Scales Gender T-shirt Numbering Top 100 Grading Temperature Length Height Team Size
  • 49. Exercise 2
    • Cost is usually a measure with ratio scale
    • 50. Quality is only ordinal (rarely interval)
    • 51. Judgment in terms of value
      • Quality per unit of cost
      • 52. Should we pay 2x for 2x quality?
    • Combining cost measure on a ratio scale with quality measure on ordinal scale , what scale do you get ?
  • 53. In This Course: Metric = Measure
  • 54. Software Metric is a measure of anything directly related to software or its production
  • 55. Agenda
    • What is a software metric?
    • 56. Examples of software metrics
      • Most famous :)
  • 57. Can anybody name any software metric?
  • 58. Lines Of Code (LOC) – Product Size
    • 12
    • 14
    • 18
  • 59. Lines Of Code
  • 60. Lines Of Code – Summary
    • Accurate, easy to measure
    • How to interpret ...
      • Empty lines
      • 61. Comments
      • 62. Several statements on one line
    • Language dependent
    • 63. Doesn't respect complexity and content
  • 64. McCabe's Cyclomatic Complexity
    • Thomas McCabe, 1976
    • 65. Complexity of a program
      • Number of linearly independent paths through a function
      • 66. Usually calculated using flow graph
    • V(G) = e – n + 2p
      • e – num of edges, n – num of vertices, p – num of unconnected parts of graph
  • 67. McCabe's Cyclomatic Complexity
  • 68. McCabe's Cyclomatic Complexity
  • 72. Cyclomatic Complexity – Summary
    • Automated
    • 73. Maintainability
      • V(G) > 10 -> Probability of defects rises
    • Testability
      • V(G) is an upper bound for the branch coverage
        • Each control structure was evaluated both to true and false
      • V(G) is a lower bound for the path coverage
        • All possible paths were executed
    • Doesn't respect other types of complexity
      • Data structure, data flow, interfaces
  • 74. Exercise 3
    • Calculate LOC
    • 75. Draw a flow graph
    • 76. Calculate McCabe's cyclomatic complexity
    Code snippet
  • 77. Agenda
    • What is a software metric?
    • 78. Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • 79. Object oriented metrics
  • 80. Object Oriented Metrics
    • Shiyam Chidamber and Chris Kemerer , 1994
    • Metrics based on firm theoretical basis and experience of professional software developers
    • 81. Measure unique aspects of the object oriented approach
  • 82. Inheritance Metrics
    • Depth of inheritance tree (DIT)
      • Depth of the class in the inheritance tree
    • Number of children (NOC)
      • Number of immediate descendants
    NOC: 2 DIT: 2 NOC: 3 DIT: 1
  • 83. Complexity
    • Weighted method count (WMC)
      • Sum of McCabe's cyclomatic complexities of all methods
    • Response for a class (RFC)
      • Number of public methods in a class and methods directly called by these
  • 84. Complexity – Example RFC = 6, WMC = 1 + 2 + 1 = 4
  • 85. Coupling
    • Coupling between object classes (CBO)
      • Number of classes given class is coupled to
    • Lack of cohesion in methods (LCOM)
      • Number of method pairs that do not share instance variables vs number of methods that share at least one instance variable
  • 86. Coupling – Example CBO = 2, LCOM = 3 – 0 = 3
  • 87. Coupling – Example LCOM = 2 – 1 = 1
  • 88. Agenda
    • What is a software metric?
    • 89. Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • 90. Object oriented metrics
      • 91. Object oriented design quality metrics
  • 92. Object Oriented Design
    • Bad design symptoms:
      • Rigidity, fragility, immobility, viscosity
    • Class design principles
      • Open closed principle , Liskov substitution principle , ...
    • Package architecture principles
      • Stable dependencies principle , Stable abstractness principle , …
  • 93. OO Design Quality Metrics
    • Robert Martin (aka Uncle Bob) , 1994
    • 94. Measure quality of an object oriented design
  • 95. Can we divide dependencies into “ good ” and “ bad ”? Dependencies Between Classes
  • 96. Dependencies
    • Stable (good) vs unstable (bad) class
    • 97. Stable
      • No need to change = independent
      • 98. Hard to change = many dependents = responsible
    • Unstable
      • Depends on many = dependent
      • 99. Easy to change = no dependents = irresponsible
  • 100. Class Category
    • Class category – group of highly cohesive classes
      • Closed and open to changes together
      • 101. Reused together
      • 102. Same goal
    • Packages in Java, namespaces in C#
  • 103. Dependency Metrics
    • Afferent Coupling (Ca) – number of classes outside the category depending on the classes inside the category
      • Incoming dependencies
    • Efferent Coupling (Ce) – number of classes inside the category depending on the classes outside the category
      • Outgoing dependencies
  • 104. Example - Coupling Package One Package Two Package Three Ca(Package One) = 1, Ce(Package One) = 2
  • 105. Instability (I)
    • Ratio of outgoing dependencies to total number of dependencies
    • I = Ce / (Ca + Ce)
    • 106. Stable -> I = 0 -> Ce = 0
    • 107. Unstable -> I = 1 -> Ca = 0, Ce > 0
  • 108. Should all categories be stable ?
  • 109. Why a stable category needs to be extensible ? How?
  • 110. Abstractness (A)
    • Degree to which a category is abstract
      • Ratio of abstract classes to the total number of classes in category
    • Completely abstract -> A = 1 -> all classes are abstract
    • 111. Completely concrete -> A = 0 -> no abstract classes in category
  • 112. Is there a relationship between Instability and Abstractness ?
  • 113. Main Sequence
  • 114. Distance From Main Sequence
    • D' = |A + I – 1|
      • Normalized to range from [0, 1]
  • 115. Agenda
    • What is a software metric?
    • 116. Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • 117. Object oriented metrics
      • 118. Object oriented design quality metrics
      • 119. Developer and team metrics
  • 120. Developer and Team Metrics
    • Productivity
      • How active developers are, how much work is being done
    • Knowledge
      • How much developers know the software they are working on
    • Expertise
      • What kind of tools and libraries developers use
    • Team “healthiness”
      • Communication and knowledge sharing
  • 121. Productivity: Code Churn Metrics
    • Amount of code changed in the software during the period of time
    • 122. Churned LOC – number of added, modified and deleted lines of code
    • 123. Churn Count – number of changes made to a file
    • 124. Files Churned – number of changed files
  • 125.  
  • 126.  
  • 127.  
  • 128.  
  • 129.  
  • 130.  
  • 131.  
  • 132. Code Churn Metrics
    • Overview of activity and productivity
    • Increase in relative code churn metrics -> increase in defect density
      • Number of defects per line of code
    • Vulnerable files have higher code churn metrics
      • Vulnerability – instance of violation of the security policy
  • 133. Agenda
    • What is a software metric?
    • 134. Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • 135. Object oriented metrics
      • 136. Object oriented design quality metrics
      • 137. Developer and team metrics
  • 139. Knowledge Metrics
    • Which parts of the software developer is comfortable working with?
      • Better planning
    • Does developer share her knowledge with colleagues?
      • Risk management
  • 140.  
  • 141.  
  • 142.  
  • 143.  
  • 144.  
  • 145. Unique: 2 / 5 -> 40% Shared: 1 / 5 -> 20% Unique: 1 / 5 -> 20% Shared: 1 / 5 -> 20%
  • 146. Example
    • If developer decides to leave – all his unique knowledge is lost for the team
    Unique - 35% Shared - 10% Unique - 10% Shared - 35%
  • 147. Agenda
    • What is a software metric?
    • 148. Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • 149. Object oriented metrics
      • 150. Object oriented design quality metrics
      • 151. Developer and team metrics
      • 152. Project size metrics
  • 153. How would you measure product size?
  • 154. Perfect Hours
    • One hour of ideal engineering
      • How many perfect hours in a work day?
    • Relative measure of effort
      • “How many ideal engineering hours required to complete the feature”
    • Team specific
    • 155. Applied early ↔ Manual and subjective
  • 156. Points
    • Generalization of a perfect hour
      • Relative measure of effort required to complete the feature
    • Not tied to time
    • 157. Team specific
    • 158. Applied early ↔ Manual and subjective
  • 159. Velocity
    • How much work can a team complete per iteration
    Completed Points Iterations
  • 160. Function Points
    • Will be covered during next workshop
  • 161. Agenda
    • What is a software metric?
    • 162. Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • 163. Object oriented metrics
      • 164. Object oriented design quality metrics
      • 165. Developer and team metrics
      • 166. Project size metrics
      • 167. Quality metrics
  • 168. What does “ high quality ” mean? Quality Metrics
  • 169. Quality Metrics
    • Many different “ models ”, “ checklists ”
      • McCall's, FRUPS, ISO 9126
      • 170. Functionality, reliability, usability, portability, …
    • Cannot be measured directly -> derived from other metrics
  • 171.  
  • 172. Quality – Developer's Perspective
    • Comprehensibility
      • Style and cleanness of source code
      • 173. Architecture and design
      • 174. Used technologies and libraries
    • Testability + Existing tests
      • Easiness of automated testing
      • 175. Code coverage with tests
  • 176. Quality – PM's Perspective
    • Predictability
      • Effort required for development, testing, ...
      • 177. Delivery planning
      • 178. Additional costs
    • Correctness
      • Satisfies specification
      • 179. Serves customer needs
  • 180. Quality – Customer's Perspective
    • Value for money
      • Supports organizational goals
      • 181. Return on investment
    • Transparency
      • Partner's effort is recognizable
      • 182. Delays and troubles are not hidden
  • 183. Quality – User's Perspective
    • Usability
      • Ease of use
      • 184. Comprehensibility
    • Performance
      • Responsive
      • 185. Critical functionality is quick
    • Functionality
      • Software does the right thing
  • 186. Example: Defect Detection Percentage
    • Efficiency of quality assurance procedures
      • How many bugs were “ delivered ” to customer
    • DDP = E / (E + D)
      • E – errors found before delivery
      • 187. D – defects = errors found after delivery
    • What would be an ideal situation?
  • 188. Example: Time Between Escaped Defects
    • How often new defects are found in delivered versions of the product
    • How would you use this metric?
  • 189. Course: IDY0204 “ Software Quality and Standards”
  • 190. Agenda
    • What is a software metric?
    • 191. Examples of software metrics
    • 192. Classification of software metrics
  • 193. Classification of Software Metrics Subject of measurement
  • 194. Subject: Development Process
    • Measuring the efficiency of process application
    • 195. On the organizational level – strategic purposes
    • 196. On the project level – tactical purposes
    • Examples of metrics
      • Length of (development) iteration
      • 197. Number of changes in requirements
      • 198. Number of finished tasks
      • 199. Defect detection percentage
  • 200. Subject: Resources
    • Measuring usage of personnel & resources and their properties
    • Examples of metrics
      • Developer competency
      • 201. Developer fluctuation
      • 202. Developer productivity and know-how in the project
      • 203. Maturity of the code written by developer
  • 204. Subject: Product
    • Measuring product attributes
      • Size, complexity, scalability
    • Examples of metrics
      • LOC, commented lines of code, function points
      • 205. McCabe's cyclomatic complexity
      • 206. Code coverage with test
      • 207. Code stability
  • 208. Classification – Overview What is measured?
  • 209. Classification of Software Metrics “ Lines of Code” vs “Quality”
  • 210. Direct Metrics
    • Directly measurable
    • Examples of metrics:
      • LOC, function points
      • 211. McCabe's cyclomatic complexity
      • 212. Number of requirements
  • 213. Indirect Metrics
    • Not possible to measure directly
      • Derived from other properties
    • Examples of metrics
      • Code quality, code readability
      • 214. Developer productivity, efficiency
      • 215. Reliability
  • 216. Classification – Overview What is measured? Is it measurable?
  • 217. Classification of Software Metrics (In)dependency on the measurement context
  • 218. Internal Attributes
    • Measurement context/environment is not relevant
    • Examples of metrics
      • LOC
      • 219. McCabe's cyclomatic complexity
      • 220. Code coverage with tests
  • 221. External Metrics
    • Measured with respect to environment/context
    • Examples of metrics
      • Software reliability
      • 222. Developer productivity
      • 223. Source code comprehensibility
      • 224. Usability
  • 225. Classification – Overview What is measured? Is it measurable? Is context dependent?
  • 226. Classification – Example What is measured? Is it measurable? Is context dependent?
  • 227. References
    • G. Ford, Measurement theory for software engineers
      • http://courses.cs.ut.ee/2010/se/uploads/Main/measurement-theory.pdf
    • Wikipedia
      • http://en.wikipedia.org/wiki/Software_metrics
    • C. Lange, Metrics in software architecting
      • http://www.win.tue.nl/~mchaudro/sa2007/Metrics%20Architecting%202005.pdf
    • M. Gökmen, Software process and project metrics
      • http://www3.itu.edu.tr/~gokmen/SE-lecture-2.pdf
    • H. Nestra, Metrics, Software engineering 2005
      • http://courses.cs.ut.ee/2005/tvt/uploads/Main/software_engineering_21.pdf
    • Lines of code
      • http://en.wikipedia.org/wiki/Source_lines_of_code
  • 228. References II
    • McCabe's cyclomatic complexity
      • http://en.wikipedia.org/wiki/Cyclomatic_complexity
      • 229. http://www.stsc.hill.af.mil/crosstalk/1994/12/xt94d12b.asp
      • 230. http://www.answers.com/topic/cyclomatic-complexity
    • S. Chidamber and C. Kemerer, A metrics suite for object oriented des.
      • http://bit.ly/2xY21F
    • C. Martin, OO Quality design metrics
      • http://cs.allegheny.edu/wiki/cs290F2004/uploads/123/oodmetrc.pdf
    • R. Pressman, Software engineering: a practitioner's approach
      • http://bit.ly/gCWvm
    • More
      • http://www.laynetworks.com/Software%20Engineering.htm
      • 231. http://www.parlezuml.com/metrics/OO%20Design%20Principles%20&%20Metrics.pdf
      • 232. http://www.parlezuml.com/metrics/Metrics%20Definitions.pdf
  • 233. Home Reading David Longstreet “ Function Point Manual”
  • 234. Thank you for your time and attention!