Software Measurement: Lecture 1. Measures and Metrics
Upcoming SlideShare
Loading in...5
×
 

Software Measurement: Lecture 1. Measures and Metrics

on

  • 13,476 views

Materials of the lecture on metrics and measures held by Programeter leadership during the Software Economics course at Tartu University: courses.cs.ut.ee/2010/se

Materials of the lecture on metrics and measures held by Programeter leadership during the Software Economics course at Tartu University: courses.cs.ut.ee/2010/se

Statistics

Views

Total Views
13,476
Views on SlideShare
13,474
Embed Views
2

Actions

Likes
7
Downloads
430
Comments
2

1 Embed 2

http://www.programeter.com 2

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Software Measurement: Lecture 1. Measures and Metrics Software Measurement: Lecture 1. Measures and Metrics Presentation Transcript

  • Software Measurement Software Economics 2010
  • Anton Litvinenko
    • Co-founder and CTO at
      • Metrics tracking kit for software development
      • Key competence: software measurement and metrics
    • 9 years of software development at Programeter, Mobi, and MicroLink
    • MSc in computer science at Tartu University
  • Agenda
    • What is a software metric?
  • Are software metrics “ good ” or “ bad ”?
  • What is a “ measure ”?
  • What is a “measure”? “ Way of associating a number with some attribute of a physical object ” height -> meters temperature -> degrees Celsius
  • What is “measure”? One-to-one mapping between physical objects and formal objects (e.g numbers)
  • Same Stuff Formally
    • Relational System – tuple consisting of
      • Set of objects ( e.g. collections of apples )
      • Relations on these objects ( e.g. more, equal )
      • Binary operations on these objects ( e.g. combine, put together )
    • What would be the corresponding formal relational system?
  • Relationships and Operations
    • Apples:
      • Steve Jobs has 7 apples
      • Steve Ballmer has 4 apples
    • Jobs has more apples
    • Jobs and Ballmer can cooperate and put their apples together to have a larger pile
  • We defined a “ complete transition ” from “ real world ” into “ formal world ”
  • Same Stuff Formally... again
    • Let A be a relational system of physical objects (e.g. apples)
    • B be a relational system of formal objects ( e.g. numbers)
    • m be a measure from A to B then
    • Tuple A , B and m is a scale if
      • Relations from A equivalent to relations from B
      • For each operator in A there is a corresponding operator in B
  • Why is this important?
  • What Can You Say?
  • Why is this important?
    • Software design:
      • 10 modules with complexity 20 – 30 range
      • 20 modules with complexity 10 – 30 range
    • Which one is less complex?
    • We don't have intuition for such cases
  • Intelligence Barrier
  • Example: Temperature
    • Facts :
      • Steve : today is 40 ºF, yesterday was 80ºF
      • Anton : today is 4ºC, yesterday was 27ºC
    • Statements :
      • Steve : Yesterday was warmer than today
      • Anton : Yesterday was warmer than today
  • Example: Temperature
    • Facts :
      • Steve : today is 40 ºF, yesterday was 80ºF
      • Anton : today is 4ºC, yesterday was 27ºC
    • Statements :
      • Steve : Yesterday was 2x times warmer
      Is this a meaningful statement about temperature?
  • Statement is meaningful when it gives same result on all similar scales
  • Scales are similar when there is a transformation from one scale to another that retains all defined relations and operations
  • Nominal Scale
    • Giving “ names ” to objects
      • Equality
    • Gender
      • Any naming is similar to any other
    • Numbers on t-shirts of football players
      • Any unique numbering is similar to any other
  • View from 3000 feet :) Nominal Scales Gender T-shirt Numbering
  • Ordinal Scale
    • Giving “ names ” in particular order
      • More .... than ....
      • Middle element – median
    • Rating of tennis players
      • Similar: any other rating that retains the order
  • All Ordinal Scales Are Nominal Nominal Ordinal Gender T-shirt Numbering Top 100 Grading
  • Interval Scale
    • Assigning numbers so that interval is also meaningful
      • Both median and arithmetic mean
      • Similar – reachable via positive linear transformation : t(x) = ax + b
    • Temperature in Celsius scale
      • Similar: Fahrenheit scale
  • Interval Scales Are Ordinal Nominal Ordinal Interval Gender T-shirt Numbering Top 100 Grading Temperature
  • Ratio Scale
    • Ratio of two measures is meaningful
      • All statistical measures
      • Similar – reachable via positive linear transformation in form of t(x) = ax
    • Length, height, ...
      • Similar: Imperial units
  • Ratio Scales Are Interval Nominal Ordinal Interval Ratio Gender T-shirt Numbering Top 100 Grading Temperature Length Height
  • Absolute Scale
    • Only one way of measuring objects !
      • Similar – identity transformation: t(x) = x
    • Counting:
      • My team has 5 members
      • My software is 25 lines of code
  • Absolute Scales Are Ratio Nominal Ordinal Interval Ratio Absolute Scales Gender T-shirt Numbering Top 100 Grading Temperature Length Height Team Size
  • Exercise 2
    • Cost is usually a measure with ratio scale
    • Quality is only ordinal (rarely interval)
    • Judgment in terms of value
      • Quality per unit of cost
      • Should we pay 2x for 2x quality?
    • Combining cost measure on a ratio scale with quality measure on ordinal scale , what scale do you get ?
  • In This Course: Metric = Measure
  • Software Metric is a measure of anything directly related to software or its production
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • Most famous :)
  • Can anybody name any software metric?
  • Lines Of Code (LOC) – Product Size
    • 12
    • 14
    • 18
  • Lines Of Code
  • Lines Of Code – Summary
    • Accurate, easy to measure
    • How to interpret ...
      • Empty lines
      • Comments
      • Several statements on one line
    • Language dependent
    • Doesn't respect complexity and content
  • McCabe's Cyclomatic Complexity
    • Thomas McCabe, 1976
    • Complexity of a program
      • Number of linearly independent paths through a function
      • Usually calculated using flow graph
    • V(G) = e – n + 2p
      • e – num of edges, n – num of vertices, p – num of unconnected parts of graph
  • McCabe's Cyclomatic Complexity
  • McCabe's Cyclomatic Complexity
    • e = 7
    • n = 6
    • p = 1
    • V(G) = 3
  • Cyclomatic Complexity – Summary
    • Automated
    • Maintainability
      • V(G) > 10 -> Probability of defects rises
    • Testability
      • V(G) is an upper bound for the branch coverage
        • Each control structure was evaluated both to true and false
      • V(G) is a lower bound for the path coverage
        • All possible paths were executed
    • Doesn't respect other types of complexity
      • Data structure, data flow, interfaces
  • Exercise 3
    • Calculate LOC
    • Draw a flow graph
    • Calculate McCabe's cyclomatic complexity
    Code snippet
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • Object oriented metrics
  • Object Oriented Metrics
    • Shiyam Chidamber and Chris Kemerer , 1994
    • Metrics based on firm theoretical basis and experience of professional software developers
    • Measure unique aspects of the object oriented approach
  • Inheritance Metrics
    • Depth of inheritance tree (DIT)
      • Depth of the class in the inheritance tree
    • Number of children (NOC)
      • Number of immediate descendants
    NOC: 2 DIT: 2 NOC: 3 DIT: 1
  • Complexity
    • Weighted method count (WMC)
      • Sum of McCabe's cyclomatic complexities of all methods
    • Response for a class (RFC)
      • Number of public methods in a class and methods directly called by these
  • Complexity – Example RFC = 6, WMC = 1 + 2 + 1 = 4
  • Coupling
    • Coupling between object classes (CBO)
      • Number of classes given class is coupled to
    • Lack of cohesion in methods (LCOM)
      • Number of method pairs that do not share instance variables vs number of methods that share at least one instance variable
  • Coupling – Example CBO = 2, LCOM = 3 – 0 = 3
  • Coupling – Example LCOM = 2 – 1 = 1
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • Object oriented metrics
      • Object oriented design quality metrics
  • Object Oriented Design
    • Bad design symptoms:
      • Rigidity, fragility, immobility, viscosity
    • Class design principles
      • Open closed principle , Liskov substitution principle , ...
    • Package architecture principles
      • Stable dependencies principle , Stable abstractness principle , …
  • OO Design Quality Metrics
    • Robert Martin (aka Uncle Bob) , 1994
    • Measure quality of an object oriented design
  • Can we divide dependencies into “ good ” and “ bad ”? Dependencies Between Classes
  • Dependencies
    • Stable (good) vs unstable (bad) class
    • Stable
      • No need to change = independent
      • Hard to change = many dependents = responsible
    • Unstable
      • Depends on many = dependent
      • Easy to change = no dependents = irresponsible
  • Class Category
    • Class category – group of highly cohesive classes
      • Closed and open to changes together
      • Reused together
      • Same goal
    • Packages in Java, namespaces in C#
  • Dependency Metrics
    • Afferent Coupling (Ca) – number of classes outside the category depending on the classes inside the category
      • Incoming dependencies
    • Efferent Coupling (Ce) – number of classes inside the category depending on the classes outside the category
      • Outgoing dependencies
  • Example - Coupling Package One Package Two Package Three Ca(Package One) = 1, Ce(Package One) = 2
  • Instability (I)
    • Ratio of outgoing dependencies to total number of dependencies
    • I = Ce / (Ca + Ce)
    • Stable -> I = 0 -> Ce = 0
    • Unstable -> I = 1 -> Ca = 0, Ce > 0
  • Should all categories be stable ?
  • Why a stable category needs to be extensible ? How?
  • Abstractness (A)
    • Degree to which a category is abstract
      • Ratio of abstract classes to the total number of classes in category
    • Completely abstract -> A = 1 -> all classes are abstract
    • Completely concrete -> A = 0 -> no abstract classes in category
  • Is there a relationship between Instability and Abstractness ?
  • Main Sequence
  • Distance From Main Sequence
    • D' = |A + I – 1|
      • Normalized to range from [0, 1]
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • Object oriented metrics
      • Object oriented design quality metrics
      • Developer and team metrics
  • Developer and Team Metrics
    • Productivity
      • How active developers are, how much work is being done
    • Knowledge
      • How much developers know the software they are working on
    • Expertise
      • What kind of tools and libraries developers use
    • Team “healthiness”
      • Communication and knowledge sharing
  • Productivity: Code Churn Metrics
    • Amount of code changed in the software during the period of time
    • Churned LOC – number of added, modified and deleted lines of code
    • Churn Count – number of changes made to a file
    • Files Churned – number of changed files
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  • Code Churn Metrics
    • Overview of activity and productivity
    • Increase in relative code churn metrics -> increase in defect density
      • Number of defects per line of code
    • Vulnerable files have higher code churn metrics
      • Vulnerability – instance of violation of the security policy
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • Object oriented metrics
      • Object oriented design quality metrics
      • Developer and team metrics
        • Productivity
        • Knowledge
  • Knowledge Metrics
    • Which parts of the software developer is comfortable working with?
      • Better planning
    • Does developer share her knowledge with colleagues?
      • Risk management
  •  
  •  
  •  
  •  
  •  
  • Unique: 2 / 5 -> 40% Shared: 1 / 5 -> 20% Unique: 1 / 5 -> 20% Shared: 1 / 5 -> 20%
  • Example
    • If developer decides to leave – all his unique knowledge is lost for the team
    Unique - 35% Shared - 10% Unique - 10% Shared - 35%
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • Object oriented metrics
      • Object oriented design quality metrics
      • Developer and team metrics
      • Project size metrics
  • How would you measure product size?
  • Perfect Hours
    • One hour of ideal engineering
      • How many perfect hours in a work day?
    • Relative measure of effort
      • “How many ideal engineering hours required to complete the feature”
    • Team specific
    • Applied early ↔ Manual and subjective
  • Points
    • Generalization of a perfect hour
      • Relative measure of effort required to complete the feature
    • Not tied to time
    • Team specific
    • Applied early ↔ Manual and subjective
  • Velocity
    • How much work can a team complete per iteration
    Completed Points Iterations
  • Function Points
    • Will be covered during next workshop
  • Agenda
    • What is a software metric?
    • Examples of software metrics
      • LOC and McCabe's cyclomatic complexity
      • Object oriented metrics
      • Object oriented design quality metrics
      • Developer and team metrics
      • Project size metrics
      • Quality metrics
  • What does “ high quality ” mean? Quality Metrics
  • Quality Metrics
    • Many different “ models ”, “ checklists ”
      • McCall's, FRUPS, ISO 9126
      • Functionality, reliability, usability, portability, …
    • Cannot be measured directly -> derived from other metrics
  •  
  • Quality – Developer's Perspective
    • Comprehensibility
      • Style and cleanness of source code
      • Architecture and design
      • Used technologies and libraries
    • Testability + Existing tests
      • Easiness of automated testing
      • Code coverage with tests
  • Quality – PM's Perspective
    • Predictability
      • Effort required for development, testing, ...
      • Delivery planning
      • Additional costs
    • Correctness
      • Satisfies specification
      • Serves customer needs
  • Quality – Customer's Perspective
    • Value for money
      • Supports organizational goals
      • Return on investment
    • Transparency
      • Partner's effort is recognizable
      • Delays and troubles are not hidden
  • Quality – User's Perspective
    • Usability
      • Ease of use
      • Comprehensibility
    • Performance
      • Responsive
      • Critical functionality is quick
    • Functionality
      • Software does the right thing
  • Example: Defect Detection Percentage
    • Efficiency of quality assurance procedures
      • How many bugs were “ delivered ” to customer
    • DDP = E / (E + D)
      • E – errors found before delivery
      • D – defects = errors found after delivery
    • What would be an ideal situation?
  • Example: Time Between Escaped Defects
    • How often new defects are found in delivered versions of the product
    • How would you use this metric?
  • Course: IDY0204 “ Software Quality and Standards”
  • Agenda
    • What is a software metric?
    • Examples of software metrics
    • Classification of software metrics
  • Classification of Software Metrics Subject of measurement
  • Subject: Development Process
    • Measuring the efficiency of process application
    • On the organizational level – strategic purposes
    • On the project level – tactical purposes
    • Examples of metrics
      • Length of (development) iteration
      • Number of changes in requirements
      • Number of finished tasks
      • Defect detection percentage
  • Subject: Resources
    • Measuring usage of personnel & resources and their properties
    • Examples of metrics
      • Developer competency
      • Developer fluctuation
      • Developer productivity and know-how in the project
      • Maturity of the code written by developer
  • Subject: Product
    • Measuring product attributes
      • Size, complexity, scalability
    • Examples of metrics
      • LOC, commented lines of code, function points
      • McCabe's cyclomatic complexity
      • Code coverage with test
      • Code stability
  • Classification – Overview What is measured?
  • Classification of Software Metrics “ Lines of Code” vs “Quality”
  • Direct Metrics
    • Directly measurable
    • Examples of metrics:
      • LOC, function points
      • McCabe's cyclomatic complexity
      • Number of requirements
  • Indirect Metrics
    • Not possible to measure directly
      • Derived from other properties
    • Examples of metrics
      • Code quality, code readability
      • Developer productivity, efficiency
      • Reliability
  • Classification – Overview What is measured? Is it measurable?
  • Classification of Software Metrics (In)dependency on the measurement context
  • Internal Attributes
    • Measurement context/environment is not relevant
    • Examples of metrics
      • LOC
      • McCabe's cyclomatic complexity
      • Code coverage with tests
  • External Metrics
    • Measured with respect to environment/context
    • Examples of metrics
      • Software reliability
      • Developer productivity
      • Source code comprehensibility
      • Usability
  • Classification – Overview What is measured? Is it measurable? Is context dependent?
  • Classification – Example What is measured? Is it measurable? Is context dependent?
  • References
    • G. Ford, Measurement theory for software engineers
      • http://courses.cs.ut.ee/2010/se/uploads/Main/measurement-theory.pdf
    • Wikipedia
      • http://en.wikipedia.org/wiki/Software_metrics
    • C. Lange, Metrics in software architecting
      • http://www.win.tue.nl/~mchaudro/sa2007/Metrics%20Architecting%202005.pdf
    • M. Gökmen, Software process and project metrics
      • http://www3.itu.edu.tr/~gokmen/SE-lecture-2.pdf
    • H. Nestra, Metrics, Software engineering 2005
      • http://courses.cs.ut.ee/2005/tvt/uploads/Main/software_engineering_21.pdf
    • Lines of code
      • http://en.wikipedia.org/wiki/Source_lines_of_code
  • References II
    • McCabe's cyclomatic complexity
      • http://en.wikipedia.org/wiki/Cyclomatic_complexity
      • http://www.stsc.hill.af.mil/crosstalk/1994/12/xt94d12b.asp
      • http://www.answers.com/topic/cyclomatic-complexity
    • S. Chidamber and C. Kemerer, A metrics suite for object oriented des.
      • http://bit.ly/2xY21F
    • C. Martin, OO Quality design metrics
      • http://cs.allegheny.edu/wiki/cs290F2004/uploads/123/oodmetrc.pdf
    • R. Pressman, Software engineering: a practitioner's approach
      • http://bit.ly/gCWvm
    • More
      • http://www.laynetworks.com/Software%20Engineering.htm
      • http://www.parlezuml.com/metrics/OO%20Design%20Principles%20&%20Metrics.pdf
      • http://www.parlezuml.com/metrics/Metrics%20Definitions.pdf
  • Home Reading David Longstreet “ Function Point Manual”
  • Thank you for your time and attention!