CMP782
Software Metrics
A Case Study
Measuring Scrum
Implementation
Kürşat Durak
Burakcan Ekici
Kübra Akeren
December 2017, Hacettepe University
Outline
• About Scrum
• GQM Tree
• Goal 1
• Goal 2
• Goal 3
• Critics
• Conclusion
• References
What is scrum?
• Scrum is based on the principles of transparency to all stakeholders
along with continuous inspection and adaptation to changing
conditions. These result in a methodology embracing change and
promoting an environment where all members of the project team
share an equal voice regarding how the application will deliver value
to its users.
Reasons
• Are we using scrum methodology correctly?
• Investigate scrum methodology pros and cons
• Importance of measurement (while doing scrum)
• Choosing correct metrics before starting scrum
• Understanding project flow by using scrum metrics etc.
GQM TREE
Goal 1
• Investigate the results of scrum implementation from the business
point of view.
Measurement Construct
MC #:
Related IRF #:
Information Need Calculating defect density for each module in scrum project
Measurable Concept Project defect density
Applied Levels [ ] Organization [ X ] Project
Relevent Entities 1. Found defects for each module
2. Requirements of each module
Attributes 1. Module defects
2. Module requirements
Base Measures M5: Defect count for each module
M6: Number of requirements for each module
Measurement Method 1. Count found defects for each module
2. Count requirements for each module
Type of Measurement Method 1. Objective
2. Objective
Scale 1. Integers from zero to infinity
2. Integers from zero to infinity
Type of Scale 1. Ratio
2. Ratio
Unit of Measurement 1. Defects
2. Requirements
Derived Measure Defect density for each module
Measurement Function Divide Total Defect Count of Each Module to Total Requirement Count of Each Module
Indicator Module defect density relative to requirement size of a module
Model Compute defect density by dividing number of requirements over defect count for each module
Decision Criteria Defect count of a module should be less than half of the requirement count of a module
Goal 1
Goal 2
• Estimation of the project cost that use scrum methodology.
Measurement Construct
MC #:
Related IRF #:
Information Need Estimation of the project cost that use scrum methodology.
Measurable Concept Project cost – Rework density
Applied Levels [ ] Organization [ X ] Project
Relevent Entities 1. PBI values that produced by team
2. Rework PBI values that produced by team
Attributes 1. Product Backlog Item (PBI)
2. PBI Phase Number
Base Measures M20: Total PBI Count
M21: Rework PBI Count
Measurement Method 1. Count PBI for all sprints
2. Count PBI phases for all PBI’s for all sprints
Type of Measurement Method 1. Objective
2. Objective
Scale 1. Integers from zero to infinity
2. Integers from zero to infinity
Type of Scale 1. Ratio
2. Ratio
Unit of Measurement 1. PBI
2. PBI
Derived Measure PBI Rework Ratio
Measurement Function Divide Total PBI Count by Rework PBI Count
Indicator Rework Ratio of Scrum
Model Compute each distinct PBI and repeated PBI (PBI Phases)
Decision Criteria Resulting ratio should less than 0.10 to carry on more effective scrum and to reduce cost of the project
Goal 2
Goal 3
• Investigate scrum methodology from the developer point of view.
Goal 3 - Q5 : Is the team working together effectively?
• This survey tends to whether scrum process is success in scrum
member’s perspective.
• It allows to evaluate what the team thinks about scrum process they
did, and it helps to generate an idea about scrum results.
• We rated each question from 0 to 4. (Never, Seldom, Sometimes,
Often, Always)
Goal 3 - Q5 : Is the team working together effectively?
Goal 3 - Q6 : What is the general health of scrum usage?
• This survey does not tend to problems that occurred during the
sprint. It tends to causes why those problems happened.
• It allows the team to visualize what the general health of scrum is and
what the areas that need more attention from the team are. Also, the
team can evaluate where they stand.
• We rated each question from 0 to 5. (Never, Seldom, Sometimes,
Often, Usually, Always)
Goal 3 - Q6 : What is the general health of scrum usage?
Critics
• Is the scrum recorded correctly in the companies in Turkey?
 Subjective Record Rates
 Manipulation of Data
 Missing Data ... etc.
• Do we cover all the dimensions of scrum?
• Did we take enough data?
• Company confidentiality
Conclusion
• Understand a big company scrum usage
• Improve the process flow by using metrics
• Investigate the results of scrum implementation from the business
point of view.
• Estimation of the project cost that use scrum methodology.
• Investigate scrum methodology from the developer point of view.
References
• http://www.disciplinedagiledelivery.com/agile-gqm/
• http://www.prowareness.nl/wp-content/uploads/2015/05/whitepaper_agile_metrics.pdf
• http://publications.lib.chalmers.se/records/fulltext/143815.pdf
• https://www.frontrowagile.com/blog/posts/69-30-metrics-for-agile-software-development-teams
• https://josephvargheese.wordpress.com/2013/02/17/1000-performance-metrics-for-agile-scrum-process/
• http://www.scrumexpert.com/knowledge/scrum-team-metrics/
• http://agilecomplexificationinverter.blogspot.com.tr/2013/05/metrics-for-scrum-team.html
• https://luis-goncalves.com/agile-retrospectives-assessment/
• https://www.scrumalliance.org/community/articles/2015/december/agile-maturity-self-assessment-
survey
THANKS!
Any questions?

A Case Study: Measuring Scrum Implementation

  • 1.
    CMP782 Software Metrics A CaseStudy Measuring Scrum Implementation Kürşat Durak Burakcan Ekici Kübra Akeren December 2017, Hacettepe University
  • 2.
    Outline • About Scrum •GQM Tree • Goal 1 • Goal 2 • Goal 3 • Critics • Conclusion • References
  • 3.
    What is scrum? •Scrum is based on the principles of transparency to all stakeholders along with continuous inspection and adaptation to changing conditions. These result in a methodology embracing change and promoting an environment where all members of the project team share an equal voice regarding how the application will deliver value to its users.
  • 6.
    Reasons • Are weusing scrum methodology correctly? • Investigate scrum methodology pros and cons • Importance of measurement (while doing scrum) • Choosing correct metrics before starting scrum • Understanding project flow by using scrum metrics etc.
  • 7.
  • 8.
    Goal 1 • Investigatethe results of scrum implementation from the business point of view.
  • 9.
    Measurement Construct MC #: RelatedIRF #: Information Need Calculating defect density for each module in scrum project Measurable Concept Project defect density Applied Levels [ ] Organization [ X ] Project Relevent Entities 1. Found defects for each module 2. Requirements of each module Attributes 1. Module defects 2. Module requirements Base Measures M5: Defect count for each module M6: Number of requirements for each module Measurement Method 1. Count found defects for each module 2. Count requirements for each module Type of Measurement Method 1. Objective 2. Objective Scale 1. Integers from zero to infinity 2. Integers from zero to infinity Type of Scale 1. Ratio 2. Ratio Unit of Measurement 1. Defects 2. Requirements Derived Measure Defect density for each module Measurement Function Divide Total Defect Count of Each Module to Total Requirement Count of Each Module Indicator Module defect density relative to requirement size of a module Model Compute defect density by dividing number of requirements over defect count for each module Decision Criteria Defect count of a module should be less than half of the requirement count of a module
  • 10.
  • 11.
    Goal 2 • Estimationof the project cost that use scrum methodology.
  • 12.
    Measurement Construct MC #: RelatedIRF #: Information Need Estimation of the project cost that use scrum methodology. Measurable Concept Project cost – Rework density Applied Levels [ ] Organization [ X ] Project Relevent Entities 1. PBI values that produced by team 2. Rework PBI values that produced by team Attributes 1. Product Backlog Item (PBI) 2. PBI Phase Number Base Measures M20: Total PBI Count M21: Rework PBI Count Measurement Method 1. Count PBI for all sprints 2. Count PBI phases for all PBI’s for all sprints Type of Measurement Method 1. Objective 2. Objective Scale 1. Integers from zero to infinity 2. Integers from zero to infinity Type of Scale 1. Ratio 2. Ratio Unit of Measurement 1. PBI 2. PBI Derived Measure PBI Rework Ratio Measurement Function Divide Total PBI Count by Rework PBI Count Indicator Rework Ratio of Scrum Model Compute each distinct PBI and repeated PBI (PBI Phases) Decision Criteria Resulting ratio should less than 0.10 to carry on more effective scrum and to reduce cost of the project
  • 13.
  • 14.
    Goal 3 • Investigatescrum methodology from the developer point of view.
  • 15.
    Goal 3 -Q5 : Is the team working together effectively? • This survey tends to whether scrum process is success in scrum member’s perspective. • It allows to evaluate what the team thinks about scrum process they did, and it helps to generate an idea about scrum results. • We rated each question from 0 to 4. (Never, Seldom, Sometimes, Often, Always)
  • 16.
    Goal 3 -Q5 : Is the team working together effectively?
  • 17.
    Goal 3 -Q6 : What is the general health of scrum usage? • This survey does not tend to problems that occurred during the sprint. It tends to causes why those problems happened. • It allows the team to visualize what the general health of scrum is and what the areas that need more attention from the team are. Also, the team can evaluate where they stand. • We rated each question from 0 to 5. (Never, Seldom, Sometimes, Often, Usually, Always)
  • 18.
    Goal 3 -Q6 : What is the general health of scrum usage?
  • 19.
    Critics • Is thescrum recorded correctly in the companies in Turkey?  Subjective Record Rates  Manipulation of Data  Missing Data ... etc. • Do we cover all the dimensions of scrum? • Did we take enough data? • Company confidentiality
  • 20.
    Conclusion • Understand abig company scrum usage • Improve the process flow by using metrics • Investigate the results of scrum implementation from the business point of view. • Estimation of the project cost that use scrum methodology. • Investigate scrum methodology from the developer point of view.
  • 21.
    References • http://www.disciplinedagiledelivery.com/agile-gqm/ • http://www.prowareness.nl/wp-content/uploads/2015/05/whitepaper_agile_metrics.pdf •http://publications.lib.chalmers.se/records/fulltext/143815.pdf • https://www.frontrowagile.com/blog/posts/69-30-metrics-for-agile-software-development-teams • https://josephvargheese.wordpress.com/2013/02/17/1000-performance-metrics-for-agile-scrum-process/ • http://www.scrumexpert.com/knowledge/scrum-team-metrics/ • http://agilecomplexificationinverter.blogspot.com.tr/2013/05/metrics-for-scrum-team.html • https://luis-goncalves.com/agile-retrospectives-assessment/ • https://www.scrumalliance.org/community/articles/2015/december/agile-maturity-self-assessment- survey
  • 22.