Your SlideShare is downloading. ×
Tutorial joaopascoalfaria-2confcmmiportugal-v1-3-split
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Tutorial joaopascoalfaria-2confcmmiportugal-v1-3-split

130
views

Published on

Tutorial: Quality Management and Process Improvement with the Team Software Process - João Pascoal Faria (FEUP)

Tutorial: Quality Management and Process Improvement with the Team Software Process - João Pascoal Faria (FEUP)

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
130
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Increased complexity, global systems of systems, and need for scalability and interoperability;Increased needs to accommodate COTS, software services, and legacy systems;Increasingly large volumes of data and ways to learn from them;Increased emphasis on users and end value;Computational plenty and multicore chips;Increasing integration of software and systems engineering;Increasing software autonomy; Combinations of biology and computing.
  • Reduces “inventory” and “work in progress” and enables more efficient and reliable processes (lean)Cycles may have variable duration and structure
  • Slide notes: explain defect classification and phases; explain defect data usage (checklists, defect prevention, …)
  • Transcript

    • 1. Portugal Quality Management and Process Improvement with the Team Software Process João Pascoal Faria Assistant Professor FEUP 2012-07-06
    • 2. Questions1. Do you have problems/concerns with software quality?2. Do you have problems/concerns with cost of quality?3. Do you know your current quality and cost of quality levels?4. How do they compare with industry benchmarks?5. How can you improve quality and reduce the cost of quality? 2 2
    • 3. AgendaMotivationTSP Performance ResultsFactor I – Bottom-Up Performance ImprovementFactor II - Personal ResponsibilityFactor III – Feedback Loops for Continual ImprovementFactor IV – Cost-Effective Defect Removal MethodsFactor V - Measurement and Quantitative MethodsConclusions 3 3
    • 4. Motivation 4 4
    • 5. Top 10 software engineeringtrends Barry Boehm 20111. Increasing emphasis on rapid development and adaptability;2. Increasing software criticality and need for assurance;… 5 5
    • 6. Increasing software criticalityand need for assurance Barry Boehm 2011“Although people’s, systems’, and organizations’ dependencyon software is becoming increasingly critical, dependability isgenerally not the top priority for software producers.” 6 6
    • 7. Increasing software criticalityand need for assurance Barry Boehm 2011“Although people’s, systems’, and organizations’ dependencyon software is becoming increasingly critical, dependability isgenerally not the top priority for software producers.”“(In fact) the IT industry spends the bulk of its resources, bothfinancial and human, on rapidly bringing products to market.” 7 7
    • 8. Increasing software criticalityand need for assurance Barry Boehm 2011“Although people’s, systems’, and organizations’ dependencyon software is becoming increasingly critical, dependability isgenerally not the top priority for software producers.”“(In fact) the IT industry spends the bulk of its resources, bothfinancial and human, on rapidly bringing products to market.”“This situation will likely continue until a major software-induced systems catastrophe similar in impact on worldconsciousness to the 9/11 World Trade Center catastrophestimulates action toward establishing accountability forsoftware dependability.” 8 8
    • 9. Why should quality be the top priority? 9 9
    • 10. Why should quality be the top priority?Increasing dependency on software systems 10 10
    • 11. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002] 11 11
    • 12. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software 12 12
    • 13. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, … 13 13
    • 14. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities 14 14
    • 15. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc. 15 15
    • 16. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors 16 16
    • 17. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors – Direct costs of sw errors represent 0,6% of USA’s GNP [NIST, 2002] 17 17
    • 18. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors – Direct costs of sw errors represent 0,6% of USA’s GNP [NIST, 2002]Quality work saves time and money 18 18
    • 19. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors – Direct costs of sw errors represent 0,6% of USA’s GNP [NIST, 2002]Quality work saves time and money – Defect prevention and early defect removal reduce rework costs 19 19
    • 20. Quality economics: Cost of Quality (COQ) 20 20
    • 21. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) 21 21
    • 22. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs 22 22
    • 23. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement 23 23
    • 24. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs 24 24
    • 25. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs • reviews and inspections • test development and execution (once) 25 25
    • 26. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs • reviews and inspections • test development and execution (once) – Internal failure costs 26 26
    • 27. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs • reviews and inspections • test development and execution (once) – Internal failure costs • repair & rework before delivery – External failure costs • repair & rework plus any scrap after delivery • law suites, loss of customers 27 27
    • 28. Example: Security vulnerabilities 28 28
    • 29. Example: Security vulnerabilities 29 29
    • 30. Example: Security vulnerabilities What happened? 30 30
    • 31. Example: Security vulnerabilities What happened? What is the cause? 31 31
    • 32. Example: Security vulnerabilities What happened? What is the cause? 5 min 32 32
    • 33. TSP Performance Results 33 33
    • 34. Better product quality (industrial results) Average Defect Density of Delivered Software 8 7,5 7 6,24 6 4,73 Defecst/KLOC 5 4 3 2,28 2 1,05 1 0,06 0 CMM L1 CMM L2 CMM L3 CMM L4 CMM L5 TSP TSP impact study Capers Jones, 2000 2003, 20 projects in 13 organizations, several maturity levels 34 34
    • 35. Better product quality (training results) PSP Training results, SEI, 298 developers Defects Mean Number of Compile and Test 120 110 100 Defects Per KLOC 90 80 70 60 50 PSP0 PSP1 PSP2 1/3 with 0 defects 40 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 Program Number 35 35
    • 36. Reduced security vulnerabilities (Microsoft) TSP + secure coding standards, security design patterns, static analysis tools, secure code inspections and reviews 8-person software development team Created 30 thousand lines of new and modified code in 7 months Source: TSP Secure, Noopur Davis et all, TSP Symposium 2009, New Orleans 36 36
    • 37. Better, faster, cheaper (Microsoft IT) TSP versus non-TSP projects 4 releases with TSP compared to previous releases without TSP Source: The TSP Story @ Microsoft IT, TSP Symposium, Phoenix, 2008 37 37
    • 38. Reduced cost of quality (Adobe) J.Sartain, Senior Director, Quality, Adobe 2009 38 38
    • 39. Factor I – Bottom-UpPerformanceImprovement 39 39
    • 40. Build high-performance teams from thebottom-up TeamInstance ofhigh-maturity Management Teampractices for Softwareteams Process Team BuildingInstance ofhigh-maturity Personal Teampractices for Software Memberindividuals Process Skills http://www.sei.cmu.edu/tsp/ 40 40
    • 41. Build high-performance teams from thebottom-up TeamInstance ofhigh-maturity Management Teampractices for Softwareteams Process Team BuildingInstance ofhigh-maturity Personal Team Process disciplinepractices for Software Member Performance measuresindividuals Process Skills Estimating & planning skills Quality management skills http://www.sei.cmu.edu/tsp/ 41 41
    • 42. Build high-performance teams from thebottom-up TeamInstance ofhigh-maturity Management Teampractices for Softwareteams Process Goal setting Team Role assignment Building Tailored team process Detailed balanced plansInstance ofhigh-maturity Personal Team Process disciplinepractices for Software Member Performance measuresindividuals Process Skills Estimating & planning skills Quality management skills http://www.sei.cmu.edu/tsp/ 42 42
    • 43. Build high-performance teams from thebottom-up Team communication Team Team coordinationInstance ofhigh-maturity Management Project tracking Teampractices for Risk analysis Softwareteams Process Goal setting Team Role assignment Building Tailored team process Detailed balanced plansInstance ofhigh-maturity Personal Team Process disciplinepractices for Software Member Performance measuresindividuals Process Skills Estimating & planning skills Quality management skills http://www.sei.cmu.edu/tsp/ 43 43
    • 44. Build personal performance from thebottom-upPersonal practices areintroduced stepwise, througha sequence of small projectsin a training environment Defect & yield management; – The objective is to convince design people by seeing their own practices performance improving as they practice Size and effort estimationTeam practices are Establish a measuredintroduced in real projects performance baselinewith the help of a coach 44 44
    • 45. Example: Personal PSP training experience 45 45
    • 46. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as before 46 46
    • 47. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/days 47 47
    • 48. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates 48 48
    • 49. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) 49 49
    • 50. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity 50 50
    • 51. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity – 5% effort estimation error 51 51
    • 52. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity – 5% effort estimation error – 2 defects in unit testing, caused by requirements problems! 52 52
    • 53. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity – 5% effort estimation error – 2 defects in unit testing, caused by requirements problems! – Next step: improve requirements analysis! 53 53
    • 54. Factor II – PersonalResponsibility 54 54
    • 55. Personal responsibility: project managementThere is only one way to manage knowledge workers:they must manage themselves [W. Humphrey] 55 55
    • 56. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader Traditional teamThe leader plans, directs, and tracks the team’s work. 56 56
    • 57. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader TSP Team Leader Coach Traditional teamThe leader plans, directs, and tracks the team’s work. Self-directed team All team members participate in planning, managing, and tracking their own work. 57 57
    • 58. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader TSP Team Leader Coach Planning Test Manager Manager Process Implementation Manager Manager Design Quality Traditional team Manager ManagerThe leader plans, directs, Support Costumer Interface and tracks the team’s Manager Manager work. Self-directed team All team members participate in planning, managing, and tracking their own work. 58 58
    • 59. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader TSP Team Leader Coach Planning Test Manager Ownership Manager Commitment Motivation Implementation Process Performance Manager Manager Design Quality Traditional team Manager ManagerThe leader plans, directs, Support Costumer Interface and tracks the team’s Manager Manager work. Self-directed team All team members participate in planning, managing, and tracking their own work. 59 59
    • 60. Personal responsibility: quality managementThe only way to build high-quality products in a cost-effectiveway, is by having developers being personally responsible forthe quality of their products.Since defects can best be managed where they are injected,developers should – remove their own defects – determine the causes of their defects – learn to prevent those defects 60 60
    • 61. Factor III – FeadbackLoops for ContinualImprovement 61 61
    • 62. Feedback loops: project lifecycle Business Release planning and Launch technical goals Estimates, plans, process, Iteration planning commitment Re-launch Developm Status Lessons, new goals, phase Development phase or cycle new requirements, new orphase cycle risks, etc. or cycle Weekly team meetings Phase or cycle Updated status Retrospective Work products, and plans status, metrics, results Project Retrospective 62 62
    • 63. Feedback loops: estimating & planning frmwkCostumer Define need requirements Items PROBE method Produce conceptual (PROxy Based Estimating) Tasks design (PBS) Estimate Size size database Estimate resources ProductivityCostumer (effort) database Produce task list, Hist. distrib., distribute effort Process def. Produce Resources Management schedule availableProduct Develop Size, resource, Process Trackingdelivery product schedule data analysis reports 63 63
    • 64. Feedback loops: quality frameworkLearn from past errors, because we tend to repeat the same errors! Development (and defect Scripts, defect injection) prevention Standards, phase Awareness Defect removal Checklists, phase early defect Test Criteria, … detection & removal Subsequent Defect Process/Data phases late defect Data analysis detection & removal 64 64
    • 65. Feedback loops: coaching (& leadership)A coach helps improving individual andteam performance through a continuouscoaching/learning cycle External Independent voice 2. Conscious 1. Unconscious Incompetence Incompetence Guidance Next and Stage Feedback 3. Conscious 4. Unconscious Competence Competence Practice 65 65
    • 66. Example: Historical defect data analysis 66 66
    • 67. Example: Historical defect data analysis 67 67
    • 68. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment 68 68
    • 69. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking 80 – Function Most frequent 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment 69 69
    • 70. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking Most frequent Most expensive* 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment 70 70
    • 71. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking Most frequent Most expensive* 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment (*) further analyses showed that they were mainly injected in design and removed in test. 71 71
    • 72. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking Most frequent Most expensive* 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment (*) further analyses showed that they were mainly injected in design and removed in test. Improve design review checklists Improve design standards and scripts Produce test spec after design spec and before design review Minimize documentation and use spell checker 72 72
    • 73. Factor IV - Cost-EffectiveDefect Removal Methods 73 73
    • 74. Defect removal methods (V-Model)Requirements REQ Team System & ST Spec Inspection Test (ST) High-Level Integration HLD Team Design Test (IT) Inspection & IT Spec Detailed DLD Code Team Design Personal DLD Team Inspection Inspection & UT Spec Review Code Unit Test Code Personal Compile (UT) Review 74 74
    • 75. Efficiency of defect removal methodsEven experienced developers introduce ~100 defects/KLOCSuch high-number of defects must be removed using the most cost-effective methods, such as personal reviews and team inspections(Continue to use unit, integration and system testing) Minutes to Find and Resolve a Defect by Discovery Activity Source: Jim Sartain, Adobe, 2009 75 75
    • 76. Use checklists derived from historical defectdata Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 76 76
    • 77. Use checklists derived from historical defectdataMake the review more effective Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 77 77
    • 78. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problems Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 78 78
    • 79. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 79 79
    • 80. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problems Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 80 80
    • 81. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 81 81
    • 82. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 82 82
    • 83. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 83 83
    • 84. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential – keep it simple and short people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 84 84
    • 85. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential – keep it simple and short people of 2008” – keep it specific for the technology, language, etc. (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 85 85
    • 86. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential – keep it simple and short people of 2008” – keep it specific for the technology, language, etc. (Time magazin) – combine with other analysis techniques http://www.youtub e.com/watch?v=YK m8NUmPg58 86 86
    • 87. Take enough review timeThe review rate (size reviewed per hour) is one of the bestleading indicators of review effectiveness (review yield)Recommended code review rate: about 200LOC/hourRevewing too slow or too fast is a waste of timeAlso recommended to review in a quiet environment take a break between producing and reviewing 87 87
    • 88. Use multiple inspectors to improve yield(yield = % of defects detected) 88 88 (source: Introduction to the Team Software Process, Watts Humphrey)
    • 89. Estimate defects remaining with the capture-recapture methodBased on the degree of overlapping between different reviewers Common defects: C=A B=4 Total defects: T A*B/C = 12 Found Found by A = 8 by B = 6 Remaining defects: R A*B/C-(A+B-C) = 2 http://en.wikipedia.org/wiki/Mark_and_recapture 89 89
    • 90. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Effort Size (review, rework) (of artifact reviewed) 90 90
    • 91. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Defect removal rate (indicator of review efficiency) Effort Size (review, rework) (of artifact reviewed) 91 91
    • 92. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Defect removal rate Defect density (indicator of review (indicator of product quality) efficiency) Effort Size (review, rework) (of artifact reviewed) 92 92
    • 93. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Defect removal rate Defect density (indicator of review (indicator of product quality) efficiency) Effort Size (review, rework) (of artifact reviewed) Review rate (leading indicator of review effectiveness) 93 93
    • 94. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Review yield (lagging indicator of review effectiveness) Defects found (number, type, description, severity) Defect removal rate Defect density (indicator of review (indicator of product quality) efficiency) Effort Size (review, rework) (of artifact reviewed) Review rate (leading indicator of review effectiveness) 94 94
    • 95. Factor V - Measurementand Quantitative Methods 95 95
    • 96. The need for measurement 96 96
    • 97. The need for measurementIn God we trust, all others bring data.W. Edwards Deming 97 97
    • 98. The need for measurementIn God we You cant trust, all manage others and bring improve data. what you dont measure.W. Edwards Deming Tom DeMarco 98 98
    • 99. The need for measurementIn God we You cant When performance is unmeasured or trust, all manage improperly measured, the results are others and often disappointing and can even be bring improve disastrous. Unless your measures cover data. what you all important aspects, you will likely dont motivate counterproductive action. measure.W. Edwards Deming Tom DeMarco Watts Humphrey 99 99
    • 100. Base measuresIn the TSP, 4 core measures are the basis for quantitative project management, quality management, and process improvementat the personal, team and organization level. Size Effort Actual and Planned Quality Schedule 100 100
    • 101. Defect trackingDefects are the measure of quality in the TSP.Any change to an interim or final work product, made toensure proper subsequent utilization is considered a defect. 101 101
    • 102. Quality planning: Defect injection andremoval plan Removed• Defects injected based on historical defects injected per size unit• Yields (% of defects removed) based on historical data or benchmarks 102 102
    • 103. Quality tracking: Defect removal profileThe defect removal profile shows – plan and actual defects removed by phase – early vs. late defect removal plan Defects Removed by Phase for Assembly SYSTEM 900.0 800.0Defects Removed by Phase 700.0 600.0 500.0 Plan 400.0 Actual 300.0 200.0 100.0 0.0 n st le st n n n e st w w ti o tio tio Te tio od pi Te ie Te ie ec om ec ec ev ec C ev n t em sp ni io sp sp sp R R C U at In LD In In In st e gr od Sy EQ LD LD e D te od C In R H D C d an i ld Bu Phase 103 103
    • 104. Quality tracking: Defect removal profileThe defect removal profile shows – plan and actual defects removed by phase – early vs. late defect removal plan Defects Removed by Phase for Assembly SYSTEM 900.0 800.0Defects Removed by Phase 700.0 600.0 500.0 Plan 400.0 Actual 300.0 200.0 100.0 Typical 0.0 software n st le st n n n e st w w ti o tio tio Te tio od pi Te ie Te ie ec project om ec ec ev ec C ev n t em sp ni io sp sp sp R R C U at In LD In In In st e gr od Sy EQ LD LD e D te od C In R H D C d an i ld Bu Phase 104 104
    • 105. Quality Tracking: Quality profileIt examines Quality Profile for Assembly Common Query Changes (BE)criteria thatare effective Design Qualitypredictors of Design/Code Time 1 Quality Profile for Assembly Common Query Changes (BE) 1system testand post- 0.8release Design/Code Time 0.6component Design Review 1 Code Review Quality 0.4 Qualityquality Design Review Time 0.8 Code Review Time ½ design time 0.6 0.2 ½ coding time 0.4 0 Design Review Time Code Review Time 0.2 Plan 0 Actual Program Quality Code Quality Unit Test Ddefects/KLOC Compile Defects/KLOC 4 10 Unit Test Ddefects/KLOC Compile Defects/KLOC 105 105
    • 106. Conclusions: TSP Quality principles 106 106
    • 107. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work. 107 107
    • 108. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater. 108 108
    • 109. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater.It is more efficient to prevent defects than to find and fix them. 109 109
    • 110. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater.It is more efficient to prevent defects than to find and fix them.The right way is always the fastest and cheapest way to do ajob. 110 110
    • 111. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater.It is more efficient to prevent defects than to find and fix them.The right way is always the fastest and cheapest way to do ajob.To maximize productivity, focus on quality first. 111 111
    • 112. ConclusionsThe TSP approach for producing high-quality software in acost-effective way is based on learning from your errorsand using cost-effective defect removal methodsData shows that TSP teams produce software with very lowdefect density and reduced cost of qualityIf you have the need to develop top quality software, youshould take a look at TSP principles and practices 112 112
    • 113. ReferencesSoftware Assessments, Benchmarks, and Best Practices, Capers Jones,Addison-Wesley, 2000Winning with Software, Watts Humphrey, 2002Introduction to the Team Software Process, Watts S. Humphrey, 2000TSP: Coaching Development Teams, Watts S. Humphrey, Addison-Wesley,2006PSP: A Self-Improvement Process for Software Engineers, Watts S.Humphrey, 2005“The Economic Impacts of Inadequate Infrastructure for Software Testing”,National Institute of Standards and Technology (NIST), 2002“Inspiring, enabling and driving the Evolution of Quality at Adobe leveragingthe TSP”, Jim Sartain, Senior Director, Quality at Adobe Systems, TSPSymposium 2009“Some Future Software Engineering Opportunities and Challenges”, BarryBoehm, in The Future of Software Engineering, Springer, 2011 113 113
    • 114. PortugalThank you!Contact information:João Pascoal Faria (*)Departamento de Engenharia InformáticaFaculdade de Engenharia da Universidade do PortoRua Dr. Roberto Frias, s/n 4200-465 Porto, PORTUGALEmail: jpf@fe.up.ptPhone: +351225082134Mobile: +351966494914(*) SEI Qualified PSP Developer, PSP Instructor, TSP Coach