Portugal              Quality Management and            Process Improvement with           the Team Software Process      ...
Questions1. Do you have problems/concerns with software quality?2. Do you have problems/concerns with cost of quality?3. D...
AgendaMotivationTSP Performance ResultsFactor I – Bottom-Up Performance ImprovementFactor II - Personal ResponsibilityFact...
Motivation             4                 4
Top 10 software engineeringtrends                                           Barry Boehm                                   ...
Increasing software criticalityand need for assurance                                               Barry Boehm           ...
Increasing software criticalityand need for assurance                                                Barry Boehm          ...
Increasing software criticalityand need for assurance                                                Barry Boehm          ...
Why should quality be the top priority?                                          9                                        ...
Why should quality be the top priority?Increasing dependency on software systems                                          ...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Why should quality be the top priority?Increasing dependency on software systems   – 80%+ of functionality of modern aircr...
Quality economics: Cost of Quality (COQ)                                           20                                     ...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)                    ...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)    – Prevention cos...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)    – Prevention cos...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)    – Prevention cos...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)    – Prevention cos...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)    – Prevention cos...
Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ)    – Prevention cos...
Example: Security vulnerabilities                                    28                                     28
Example: Security vulnerabilities                                    29                                     29
Example: Security vulnerabilities                                      What                                    happened?  ...
Example: Security vulnerabilities                                      What                                    happened?  ...
Example: Security vulnerabilities                                      What                                    happened?  ...
TSP Performance Results                          33                           33
Better product quality (industrial results)                             Average Defect Density of Delivered Software      ...
Better product quality (training results)                        PSP Training results, SEI, 298 developers                ...
Reduced security vulnerabilities (Microsoft)                        TSP + secure coding standards,                        ...
Better, faster, cheaper (Microsoft IT)                                                                    TSP             ...
Reduced cost of quality (Adobe)                                  J.Sartain, Senior                                  Direct...
Factor I – Bottom-UpPerformanceImprovement                       39                        39
Build high-performance teams from thebottom-up                             TeamInstance ofhigh-maturity                   ...
Build high-performance teams from thebottom-up                               TeamInstance ofhigh-maturity                 ...
Build high-performance teams from thebottom-up                               TeamInstance ofhigh-maturity                 ...
Build high-performance teams from thebottom-up                                          Team communication                ...
Build personal performance from thebottom-upPersonal practices areintroduced stepwise, througha sequence of small projects...
Example: Personal PSP training experience                                            45                                   ...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with th...
Factor II – PersonalResponsibility                       54                        54
Personal responsibility: project managementThere is only one way to manage knowledge workers:they must manage themselves [...
Personal responsibility: project management  There is only one way to manage knowledge workers:  they must manage themselv...
Personal responsibility: project management  There is only one way to manage knowledge workers:  they must manage themselv...
Personal responsibility: project management  There is only one way to manage knowledge workers:  they must manage themselv...
Personal responsibility: project management  There is only one way to manage knowledge workers:  they must manage themselv...
Personal responsibility: quality managementThe only way to build high-quality products in a cost-effectiveway, is by havin...
Factor III – FeadbackLoops for ContinualImprovement                        61                         61
Feedback loops: project lifecycle   Business      Release planning   and              Launch   technical   goals          ...
Feedback loops: estimating & planning frmwkCostumer        Define  need       requirements                                ...
Feedback loops: quality frameworkLearn from past errors, because we tend to repeat the same errors! Development (and      ...
Feedback loops: coaching (& leadership)A coach helps improving individual andteam performance through a continuouscoaching...
Example: Historical defect data analysis                                           66                                     ...
Example: Historical defect data analysis                                           67                                     ...
Example: Historical defect data analysis                                      Defect Types                                ...
Example: Historical defect data analysis                                      Defect Types                                ...
Example: Historical defect data analysis                                         Defect Types                             ...
Example: Historical defect data analysis                                         Defect Types                             ...
Example: Historical defect data analysis                                                                Defect Types      ...
Factor IV - Cost-EffectiveDefect Removal Methods                             73                              73
Defect removal methods (V-Model)Requirements REQ Team                                                  System  & ST Spec  ...
Efficiency of defect removal methodsEven experienced developers introduce ~100 defects/KLOCSuch high-number of defects mus...
Use checklists derived from historical defectdata                                   Peter Pronovost                       ...
Use checklists derived from historical defectdataMake the review more effective                                   Peter Pr...
Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent pro...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Use checklists derived from historical defectdataMake the review more effective  – focus the attention on most frequent pr...
Take enough review timeThe review rate (size reviewed per hour) is one of the bestleading indicators of review effectivene...
Use multiple inspectors to improve yield(yield = % of defects detected)                                                   ...
Estimate defects remaining with the capture-recapture methodBased on the degree of overlapping between different reviewers...
Measure the review process and use data toimprove the reviews                         Escaped defects                     ...
Measure the review process and use data toimprove the reviews                           Escaped defects                   ...
Measure the review process and use data toimprove the reviews                           Escaped defects                   ...
Measure the review process and use data toimprove the reviews                           Escaped defects                   ...
Measure the review process and use data toimprove the reviews                           Escaped defects                   ...
Factor V - Measurementand Quantitative Methods                           95                            95
The need for measurement                           96                            96
The need for measurementIn God we trust, all  others   bring   data.W. Edwards Deming                            97       ...
The need for measurementIn God we           You cant trust, all         manage  others              and   bring           ...
The need for measurementIn God we           You cant     When performance is unmeasured or trust, all         manage      ...
Base measuresIn the TSP, 4 core measures are the basis for quantitative    project management,    quality management, and ...
Defect trackingDefects are the measure of quality in the TSP.Any change to an interim or final work product, made toensure...
Quality planning: Defect injection andremoval plan                                                  Removed• Defects injec...
Quality tracking: Defect removal profileThe defect removal profile shows   – plan and actual defects removed by phase   – ...
Quality tracking: Defect removal profileThe defect removal profile shows   – plan and actual defects removed by phase   – ...
Quality Tracking: Quality profileIt examines                               Quality Profile for Assembly Common Query Chang...
Conclusions: TSP Quality principles                                      106                                       106
Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality...
Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality...
Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality...
Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality...
Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality...
ConclusionsThe TSP approach for producing high-quality software in acost-effective way is based on learning from your erro...
ReferencesSoftware Assessments, Benchmarks, and Best Practices, Capers Jones,Addison-Wesley, 2000Winning with Software, Wa...
PortugalThank you!Contact information:João Pascoal Faria (*)Departamento de Engenharia InformáticaFaculdade de Engenharia ...
Upcoming SlideShare
Loading in...5
×

Tutorial joaopascoalfaria-2confcmmiportugal-v1-3-split

158

Published on

Tutorial: Quality Management and Process Improvement with the Team Software Process - João Pascoal Faria (FEUP)

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
158
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Increased complexity, global systems of systems, and need for scalability and interoperability;Increased needs to accommodate COTS, software services, and legacy systems;Increasingly large volumes of data and ways to learn from them;Increased emphasis on users and end value;Computational plenty and multicore chips;Increasing integration of software and systems engineering;Increasing software autonomy; Combinations of biology and computing.
  • Reduces “inventory” and “work in progress” and enables more efficient and reliable processes (lean)Cycles may have variable duration and structure
  • Slide notes: explain defect classification and phases; explain defect data usage (checklists, defect prevention, …)
  • Tutorial joaopascoalfaria-2confcmmiportugal-v1-3-split

    1. 1. Portugal Quality Management and Process Improvement with the Team Software Process João Pascoal Faria Assistant Professor FEUP 2012-07-06
    2. 2. Questions1. Do you have problems/concerns with software quality?2. Do you have problems/concerns with cost of quality?3. Do you know your current quality and cost of quality levels?4. How do they compare with industry benchmarks?5. How can you improve quality and reduce the cost of quality? 2 2
    3. 3. AgendaMotivationTSP Performance ResultsFactor I – Bottom-Up Performance ImprovementFactor II - Personal ResponsibilityFactor III – Feedback Loops for Continual ImprovementFactor IV – Cost-Effective Defect Removal MethodsFactor V - Measurement and Quantitative MethodsConclusions 3 3
    4. 4. Motivation 4 4
    5. 5. Top 10 software engineeringtrends Barry Boehm 20111. Increasing emphasis on rapid development and adaptability;2. Increasing software criticality and need for assurance;… 5 5
    6. 6. Increasing software criticalityand need for assurance Barry Boehm 2011“Although people’s, systems’, and organizations’ dependencyon software is becoming increasingly critical, dependability isgenerally not the top priority for software producers.” 6 6
    7. 7. Increasing software criticalityand need for assurance Barry Boehm 2011“Although people’s, systems’, and organizations’ dependencyon software is becoming increasingly critical, dependability isgenerally not the top priority for software producers.”“(In fact) the IT industry spends the bulk of its resources, bothfinancial and human, on rapidly bringing products to market.” 7 7
    8. 8. Increasing software criticalityand need for assurance Barry Boehm 2011“Although people’s, systems’, and organizations’ dependencyon software is becoming increasingly critical, dependability isgenerally not the top priority for software producers.”“(In fact) the IT industry spends the bulk of its resources, bothfinancial and human, on rapidly bringing products to market.”“This situation will likely continue until a major software-induced systems catastrophe similar in impact on worldconsciousness to the 9/11 World Trade Center catastrophestimulates action toward establishing accountability forsoftware dependability.” 8 8
    9. 9. Why should quality be the top priority? 9 9
    10. 10. Why should quality be the top priority?Increasing dependency on software systems 10 10
    11. 11. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002] 11 11
    12. 12. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software 12 12
    13. 13. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, … 13 13
    14. 14. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities 14 14
    15. 15. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc. 15 15
    16. 16. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors 16 16
    17. 17. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors – Direct costs of sw errors represent 0,6% of USA’s GNP [NIST, 2002] 17 17
    18. 18. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors – Direct costs of sw errors represent 0,6% of USA’s GNP [NIST, 2002]Quality work saves time and money 18 18
    19. 19. Why should quality be the top priority?Increasing dependency on software systems – 80%+ of functionality of modern aircrafts provided by sw [Humphrey, 2002]Increasing criticality of functions provided by software – e-banking, aircraft control software, medical device software, …Software defects are a primary cause of security vulnerabilities – Buffer overflow, SQL injection, etc.Huge economic impact of software errors – Direct costs of sw errors represent 0,6% of USA’s GNP [NIST, 2002]Quality work saves time and money – Defect prevention and early defect removal reduce rework costs 19 19
    20. 20. Quality economics: Cost of Quality (COQ) 20 20
    21. 21. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) 21 21
    22. 22. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs 22 22
    23. 23. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement 23 23
    24. 24. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs 24 24
    25. 25. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs • reviews and inspections • test development and execution (once) 25 25
    26. 26. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs • reviews and inspections • test development and execution (once) – Internal failure costs 26 26
    27. 27. Quality economics: Cost of Quality (COQ)For producers, the goal is to reduce the cost of quality (COQ) – Prevention costs • identifying and resolving defect causes • formal spec, prototyping, and design work • process analysis and improvement – Appraisal (defect detection) costs • reviews and inspections • test development and execution (once) – Internal failure costs • repair & rework before delivery – External failure costs • repair & rework plus any scrap after delivery • law suites, loss of customers 27 27
    28. 28. Example: Security vulnerabilities 28 28
    29. 29. Example: Security vulnerabilities 29 29
    30. 30. Example: Security vulnerabilities What happened? 30 30
    31. 31. Example: Security vulnerabilities What happened? What is the cause? 31 31
    32. 32. Example: Security vulnerabilities What happened? What is the cause? 5 min 32 32
    33. 33. TSP Performance Results 33 33
    34. 34. Better product quality (industrial results) Average Defect Density of Delivered Software 8 7,5 7 6,24 6 4,73 Defecst/KLOC 5 4 3 2,28 2 1,05 1 0,06 0 CMM L1 CMM L2 CMM L3 CMM L4 CMM L5 TSP TSP impact study Capers Jones, 2000 2003, 20 projects in 13 organizations, several maturity levels 34 34
    35. 35. Better product quality (training results) PSP Training results, SEI, 298 developers Defects Mean Number of Compile and Test 120 110 100 Defects Per KLOC 90 80 70 60 50 PSP0 PSP1 PSP2 1/3 with 0 defects 40 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 Program Number 35 35
    36. 36. Reduced security vulnerabilities (Microsoft) TSP + secure coding standards, security design patterns, static analysis tools, secure code inspections and reviews 8-person software development team Created 30 thousand lines of new and modified code in 7 months Source: TSP Secure, Noopur Davis et all, TSP Symposium 2009, New Orleans 36 36
    37. 37. Better, faster, cheaper (Microsoft IT) TSP versus non-TSP projects 4 releases with TSP compared to previous releases without TSP Source: The TSP Story @ Microsoft IT, TSP Symposium, Phoenix, 2008 37 37
    38. 38. Reduced cost of quality (Adobe) J.Sartain, Senior Director, Quality, Adobe 2009 38 38
    39. 39. Factor I – Bottom-UpPerformanceImprovement 39 39
    40. 40. Build high-performance teams from thebottom-up TeamInstance ofhigh-maturity Management Teampractices for Softwareteams Process Team BuildingInstance ofhigh-maturity Personal Teampractices for Software Memberindividuals Process Skills http://www.sei.cmu.edu/tsp/ 40 40
    41. 41. Build high-performance teams from thebottom-up TeamInstance ofhigh-maturity Management Teampractices for Softwareteams Process Team BuildingInstance ofhigh-maturity Personal Team Process disciplinepractices for Software Member Performance measuresindividuals Process Skills Estimating & planning skills Quality management skills http://www.sei.cmu.edu/tsp/ 41 41
    42. 42. Build high-performance teams from thebottom-up TeamInstance ofhigh-maturity Management Teampractices for Softwareteams Process Goal setting Team Role assignment Building Tailored team process Detailed balanced plansInstance ofhigh-maturity Personal Team Process disciplinepractices for Software Member Performance measuresindividuals Process Skills Estimating & planning skills Quality management skills http://www.sei.cmu.edu/tsp/ 42 42
    43. 43. Build high-performance teams from thebottom-up Team communication Team Team coordinationInstance ofhigh-maturity Management Project tracking Teampractices for Risk analysis Softwareteams Process Goal setting Team Role assignment Building Tailored team process Detailed balanced plansInstance ofhigh-maturity Personal Team Process disciplinepractices for Software Member Performance measuresindividuals Process Skills Estimating & planning skills Quality management skills http://www.sei.cmu.edu/tsp/ 43 43
    44. 44. Build personal performance from thebottom-upPersonal practices areintroduced stepwise, througha sequence of small projectsin a training environment Defect & yield management; – The objective is to convince design people by seeing their own practices performance improving as they practice Size and effort estimationTeam practices are Establish a measuredintroduced in real projects performance baselinewith the help of a coach 44 44
    45. 45. Example: Personal PSP training experience 45 45
    46. 46. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as before 46 46
    47. 47. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/days 47 47
    48. 48. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates 48 48
    49. 49. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) 49 49
    50. 50. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity 50 50
    51. 51. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity – 5% effort estimation error 51 51
    52. 52. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity – 5% effort estimation error – 2 defects in unit testing, caused by requirements problems! 52 52
    53. 53. Example: Personal PSP training experienceGoal: achieve 0 defects in unit testing and near 0% timeestimation error, with the same productivity as beforePSP Fundamentals: Steady reduction of time estimation error, from70% to 1% in 4 projects/daysPSP Advanced: Initial decrease of productivity because of theintroduction of design documentation templates8th project: compute the minimum difference between two Javasource files, in lines added, delete and modified, ignoring commentsand blank lines (considering that deletions have null cost) – Almost recovered the initial productivity – 5% effort estimation error – 2 defects in unit testing, caused by requirements problems! – Next step: improve requirements analysis! 53 53
    54. 54. Factor II – PersonalResponsibility 54 54
    55. 55. Personal responsibility: project managementThere is only one way to manage knowledge workers:they must manage themselves [W. Humphrey] 55 55
    56. 56. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader Traditional teamThe leader plans, directs, and tracks the team’s work. 56 56
    57. 57. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader TSP Team Leader Coach Traditional teamThe leader plans, directs, and tracks the team’s work. Self-directed team All team members participate in planning, managing, and tracking their own work. 57 57
    58. 58. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader TSP Team Leader Coach Planning Test Manager Manager Process Implementation Manager Manager Design Quality Traditional team Manager ManagerThe leader plans, directs, Support Costumer Interface and tracks the team’s Manager Manager work. Self-directed team All team members participate in planning, managing, and tracking their own work. 58 58
    59. 59. Personal responsibility: project management There is only one way to manage knowledge workers: they must manage themselves [W. Humphrey] Team Leader TSP Team Leader Coach Planning Test Manager Ownership Manager Commitment Motivation Implementation Process Performance Manager Manager Design Quality Traditional team Manager ManagerThe leader plans, directs, Support Costumer Interface and tracks the team’s Manager Manager work. Self-directed team All team members participate in planning, managing, and tracking their own work. 59 59
    60. 60. Personal responsibility: quality managementThe only way to build high-quality products in a cost-effectiveway, is by having developers being personally responsible forthe quality of their products.Since defects can best be managed where they are injected,developers should – remove their own defects – determine the causes of their defects – learn to prevent those defects 60 60
    61. 61. Factor III – FeadbackLoops for ContinualImprovement 61 61
    62. 62. Feedback loops: project lifecycle Business Release planning and Launch technical goals Estimates, plans, process, Iteration planning commitment Re-launch Developm Status Lessons, new goals, phase Development phase or cycle new requirements, new orphase cycle risks, etc. or cycle Weekly team meetings Phase or cycle Updated status Retrospective Work products, and plans status, metrics, results Project Retrospective 62 62
    63. 63. Feedback loops: estimating & planning frmwkCostumer Define need requirements Items PROBE method Produce conceptual (PROxy Based Estimating) Tasks design (PBS) Estimate Size size database Estimate resources ProductivityCostumer (effort) database Produce task list, Hist. distrib., distribute effort Process def. Produce Resources Management schedule availableProduct Develop Size, resource, Process Trackingdelivery product schedule data analysis reports 63 63
    64. 64. Feedback loops: quality frameworkLearn from past errors, because we tend to repeat the same errors! Development (and defect Scripts, defect injection) prevention Standards, phase Awareness Defect removal Checklists, phase early defect Test Criteria, … detection & removal Subsequent Defect Process/Data phases late defect Data analysis detection & removal 64 64
    65. 65. Feedback loops: coaching (& leadership)A coach helps improving individual andteam performance through a continuouscoaching/learning cycle External Independent voice 2. Conscious 1. Unconscious Incompetence Incompetence Guidance Next and Stage Feedback 3. Conscious 4. Unconscious Competence Competence Practice 65 65
    66. 66. Example: Historical defect data analysis 66 66
    67. 67. Example: Historical defect data analysis 67 67
    68. 68. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment 68 68
    69. 69. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking 80 – Function Most frequent 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment 69 69
    70. 70. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking Most frequent Most expensive* 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment 70 70
    71. 71. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking Most frequent Most expensive* 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment (*) further analyses showed that they were mainly injected in design and removed in test. 71 71
    72. 72. Example: Historical defect data analysis Defect Types 10 - Documentation 60 – Checking Most frequent Most expensive* 80 – Function 40 - Assignment 70 - Data 50 - Interface 30 - Build 100 - Environment (*) further analyses showed that they were mainly injected in design and removed in test. Improve design review checklists Improve design standards and scripts Produce test spec after design spec and before design review Minimize documentation and use spell checker 72 72
    73. 73. Factor IV - Cost-EffectiveDefect Removal Methods 73 73
    74. 74. Defect removal methods (V-Model)Requirements REQ Team System & ST Spec Inspection Test (ST) High-Level Integration HLD Team Design Test (IT) Inspection & IT Spec Detailed DLD Code Team Design Personal DLD Team Inspection Inspection & UT Spec Review Code Unit Test Code Personal Compile (UT) Review 74 74
    75. 75. Efficiency of defect removal methodsEven experienced developers introduce ~100 defects/KLOCSuch high-number of defects must be removed using the most cost-effective methods, such as personal reviews and team inspections(Continue to use unit, integration and system testing) Minutes to Find and Resolve a Defect by Discovery Activity Source: Jim Sartain, Adobe, 2009 75 75
    76. 76. Use checklists derived from historical defectdata Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 76 76
    77. 77. Use checklists derived from historical defectdataMake the review more effective Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 77 77
    78. 78. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problems Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 78 78
    79. 79. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 79 79
    80. 80. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problems Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 80 80
    81. 81. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 81 81
    82. 82. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist) “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 82 82
    83. 83. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 83 83
    84. 84. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential – keep it simple and short people of 2008” (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 84 84
    85. 85. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential – keep it simple and short people of 2008” – keep it specific for the technology, language, etc. (Time magazin) http://www.youtub e.com/watch?v=YK m8NUmPg58 85 85
    86. 86. Use checklists derived from historical defectdataMake the review more effective – focus the attention on most frequent problemsMake the review more efficient – don’t waste time looking for too rare problemsReduce the risk of missing critical issues Peter Pronovost – even experts benefit from checklists (Dr. Checklist)But: “Most influential – keep it simple and short people of 2008” – keep it specific for the technology, language, etc. (Time magazin) – combine with other analysis techniques http://www.youtub e.com/watch?v=YK m8NUmPg58 86 86
    87. 87. Take enough review timeThe review rate (size reviewed per hour) is one of the bestleading indicators of review effectiveness (review yield)Recommended code review rate: about 200LOC/hourRevewing too slow or too fast is a waste of timeAlso recommended to review in a quiet environment take a break between producing and reviewing 87 87
    88. 88. Use multiple inspectors to improve yield(yield = % of defects detected) 88 88 (source: Introduction to the Team Software Process, Watts Humphrey)
    89. 89. Estimate defects remaining with the capture-recapture methodBased on the degree of overlapping between different reviewers Common defects: C=A B=4 Total defects: T A*B/C = 12 Found Found by A = 8 by B = 6 Remaining defects: R A*B/C-(A+B-C) = 2 http://en.wikipedia.org/wiki/Mark_and_recapture 89 89
    90. 90. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Effort Size (review, rework) (of artifact reviewed) 90 90
    91. 91. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Defect removal rate (indicator of review efficiency) Effort Size (review, rework) (of artifact reviewed) 91 91
    92. 92. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Defect removal rate Defect density (indicator of review (indicator of product quality) efficiency) Effort Size (review, rework) (of artifact reviewed) 92 92
    93. 93. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Defects found (number, type, description, severity) Defect removal rate Defect density (indicator of review (indicator of product quality) efficiency) Effort Size (review, rework) (of artifact reviewed) Review rate (leading indicator of review effectiveness) 93 93
    94. 94. Measure the review process and use data toimprove the reviews Escaped defects (collected later) Review yield (lagging indicator of review effectiveness) Defects found (number, type, description, severity) Defect removal rate Defect density (indicator of review (indicator of product quality) efficiency) Effort Size (review, rework) (of artifact reviewed) Review rate (leading indicator of review effectiveness) 94 94
    95. 95. Factor V - Measurementand Quantitative Methods 95 95
    96. 96. The need for measurement 96 96
    97. 97. The need for measurementIn God we trust, all others bring data.W. Edwards Deming 97 97
    98. 98. The need for measurementIn God we You cant trust, all manage others and bring improve data. what you dont measure.W. Edwards Deming Tom DeMarco 98 98
    99. 99. The need for measurementIn God we You cant When performance is unmeasured or trust, all manage improperly measured, the results are others and often disappointing and can even be bring improve disastrous. Unless your measures cover data. what you all important aspects, you will likely dont motivate counterproductive action. measure.W. Edwards Deming Tom DeMarco Watts Humphrey 99 99
    100. 100. Base measuresIn the TSP, 4 core measures are the basis for quantitative project management, quality management, and process improvementat the personal, team and organization level. Size Effort Actual and Planned Quality Schedule 100 100
    101. 101. Defect trackingDefects are the measure of quality in the TSP.Any change to an interim or final work product, made toensure proper subsequent utilization is considered a defect. 101 101
    102. 102. Quality planning: Defect injection andremoval plan Removed• Defects injected based on historical defects injected per size unit• Yields (% of defects removed) based on historical data or benchmarks 102 102
    103. 103. Quality tracking: Defect removal profileThe defect removal profile shows – plan and actual defects removed by phase – early vs. late defect removal plan Defects Removed by Phase for Assembly SYSTEM 900.0 800.0Defects Removed by Phase 700.0 600.0 500.0 Plan 400.0 Actual 300.0 200.0 100.0 0.0 n st le st n n n e st w w ti o tio tio Te tio od pi Te ie Te ie ec om ec ec ev ec C ev n t em sp ni io sp sp sp R R C U at In LD In In In st e gr od Sy EQ LD LD e D te od C In R H D C d an i ld Bu Phase 103 103
    104. 104. Quality tracking: Defect removal profileThe defect removal profile shows – plan and actual defects removed by phase – early vs. late defect removal plan Defects Removed by Phase for Assembly SYSTEM 900.0 800.0Defects Removed by Phase 700.0 600.0 500.0 Plan 400.0 Actual 300.0 200.0 100.0 Typical 0.0 software n st le st n n n e st w w ti o tio tio Te tio od pi Te ie Te ie ec project om ec ec ev ec C ev n t em sp ni io sp sp sp R R C U at In LD In In In st e gr od Sy EQ LD LD e D te od C In R H D C d an i ld Bu Phase 104 104
    105. 105. Quality Tracking: Quality profileIt examines Quality Profile for Assembly Common Query Changes (BE)criteria thatare effective Design Qualitypredictors of Design/Code Time 1 Quality Profile for Assembly Common Query Changes (BE) 1system testand post- 0.8release Design/Code Time 0.6component Design Review 1 Code Review Quality 0.4 Qualityquality Design Review Time 0.8 Code Review Time ½ design time 0.6 0.2 ½ coding time 0.4 0 Design Review Time Code Review Time 0.2 Plan 0 Actual Program Quality Code Quality Unit Test Ddefects/KLOC Compile Defects/KLOC 4 10 Unit Test Ddefects/KLOC Compile Defects/KLOC 105 105
    106. 106. Conclusions: TSP Quality principles 106 106
    107. 107. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work. 107 107
    108. 108. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater. 108 108
    109. 109. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater.It is more efficient to prevent defects than to find and fix them. 109 109
    110. 110. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater.It is more efficient to prevent defects than to find and fix them.The right way is always the fastest and cheapest way to do ajob. 110 110
    111. 111. Conclusions: TSP Quality principlesTo produce quality products, developers must feel personallyresponsible for the quality of their products. Superiorproducts are not produced by accident; developers muststrive to do quality work.It costs less to find and fix defects earlier in a process thanlater.It is more efficient to prevent defects than to find and fix them.The right way is always the fastest and cheapest way to do ajob.To maximize productivity, focus on quality first. 111 111
    112. 112. ConclusionsThe TSP approach for producing high-quality software in acost-effective way is based on learning from your errorsand using cost-effective defect removal methodsData shows that TSP teams produce software with very lowdefect density and reduced cost of qualityIf you have the need to develop top quality software, youshould take a look at TSP principles and practices 112 112
    113. 113. ReferencesSoftware Assessments, Benchmarks, and Best Practices, Capers Jones,Addison-Wesley, 2000Winning with Software, Watts Humphrey, 2002Introduction to the Team Software Process, Watts S. Humphrey, 2000TSP: Coaching Development Teams, Watts S. Humphrey, Addison-Wesley,2006PSP: A Self-Improvement Process for Software Engineers, Watts S.Humphrey, 2005“The Economic Impacts of Inadequate Infrastructure for Software Testing”,National Institute of Standards and Technology (NIST), 2002“Inspiring, enabling and driving the Evolution of Quality at Adobe leveragingthe TSP”, Jim Sartain, Senior Director, Quality at Adobe Systems, TSPSymposium 2009“Some Future Software Engineering Opportunities and Challenges”, BarryBoehm, in The Future of Software Engineering, Springer, 2011 113 113
    114. 114. PortugalThank you!Contact information:João Pascoal Faria (*)Departamento de Engenharia InformáticaFaculdade de Engenharia da Universidade do PortoRua Dr. Roberto Frias, s/n 4200-465 Porto, PORTUGALEmail: jpf@fe.up.ptPhone: +351225082134Mobile: +351966494914(*) SEI Qualified PSP Developer, PSP Instructor, TSP Coach
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×