Your SlideShare is downloading. ×
0
3 CMMI® Views<br />Rick Hefner<br />Director, Process Assurance<br />Northrop Grumman Corporation<br />Southern California...
Background<br />Many published results show improved cost and schedule performance from adopting CMMI®<br />Despite these ...
Agenda<br />Underlying CMMI® Principles<br />CMMI®relationship to productivity, predictability and speed<br />Does CMMI Be...
Projects Have Historically Suffered from Mistakes<br />People-Related Mistakes<br /> 1. Undermined motivation<br /> 2. Wea...
 15% failures
 51% overruns</li></ul>Reference: Steve McConnell, Rapid Development<br />4<br />
Many Approaches to Solving the Problem<br />Which weaknesses are causing my problems?<br />Which strengths may mitigate my...
Approaches to Process Improvement<br />Data-Driven (e.g., Six Sigma, Lean)<br />Clarify what your customer wants (Voice of...
What Is the CMMI® Trying to Achieve?<br />A model is a simplified representation of the world. Capability Maturity Models ...
How Do Mature Processes Help?<br />Process maturity gets at one source of the problem, e.g., <br />Are we using proven ind...
Two-fold increase in productivity</li></ul>8<br />
Agenda<br />Underlying CMMI® Principles<br />Does CMMI Benefit the Customer?<br />Cost of implementing CMMI-compliant proc...
CMMI® Provides Several Related Benefits<br />10<br />Project Performance<br />Organizational Performance<br />Institutiona...
Project Performance<br />CMMI<br />Project performance problems often arise because of incomplete or unrealistic planning<...
Project managers (not organizational staff) must be responsible for implementing the improved processes
Demand realistic, data-driven estimates</li></ul>11<br />
Organizational Performance<br />CMMI<br />Each project’s processes are unique<br />Personnel must re-learn with each proje...
Capture and use historical data (measurement repository)
Capture and share project documents (process asset library)</li></ul>12<br />
Rework/Quality<br />CMMI<br />Focus on “faster and cheaper” leads to skipping of essential steps<br />Key steps are not ob...
Measure the cost of finding and fixing defects
Invest time in learning different methods of peer review and when each is effective</li></ul>13<br />
Institutionalization<br />CMMI<br />Some improvement efforts focus on quick fixes<br />Driven by yearly budget cycles<br /...
Senior management must demand that everyone follow the new processes
QA can be the organization’s strongest tool  – if they are focused!</li></ul>14<br />
Benefits<br />The typical benefits are:<br />Reduced cost<br />Faster schedules<br />Greater productivity<br />Higher qual...
Typical CMMI Benefits Cited in Literature<br />Reduced Costs<br />33% decrease in the average cost to fix a defect (Boeing...
Cost vs. Benefit<br />Both theoretical models and industry data suggests that CMMI-compliant projects achieve a cost reduc...
When Good Organizations Go Bad<br />Some organizations are driven to achieve a maturity level only for it’s marketing valu...
What Does a CMMI Level Guarantee?<br />Decisions made on the basis of maturity level ratings are only valid if the ratings...
How Does Level 4 & 5 Benefit the Customer?<br />Better Products and Services Produced Faster And Cheaper<br />Rick Hefner,...
The Project Manager’s Dilemma at Level 3<br />I want to use the organization’s standard process, but…<br />… Does it’s per...
 Understanding the Process<br />22<br />Managing by Variation<br />How many errors are typically found in reviewing an int...
Typical Choices in Industry<br />Most customers care about:<br />Delivered defects<br />Cost and schedule<br />So organiza...
What Can a Level 4 Organization Do?<br />Determine whether processes are behaving consistently or have stable trends (i.e....
Lessons Learned<br />Based on over 20 Northrop Grumman CMMI Level 5 organizations<br />Six Sigma is an enabler for higher ...
Agenda<br />Underlying CMMI® Principles<br />Does CMMI Benefit the Customer?<br />How Projects Fail<br />Start up problems...
Where Could Problems Arise?<br />The projects within the organization may not live up to the capability<br />Start-up prob...
The First Three Months: Essential Project Start-Up Activities<br />Many process-related problems arise in the first few mo...
Preventing Back-Sliding<br />29<br />The CMMI generic practices ensure that processes are institutionalized – sustained ov...
Sampling Bias<br />	The size and number of instantiations investigated should be selected to form a valid sample of the or...
Organizational Sampling<br />An organization with 50 projects at multiple sites may select 4-5 sample projects<br />Are th...
Appraisal Inaccuracies<br />Methodology<br />SCAMPI A appraisals are the only approach that provides benchmark quality app...
Fiction or Non-Fiction:  How to Read Appraisal Results for Fun and Profit<br />	The ADS is a summary statement describing ...
How to Write a Better RFP<br />Acquirers seeking to ensure that the proposed project will implement mature practices shoul...
Agenda<br />Underlying CMMI® Principles<br />Does CMMI Benefit the Customer?<br />How Projects Fail<br />How to Get Contra...
Background<br />There is a marked difference between organizations that truly want to implement CMMI®, and those who simpl...
Where Could Problems Arise?<br />	Assuming the contractor’s CMMI® rating is accurate, and applicable to the team doing the...
Areas Outside of the CMMI®<br />38<br />Process<br />Technology<br />Domain-specific<br />Maturity<br />Tools<br />People<...
Top Five System Engineering Issues<br />Lack of awareness of the importance, value, timing, accountability, and organizati...
Top Software Engineering Issues<br />The impact of requirements upon software is not consistently quantified and managed i...
Start-Up Issues<br />Project Planning starts in the proposal phase, is refreshed at contract start, and re-occurs througho...
42<br />CMMI® Project Planning - Goal 1<br />
43<br />CMMI® Project Planning - Goal 2<br />
Upcoming SlideShare
Loading in...5
×

Applicability of CMMI for Small to Medium Enterprises

1,852

Published on

There are many reasons why CMMI is difficult to implement in small organizations and small projects -- the fixed costs of establishing the necessary infrastructure; the large number of roles which must be filled by a limited number of people; the quantity of information that must be absorbed to properly interpret the model. Similar problems are experienced when applying the CMMI to short duration projects.
This tutorial will outline the challenges in applying CMMI in small settings, and present practical strategies for overcoming them. Specific techniques for infrastructure, adoption, and appraisals will be presented. In addition, a guide for interpreting each CMMI practice in small settings will be provided.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,852
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
93
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Applicability of CMMI for Small to Medium Enterprises"

  1. 1. 3 CMMI® Views<br />Rick Hefner<br />Director, Process Assurance<br />Northrop Grumman Corporation<br />Southern California SPIN<br />4 June 2010<br />
  2. 2. Background<br />Many published results show improved cost and schedule performance from adopting CMMI®<br />Despite these results, there is still community debate over the value of CMMI®, and whether CMMI® ratings provide sufficient guarantees of program performance.<br />This program will explore three factors contributing to the confusion:<br />Inaccurate CMMI® ratings<br />Over-estimating the benefits that CMMI® provides a customer<br />Contractors not living up to their CMMI® rating <br />2<br />
  3. 3. Agenda<br />Underlying CMMI® Principles<br />CMMI®relationship to productivity, predictability and speed<br />Does CMMI Benefit the Customer?<br />How Projects Fail<br />How to Get Contractors to Live Up to Their CMMI Ratings<br />3<br />SM SCAMPI, SCAMPI Lead Appraiser, and SEI are service marks of Carnegie Mellon University. <br />® Capability Maturity Model Integration and CMMI® are registered in the U.S. Patent & Trademark Office.<br />
  4. 4. Projects Have Historically Suffered from Mistakes<br />People-Related Mistakes<br /> 1. Undermined motivation<br /> 2. Weak personnel<br /> 3. Uncontrolled problem<br /> employees<br /> 4. Heroics<br /> 5. Adding people to a late project<br /> 6. Noisy, crowded offices<br /> 7. Friction between developers<br /> and customers <br /> 8. Unrealistic expectations <br /> 9. Lack of effective project<br /> sponsorship <br />10. Lack of stakeholder buy-in <br />11. Lack of user input <br />12. Politics placed over substance <br />13. Wishful thinking <br />Process-Related Mistakes<br />14. Overly optimistic schedules <br />15. Insufficient Risk Management<br />16. Contractor failure Insufficient<br />planning <br />17. Abandonment of planning<br />under pressure <br />18. Wasted time during the fuzzy front end <br />19. Shortchanged upstream<br />activities <br />20. Inadequate design <br />21. Shortchanged quality<br />assurance <br />22. Insufficient management<br />controls <br />23. Premature or too frequent<br />convergence <br />25. Omitting necessary tasks from estimates <br />26. Planning to catch up later<br />27. Code-like-hell programming <br />Product-Related Mistakes<br />28. Requirements gold-plating <br />29. Feature creep <br />30. Developer gold-plating <br />31. Push me, pull me negotiation<br />32. Research-oriented<br />development <br />Technology-Related Mistakes<br />33. Silver-bullet syndrome <br />34. Overestimated savings from<br />new tools or methods <br />35. Switching tools in the middle<br />of a project <br />36. Lack of automated<br />source-code control<br />Standish Group survey of 13,000 projects (2003)<br /><ul><li> 34% successes
  5. 5. 15% failures
  6. 6. 51% overruns</li></ul>Reference: Steve McConnell, Rapid Development<br />4<br />
  7. 7. Many Approaches to Solving the Problem<br />Which weaknesses are causing my problems?<br />Which strengths may mitigate my problems?<br />Which improvement investments offer the best return?<br />People<br />Business<br />Environment<br />Management<br />Structure<br />Product<br />Tools<br />Methods<br />One solution!<br />Technology<br />Process<br />5<br />
  8. 8. Approaches to Process Improvement<br />Data-Driven (e.g., Six Sigma, Lean)<br />Clarify what your customer wants (Voice of Customer)<br />Critical to Quality (CTQs)<br />Determine what your processes can do (Voice of Process)<br />Statistical Process Control<br />Identify and prioritize improvement opportunities<br />Causal analysis of data<br />Determine where your customers/competitors are going (Voice of Business)<br />Design for Six Sigma<br />Model-Driven (e.g., CMM®, CMMI®)<br />Determine the industry best practice<br />Benchmarking, models<br />Compare your current practices to the model<br />Appraisal, education<br />Identify and prioritize improvement opportunities<br />Implementation<br />Institutionalization<br />Look for ways to optimize the processes<br />6<br />
  9. 9. What Is the CMMI® Trying to Achieve?<br />A model is a simplified representation of the world. Capability Maturity Models (CMM®s) contain the essential elements of effective processes for one or more bodies of knowledge. These elements are based on the concepts developed by Crosby, Deming, Juran, and Humphrey. -Introduction, CMMI®<br />CMMI® provides a model of industry best practices<br />Following these practices has shown to produce software and systems faster, better, and cheaper, when properly applied<br />The main benefits cited by CMMI® users are:<br />More predictable adherence to budgets and schedules<br />Reduced re-work (which can reduce cost and schedule)<br />Reduced risk<br />7<br />
  10. 10. How Do Mature Processes Help?<br />Process maturity gets at one source of the problem, e.g., <br />Are we using proven industry practices?<br />Does the staff have the resources needed to execute the process?<br />Is the organization providing effective project support?<br />The main benefits typically seen are:<br />Improved predictability of project budgets and schedules<br />Improved management awareness of problems<br />Reduced re-work, which improves predictability, cost, and schedule<br />J. Herbsleb and D. Zubrow, “Software Process Improvement: An Analysis of Assessment Data and Outcomes”<br />13 organizations<br />ROI of 4:1 to 9:1<br />Improved quality, error rates, time to market, productivity<br />R. Dion, “Process Improvement and the Corporate Balance Sheet”<br /><ul><li>ROI of 7.7:1: Reduced re-work, improved quality
  11. 11. Two-fold increase in productivity</li></ul>8<br />
  12. 12. Agenda<br />Underlying CMMI® Principles<br />Does CMMI Benefit the Customer?<br />Cost of implementing CMMI-compliant processes<br />Timelines for impacting program performance<br />Practical tips and techniques for realizing the benefits<br />How Projects Fail<br />How to Get Contractors to Live Up to Their CMMI Ratings<br />9<br />SM SCAMPI, SCAMPI Lead Appraiser, and SEI are service marks of Carnegie Mellon University. <br />® Capability Maturity Model Integration and CMMI® are registered in the U.S. Patent & Trademark Office.<br />
  13. 13. CMMI® Provides Several Related Benefits<br />10<br />Project Performance<br />Organizational Performance<br />Institutionalization<br />Quality/Rework<br />Rick Hefner, “Achieving the Promised Benefits of CMMI,” CMMI Technology Conference & User Group, Denver, CO, 14-17 Nov 2005<br />
  14. 14. Project Performance<br />CMMI<br />Project performance problems often arise because of incomplete or unrealistic planning<br />Forgotten activities<br />Unconscious decisions<br />Overly-optimistic estimates<br />When cost/schedule pressure arises, people abandon the plans, leading to more problems<br />Individual judgment versus best use of resources<br />Identifies the elements of good planning<br />Proven engineering processes<br />Estimates based on historical data, using these processes<br />When cost/schedule pressure arises, CMMI®practices track and correct<br />Reactive (L2)<br />Proactive, risk management (L3)<br />Quantitative management (L4)<br />QA, management ensures processes/plans are followed<br /><ul><li>Train project managers on how to use the tools (estimation, earned value, risk management)
  15. 15. Project managers (not organizational staff) must be responsible for implementing the improved processes
  16. 16. Demand realistic, data-driven estimates</li></ul>11<br />
  17. 17. Organizational Performance<br />CMMI<br />Each project’s processes are unique<br />Personnel must re-learn with each project<br />Difficulty moving people from project to project<br />Historical data of little use in estimation<br />No way to compare project-to-project<br />Which process was best?<br />What did we learn?<br />Standard organizational process, tailored to fit each project<br />Can be documented, trained, supported by templates<br />Over time, people learn the process<br />Common processes/measures allow better use of historical data<br />Calibrate cost estimation models<br />Project to project comparisons<br />Over time, the organization can optimize the process<br /><ul><li>Develop an organizational process(es) which fits the full range of your projects (small/large, all life cycles and project types)
  18. 18. Capture and use historical data (measurement repository)
  19. 19. Capture and share project documents (process asset library)</li></ul>12<br />
  20. 20. Rework/Quality<br />CMMI<br />Focus on “faster and cheaper” leads to skipping of essential steps<br />Key steps are not obvious, often counter intuitive<br />Fixing latent defects often accounts for 30-40% of project cost<br />The cost of defects (rework) is seldom measured<br />A disciplined engineering and management process<br />Do it right the first time<br />CMMI identifies the essential steps<br />Peer reviews find defects early, where it is cost effective to fix them<br />Requirements, designs, code, plans, etc.<br />Often more efficient and effective than testing<br />Many types (Fagan inspections, walkthroughs, desk checks, etc.)<br /><ul><li>Focus on eliminating defects, not on faster and cheaper
  21. 21. Measure the cost of finding and fixing defects
  22. 22. Invest time in learning different methods of peer review and when each is effective</li></ul>13<br />
  23. 23. Institutionalization<br />CMMI<br />Some improvement efforts focus on quick fixes<br />Driven by yearly budget cycles<br />Expectation that results will be immediate<br />It is tempting to reduce overhead to reduce cost<br />Training<br />Staff support to projects<br />Use of outside process experts<br />Short-term investment for long-term gain<br />Initial investment in the cost of change, learning curve, new overhead structures<br />Long-term benefits in increased productivity<br />Organizational infrastructure exists to support the policies and process<br />Measurement repositories<br /><ul><li>Expect 18-24 months before benefits begin to be realized
  24. 24. Senior management must demand that everyone follow the new processes
  25. 25. QA can be the organization’s strongest tool – if they are focused!</li></ul>14<br />
  26. 26. Benefits<br />The typical benefits are:<br />Reduced cost<br />Faster schedules<br />Greater productivity<br />Higher quality<br />Increased customer satisfaction<br />Over 40 published studies on the benefits of SW-CMM®<br />DoD DACS website: http://www.thedacs.com/databases/roi/<br />Similar results starting to be seen for CMMI®<br />“Demonstrating the Impact and Benefits of CMMI: An Update and Preliminary Results,” Software Engineering Institute, CMU/SEI-2003-SR-009, Oct 2003<br />http://www.sei.cmu.edu/cmmi/results/results-by-category.html<br />15<br />
  27. 27. Typical CMMI Benefits Cited in Literature<br />Reduced Costs<br />33% decrease in the average cost to fix a defect (Boeing)<br />20% reduction in unit software costs (Lockheed Martin)<br />Reduced cost of poor quality from over 45 percent to under 30 percent over a three year period (Siemens)<br />10% decrease in overall cost per maturity level (Northrop Grumman) <br />Faster Schedules<br />50% reduction in release turnaround time (Boeing)<br />60% reduction in re-work following test (Boeing)<br />Increase from 50% to 95% the number of milestones met (General Motors)<br />Greater Productivity<br />25-30% increase in productivity within 3 years (Lockheed Martin, Harris, Siemens)<br />Higher Quality<br />50% reduction of software defects (Lockheed Martin)<br />Customer Satisfaction<br />55% increase in award fees (Lockheed Martin)<br />16<br />
  28. 28. Cost vs. Benefit<br />Both theoretical models and industry data suggests that CMMI-compliant projects achieve a cost reduction of 10% per level, i.e., Level 3 is 20% cheaper than Level 1<br />The key is reducing rework<br />Knox Model – Theoretical Benefits<br />17<br />COCOMO predicts similar benefits based on current industry data<br />
  29. 29. When Good Organizations Go Bad<br />Some organizations are driven to achieve a maturity level only for it’s marketing value<br />Focus on passing the appraisal, not understanding and deciding among possible interpretations<br />Improvement goals are not set realistically (“Level 5 in ’05”)<br />Practitioners/customers perceive CMMI as more expensive<br />Only some of the projects participate in the improvement effort<br />The remaining projects don’t implement<br />Only some of the projects get appraised<br />People don’t learn or become proficient in the new behaviors<br />Insufficient resources (e.g., training, QA, metrics, consultants)<br />Benefits are not realized because projects do not start up effectively<br />Management doesn’t enforce using processes on new programs<br />Rick Hefner, “CMMI Horror Stories: When Good Projects Go Bad,” Software Engineering Process Group Conference , 6-9 March 2006<br />18<br />
  30. 30. What Does a CMMI Level Guarantee?<br />Decisions made on the basis of maturity level ratings are only valid if the ratings are based on known criteria.<br /><ul><li>SCAMPI A Method Description Document</li></ul>A CMMI appraisal indicates the organization’s capacity to perform the next project, but cannot guarantee that each new project will perform in that way<br />The CMMI methodology assumes the organization will propagating their processes to every new project <br />An organization that gets appraised solely to demonstrate a maturity level might not have that intent<br />Organizations may not have developed the skills to roll out their processes effectively<br />A CMMI appraisal judges the maturity of the organization’s processes – based upon the projects sampled<br />New projects must embrace the new processes<br />19<br />
  31. 31. How Does Level 4 & 5 Benefit the Customer?<br />Better Products and Services Produced Faster And Cheaper<br />Rick Hefner, “How Does High Maturity Benefit the Customer?,” Systems & Software Technology Conference, 18-22 April 2005<br />20<br />
  32. 32. The Project Manager’s Dilemma at Level 3<br />I want to use the organization’s standard process, but…<br />… Does it’s performance and quality meet my customer’s expectations?<br />… If not, how should I tailor the process?<br />21<br />
  33. 33. Understanding the Process<br />22<br />Managing by Variation<br />How many errors are typically found in reviewing an interface specification?<br />Useful in evaluating future reviews<br />Was the review effective?<br />Was the process different?<br />Is the product different?<br />Average<br />Expected<br />Variation<br />Corrective and preventative actions<br />
  34. 34. Typical Choices in Industry<br />Most customers care about:<br />Delivered defects<br />Cost and schedule<br />So organizations try to predict:<br />Defects found throughout the lifecycle<br />Effectiveness of peer reviews, testing<br />Cost achieved/actual (Cost Performance Index – CPI)<br />Schedule achieved/actual (Schedule Performance Index – SPI)<br />23<br />
  35. 35. What Can a Level 4 Organization Do?<br />Determine whether processes are behaving consistently or have stable trends (i.e., are predictable)<br />Identify processes where the performance is within natural bounds that are consistent across process implementation teams<br />Establish criteria for identifying whether a process or process element should be statistically managed, and determine pertinent measures and analytic techniques to be used in such management<br />Identify processes that show unusual (e.g., sporadic or unpredictable) behavior<br />Identify any aspects of the processes that can be improved in the organization's set of standard processes<br />Identify the implementation of a process which performs best<br />24<br />
  36. 36. Lessons Learned<br />Based on over 20 Northrop Grumman CMMI Level 5 organizations<br />Six Sigma is an enabler for higher maturity<br />Focus on data, measurement systems, process improvement<br />Tying improvements to business goals<br />Tools and methods support the Level 4/5 analysis tasks <br />Level 3 metrics, measurement processes, and goal setting are generally inadequate for Levels 4 and 5<br />Better definitions of the measures<br />Lower level metrics of lower level subprocesses<br />Having all the tools at Level 5 gives you the insight to manage each project the way the customer needs it to be managed<br />25<br />
  37. 37. Agenda<br />Underlying CMMI® Principles<br />Does CMMI Benefit the Customer?<br />How Projects Fail<br />Start up problems<br />Appraisal inaccuracies<br />How to Get Contractors to Live Up to Their CMMI Ratings<br />26<br />
  38. 38. Where Could Problems Arise?<br />The projects within the organization may not live up to the capability<br />Start-up problems with planning, subcontractors, and infrastructure<br />Problems with staffing, either as the prime or with subcontractors<br />Differences in domain experience<br />Back-sliding<br />The appraisal results may not be an accurate reflection of the organization’s capability<br />Sampling bias<br />Appraisal inaccuracies<br />Organization’s inability to immediately apply their appraised processes<br />Note that all of these problems are equally possible with both the staged and continuous representations<br />27<br />
  39. 39. The First Three Months: Essential Project Start-Up Activities<br />Many process-related problems arise in the first few months of a project<br />New relationships are established<br />Personnel changes and shortfalls<br />Pressure to produce quickly<br />Gaps between the planned processes and what was bid<br />If a project is going to live up to the organization’s process capability, it is essential to fully implement the processes from the beginning<br />Processes should be defined during the proposal, by tailoring the organization’s standard process<br />Estimates should be based on historical data from the organization’s measurement repository<br />Process assets (e.g., templates) should support detailed planning to ensure consistency with the organization’s best practices<br />Evidence reviews should be used to ensure CMMI compliance<br />28<br />
  40. 40. Preventing Back-Sliding<br />29<br />The CMMI generic practices ensure that processes are institutionalized – sustained over time<br />The approach for implementing the generic practices must reflect:<br />Efficiency<br />Effectiveness<br />Applicability to ALL projects<br />Frequent appraisals can be used to assess the effectiveness of the institutionalization<br />Geoff Draper and Rick Hefner, “Applying CMMI Generic Practices with Good Judgment,” SEPG Conference, 2004.<br />
  41. 41. Sampling Bias<br /> The size and number of instantiations investigated should be selected to form a valid sample of the organizational unit to which the results will be attributed.<br />- SCAMPI A Method Description Document<br />The Lead Appraiser is permitted to select sample projects as “representative” of the organization as a whole<br />Little guidance in the MDD<br />Wide variation among Lead Appraisers<br />If an organization is only interested in a good appraisal result, they will appraise large organizations with a handful of samples, and/or exclude/hide inferior projects <br />This potential abuse exist with both staged and continuous representations<br />The remaining projects don’t implement<br />Only some of the projects get appraised<br />30<br />
  42. 42. Organizational Sampling<br />An organization with 50 projects at multiple sites may select 4-5 sample projects<br />Are the appraisal results representative of the organization?<br />Sampled projects<br />31<br />
  43. 43. Appraisal Inaccuracies<br />Methodology<br />SCAMPI A appraisals are the only approach that provides benchmark quality appraisal results<br />SCAMPI B, C, and other appraisal methods may be useful, but they are not designed to provide the same accuracy<br />Appraiser Skill<br />There is wide variation in appraiser skill, experience and insight<br />Although appraisal experience is a crucial contributor to accuracy, the appraisal methods do little to ensure sufficient experience<br />There is also wide variation in how the model is interpreted<br />Appraiser Independence<br />Appraiser independence in needed to ensure unbiased results<br />It is difficult to establish a completely independent situation <br />32<br />
  44. 44. Fiction or Non-Fiction: How to Read Appraisal Results for Fun and Profit<br /> The ADS is a summary statement describing the appraisal results that includes the conditions and constraints under which the appraisal was performed. It contains information considered essential to adequately interpret the meaning of assigned maturity level or capability level ratings.<br />- SCAMPI A Method Description Document<br />The Appraisal Disclosure Statement (ADS) provides keys to assessing an appraisal’s accuracy<br />Organizational unit appraised (the unit to which the ratings are applicable and the domains examined)<br />Appraisal team leader and appraisal team members and their organizational affiliations<br />Process areas rated and process areas not rated<br />Dates of on-site activity<br />Not included - sampling approach or percentage of projects sampled<br />33<br />
  45. 45. How to Write a Better RFP<br />Acquirers seeking to ensure that the proposed project will implement mature practices should request the following:<br />SCAMPI A Appraisal Disclosure Statement<br />Organizational unit appraised <br />Appraisal team leader affiliation <br />Process areas rated and not rated<br />Dates of on-site activity<br />Explanation of sampling approach used in appraisal<br />Approach to be used to ensure proper project start-up<br />Data to demonstrate the speed with which new projects adopt and execute the organization’s processes<br />Approach to be used to prevent back-sliding<br />34<br />
  46. 46. Agenda<br />Underlying CMMI® Principles<br />Does CMMI Benefit the Customer?<br />How Projects Fail<br />How to Get Contractors to Live Up to Their CMMI Ratings<br />Contenders and Pretenders <br />35<br />
  47. 47. Background<br />There is a marked difference between organizations that truly want to implement CMMI®, and those who simply want a “certificate”<br />Contenders invest time and energy on understanding the industry best practices in the model, fitting them to their projects and organization, and improving their effectiveness and efficiency<br />Pretenders simply do enough to convince an appraiser to give them the maturity level -- along the way, they de-motivate their staff with bureaucratic processes, disappoint their customers with inconsistent performance, and generally give the model a bad name <br />36<br />
  48. 48. Where Could Problems Arise?<br /> Assuming the contractor’s CMMI® rating is accurate, and applicable to the team doing the work, where could problems arise?<br />Areas outside of the CMMI®<br />Start-up problems<br />Back-sliding<br />37<br />
  49. 49. Areas Outside of the CMMI®<br />38<br />Process<br />Technology<br />Domain-specific<br />Maturity<br />Tools<br />People<br />Domain knowledge<br />Sufficient quantity<br />Motivation<br />
  50. 50. Top Five System Engineering Issues<br />Lack of awareness of the importance, value, timing, accountability, and organizational structure of SE on programs<br />Adequate, qualified resources are generally not available within Government and industry for allocation on major programs<br />Insufficient SE tools and environments to effectively execute SE on programs<br />Requirements definition, development and management is not applied consistently and effectively<br />Poor initial program formulation<br />39<br />“Top Five Systems Engineering Issues In Defense Industry”, NDIA Systems Engineering Division Task Group Report, Jan, 2003<br />
  51. 51. Top Software Engineering Issues<br />The impact of requirements upon software is not consistently quantified and managed in development or sustainment<br />Fundamental system engineering decisions are made without full participation of software engineering.<br />Software life-cycle planning and management by acquirers and suppliers is ineffective.<br />The quantity and quality of software engineering expertise is insufficient to meet the demands of government and the defense industry.<br />Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems.<br />There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments.<br />Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk.<br />40<br />“Top Software Engineering Issues In Defense Industry”, NDIA Systems Engineering Division and Software Committee, Sep 2006<br />
  52. 52. Start-Up Issues<br />Project Planning starts in the proposal phase, is refreshed at contract start, and re-occurs throughout the project lifecycle<br />Contenders extend their CMMI practices to proposal teams and re-planning efforts<br />Pretenders focus on contract start<br />Costs and schedules defined at proposals may be immature and overly-aggressive<br />Re-planning may be ad hoc<br />Mature estimates may also be overruled by business interests<br />41<br />
  53. 53. 42<br />CMMI® Project Planning - Goal 1<br />
  54. 54. 43<br />CMMI® Project Planning - Goal 2<br />
  55. 55. 44<br />CMMI® Project Planning – Goal 3 <br />
  56. 56. Keys to Success<br />Ask suppliers to show how they extend the CMMI practices to proposal activities<br />Request planning documents with the proposal<br />During re-planning, ask suppliers to show how they performed the CMMI practices<br />45<br />
  57. 57. Back-Sliding: A Failure of Institutionalization<br />Institutionalization: The ingrained way of doing business that an organization follows routinely as part of its corporate culture.<br />- CMMI-DEV v1.2<br />GG 2 Institutionalize a Managed Process<br />GP 2.1 Establish an Organizational Policy<br />GP 2.2 Plan the Process<br />GP 2.3 Provide Resources<br />GP 2.4 Assign Responsibility<br />GP 2.5 Train People<br />GP 2.6 Manage Configurations<br />GP 2.7 Identify and Involve Relevant Stakeholders<br />GP 2.8 Monitor and Control the Process<br />GP 2.9 Objectively Evaluate Adherence<br />GP 2.10 Review Status with Higher Level Management<br />GG 3 Institutionalize a Defined Process <br />GP 3.1 Establish a Defined Process<br />GP 3.2 Collect Improvement Information<br />When mentioned in the generic goal and generic practice descriptions, institutionalization implies that the process is ingrained in the way the work is performed and there is commitment and consistency to performing the process.<br />An institutionalized process is more likely to be retained during times of stress.<br />46<br />
  58. 58. Common Features – A Lost Perspective in CMMI ®v1.2!<br />47<br />
  59. 59. Organizational Support<br />Contenders<br />Fully support the CMMI ® -based improvement program by providing training, templates, tools, process assets libraries, measurement repositories and other work aids focused on improving the ability of practitioners to competently adopt the model <br />Pretenders<br />Largely ignore organizational support, often to save money<br />Where required by the model, they establish process asset libraries and measurement repositories, but they are largely shelfware<br />48<br />
  60. 60. 49<br />Organizational Infrastructure Required for CMMI® Level 3<br />Policies, Processes,Templates & Tools<br />Process Group<br />Process Improvement<br />Training Program<br />Best-Practice Libraries<br />Audits & Appraisals<br />Measurement Repositories<br />Predictive Modeling<br />Communications<br />Developing and maintaining mature processes requires significant time and investment in infrastructure<br />
  61. 61. Organizational Culture<br />A pattern of shared basic assumptions that the group learned as it solved its problems of external adaptation and internal integration, that has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way you perceive, think, and feel in relation to those problems. <br />Artifacts<br />The practices that can be observed in such areas as dress code, leadership style, communication processes<br />Espoused values<br />The elements the organization says it believes in, the factors that it says influence the practices in which it engages<br />Basic underlying assumptions<br />Unstated beliefs the organization has come to accept and abide by<br />Organizational Culture & Leadership, Edgar H Schein, used with permission<br />50<br />
  62. 62. Management Commitment and Support<br />Understands the key messages<br />Is willing to take actions to reinforce them<br />Provides resources to support/sustain process improvement efforts<br />Sets expectations that essential project functions will be funded and processes will be followed<br />Project planning, estimation, tailoring, CM, QA, etc.<br />Supports process improvement and sustainment, rather than passing appraisals<br />Rewards mature processes development and sustainment rather than individual heroics<br />51<br />Rick Hefner, “Sustaining CMMI Compliance,” 2006 CMMI Technology Conference and User Group<br />
  63. 63. Keys to Success<br />Ask suppliers to show how they perform the CMMI generic practices<br />When problems occur, ask why the CMMI practices were not effective in sustaining the desired behavior, and what will be done to prevent future problems<br />52<br />
  64. 64. Summary<br />There is a marked difference between organizations that truly want to implement CMMI®, and those who are simply try to get a “certificate”<br />By discussing the differences, we hope to help the CMMI® community the true value of CMMI®<br />53<br />
  65. 65. 54<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×