Root Cause Analysis in the         Department of Defense                     February 10th, 2011David J. NichollsDavid.Nic...
AcronymsSME – subject matter expertSEP – systems engineering planDAES – Defense Acquisition Executive SummaryPARCA – P...
PARCA IntroductionPARCA’s institutional role – Provides USD(AT&L) execution-phase situational   awareness of major defens...
PARCA Does NOT . . .Forecast program requirements – Funding requirements – Total Life Cycle CostEvaluate alternative mea...
OverviewPARCA root cause analysisProgress so farAreas to pursue                             4
Root Cause Analysis FunctionsStatutory duties defined in WSARA 09 – Conduct root cause analyses for major defense   acqui...
Analytical Framework                Unrealistic                performance                expectations Unrealistic cost   ...
RCA OperationsPARCA deliberately Analytical Steps limited in organic                      Initial Investigation resources...
Initial InvestigationDefine the cost and/or schedule growthGather documentation – Program content and   guidance – Acqui...
Fact Finding    Interviews     – Program Office and Contractors     – Organizations providing functional       support to...
AnalysisExact analytical methods are case dependentBlend of quantitative and                                     Product...
ProductsOne-page memorandum for Nunn-McCurdy certification process – Describes breach and its root causes – Supporting do...
OverviewPARCA root cause analysisProgress so farAreas to pursue                             12
Root Cause Analyses in 2010 Performed six root cause analyses as  part of Nunn-McCurdy breach process  – F-35            ...
PreliminaryTrendsInception IssuesUnrealistic cost or schedule estimates    X        X       X   X X XImmature technology, ...
ObservationsInception IssuesUnrealistic cost or schedule estimates   X   X       X   X X XImmature technology, excessivema...
ObservationsExecution issuesChanges in procurement quantity           X X X       X X XInadequate funding/ funding instabi...
Assessing Management An important evaluation  – Assessed to be a significant factor in    several cases  – Area for “acti...
Defining Systems & InterfacesComplex process which demands SME input Poorly developed or defined            Insufficien...
Effective Contracting StrategyTwo elements – Blocking and tackling – Incentive strategy effectivenessIncentive evaluatio...
Situational AwarenessTailored metrics – Relevant metrics e.g. identification AND tracking of “bets” – Effective metrics e...
OverviewPARCA root cause analysisProgress so farAreas to pursue                             21
Ongoing DevelopmentAnalytical methodologies – Impact of funding constraints in   development – Evaluation of incentive st...
Policies, Procedures,                      and GuidanceState of play –WSARA 09 requires PARCA to address – Virtually no c...
Concluding ThoughtsIndependenceAnalysis transparently based on factsClear and accurate definition of the predominant is...
Upcoming SlideShare
Loading in …5
×

Nichols.david

13,598 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
13,598
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Nichols.david

  1. 1. Root Cause Analysis in the Department of Defense February 10th, 2011David J. NichollsDavid.Nicholls@osd.mil 0
  2. 2. AcronymsSME – subject matter expertSEP – systems engineering planDAES – Defense Acquisition Executive SummaryPARCA – Performance Assessments and Root Cause AnalysesWSARA – Weapon System Acquisition Reform ActMGB – Milestone B 1
  3. 3. PARCA IntroductionPARCA’s institutional role – Provides USD(AT&L) execution-phase situational awareness of major defense acquisition programs – Performs forensics for troubled programs – Reports activities annually to the four defense committeesThree functions of PARCA office – “Performance Assessment”; function in statute – “Root Cause Analysis”; function in statute – “EVMS”; function not in statute 2
  4. 4. PARCA Does NOT . . .Forecast program requirements – Funding requirements – Total Life Cycle CostEvaluate alternative means to execute – Acquisition strategies – Contracting terms / incentivesCompare alternative means to achieve an end – Evaluations of alternate approaches – Cost-effectiveness Necessary for independence 3
  5. 5. OverviewPARCA root cause analysisProgress so farAreas to pursue 4
  6. 6. Root Cause Analysis FunctionsStatutory duties defined in WSARA 09 – Conduct root cause analyses for major defense acquisition programs  As part of the Nunn-McCurdy breach certification process  When requested by designated officials. – Issue policies, procedures, and guidance governing the conduct of root cause analyses.Identification of lessons learned for the benefit of acquisition community 5
  7. 7. Analytical Framework Unrealistic performance expectations Unrealistic cost M a or schedule Inadequate risk n estimates assessment a Cost, g schedule and Any other e matters Unanticipated m performance e impact technological or n manufacturing issues t Quantity change Funding instability or inadequacyProblems will occur: why they occur and our response are crucial issues for root cause analysis 6
  8. 8. RCA OperationsPARCA deliberately Analytical Steps limited in organic Initial Investigation resourcesDepends on other Fact Finding OSD organizations for Analysis functional expertiseDepends on FFRDCs Products and academia for analytical support 7
  9. 9. Initial InvestigationDefine the cost and/or schedule growthGather documentation – Program content and guidance – Acquisition documents – General backgroundProgram chronologyChange analysisPrepare for fact finding Define the problem 8
  10. 10. Fact Finding Interviews – Program Office and Contractors – Organizations providing functional support to program Analysis of performance reporting data – EVMS – Cost reporting – Performance metrics Program documentation, studies, reportsComprehend perspectives 9
  11. 11. AnalysisExact analytical methods are case dependentBlend of quantitative and Product qualitative (but fact based) methodologies ProgrammaticDecision-maker concerns – Actionability Acquisition System – Exogenous vs endogenous – Effectiveness Develop and evaluate hypotheses 10
  12. 12. ProductsOne-page memorandum for Nunn-McCurdy certification process – Describes breach and its root causes – Supporting document in the program certification package sent to Congress – Briefing summarizing resultsLessons for acquisition community – Reports – Analytical toolsContribution for PARCA’s annual report to Congress 11
  13. 13. OverviewPARCA root cause analysisProgress so farAreas to pursue 12
  14. 14. Root Cause Analyses in 2010 Performed six root cause analyses as part of Nunn-McCurdy breach process – F-35 – RMS – DDG-1000 – Apache Block III – ATIRCM/CMWS – WGS Seven root cause analyses are ongoing – Some “on request” – Variety of programsExpect about 20 root cause analyses per year 13
  15. 15. PreliminaryTrendsInception IssuesUnrealistic cost or schedule estimates X X X X X XImmature technology, excessivemanufacturing, integration risk XUnrealistic performance expectations XExecution issuesChanges in procurement quantity X X X X X XInadequate funding/ funding instabilityUnanticipated design, engineering,manufacturing or technology issues X X XOther X XPoor Performance X X X X 14
  16. 16. ObservationsInception IssuesUnrealistic cost or schedule estimates X X X X X XImmature technology, excessivemanufacturing, integration risk XUnrealistic performance expectations XMost likely problem “to be discovered” by program is an unrealistic cost or schedule estimateOther factors are apparently not common or perhaps need better tools to detect Problems to be discovered in execution 15
  17. 17. ObservationsExecution issuesChanges in procurement quantity X X X X X XInadequate funding/ funding instabilityUnanticipated design, engineering,manufacturing or technology issues X X XOther X XQuantity changes a common factor but usually accompanied by (or caused by) other factorsPrograms likely to encounter the unexpectedSuprisingly, funding has not been a major factor Quantity change as a cause must be questioned 16
  18. 18. Assessing Management An important evaluation – Assessed to be a significant factor in several cases – Area for “actionable” root causes M a A difficult area to evaluate n a – Quantitative impact of poor mgt? g Cost, and e schedule – Likely contentious m impact e Approach is to evaluate pieces n t – Defining system and interfaces – Appropriate contracting strategy – Maintaining situational awarenessKey evaluation that includes contractor, PMO, and governance 17
  19. 19. Defining Systems & InterfacesComplex process which demands SME input Poorly developed or defined  Insufficient or incomplete SEP requirements  Lack of a well developed Integrated Undefined dependencies Master Schedule (IMS) Insufficient interoperability  Program Office defaults planning control/responsibility to contractor Requirements instability  Reliance on schedule over Lack of stakeholder involvement performance Poorly developed or under  Poor integration test planning specified MS exit/entrance criteria  Lack of integrated planningTwo key questions – Does Government know what it wants/able to define it ? – Is schedule based upon engineering realities? 18
  20. 20. Effective Contracting StrategyTwo elements – Blocking and tackling – Incentive strategy effectivenessIncentive evaluation – Aligned with program goals and challenges – Demanding yet achievable – Sufficient to motivate – No perverse effects – Consistent with overall incentives – Signal sent and received 19
  21. 21. Situational AwarenessTailored metrics – Relevant metrics e.g. identification AND tracking of “bets” – Effective metrics e.g. avoiding integration effectsEarned Value Management – Compliance – Effectiveness PA Situational awarenessIntegrated with PARCA’s program assessments Tailored RCA – Semi-annual, statutory reviews after concerns critical Nunn-McCurdy breach – As part of DAES process 20
  22. 22. OverviewPARCA root cause analysisProgress so farAreas to pursue 21
  23. 23. Ongoing DevelopmentAnalytical methodologies – Impact of funding constraints in development – Evaluation of incentive strategies – Systems engineering assessment – EV metrics and data presentationRelationships – SME sources for specific issues – Knowledge-sharing of analytical approaches e.g. UK MoD – Effective use of root cause analysis results – Effective mechanisms for sharing lessons for benefit of acquisition community 22
  24. 24. Policies, Procedures, and GuidanceState of play –WSARA 09 requires PARCA to address – Virtually no current guidance for root cause analysis – Other organizations have done some root cause analysis – Key questions exist e.g. is it enough to say that estimate was unrealistic (WSARA 09 category) or should we ask why?Three different purposes – Documentation of PARCA processes – Service conduct of root cause analyses – Program Office conduct of root cause analyses Documentation of organizational lessons 23
  25. 25. Concluding ThoughtsIndependenceAnalysis transparently based on factsClear and accurate definition of the predominant issues – Actionable – Relevant trendsTimely results A commitment to not repeating mistakes 24

×