Challenges in partnering on major researchplatforms and facilities                                      William L. Miller,...
Scope         • High cost and complexity of major projects           have drive partnering between U.S. agencies          ...
Science platforms and facilities: a highly varied class…PMC2010: Partnering challenges on large projects   W. Miller   3
Big science projects engage many stakeholders…                                                     National               ...
Comparison of project practices at DOE, NASA and NSF                                               Organizations and portf...
Comparative Approach•       Framework for agency comparison along many dimensions                Organizational structure...
Comparative anatomy of agency science structures                       $33.8B DOE                              $17.8B NASA...
Development process framework  look at elements Ongoing community input, horizon planning, reprioritization, and portfoli...
Development processes: alignment and terminology                                    Initiation                Definition  ...
Oversight: two modes of assessment                                             Preconstruction                 Conception ...
Review-based oversight and decision/approval                        Independent                          Internal decision...
Baseline establishment – with focus on cost   During construction,   projects performance                                 ...
Funding                         Initiation         Definition                               Execution                     ...
Scope          How do partner processes and practices           align? (Start with agencies)          What are the chall...
Some recent cases of major partnered projects                                                   NASA DOE   NSF   Internati...
Examples of interagency Joint Oversight Groups (JOG)   • DOE-NASA: FERMI, Joint Dark Energy Mission   • NSF-DOE: U.S. Larg...
Partnering  Levels of impedance (mis)matches Capabilities & Practices  Strategic  valuation  Technical  domainsManagement...
Challenges: Strategic Management Technical• Valuation. How highly does each partner prioritize the  project – and how much...
Challenges: Strategic Management Technical• Role of project management. Is project management valued? Are  managers well c...
Challenges: Strategic Management Technical• Complementarity. Synergies among partners in technical  know-how relative to r...
Synchronizing “On-ramps”: when does a project start?                                                                   Pre...
Reported best practices – responses to challenges• Early engagement. Agree on clear goals, timeframe, and effective  membe...
AcronymsCD           Critical Decision (DOE)                 OECM   Office of Engineering and Construction                ...
Upcoming SlideShare
Loading in …5
×

William.miller.pmc2010

14,718 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
14,718
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

William.miller.pmc2010

  1. 1. Challenges in partnering on major researchplatforms and facilities William L. Miller, Ph.D. AAAS Science and Technology Policy Fellow National Science Foundation Used with permission
  2. 2. Scope • High cost and complexity of major projects have drive partnering between U.S. agencies and with international entities.  How do partner processes and practices align? (Start with agencies)  What are the challenges and best practices for partnering?PMC2010: Partnering challenges on large projects W. Miller 2
  3. 3. Science platforms and facilities: a highly varied class…PMC2010: Partnering challenges on large projects W. Miller 3
  4. 4. Big science projects engage many stakeholders… National Priorities Science Technology Enterprise Enterprise Large Science Infrastructure Interagency International Partnerships Cooperation  lots of interest in process, performance and outcomes…PMC2010: Partnering challenges on large projects W. Miller 4
  5. 5. Comparison of project practices at DOE, NASA and NSF Organizations and portfolios studied: DOE Office of Science “Science User Facilities” NASA Science Missions Directorate (SMD) “Robotic Science Missions” NSF Science & Engineering Directorates, OPP “Large Facilities” (MREFCs) • Reviewed key agency guidelines, external reports & analyses… • “Looked under the hood”: ~45 stakeholder interviews; site visits; direct observations… W. Miller, PRECONSTRUCTION PLANNING FOR LARGE SCIENCE INFRASTRUCTURE PROJECTS: A comparative analysis of practices and challenges at DOE, NASA and NSF, November 3, 2009PMC2010: Partnering challenges on large projects W. Miller 5
  6. 6. Comparative Approach• Framework for agency comparison along many dimensions Organizational structures Development processes Oversight and decision/approval Funding and acquisition etc…. • Identify partnering issues along these and other dimensions…PMC2010: Partnering challenges on large projects W. Miller 6
  7. 7. Comparative anatomy of agency science structures $33.8B DOE $17.8B NASA $6.5B NSF NSB Secretary Administrator Director Administration Management Budget, Finance Chief Support Office & Award Mgmt Engineer OECM LFO $4.8B UnderSec $4.5B Science Missions Directorates $5.3B Ofc of Science Directorate & Offices Programs Program Offices OPA Divisions Divisions Divisions Programs SRBs Programs Panels FPDs 42 U.S.C.§1873b Projects Laboratories $2.0B Centers Awardees (“shall not operate laboratories”) Projects Projects Projects “Project assurance” bodies FPD Federal Project Director LFO Large Facilities Office Provides policy OECM Office of Engineering and Construction Mgmt NSB National Science Board OPA Office of Project Assessment S&E Science and engineering Provides independent review SRB Standing Review Board Budgets: FY 2009 plans (no ARRA), from FY2010 RequestsPMC2010: Partnering challenges on large projects W. Miller 7
  8. 8. Development process framework  look at elements Ongoing community input, horizon planning, reprioritization, and portfolio adjustment Early Early acquisitions operations Conceptualization Preconstruction Planning* Final Construction Operations D & Prioritization Conceptual Preliminary design Development Begin major Project Terminate Investment Closeout Facility • Plan Management/governance plans, WBS  assemble project team • Design Goals, requirements  iterative design  bring to readiness • Invest R&D, necessary technologies  bring to readiness • Estimate Effort, cost, schedule, reserves, risks  refine to believability • Govern Progress oversight and decision-making *Also called definition, formulation, front-end planning…PMC2010: Partnering challenges on large projects W. Miller 8
  9. 9. Development processes: alignment and terminology Initiation Definition Execution Pre-conceptual Conceptual Preliminary Trans/Closeout Final Design Construction Operations Planning Design Design DOE-SC CD-0 CD-1 CD-2 CD-3 CD-4 Ref: DOE O 413.3A IPR** CDR EIR FDR ORR/RA CDR Conceptual Design Review PDR IPR/EIR** EIR External Independent Review (OECM) Critical Decision (CD) approvals FDR Final Design Review CD-0 Approve mission need IPR Independent Project Review (SC) CD-1 Approve Alternatives selection & cost range ORR Operations Readiness Review CD-2 Approve Performance baseline PDR Preliminary Design Review CD-3 Approve Construction start RA Readiness Assessment CD-4 Approve Operations start **CD-0 IPR and CD-3 EIR for >$750M projects Formulation Implementation Pre-Phase A Phase A Phase B Phase C Phase D Phase E Assembly, Concept Concept & Prelim Design & Final Design & Integ & Test, Operations Studies Tech Devel Tech Completion Fabrication* Launch NASA-SMD KDP-A KDP-B KDP-C KDP-D KDP-E Ref: NASA NPR 7120.5D MCR MDR PDR CDR SIR FRR CERR SRR LRR CDR Critical Design Review ORR Operational Readiness Review ORR PLAR CERR Critical Events Readiness Review PDR Preliminary Design Review Key Decision Point (KDP) approvals FRR Flight Readiness Review PLAR Post-Launch Assessment Review KDP-C Approve Implementation *elongated to visually align NASA, NSF & DOE LRR Launch Readiness Review SDR System Definition Review MCR Mission Concept Review SIR System Integration Review equivalent events. Sequences on this chart do MDR Mission Definition Review SRR System Requirements Review not represent typical or relative phase durations. Readiness NSB Approved Horizon planning and Preliminary Final Design Construction Operations Conceptual Design Design NSF Ref: NSF 0738 CDR PDR FDR Ops CDR Conceptual Design Review Approvals FDR Final Design Review Post-CDR Approve advance to Readiness PDR Preliminary Design Review Post-PDR Approve submission to Nat. Science Board (NSB) Review Decision Post-FDR Congress appropriates MREFC fundsW.MillerOperations Review & Sponsor Processes OpsPMC2010:Big Projectschallenges on large projects - Partnering Post-Ops Approve Operations start ProjSci Oct 22, 2009 W. Miller 9 WLM Rev2.05052009
  10. 10. Oversight: two modes of assessment Preconstruction Conception Construction Operations D Planning Initiate Ready Finished  Qualitative assessment (Reviews at major Review & milestones) Approve  Quantitative Tracking 0.40 • EVM required by OMB for 0.20 CV 0.00 major projects -0.20 • Industry standards -0.40 SV • Encourages baselining -0.60 • Fuller assessment of outcomesPMC2010: Partnering challenges on large projects W. Miller 10
  11. 11. Review-based oversight and decision/approval Independent Internal decision Sign Off Review support (e.g. Implement Project) PDR Energy Systems Acquisition CD-2 DOE by OPA Advisory Board (ESAAB) Acquisition Exec (AE) “Lehman Review” (DepSec, US Sci, SC Dir/AD) • OPA briefs AE, calls ESAAB (also: EIR by OECM) Approve Performance Baseline • Federal Project Director (Enter Final Design Stage) PDR Program Mgmt Council KDP-C NASA by Standing Review Decision Authority (DA) Board • Program Manager (AA for Cat 1, otherwise MDAA) • Center Mgmt Council Approve Implementation • Technical Authority (Enter Phase C – Final Design & Fab) • Project Manager PDR NSF Director NSB Approval NSF by Review Panel for inclusion in a future MREFC* Panel budget in MREFC* • CFO, DDLFP construction account • Directorate/Division • Program OfficerPMC2010: Partnering challenges on large projects * Major Research Equipment and Facilities Construction W. Miller 11
  12. 12. Baseline establishment – with focus on cost During construction, projects performance Cost assessed relative to Cost baseline… “Baseline” “Outcome” (what you were promised) (what you got) Cost estimation toward the baseline: DOE • “Bottom-up" Independent Cost Estimate (ICE) or Independent Cost Review as part of External Independent Review for CD-2. NASA • Project’s bottom-up Life-Cycle Cost Estimate (LCCE). • ICE prepared and owned by the Standing Review Board (SRB), (generally parametric, using same definitions as LCCE)  Project must reconcile ICE with LCCE at Preliminary Design Review. NSF • Bottom-up cost estimate in proposal. • NSF may acquire an independent top-down estimate for comparison.PMC2010: Partnering challenges on large projects W. Miller 12
  13. 13. Funding Initiation Definition Execution Pre-conceptual Conceptual Preliminary Construction Trans/Closeout Final Design Planning Design Design Operations CD-0 CD-1 CD-2 CD-3 CD-4 DOE-SC LIC Prelim Engr & Design (PED) LIC Construction Program Funds Program Ops Funds Major Item of Equipment (MIE) Two modes for intramural projects: Line Item Construction (LIC) or • Separate funds support research Major Item of Equipment (MIE, no major construction) Formulation Implementation Pre-Phase A Phase A Phase B Phase C Phase D Phase E Assembly, Concept Concept & Prelim Design & Final Design & Integ & Test, Operations Studies Tech Devel Tech Completion Fabrication* Launch KDP-A KDP-B KDP-C KDP-D KDP-ENASA-SMD Program Funds, project line-item MO&DA Mission Operations and Data Analysis • Separate funds support research (Research and Analysis, R&A) Readiness NSB Approved Horizon planning and Preliminary Final Design Construction Operations Conceptual Design Design NSF Program Funds (Research & Related Activity, R&RA) MREFC R&RA Major Research Equipment and Facilities Construction • Same funds support research (Research & Related Activities, R&RA) Requires separate appropriationPMC2010: Partnering challenges on large projects W. Miller 13
  14. 14. Scope  How do partner processes and practices align? (Start with agencies)  What are the challenges and best practices for partnering?  Try to “systematize” Sources: • Interviews with ~45 stakeholders for study • “Lessons learned” and other reportsPMC2010: Partnering challenges on large projects W. Miller 14
  15. 15. Some recent cases of major partnered projects NASA DOE NSF InternationalUlysses ▲ ESACassini ▲ ESA, ASITropical Rainfall Measuring Mission (TRMM) ▲ Japan/JAXAGlobal Precipitation Measurement (GPM) ▲ Japan/JAXAFermi (operating) ▲ ▲ CERNLHC (operating) ▲ ▲ membersInternational Ocean Drilling Program (IODP) ▲ Japan/Mext Europe,Atacama Large Millimeter Array ▲ Japan, AsiaPMC2010: Partnering challenges on large projects W. Miller 15
  16. 16. Examples of interagency Joint Oversight Groups (JOG) • DOE-NASA: FERMI, Joint Dark Energy Mission • NSF-DOE: U.S. Large Hadron Collider (LHC), Deep Underground Science and Engineering Laboratory (DUSEL, in planning)PMC2010: Partnering challenges on large projects W. Miller 16
  17. 17. Partnering  Levels of impedance (mis)matches Capabilities & Practices Strategic valuation Technical domainsManagement practices  Better planning could involve assessment of COMPLEMENTARITY across capabilities, practices, lessons learned, etc…) -Source: R. Staffin, 14Feb2006, FY06 presentation to HEPAP, www.er.doe.gov/hep/files/pdfs/HEPAPFeb142005Staffin.pdfPMC2010: Partnering challenges on large projects W. Miller 17
  18. 18. Challenges: Strategic Management Technical• Valuation. How highly does each partner prioritize the project – and how much is it willing to spend based on that priority? What does it expect as an outcome (science, jobs, prestige) and when? Where does it fit in the partners national plan?• Goals and roles. Are the overall goals and specific objectives shared? Does each partner want a leadership or participatory role? How committed are the commitments?• Approval and allocation. Which hoops does each partner have to jump through to obtain funding and approval? What are the pressures? What is the funding source and cycle? How long does prioritization and approval typically take?PMC2010: Partnering challenges on large projects W. Miller 18
  19. 19. Challenges: Strategic Management Technical• Role of project management. Is project management valued? Are managers well chosen, given sufficient authority? Who do they report to?• Project definition and baselining. How are requirements, risks and baseline elements defined, documented and managed? Change control? How much emphasis placed on system I&T and operations planning?• Budgeting. Can be large differences in costing – e.g. labor cost inclusion in project budgets. Can funds be used early, carried-over? Contribution approach (in kind, etc.)? How are contingency and reserve handled?• Oversight. Partners may not perform reviews with the same rigor or frequency. Identified problems may be addressed with less urgency. What systems do partners use for tracking and quantifying performance?• Cultural differences. Sense of urgency to obtain desired outcome? Consequences of (and solutions adopted for) cost overruns, late development, poor performance, outright failure?PMC2010: Partnering challenges on large projects W. Miller 19
  20. 20. Challenges: Strategic Management Technical• Complementarity. Synergies among partners in technical know-how relative to requirements of the project? Is at least one partner strong in each area? What are the gaps?• Design process. What approaches are employed by each partner? How are software and IT systems developed?• Systems engineering. To what degree is systems engineering valued by each partner? Is each partner familiar with establishing interfaces? With I&T planning?• HR. How do partners obtain their skill contributions – via contractors, in-house staff, exchanges of personnel from member academic institutions…? Workforce stablity?PMC2010: Partnering challenges on large projects W. Miller 20
  21. 21. Synchronizing “On-ramps”: when does a project start? Preconstruction Planning Conceptualization Conceptual Preliminary Final ? “Mission-driven” – projects determined via strategically-defined goals and priorities DOE • Projects identified in a SC 20-year prioritized facility plan1 • Establish “mission need”, feasibility at CD-0  Definition phase NASA • Strategic and Science Plans2 based on Decadal Surveys, roadmaps. Two flavors: • Strategic missions are internally initiated (KDP-A) and managed • “PI-led” missions are competed in Phase A  selected  Phase B “Community-driven” – projects “bubble up” from the scientific disciplines NSF • Peer-reviewed unsolicited proposals, workshops, studies, etc. • Evolved concepts may be brought to development 1. Facilities for the Future of Science, A twenty year outlook, DOE/SC-0078, Dec 2003; and Four Years Later: an Interim Report on Facilities for the Future…, Aug 2007. 2. NASA Strategic Plan, 2006; and NASA Science Plan 2007–2016PMC2010: Partnering challenges on large projects W. Miller 21
  22. 22. Reported best practices – responses to challenges• Early engagement. Agree on clear goals, timeframe, and effective membership rules and governance structure. Having a dominant partner may work best…• Communication. Identify the appropriate interfaces at all levels. Maintain open, frequent and honest communication.• Complementarity. Learn partner enterprises and realities. Identify technical domains required and respective partner(s) capabilities.• Strong project management. Devise clear process for project management and oversight. Aim for integrated a single project management team, in place before funding begins and with effective budget authority (not just a coordinator).• Budgeting. Aim to adopt standard costing and budgeting techniques.PMC2010: Partnering challenges on large projects W. Miller 22
  23. 23. AcronymsCD Critical Decision (DOE) OECM Office of Engineering and Construction Management (DOE)CDR Conceptual Design Review (DOE, NSF) Critical Design Review (NASA) OPA Office of Program Assessment (DOE/SC)DDLFP Deputy Director for Large Facility Projects, PED Preliminary Engineering and Design head of LFO (NSF) funding account (DOE)EIR External Independent Review (DOE PDR Preliminary Design Review OECM) PNAR Preliminary Non-Advocate Review (MDR,IPAO Independent Program Assessment & NASA) Oversight Office (NASA) R&RA Research and Related Activities fundingKDP Key Decision Point (NASA) account (NSF)LFO Large Facilities Office (NSF) R&A Research and Analysis funding account (NASA)MO&DA Mission Operations and Data Analysis funding account (NASA) SC Office of Science (DOE)MREFC Major Research Equipment and Facilities SRB Standing Review Board (NASA) Construction funding account (NSF) SMD Science Missions Directorate (NASA)NAR Non-Advocate Review (PDR, NASA)PMC2010: Partnering challenges on large projects W. Miller 23

×