Orion PDR Process - What in the World Happened? NASA PM Challenge 2010 Used with PermissionCharles ArmstrongNational Aeronautics and Space AdministrationOrion VI&D Plans & Processes Managercharles.email@example.comTony ByersLockheed Martin Space Systems CompanySystems Engineering – Program Integration Manageranthony.firstname.lastname@example.org 1
Management Challenges Project Orion• Planning Challenges • Develop and execute a Preliminary Design Review (PDR) process that ensures NASA’s next generation spacecraft can meet all mission objectives and satisfy requirements with acceptable risk • Develop a multi-tiered approach that ensures the integrated subsystems, modules and vehicle are reviewed to ensure the design approach is at a sufficient level of maturity to proceed to the detailed design phase. • Balance thoroughness and schedule. The desire to review every aspect of the design down to the lowest possible level must be balanced against the available time to execute the review as to not delay the development activities unnecessarily. • Balance inclusion and efficiency. Develop a process that ensures all stakeholders have an adequate time to participate and provide input helping to shape the design, ensure mission success, and ultimately provide a safe crew exploration vehicle. 3
Management Challenges Project Orion• Execution Challenges • Communicate the expectations to a diverse 2000+ personnel organization consisting of NASA, prime contractor (LM), and many support contractors • Balance the desire to improve the design and need for a design baseline. From the time the PDR baseline is struck, a lot of potential energy develops to improve the requirements, architecture, design, and configuration. This energy must be managed to ensure that PDR products align as close as possible to the PDR baseline. • Monitor and control process execution throughout the duration of the effort 4
Operational Excellence Project Orion• Teamwork – Created a joint NASA/LM PDR Integration Team (PIT) to ensure common expectations are communicated across the NASA, LM, and support contractor teams • Ensured key personnel from each Module and IPT were included on team • Enlisted the support of the Orion planning organizations• Communication – Focused on communicating early and often – Conducted a series of TIMs with each of the subsystem leads to develop common expectations and identify issues needing resolution prior to PDR – Instituted multiple statuses and forums targeting all stakeholders• Commitment – With the goal of “Get it right the 1st time”, the Orion Management team routinely stressed the importance of attaining the PDR level of design maturity – Management routinely communicated the PDR expectations and monitored status weekly –• Accountability – Each Subsystem Lead took ownership of their respective Subsystem Design Review preparation and execution – A detailed schedule was developed to manage the program DRD review and delivery – Detailed metrics were developed for tracking all deliverables by System and Subsystem
PDR Process & Data Package Project Orion• PDR Definition – A technically-focused design review which demonstrates that the preliminary system, module and subsystem design of the Orion integrated vehicle, its concept of operations, associated processes and applicable technical plans meet all system requirements with acceptable risk and within cost and schedule constraints.• PDR Process – The PDR process included a series reviews which provide a solid foundation for determining if the current design is at PDR level of maturity. • 184 Technical Reviews • 104 Peer Reviews • 10 Combined Tech/Peer Reviews • 18 Subsystem Design Reviews • 1 System and Module Review • 8 RID Teams responsible for RID Screening & Disposition • 1 PDR Pre-Board • 1 PDR Board• PDR Data Package – 170 DRDs delivered and reviewed – 53 “Drafts Available” DRDs provided to NASA for comment
PDR Objectives Project Orion• Demonstrate that the Orion Project’s design baseline meets all Constellation Program and Orion system requirements to the subsystem level and does so with acceptable risk and within the cost and schedule constraints• Ensure that all system requirements have been allocated to the component level, the requirements are complete, and flow down is adequate to verify system performance• Ensure all physical and functional interfaces for Orion system and subsystem design have been identified and defined• Demonstrate sufficient definition of integration and verification activities to allow the Project to proceed with detailed planning of test, integration, and the manufacturing of test articles• Confirm that verification method requirements have been developed to the subsystem level and are consistent with the requirements and the vehicle design• Demonstrate that the PDR vehicle configuration is a closed vehicle and within all specified margins• Demonstrate that the vehicle hazards and hazard control methods are identified and assessed, including safety features of the design and crew survival features of the vehicle 8
PDR Objectives Project Orion• Establish the preliminary design, thus enabling the Project to baseline subsystem specifications• Ensure that technical and programmatic risks are identified, are categorized by criticality, and that risk mitigation plans are approved and are being managed• Ensure that the Project is meeting cost and schedule baselines and has the infrastructure necessary to support management and reporting of cost and schedule data• Show appropriate maturity in the proposed design approach to proceed to critical design• Establish the basis and a viable path for proceeding to the Critical Design Review (CDR) including interim milestones 9
PDR Entry & Success Criteria Project Orion• Performed a detailed analysis and mapping of Entrance and Success Criteria • Orion is required to meet both NASA NPR 7123.1A and LM 2.1.6-T1-PgmMgt-1.2-P- C3 PDR Entrance and Success/Exit Criteria • Evaluated both sets of entrance and exit (success) criteria • Integrated into an “Orion” set of PDR Entrance and Success Criteria • Documented in CxP 72212 Orion PDR Process Plan • Mapped PDR Products/DRDs to criteria • Criteria reviewed by all NASA and LM Module/IPT Leads, Subsystem Leads, and key others prior to formal documentation in CxP 72212 • Criteria reviewed and approved by LM SSC Central Systems Engineering• NASA & LM IPTs developed a detailed self assessment against the entry and success criteria • Allowed team to better resolve issues prior to the PDR Board • Assisted program management in documenting open issues needing to be resolved post PDR• First evaluation of PDR Entry and Success Criteria occurred at the PDR Pre-Board • Final evaluation of the Success Criteria occured at the PDR Board 10
CxP 72212 Orion PDR Plan Project Orion• Comprehensive PDR plan documented in CxP 72212 and placed under configuration control • All significant stakeholders participated in review of document prior to release • Formalized the overall PDR process• Documented the requirements and associated documentation baseline for PDR• Incorporated tailored NPR 7123.1A criteria into plan • Served as the primary method of documenting all agreements regarding method to meet the SSDR and PDR criteria• Used as a reference throughout the execution to ensure the process was adhered to 11
Road to August 2009 PDR Project Orion 2008 2009 Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July Aug Sep Ares LC4 Results Final Drop 12/20 Remaining Ares LC4 Loads Analysis Update Ares LA4 Loads Analysis Data Dropped 1/23 1/30 Lunar Data Drop 8/14 8/31 (TBR) Orion LC-4 R&U Update Will be updated after Orion/Ares Coordination Orion Rev & Update Sizing / Subsystem FEMs LA4 FEM Develop Sizing / SS FEM LC5 FEM Develop Szg Orion Uncoupled Orion Uncoupled Orion Uncoupled Orion Uncoupled DAC-3 Initialization Loads Loads Update (Test Data Updates) 606E Closeout ERB (DAC-2 Loads + 6 wk Loads Update (LC-4, Landing) 2/24-27 PDR Loads LOC/LOM ERB Update for Landing, GN&C SSDR Subsystem Design Reviews Status Module Struct, Mech, & Sec Environments) MELs Pyro, & TPS 10/23 10/24 Midterm 606F Integrated Loads SWELs Closeout 7/8 T-008 SAP Complete MEL Update 9/29,30 ERB 606F Products ERB Complete ERB 606G T-009, T-010 DAC-3 DAC-3 POD Delivered 9/9 & 10/2 SBR ERB 11/19-20 12/22 Analysis Reports Due to 8/1 8/15 2/12-13 3/24 4/14 5/15 6/11&12 NASA KO PDR Board 6-wk Asmts & 7/13 7/24 8/31 DAC-2 Sys Integ DAC-3 Sys Integ DAC/PDR Planing SMR Pre-Board Integrated vehicle 10/29 Integrated vehicle Integrated vehicle products on ProjectLink T-009, T-010 DAC-2 products available products available 2 wk Sys & Mod 1 week prior to ERB Analysis Reports Due to NASA Presentations 8/31Tech Reviews & Peer Reviews S/C, Mod, & AV&SW Spec DRD Rev & RID All RIDs S/C & Module Spec (DAC-2) 12/5 (C&T & Mid-DAC-3) – 5/5 Dispositioned 7/7 S/C & Module Spec Development Sys Product Develop Final PDR DRDs Subsystem Spec Development delivered to NASA Subsystem IRD & ICD Development Subsystem Design and Databook Development 1-wk DRD Master Verification Plan Development Delivery Margin Drawing Release Ph 1 TIM #2 1/20-23 2/9-13 Ph 1 3/30-4/3 Ph 1 5/4-8 Ph 1 6/15-26 Ph 1 TIM #6 Delta CSERP Groundrules TIM 10/1 11/3-7 Ph 1 TIM #1 TIM #3 TIM #4 TIM #5 7/27-8/7 CSERP Review Safety & Mission Assurance Spiral Data Spiral Spiral Cradle Snapshot 8/15 C&T SS Review 12/4-5 Pkg Delivery Review Data Rev C&T SRR 10/29 2/28 Sys & Subsystems Cradle Snapshot 11/20 12/7-11 Board 1/22/10 Avionics & Software Activities AA-2 7/16 PA1 DD250 3/3 4/10 T&V Sys B/L Rev PTR2 EGSE DD250 2/11 CAIL PR 6/30 Test 12 Verification Activities and 12
Orion PDR Process Project Orion Project Orion June July August Sep OctSubsystem Design 2-Wk SMR Prep 8/4 Board Dry-Reviews (SSDRs) SSDRs Decision Gate Runs Orion PDR PDR KO 7/13 7/21 8/10 Forward System & Daily Board Decision Gates Module Presentations RIDs Captured SMR Against Design DRDs Delivered RID Tool Close (all RIDs Sponsored) RID Tool Open 7/7 PDR -45 7/31 TBD 4 wk DRD Review DRD Waterfall / Early Review (RID Creation & S (Tech Reviews & Peer Reviews) Sponsor) PIT Lead with Module/IPT Support Integ PIT War Room Starts 8/27 Team 8/10 NASA Screening NASA Team Concludes RID Screening NASA Lvl III Review & Filter RIDs Screening All RIDs receive product (Work disconnects with external stakeholders) Team owner or lead recommended (LM SEIT led SME team available to answer questions) Disposition 8/17 Starts 8/3 Product Owner Recommend All RIDs Disposition Dispositioned 8/20 Starts Review Team 8/12 Disposition DRDs 8/31-9/1 PDR Board Redelivered Starts 8/17 Pre Decision to Proceed to DAC-4 NLT 2/1/10 Board 5 1/2 Day Pre-Board to work issues in parallel with completion of RID Disposition DRD Update & Redeliver (Type 1 + Final Type 2)
Orion Subsystem Design Reviews Project Orion Project Orion June 2009 Monday Tuesday Wednesday Thursday Friday 1 2 3 4 5 SSDR KO/ROE (2 ea.) 8 9 SSDR KO/ROE (2 ea.) 10 Chairs ROE (Makeup) 11 12 Chairs ROE (Makeup) Houston 2:00 PM MT 2:00 PM MT SCRAM Review Sacramento, CA LAS Jettison Motor SSDR DAC-3 Closeout ERB 15 16 17 18 19 Includes Denver Mechanisms SSDR Structures SSDR Sat 6/20 Denver Crew Systems SSDR Denver Wiring SSDR2 wk Approach Denver Denver Ends Passive Thermal Control SSDR Landing and Recovery System SSDR Sat 6/20 Denver Propulsion SSDR 22 23 24 25 26 Denver LAS focus 6/20 & 6/22 AM Structures SSDR Overflow Denver Environmental Control and Life Support SSDR Denver Avionics and Software SSDR Denver Denver Ends Electrical Power System SSDR & Voltage/Distributed Architecture Updates Thermal Protection System SSDR Sat 6/27 Promontory, Utah LAS Abort Motor SSDR 29 30 July 1 2 3 - Holiday Houston PDU Review Houston Pyrotechnics SSDR Elkton, MD LAS Attitude Control Motor SSDR GN&C SSDR Conducted 2/24-27 LAS SEIT, Avionics & Power SSDR Conducted 4/14-15
RID Criteria & Ground Rules Project OrionCriteria• A design-related issue intended to document: • Inability of the preliminary design to meet requirements (e.g., design compliance issues) • Incomplete or incorrect requirement flow down or traceability • Incomplete or inadequate maturity for PDR (e.g., incomplete/inadequate trades or models) • Undefined or inadequately defined technical interfaces • Inconsistency between design, analysis, requirements, or operations concepts • Incorrect or inadequate verification methods in support of the preliminary design • Inconsistency with the primary characteristics that should factor into each design decision as identified in CxP 70000, Constellation Architecture Requirements Document, Section 3.1.2. These primary characteristics are: Safety and Mission Success, Programmatic Risk, Extensibility and Flexibility, and Life Cycle Cost • Unsafe design features • Unidentified or inadequately mitigated risk • Inconsistency with Project objectives 15
RID Criteria & Ground Rules Project OrionGround Rules• RIDs can be written against RIDable documents or against any aspect of the Orion vehicle design and technical plans within the purview of the Orion Project • RIDs will be recommended for withdrawal if the initiator is identifying a deficiency that is out of scope of this review or does not meet the RID criteria listed above• An assertion via RID that a design feature is inconsistent or deficient with respect to the primary characteristics expressed in the Orion Requirements Baseline should be supported by rationale that provides a credible case for the existence of a more favorable alternative• Where possible, RIDs should be tied to functional or performance requirements or constraints• RIDs will also be recommended for withdrawal if the requestor is identifying an area of “forward work” • “Forward work” is defined as work that is necessary for successful completion of the Orion vehicle but not required to satisfy the success criteria for PDR• Lack of sufficient information is a basis for a RID only if the RID initiator has exhausted all reasonable means to obtain the information or if the information has not been made available as required by the entrance criteria 16
RID Criteria & Ground Rules Project OrionGround Rules• Editorial RIDs will be demoted to comments and forwarded to the document developer as information at the end of the review• Initiators are expected to review the errata sheet associated with a specific DRD prior to reviewing the DRD and/or prior to entering a RID • RIDs are intended to identify design issues and are not intended to document known issues as documented in the errata • RIDs may be written if the errata description or details are insufficient to provide confidence that the issue has been addressed or will be addressed in the appropriate manner and timeframe 17
What is a Good RID? Project Orion• What is a good RID? – A “new” design related issue – Accurately describes the design related issue consistent with the RID criteria – Provides the details of where the issue was found • For DRD related RIDs (section, page, etc) • For Architecture Element RIDs with no DRD referenced (chart, conversation, etc.) – Provides a sufficiently detailed recommended resolution – Stays within the scope of PDR expectations – For requirements, provides correct “from” language and recommends “to” language An issue that has not previously been identified.• What is a bad RID? – Incomplete issue identification – No recommended resolution – Recommended resolution of “fix this” – Editorial comment (spelling, grammar, etc.) – RID that expands on errata or duplicates a Pre-Declared RID – Post PDR level of maturity / unrealistic expectations – RID written against a baselined or reference document (Examples: SRD, S/C Spec) – RID entered by someone other than the original initiator without documenting the original initiator – RID written against a previously made project decision (VICB, CPCB, etc.) without new data or evidence to challenge the decision Don’t tell us something we already know!!!
PDR Assessment Project Orion• Design Issues – A detailed pre-assessment was performed by each of the module prior to entering PDR – All known issues were presented at each SSDR and the SMR • No new significant issues surfaced during the reviews – Significant focus was applied to ensure known issues had agreed-to plans to proceed – Top issues contributing to non-green assessments • Mechanisms – FBC R&R – FBC Recontact, CM/SM R&R Maturity Parachute Architecture Study was underway prior to • LRS – FBC Recontact / Parachute Architecture & Testability entering PDR. Assessment concluded in October • TPS & Structures – Parachute Architecture Impacts to Baseline 2009. • EPS – 120 V Incorporation Driven by significant post SDR requirements changes • AV&SW – C&T Incorporation, BFCS Maturity • ECLS – ARS Suit Loop, Procurement Schedule Procurement schedules were being updated to align with • CM Prop – Procurement Schedule project funding profile • Pyro – Devices Dependent on Mechanism Maturity • LAS ACM – Production Motor Maturity• 18 Subsystem Design Reviews (SSDRs) executed successfully – Each SSDR assessed against a common set of entry and success criteria plus a joint set of NASA/LM agreed to objective evidence customized for each Subsystem – 59 Total Review Days & approximately 500 review participants• PDR executed successfully – Each module assessed against a common set of entry and success criteria; ITAs assessment integrated – 56 days of DRD review, 7 day System & Module review, 1 week pre-Board and a Board – Approximately 1200 reviewers • Project Management assessed PDR a success and authorized the program to proceed Yellow / Green to the detailed design phase • Assessment based on a Red, Yellow, Green scale
PDR Data Package Delivery Project OrionAs of July 8, 2009 PDR Data Package • NASA/LM jointly assessed PDR deliverables to determine which DRDs were necessary to meet PDR expectations and satisfy criteria • PDR – • 146 Deliverable Documents • SSDR & PDR – • 26 Design & Data Books • Data Package Total • 172 Total PDR Documents • 53 “Drafts Available” DRDs provided to NASA for comment Waterfalled Review and Delivery • Managed via the PDR Integration PDR -45 days Team • Captured in a 2000+ line MS Project Schedule • Metrics generated & reported on a weekly basis • Deliveries culminated on 7/7/09, PDR-45 Days
Action Item & RID Closure Project Orion SSDR Actions PDR RIDs • Detailed Action and RIDAV&SW 140 150 screening occurred LASCrew Systems 39 throughout the SSDR & CM 241ECLSS 49 PDR process SM 90EPS 97 • All actions and RIDsGN&C 26 Av&Pwr&Wire 444 dispositioned and closureLAS AM 60 38 Software aligned with project postLAS ACM 96 T&V, Fac, GSE 411 PDR work planLAS JM 60 AI&P 18LAS SE&I AVP 12 • All RIDs approved by PDRLRS 42 SR&QA 91 BoardMechanisms 56 566 SystemsPropulsion 150 • Project approved closure 33 Total 2049 plan implementedPTCPyro 26Structures 141 • Approximately 100,000 pages of design, process,TPS 20 and plan documentation reviewedWiring 14 • Included requirements, interface definition,Total 1061 verification details, general plans, etc.
Key PDR Lessons Learned Project Orion• SSDRs – SSDRs executed in parallel and in a common location worked well • Key personnel were in the building and available and could easily bounce between reviews when needed – Focused review teams allowed the reviews to move at a sufficient pace while ensuring all significant information was reviewed to an adequate level • On-site review teams of 50 to 100 personnel is appropriate • Overflow rooms and remote participation were not an ideal way of participating in the on-site review – Data Packages consisted of the core information necessary to evaluate design maturity • Minimizing the data package to only those DRDs necessary to establish design maturity supported a streamlined review process – Early technical and peer reviews facilitated improved design awareness prior to the review – Need to strengthen Software integration into SSDRs • Evaluate what additional products need to be added to the SSDR Data Packages • Note software was addressed to level required for NPR 7123.1a (project wants additional focus for CDR) – Need to ensure adequate spacing between tech reviews and SSDRs to allow for sufficient time to close TR actions • Need SSDR co-chairs to review open TR actions prior to SSDRs – Transferring SSDR actions to RIDs created a dual paper trail that required unnecessary effort to resolve – SSDR Dry-runs need to be scheduled further in advance to allow subsystem team to make adjustments prior to the review – SSDR Caucuses were most effective when kept to a small attendance – SSDR color ratings took more time than expected given that issues were identified ahead of the SSDR
Key PDR Lessons Learned Project Orion• PDR – Data Package Preparation, Delivery, & Review • Separate Technical and Peer Reviews were not the most efficient way to review DRDs – Peer Reviews were not viewed as where the detailed review occurs – Integrating Technical and Peer Reviews into a single review that meets the expectation of both reviews • Decouple DRDs not needed for design review from the design review – Review and accept these documents through NASA’s normal DRD acceptance process • Waterfalled delivery allows for early review of material prior to the face-to-face reviews – System & Module Review (SMR) • SMR value did not match level of effort required to prepare and execute – Most technical review and discussion occurred at the SSDRs – Weak in-person attendance • Need to change this review to an integrated vehicle review focusing on the integrate module and integrated stack design for CDR – Include Assembly, Integration & Production – Include Test & Verification – Pull the integrated analysis and design products and review them up-front prior to the SSDRs • Create a one-day CDR Kick-off that gives the requirements baseline, environments overview, etc. prior to the CDR SSDRs
Key PDR Lessons Learned Project Orion• PDR – RID Tool, ProjectLink, Portal, and Wiki • Use one tool from the beginning of the process through completion – Minimizes duplication of issues and the overhead of transferring from one tool to another • Test the RID tool earlier in the planning process • Choose a RID tool that allows for a detailed closure process tracking • Portals and Wiki were a valuable resource to all reviewer – Assign a NASA IT person to the PIT for CDR • PDR hotline (outlook email box) served as a ongoing resource when reviewers did not know who to contact regarding a process or technical issue – RID Screening and Review Teams • IPTs who integrated the screening/review effort completed activities earlier and reduced iteration when dispositioning / resolving issues • Improved direction regarding “Approve” and “Approve w/mod” needs to be developed and communicated to review teams prior to start • Training needs to better prepare the reviewers for what to review the design against (Rqmts, Docs) – Consider adding an Orion familiarization to the training – Pre-Board and Board • Overall Board execution proceeded to plan – Increased Pre-Board time needed in CDR schedule if a similar number of RIDable documents are to be reviewed during CDR • Ensure RIDs dispositioned by pre-board or board are updated in the tool needs to occur • Do not push issue decisions to later forums (VICB, CPCB, etc.). Make the decisions prior to leaving the Board even if this extends the Board
• All reviewers participate in the Orion CDR Kickoff (Orion Familiarization) to educate reviewers on the: • Design configuration Top-level CDR Plan • Coordinate what metrics are needed with RID/Action Tool prior to CDR • RID closure will be handled within IDE • Requirements baseline DRAFT • Errata and Pre-Declared RIDs all in one location (minimize number) • Environments baseline • Caucus serves the function of RID Screening Project Orion • Design Work products • Align tool with Process • Significant design issues 3 Week 1 Week 2 Week and their closure plans Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Orion CDR Kickoff DRD Review • Don’t duplicate SSDR actions and RIDs. RID Tool Opens • Issues captured in standard format throughout process (Action/RID Tool) RID Tool Closes • Standardize Etc.) Interactive Reading Room (DRDs, Drawings, process for handling cost/schedule issues • Ensure Action/RID template aligns with IDE tool • Summarize SSDR content that has been covered in component CDRs System Analysis Review (1 to 2 days) SSDRs Issue Identification (RID generation) • Replace SMR with an Integrated Vehicle Design Review SSDR Caucuses • Cover integrated module, integrated stack, AI&P, T&V, etc. IVDR Integrated Vehicle Design Review RID Disposition (Product Owners & Leads) RID Review Teams Pre-Board Board Notes from Orion Streamlining TIM 12/09 DRAFT
PDR Supporting Documentation Project Orion• PDR Plan – CxP 72212 CxP Orion PDR Plan is available on PDR Wiki or PDR Portal/Dashboard• PDR Resources – PDR Wiki • Used as a resource to all Orion personnel and PDR stakeholders to communicate the current planning status • https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=13828791 &maxRecentlyUpdatedPageCount=20 – PDR Portal • Used to formally provide all SSDR and PDR details, data package, and review results • Federal record of review • https://ice.exploration.nasa.gov/ice/site/cx/reviews_orion_pdr/
PDR Integration Team Project OrionIn addition to the core PIT members above, there were members from each IPT, Module, Planning, and Administration incorporated into the planning effort and considered active PIT members.
Entrance Criteria Project Orion Orion Entrance Criteria (CxP 72212)1) Successful completion and close-out of SDR. All prior SDR Actions and RIDs have been closed, orapproved closure plan exists for those remaining open.2) PDR plan has been approved by the CPCB, which includes the PDR entrance and success criteria,RID/Action process, Board Members, stakeholders and schedule.3) All PDR applicable and supporting products have been delivered and made accessible for stakeholdersprior to review.4) Updated baselined documentation, as required.5) Preliminary design is established for each subsystem and supporting subsystem specifications are finalwith supporting analysis and trade-studies. Spacecraft, Module and Subsystem Specifications arecomplete and traceable. Subsystem specifications include Component definition. Componentspecifications delivery and review schedules have been developed and aligned with procurement strategy,including long-lead items. Preliminary software design and supporting subsystem specifications shouldinclude a complete definition of the software architecture and preliminary database design description (ifapplicable) which should include all analysis and data used for the development/maturation of the softwarearchitecture.6) Technology development plans are updated.7) The project risks and opportunities have been developed. Acceptable risk mitigation plans have beendeveloped in accordance with the Orion Risk Management Plan.8) Cost/schedule data are available to appropriate reviewers.9) Updated logistics documentation, including initial support equipment list.
Entrance Criteria Project Orion Orion Entrance Criteria (CxP 72212)10) Applicable technical plans (e.g., technical performance measurement plan, contamination control plan,parts management plan, environments control plan, EMI/EMC control plan, producibility/manufacturabilityprogram plan, reliability program plan, quality assurance plan).11) Updated Applicable standards.12) Integrated CEV flight safety analysis (including software safety analyses) and other applicable S&MAanalysis of preliminary design (Failure Modes and Effect Analysis, flight system reliability andmaintainability analysis, Probabilistic Risk Assessment, etc.) are complete and requirements areimplemented in system, module, subsystem, and component specifications (component specifications aredeveloped per the acquisition strategy). All associated plans have been developed.13) Updated Engineering drawing tree.14) Preliminary Functional and physical interfaces are developed and documented in the Interface ControlDocuments (ICDs).15) Test and verification and validation plans are defined.16) Plans to respond to regulatory requirements (e.g., Environmental Impact Statement, as required)17) Disposal Plans defining safe disposal of assets.18) Technical resource utilization estimates and performance margins are updated.19) Preliminary limited life items list is updated (LLIL).20) Operational planning is proceeding to schedule.
Success Criteria Project Orion Orion Success Criteria (CxP 72212)1) The Spacecraft, Module, and Subsystem Requirements, TPMs and any CxP-imposed requirements areagreed upon, finalized, stated clearly and are consistent with the preliminary design.2) The flow down of verifiable requirements to the subsystems and to the appropriate components arecomplete. An adequate plan exists for timely resolution of all open items. Requirements are traceable toOrion SRD, CxP IRDs and other applicable CxP requirements including any errata.3) System, module and subsystem preliminary designs are complete and meet requirements at anacceptable level of risk. All liens have been identified and closure plans developed.4) Technical interfaces are defined, deemed adequate, and are consistent with the overall technicalmaturity and provide an acceptable level of risk.5) Adequate spacecraft technical margins exist with respect to TPMs.6) Any required new technology has been developed to an adequate state of readiness or back-up optionsexist and are supported to make them a viable alternative.7) The project risks and opportunities are understood and have been credibly assessed. Appropriate riskmitigation plans for all top risks.8) Safety and mission assurance (e.g., safety, reliability, maintainability, quality, and EEE parts) have beenadequately addressed in preliminary designs and any applicable S&MA products (e.g., PRA, system safetyanalysis, and failure modes and effects analysis) have been reviewed and concurred.9) The operational concept is technically sound, is supported by the vehicle design, and includes (whereappropriate) human factors. Appropriate requirements identified by the ConOps have been flowed downincluding requirements for ConOps execution.10) Preliminary test and verification plans are updated. Manufacturing plans are developed.11) System costs are within targets.
Product to Criteria Mapping Project Orion• Performed a detailed analysis and mapping of Entrance and Success Criteria • The PDR Data Package is mapped to the criteria and documented in Appendix C of CxP 72212 Orion PDR Process Plan
Orion PDR rev 8/26/09 – 2200 MT Project Orion Project Orion Subsystem Design Reviews (SSDRs) June 18 Subsystem Reviews July August Sep OctSubsystem Design 2-Wk • 3 SSDRs Executed Prior to the 2-wk SSDRs 8/4 BoardReviews (SSDRs) SSDRs Dry- SMR Prep • GN&C SSDR Completed on 2/24-27 Decision Gate Runs Orion PDR PDR KO• LAS SEIT, Avionics & Power SSDR Completed on 4/14-15 7/13 • LAS Jettison Motor Completed on 6/8-10 7/21 8/10 Forward System & Daily Board Decision Gates Module • Helped iron out the process and allowed the team to take advantage of RIDs Captured Presentations SMR lessons learned Against Design • 13 SSDRs completed 2-wk set of SSDRs on 6/15-26 RID Tool Open • At any given time 4Close (all RIDs Sponsored) in parallel in the Lockheed Martin DRDs Delivered RID Tool 7/31 SSDRs were occurring 7/7 PDR -45 TBD Denver WB2 facility 2 remaining SSDRs completed the last week of June • 4 wk DRD Review DRD Waterfall / Early Review (Tech Reviews & Peer Reviews) • SSDR ProductsS (RID Creation & Sponsor) • 28 Design & Data Books Delivered & Reviewed • 23 Draft Subsystem Specs, 21 Subsystem IRDs, 21 Subsystem ICDs, and PIT Lead with Module/IPT Support Integ PIT War Room Starts 8/27 Team 12 Subsystem Volumes of MVPs Available for Comment 8/10 NASA Screening • 59 Total Review Days Team Concludes RID Screening NASA • Over 1,000 review participants includingreceive product NESC, SRB, NASA Crew NASA Lvl III Review & Filter RIDs Screening All RIDs support from Office, NASA Team owner or lead recommended (Work disconnects with external stakeholders) MOD, NASA Engineering, NASA S&MA, NASA Health & Medical, (LM SEIT led SME team available to answer questions) Disposition Constellation, Orion Project, & LM Central Engineering 8/17 Redelivery date will be • 1,056 Action Items Generated Starts 8/3 Product Owner recommended on 8/21 based Recommend • All actions recorded in either the ConCERT RID Tool on number and complexity of All RIDs or LM IDE Disposition Dispositioned RIDs. 8/20 Starts 8/12 Review Team Disposition DRDs 8/31-9/1 PDR Board Redelivered Starts 8/17 Pre Decision to Proceed to DAC-4 NET 11/6 Board 5 1/2 Day Pre-Board to work issues in parallel with completion of RID Disposition DRD Update & Redeliver (Type 1 + Final Type 2)
Orion PDR rev 8/26/09 – 2200 MT Project Orion Project Orion Subsystem Design Reviews (SSDRs) Subsystem Design Reviews (SSDRs) June 18 Subsystem Reviews July August Sep OctSubsystem Design 2-Wk Each SSDR assessedBoard 2-wk SSDRs set of entry and success criteria • 3 SSDRs Executed Prior to the 8/4 against a commonReviews (SSDRs) SSDRs Dry- SMRplus a joint set of Decision Gate agreed to objective evidence customized for Prep NASA/LM • GN&C SSDR Completed on 2/24-27 Runs Orion PDR each Subsystem (Assessed either Green, Yellow,on 4/14-15 or Red) PDR KO• LAS SEIT, Avionics & Power SSDR Completed 7/13 • LAS Jettison Motor Completed on 6/8-10 7/21 8/10 Forward Crew Systems LAS SEI, AVP System & Mechanisms Mechanisms Daily Board Decision Gates • Helped iron out the process and allowed the team to take advantage of Module Structures LAS ACM LAS ACM RIDs Captured AV & SW CM Prop SM Prop SM Prop lessons learned LAS AM LAS AM Presentations LAS JM SMR Wiring Wiring Against Design GN&C ECLS ECLS • 13 SSDRs completed 2-wk set of SSDRs on 6/15-26 Pyro LRS LRS EPS EPS PTC TPS DRDs Delivered Subsystem Criteria4Close (all RIDs Sponsored) in parallel in the Lockheed Martin RID Tool • At any given time SSDRs were occurring RID Tool Open 7/7 PDR -45 7/31 1. Requirements are complete and flowed to the appropriate TBD Denver WB2 facility products. Subsystem specifications are updated. 2. Subsystem design is complete to a level of detail that is 2 remaining SSDRs completed the last week of June • 4 wk DRD Review sufficient to demonstrate that all requirements are expected to DRD Waterfall / Early Review (Tech Reviews & Peer Reviews) • SSDR ProductsS be met at an acceptable level of risk. Subsystem preliminary (RID Creation & design baseline is complete and does not exceed subsystem Sponsor) mass and• Design & Data Books Delivered & Reviewed power allocations. Subsystem preliminary design includes hardware and software considerations. Test and PIT Lead with Module/IPT Support Subsystem Specs, are verification plans are defined. Technical interfaces Subsystem IRDs, Subsystem ICDs, and Subsystem • Draft Integ PIT War Room Starts 8/27 Team defined, deemed adequate, and are consistent with the Volumesatof MVPs Available for Comment overall technical maturity an acceptable level of risk. 8/10 NASA Screening Resolution plans for any non-compliance have been including support from NESC, SRB, NASA Crew • Over 1,000 review participants Concludes RID Screening developed. NASA Team 3. The Office, NASA opportunities are understood and RIDs receive product LM Orion, & LM Central subsystem risks and MOD, NASA Engineering, NASA Orion, NASA Lvl III Review & Filter RIDs Screening All (Work disconnects with externalbeen credibly assessed. Acceptable risk mitigation owner or lead recommended have Engineering where applicable. stakeholders) plans have been developed Team • preliminary subsystem design is reliable and safe forDisposition (LM SEIT led SME team available 4. The _________ Action Items Generated to answer questions) use. 8/17 Reliability and safety analyses of the preliminary design have Redelivery date will be • _____ 8/3 requirements flowed down “Pre-declared RIDs” Actions classified as to been performed and resulting Starts Product Owner recommended on 8/21 based • _____ Actions Recommend as “Complete Prior to PDR”on number and complexity of the system, module, and subsystem. Preliminary plans for classified tests, analyses, demonstrations, and Disposition verify inspections will All RIDs the reliability_____ Actions classifiedReliability, Dispositioned RIDs. • and safety of the subsystem. Safety, as “Forward Work” and Quality Assurance (SR&QA) has been adequately 8/20 addressed in preliminary designs, and applicable SR&QA the ConCERT RID Tool or LM IDE • All actions recorded in either Starts 8/12 products (e.g., PRA, system safety hazard analysis, and Team Review Disposition failure modes and effects analysis) satisfy PDR product maturation requirements (e.g. Phase 1 Safety Review criteria) DRDs 5. Test, verification and manufacturing planning are underway. 8/31-9/1 PDR Board Redelivered Starts 8/17 Pre Decision to Proceed to DAC-4 NET 11/6 Board 5 1/2 Day Pre-Board to work issues in parallel with completion of RID Disposition DRD Update & Redeliver (Type 1 + Final Type 2)
Orion PDR rev 8/26/09 – 2200 MT Project Orion Project Orion June Data Requirements DescriptionAugust Review and Waterfall July (DRD) Sep OctSubsystem Design 2-Wk 8/4 BoardReviews (SSDRs) SSDRs Dry- SMR Prep PDR Data Package Decision Gate Runs Orion PDR • NASA/LM jointly assessed PDR deliverables to determine PDR KO which DRDs were System & necessary to meet PDR expectations and satisfy criteria 7/13 7/21 8/10 Forward Daily Board Decision Gates Module • PDR – RIDs Captured Presentations SMR146 Deliverable Documents • Against Design (Includes multiple volumes of specific DRDs, i.e. MVP, Specs, IRDs) DRDs Delivered • SSDR & PDR – Tool Close (all RIDs Sponsored) RID RID Tool Open 7/7 PDR -45 7/31 TBD • 26 Design & Data Books 4 wk DataReview • DRD Package Total DRD Waterfall / Early Review (RID Creation Total PDR Documents • 172 & S (Tech Reviews & Peer Reviews) Sponsor) 145 RIDable Documents (listed on PDR Portal) • PIT Lead with Module/IPT Support 12 Commentable Documents • Integ PIT War Room Team Starts 8/27 17 DRDs reviewed through NASA DRD Review Process Early • 8/10 NASA Screening (2 Documents Redelivered Prior to 7/7) Team Concludes RID Screening NASA • 53 “DraftsScreening NASA Lvl III Review & Filter RIDs Available” DRDsAll RIDs receive product for comment provided to NASA • Total SSDR and PDR Data Package consisted of nearly 100,000 pages of (Work disconnects with external stakeholders) Team owner or lead recommended Disposition (LM SEIT led SME team available to answer requirements, interface definition, verification details, general plans, and questions) 8/17 Redelivery date will be supporting design detail Starts 8/3 Product Owner recommended on 8/21 based Recommend All RIDs on number and complexity of Disposition Dispositioned RIDs. Waterfalled Review and Delivery 8/20 • Managed via the PDR Integration Team Starts 8/12 Review Team • Captured in a 2000+ line MS Project Schedule Disposition • Included Tech Reviews, Peer Reviews, C&DM Processing, and Delivery 8/31-9/1 PDR Board DRDs Redelivered • Metrics generated on a weekly basis NET 11/6 Starts 8/17 Pre Decision to Proceed to DAC-4 • Reported to Management team through various forums Board 4 Day Pre-Board to work issues in parallel with 7/7/09, PDR-45 Days • Deliveries culminated on completion of RID Disposition DRD Update & Redeliver (Type 1 + Final Type 2)
Orion PDR rev 8/26/09 – 2200 MT Project Orion Project Orion June Data Requirements DescriptionAugust Review and Waterfall July (DRD) Sep OctSubsystem Design 2-Wk Data Requirements Description (DRD) Review and Waterfall 8/4 BoardReviews (SSDRs) SSDRs Dry- SMR Prep PDR Data Package Decision Gate Runs Orion PDR PDR Data Packageassessed PDR deliverables to determine which DRDs were • NASA/LM jointly PDR KO • 8th, 2009 7/13 Metrics Summary – July 8/10 Forward and satisfy criteria necessary to meet PDR expectations 7/21 System & Daily Board Decision Gates Module • PDR – RIDs Captured Presentations SMR146 Deliverable Documents • Against Design (Includes multiple volumes of specific DRDs, i.e. MVP, Specs, IRDs) DRDs Delivered • SSDR & PDR – Tool Close (all RIDs Sponsored) RID RID Tool Open 7/7 PDR -45 7/31 TBD • 26 Design & Data Books 4 wk DataReview • DRD Package Total DRD Waterfall / Early Review (RID Creation Total PDR Documents • 172 & S (Tech Reviews & Peer Reviews) Sponsor) 145 RIDable Documents (listed on PDR Portal) • PIT Lead with Module/IPT Support 12 Commentable Documents • Integ PIT War Room Team Starts 8/27 17 DRDs reviewed through NASA DRD Review Process Early • 8/10 NASA Screening (2 Documents Redelivered Prior to 7/7) Team Concludes RID Screening NASA • 53 “DraftsScreening NASA Lvl III Review & Filter RIDs Available” DRDsAll RIDs receive product for comment provided to NASA (Work disconnects with external stakeholders) and PDR Data Package consisted of nearly 100,000 pages of • Total SSDR Team owner or lead recommended requirements, interface definition,Disposition (LM SEIT led SME team available to answer questions) verification details, general plans, and supporting 8/17 Redelivery date will be design Starts 8/3 detail Product Owner recommended on 8/21 based Recommend All RIDs on number and complexity of Disposition Dispositioned RIDs. Waterfalled Review and Delivery 8/20 • Managed via the PDR Integration Team Starts 8/12 Review Team • Captured in a 2000+ line MS Project Schedule Disposition • Included Tech Reviews, Peer Reviews, C&DM Processing, and Delivery 8/31-9/1 PDR Board DRDs Redelivered • Metrics generated on a weekly basis NET 11/6 Starts 8/17 Pre Decision to Proceed to DAC-4 • Reported to Management team through various forums Board 4 Day Pre-Board to work issues in parallel with 7/7/09, PDR-45 Days • Deliveries culminated on completion of RID Disposition DRD Update & Redeliver (Type 1 + Final Type 2)