Armstrong
Upcoming SlideShare
Loading in...5
×
 

Armstrong

on

  • 12,723 views

 

Statistics

Views

Total Views
12,723
Views on SlideShare
12,723
Embed Views
0

Actions

Likes
0
Downloads
10
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • CDR DRDs go through the Action Item/RID Process Each Action Item/RID will be screened during the applicable design review. Pre- and Post-CDR DRDs go through the Orion DRD Review and Approval Process Each DRD will be routed through the applicable SMEs for review Issues/Comments for incorporation into the DRD will be identified and reviewed (IPT) DRD issues/comments for incorporation into the DRD will be approved by the VICB
  • 10/21/11

Armstrong Armstrong Presentation Transcript

  • Orion CDR Process: A Streamlined Approach Charles Armstrong National Aeronautics and Space Administration Orion VIO SE&I Plans & Processes Manager [email_address] 281-244-6315 Tony Byers Lockheed Martin Space Systems Company Systems Engineering – Program Integration Manager [email_address] 303-977-4389 NASA PM Challenge 2011
  • Agenda
    • Introduction
    • Preliminary Design Review
    • CDR Streamlining Focus
      • DRDs / Documentation
      • RIDs / Actions
      • Reviews
    • Critical Design Review Process
      • Management Challenges
      • Keys to Success
      • Objectives & Criteria
      • Process
    • Closing Remarks
  • Introduction
    • Presentation Focus:
      • Historically, NASA Human Spaceflight major milestone reviews have been conducted by allowing participation by a large number of agency personnel.
        • Review a wide range of materials
        • Write a large number of candidate issues in the form of RIDs
          • Very expensive
          • Significant infrastructure and logistics
          • Large effort to review, disposition, and close
      • Using the lessons learned from the Orion PDR, the Orion Project has pursued a more streamlined approach in conducting its CDR.
        • Expands the use of subsystem design reviews
        • More fully utilizes technical reviews of design products / documentation
          • Uses highly qualified personnel
          • Changes the way issues are identified and vetted
          • Limits CDR participation to individuals with an identified ability to make a value-added contribution
    If successful, this methodology will reduce the cost of conducting CDR to about one-third that of PDR and set the new standard for the conduct of major milestone reviews within the agency.
  • Current Context
    • August 2009 – PDR successfully concluded
      • Block 1 (ISS) and Block 2 (Lunar) configurations
    • February 2010 – President’s 2011 Budget request removed CxP
      • Included the cancellation of Ares & Orion
      • 2010 Appropriations Bill prevented CxP cancellation during FY10
    • February 2010 – Completed optimization to align with FY10 funding profile
      • Refocused vehicle configuration on Block 0 (Orion 1 & Orion 2)
      • Set 2013 target launch date for Orion 1
    • August 2010 – Senate passes NASA Authorization Bill (includes MPCV)
    • September 2010 – Congress passes NASA Authorization Bill (includes MPCV)
    • September 2010 – Completed optimization needed for 2013 OFT-1
    • October 2010 – President signs NASA Authorization Act (includes MPCV)
    Throughout 2010, the Orion Project continued to stay focused on design and development. The integration team quickly adapted to the changing landscape and routinely revisited CDR expectations.
  • Preliminary Design Review
  • PDR Process Refresher
    • PDR Process
      • The PDR process included a series reviews which provide a solid foundation for determining if the current design is at PDR level of maturity.
        • 184 Technical Reviews
        • 104 Peer Reviews
        • 10 Combined Tech/Peer Reviews
        • 18 Subsystem Design Reviews (59 days)
          • Each SSDR assessed against a common set of criteria
        • 1 System and Module Review (7 days)
        • 8 RID Teams responsible for RID Screening & Disposition (8 days)
          • 56 days of total DRD review
        • 1 PDR Pre-Board (7 days)
          • Each module assessed against a common set of criteria
          • ITAs assessment integrated
        • 1 PDR Board (1 day)
    • PDR Data Package
      • 170 DRDs delivered and reviewed
      • 53 “Drafts Available” DRDs provided to NASA for comment
    500+ Review Participants 1500+ Review Participants Approximately 100,000 pages of design, process, and plan documentation reviewed. Included requirements, interface definition, verification details, drawings, general plans, etc.
  • Orion PDR Process DRD Update & Redeliver (Type 1 + Final Type 2) DRD Waterfall / Early Review (Tech Reviews & Peer Reviews) Project Orion SMR PDR KO 7/13 7/21 8/10 NASA Screening Team Concludes RID Screening All RIDs Dispositioned 8/20 4 wk DRD Review (RID Creation & Sponsor) System & Module Presentations Decision to Proceed to DAC-4 RIDs Captured Against Design DRDs Delivered 7/7 PDR -45 NASA Screening Team NASA Lvl III Review & Filter RIDs (Work disconnects with external stakeholders) (LM SEIT led SME team available to answer questions) RID Tool Open TBD RID Tool Close (all RIDs Sponsored) 7/31 DRDs Redelivered NLT 2/1/10 Pre Board 5 1/2 Day Pre-Board to work issues in parallel with completion of RID Disposition Orion PDR All RIDs receive product owner or lead recommended Disposition 8/17 Starts 8/3 Starts 8/12 Product Owner Recommend Disposition Starts 8/17 8/4 Board Decision Gate 8/10 Forward Daily Board Decision Gates S Integ Team PIT Lead with Module/IPT Support Starts 8/27 PIT War Room 8/31-9/1 PDR Board Review Team Disposition Dry- Runs Subsystem Design Reviews (SSDRs) 2-Wk SSDRs SMR Prep June July August Sep Oct
  • PDR Key Elements
    • Technical Reviews (Tabletops)
      • Provided participants an opportunity to identify and resolve design, integration, document and/or process issues prior to formal product/document review
        • Working meetings that provide participants the opportunity to work design issues real time
      • Provided a joint NASA/LM forum for horizontal integration
      • Technical Reviews can be iterative, as required
    • Peer Reviews
      • Focused primarily on detecting and eliminating defects/findings in critical work products or DRDs.
        • More formal than a Technical Review
        • Facilitated by a trained moderator
        • Subject matter experts discuss issues identified within the work products or DRDs and strive for concurrence on the identified defects and recommended solutions
        • Includes a formal closure of all actions/findings
      • Provided a joint NASA/LM forum for horizontal integration
  • Subsystem Design Reviews
    • Subsystem Design Reviews (SSDRs)
      • Provided the Orion stakeholders the opportunity to review each subsystem’s preliminary design
      • Provided stakeholders the opportunity to review each subsystem’s preliminary design, including GFE and ADP as appropriate.
      • Products reviewed at SSDRs provided objective evidence that the preliminary subsystem design meets requirements with acceptable risk
      • Additional Objectives included:
          • Reviewing the preliminary design as documented in the technical design products for each subsystem
          • Participating in a face-to-face open forum, raising issues/concerns, answering questions or clarifying design details
          • Providing forward plans for significant issues from preceding reviews
          • Demonstrating horizontal integration with other subsystems
          • Reviewing the design against functional and performance specifications and ground and mission operability characteristics
          • Evaluating the technical adequacy and maturity of design
          • Ensuring correct design options have been selected
          • Ensuring interfaces have been identified and defined
          • Reviewing the verification approach
          • Reviewing associated risks and risk management strategies
          • Evaluating the Subsystem readiness to proceed to PDR
  • System & Module Review (SMR)
    • System & Module Review (SMR)
      • Provided an overview of the DAC-3 close-out (606G) vehicle configuration and system architecture
      • Provided an opportunity to ensure all 1500+ PDR reviewers developed a common understanding of the following:
        • System architecture and PDR technical baseline
        • Ground systems and operations requirements and plans
        • Flight operations by mission phase
      • Provided a status on major open system, module, and subsystem issues
      • Provided clarification of what products were officially delivered for PDR data package review
      • Provided the RID criteria and ground rules
      • Served as the kick-off for the official DRD review and acceptance process
    • RID Review Process culminating in PDR Board
      • DRDs waterfalled over 3 months leading up to the start of the official DRD review
      • Provided NASA four weeks of dedicated DRD review
      • Eight joint NASA/LM review teams reviewed, integrated, and dispositioned 3000+ initial RIDs
        • Teams operated in parallel for 8 days resulting in 1061 RIDs (after consolidation of like issues)
      • Conducted a 7 day Pre-board chaired by the NASA Vehicle Integration Manager
        • Reviewed the final criteria assessment and worked to disposition remaining RIDs
      • Conducted a 1 day Board chaired by the NASA Orion Project Manager
  • PDR RID Process Flow
  • Orion PDR Data Package Review
    • 170 DRDs Delivered & Reviewed
      • 100,000+ pages of design, process, and plan documentation
      • Requirements, Interface Definition, Drawings, Verification Details, General Plans, etc.
    • 1470 RID Initiators (DRD Reviewers)
      • 520 unique initiators wrote a RID (35.4%)
      • 399 unique initiators with an approved RID (76.7%)
    • 3625 Sponsored RIDs
      • 2049 were approved (56.5%)
    A key CDR streamlining goal is to improve the overall quality of RIDs while reducing the number of RIDs against known, out of scope or duplicate issues.
  • PDR Lessons Learned
    • PDR Lessons Learned were solicited from all participants
      • Orion team and all reviewers
    • Significant Lessons Learned Include:
      • SSDRs were very successful; however, several could have used more time and fewer could have been conducted in parallel
      • Modify SMR to a forum similar to SSDRs
      • Merge the Screening and Review Team steps in the RID Process
      • More time was needed between Review Teams and the Board
      • Initiators rewrote Tech Review actions as RIDs – due to lack of TR feedback
      • Minimize duplication of Pre-declared RIDs, SSDR actions, and Errata
        • 597 RIDs were duplicate to Pre-Declared RIDs during PDR
      • RIDs/issues were discussed several times during the PDR Process and decisions on issues were not made effectively
      • RIDs/issues were not Sponsored effectively - out of scope issues clogged process
      • Reviewers attending by Webex and telecon were unable to support effectively
      • Improve how contractual/cost issues are addressed during CDR
    Executing a Lessons Learned process made streamlining CDR much easier. All stakeholders quickly aligned to ensure an efficient thorough CDR process was developed.
  • CDR Streamlining Focus
  • Incremental DRD Delivery
    • The DRDs are divided into four groups
        • CDR DRDs (Design Data Books, ICDs, Drawings, etc.) ~104 DRDs
        • Post-CDR DRDs (Reports, Plans, etc.) ~ 21 DRDs
        • Pre-CDR DRDs ~ 73 DRDs
        • Software CDR DRDs ~ 125 DRDs
    Action Item / RID Process Project DRD Review & Approval Process Process TBD Nearly half of all DRDs originally linked to CDR for delivery will be adjusted to allow delivery based on need (design, procurement, etc.). This provides increased project flexibility in prioritizing work and increases overall CDR efficiency and review quality. * Component Specs & Supporting documents are developed & delivered based on procurement need. Reduced core set of DRDs to be reviewed and accepted as part of the SSDRs & CDR DRDs delivered per PIT managed schedule. Reviewed and accepted through NASA’s normal C&DM process. Focus 1: Improve the effectiveness of reviewing and accepting DRDs at CDR
  • Pre-CDR DRDs CDR Start CDR + 90 Redelivery DRD Delivery Draft DRD Posted on Portal days who Note: Days are in business days. 20 business days (4 weeks) Pre-CDR DRDs Include
    • Con Ops
    • Ground Sys Reqt’s
    • Safety Docs
    • Fault Tree Analysis
    • Lunar Upgrade Plan
    • Workmanship Stds
    • Model & Sims Plans
    • Sw Dev Plan
    • IRDs
    Pre-CDR DRD Review & Delivery Flow
    • DRD Quantity: 73
    • Time to Review: 2 Weeks
    • Review Team: IPTs
    • Re-delivery: CDR +90 days
    • Comment/Action Mechanism: Excel or ConCERT
    Review & Delivery Details Tech/Peer Review Input Feedback LM CM/DM Post Portal DRD Review Consolidate Comments Comment Disposition VICB Summary Post Final Comments Contract Letter 1-2 15 5 <1 10 1 5 1 2 3 NASA & LM Tech POCs LM Book Owners LM CM/DM NASA CM/DM NASA Reviewers NASA CM/DM DRD POC (2), TCM (2), & IPT (1) VICB Outbrief NASA CM/DM & DRD POC NASA COTR & PIT Focus 1: Improve the effectiveness of reviewing and accepting DRDs at CDR
  • CDR DRDs CDR Board CDR + 90 Redelivery DRD Delivery Draft DRD Posted on PL days who CDR Start Note: Days are in business days. CDR DRDs Include
    • CAD Models
    • Drawings
    • MVP
    • ICDs
    • ADD
    • Mass Properties
    • Data & Command
    • Dictionary
    • Avionics DDB
    CDR DRD Review & Delivery Flow
    • DRD Quantity: 104
    • Time to Review: 2 Weeks
    • Review Team: SSDR Caucus
    • Re-delivery: CDR +90 days
    • Comment/Action Mechanism: IDE Action Tool
    Review & Delivery Details Tech/Peer Review Input Feedback LM CM/DM Post on Portal DRD Review SSDR Caucus Reclama Loaded as IDE Action Contract Letter 1-2 15 5 <1 5+ 5 5 1 2 3 NASA & LM Tech POCs LM Book Owners LM CM/DM NASA CM/DM NASA Tech POCs SSDR Co-Chairs & Joint PIT SSDR Co-Chairs & Tech POCs MRT & Board, As Required LM PIT & MRI NASA COTR & PIT Focus 1: Improve the effectiveness of reviewing and accepting DRDs at CDR
  • Post-CDR DRDs CDR Board DRD Delivery Draft DRD Posted on Portal days who Note: Days are in business days. 20 business days (4 weeks) Post-CDR DRDs Include
    • Ops Data Book
    • Test Procedures
    Post-CDR DRD Review & Delivery Flow
    • DRD Quantity: 21
    • Time to Review: 2 Weeks
    • Review Team: IPTs
    • Re-delivery: TBD
    • Comment/Action Mechanism: Excel or ConCERT
    Review & Delivery Details Tech/Peer Review Input Feedback LM CM/DM Post Portal DRD Review Consolidate Comments Comment Disposition VICB Summary Post Final Comments Contract Letter 1-2 15 5 <1 10 1 5 1 2 3 NASA & LM Tech POCs LM Book Owners LM CM/DM NASA CM/DM NASA Reviewers NASA CM/DM DRD POC (2), TCM (2), & IPT (1) VICB Outbrief NASA CM/DM & DRD POC NASA COTR & PIT Focus 1: Improve the effectiveness of reviewing and accepting DRDs at CDR
  • RID Quality & Quantity
    • Increase education of external initiators with Orion CDR Kick-off
      • Model after LM’s Orion Orientation Training
    • Enhance sponsor selection and training
    • Increase the quality of Pre-Declared RIDs
      • No automatic conversion of SSDR Actions not complete by PDR
    • Increase training of RID Initiators
      • Create 1-Page Quick Reference Guide emphasizing Good/Bad RID Criteria
    • Re-examine initiator list
      • 35.4% of Initiators actually submitted a RID (520 of 1470)
      • 76.7% of Initiators who submitted a RID had an approved RID (399 of 520)
    Focus 2: Improve the quality of RIDs and reduce overall quantity
  • Action & RID Creation & Disposition
    • Tech Reviews were not effective in communicating issue resolution
      • Resolution: Improved closed-loop process will communicate dispositions of issues raised at Tech Reviews to avoid repeated Action Items/RIDs during CDR
    • RIDs/issues were discussed several times during the PDR Process and decisions on issues were not made effectively
      • Resolution: All Action Items/RIDs will come through SSDRs, decreasing the amount of times issues are discussed
      • Resolution: Management Review Team created to make decisions on Action Items/RIDs while SSDRs are being held
    • RIDs/issues were not Sponsored effectively, causing out of scope issues to clog the PDR Process
      • Resolution: Improved process of raising all issues at SSDRs will greatly improve Action Item/RID Sponsoring and increase the efficiency of dispositioning during CDR
    Focus 2: Improve the quality of RIDs and reduce overall quantity
  • CDR SSDR Action Item/RID Process PROPOSED FLOW Path Legend 1. Reject 2. Reclama 3. Accept 4. Accept with Mod 5. Merge 6. Elevate for Discussion Process Details
    • Initiators review DRDs prior to SSDR and note issues
    • Initiators required to raise issues during the SSDR
    • Co-Chair ruling officially “sponsors” the Action/RID
    • SSDR Caucus makes initial disposition on Actions/RIDs
    • Recommended dispositions posted on the CDR Portal
    • Management Review Team (MRT) handles Reclamas, issues with Cost/Schedule/Risk, and “Big Red” Issues
    • CDR Board formally approves Action/RID disposition
    • Addresses Reclamas
    • - All Actions/RIDs w/ their dispositions are placed on the CDR Portal for project historical record
    • Approved Action/RIDs are input to IDE for LM action tracking
    • Updates placed on the CDR Wiki for initiator tracking
    Focus 3: Improve the efficiency of the Action/RID Process & lifecycle in CDR
  • SSDR Action Item/RID Process Flow
    • Before an Action Item/RID is written, a question must be asked at the SSDR
      • If not resolved, an Action Item/RID is documented
      • Question/Answer session is held at the end of the day to give reviewers an opportunity to ask a question that may have been missed earlier
      • Additional time is provided at the end of the week for questions/answers
    • Action Items/RIDs is dispositioned in the Caucus
    • Disposition of Action Item/RID is provided to reviewers the following day
      • If reviewer disagrees with the disposition of the Action/RID, they bring up that they disagree with the disposition with the Co-Chairs (5-10 minutes given after lunch for people to let Co-Chairs know if the disagree with the disposition)
      • If not resolved prior to the caucus, the reviewer is invited to the caucus to discuss issue
    • Once all Action Items/RIDs are dispositioned, the Co-Chairs identify the major design issues for the subsystem
      • Detailed Forward Plans are developed for the major issues
    Focus 3: Improve the efficiency of the Action/RID Process & lifecycle in CDR
  • SSDR Action Item/RID Process Flow
    • All major design issues, any reclamas/disagreements, cross-team integration issues, and cost issues (> $500,000 (TBR)) for each subsystem are discussed at the Management Review Team (IPT leads appropriate to SSDR, Chief Engineers, Crew Rep, MOD rep, chaired by Mark Geyer and Cleon Lacefield) the week following the SSDR
      • Detailed Forward Plans are discussed and are modified/approved for each major issue
      • Any reclama or disagreement is resolved in the Management Review Team
      • Pre-Board topics are identified by the Management Review Team
    • Major issues will be presented at the Pre-Board after the forward work plan has been reviewed and approved by the Management Review Team
    Focus 3: Improve the efficiency of the Action/RID Process & lifecycle in CDR
  • Action Item/RID/Comment Dispositions Reject Accept With Mod Accept Dispositions will be used in the Caucus and Review Teams All Approved Action Items/RIDs are expected to be incorporated in the CDR redelivery. This ensures a consistent and well documented technical baseline. Disposition Definition
    • Issues identified that are outside of the scope of CDR
    • - Normal forward work items will not be accepted
    • Issues that are not accepted by SSDR/IVDR/MRT/TMR/Board
    Design issue that needs to be resolved to ensure an appropriate level of CDR maturity is achieved along with resolution EXACTLY AS WRITTEN BY INTIATOR Same as accept; however, additional tailoring or editing to the action or resolution is needed Focus 3: Improve the efficiency of the Action/RID Process & lifecycle in CDR
  • Overall CDR Process
    • Enhance Technical Reviews of the design work products with the correct NASA Orion Level 3 participation
      • Design Data Books will receive significant review support and serve as the NASA Orion Level 3 chance to review the design at the subsystem level
      • DRDs are provided to TR participants 5 to 10 days prior to the review
        • Review dates are documented in a easily accessible integrated schedule
        • Schedule will be deconflicted where possible to allow for Project support
      • Minimum required stakeholders will be identified for all DRDs
        • Each DRD has an assigned NASA POC to ensure stakeholder selection is correct and the work product receives adequate review
      • TRs must be given high priority by each appropriate Orion organization
        • Appropriate SMEs are expected to support DRD TRs
      • Note: Official delivery to NASA occurs after the TRs per the J-2
    • Software CDR and Component CDRs will occur separately from the System CDR
      • The combination of System CDR and Software CDR will satisfy compliance to NPR 7123
    Focus 4: Improve the efficiency and effectiveness of the reviews.
  • SSDRs
    • Additional time is given during each SSDR to display/discuss the design in the Interactive Reading Rooms (IRRs)
      • SMEs are available in the IRRs to answer/discuss design questions
    • General Improvements to Process
      • Caucus extended into the following week
        • Reduce overload on Co-Chairs
      • Stacked/Overlapping Schedule
        • 3-4 SSDRs per week for ~5 weeks
      • SSDRs deconflicted so Co-Chairs can participate in review and Caucuses
    SSDR Chairs never chair a review on consecutive weeks. Remain focused on an SSDR until Caucus completion. Project Management available throughout process to resolve issues quickly. Focus 4: Improve the efficiency and effectiveness of the reviews.
  • Critical Design Review Process
  • CDR Management Challenges
    • Planning Challenges
      • Develop and execute a Critical Design Review (PDR) process that ensures NASA’s next generation spacecraft can meet all mission objectives and satisfy requirements with acceptable risk
      • Develop a multi-tiered approach that ensures the integrated subsystems, modules and vehicle are reviewed to ensure the design approach is at a sufficient level of maturity to production, assembly, integration, and test.
      • Balance thoroughness and schedule. The desire to review every aspect of the design down to the lowest possible level must be balanced against the available time to execute the review as to not delay the development activities unnecessarily.
      • Balance inclusion and efficiency. Develop a process that ensures all stakeholders have an adequate time to participate and provide input helping to shape the design, ensure mission success, and ultimately provide a safe crew exploration vehicle.
  • CDR Management Challenges
    • Execution Challenges
      • Communicate the expectations to a diverse 2000+ personnel organization consisting of NASA, prime contractor (LM), and many support contractors
      • Balance the desire to improve the design with the need to efficiently transition to production.
        • From the time the CDR baseline is struck, a lot of potential energy develops to revisit the requirements (did we miss something?) and improve the design (could it be better?).
          • This energy must be managed to ensure that identified issues and design refinement are necessary to ensure mission success and crew safety and not simply to resolve an “I would have did it differently” comment
          • CDR products must align as close as possible to the CDR baseline.
      • Monitor and control process execution throughout the duration of the effort
  • Keys to CDR Success (Operational Excellence)
    • Teamwork
      • Created a joint NASA/LM CDR Program Integration Team (PIT) to ensure common expectations are communicated across the NASA, LM, and support contractor teams
        • Key personnel from each Module and IPT are included in team
        • Planning and C&DM personnel incorporated into team
    • Communication
      • Focus on communicating early and often
      • Conduct a series of TIMs and Training Sessions with each of the subsystem leads to develop common expectations and identify issues needing resolution prior to CDR
      • Institute multiple statuses and forums targeting all stakeholders
    • Commitment
      • NASA and LM each assigned a manager responsible for ensuring CDR success
      • With the goal of “Get it right the 1 st time”, the Orion Management team routinely stresses the importance of attaining the CDR level of design maturity
      • Management must routinely communicate the CDR expectations and monitor status weekly
    • Accountability
      • Each Subsystem Lead must take ownership of their respective Subsystem Design Review preparation and execution
      • A detailed schedule will be developed to manage the data package review and delivery
      • Detailed metrics will be developed for tracking all deliverables by System and Subsystem
  • CDR Objectives & Criteria
    • CDR Objectives
      • Demonstrate that the maturity of the design is appropriate to support proceeding with full scale fabrication, assembly, integration, and test.
      • Determine that the technical effort is on track to complete the flight and ground system development and mission operations, and to meet mission performance requirements within the Project’s cost and schedule constraints.
      • Enable the Project to baseline the component specifications.
    • CDR Criteria
      • NASA plans to authorizing contractor to use their own Command Media criteria for a successful CDR (Significantly greater detail than NPR 7123.1A)
        • PIT developing a detailed map of NPR 7123.1A criteria to LM criteria
      • NASA/LM IPTs and Subsystems will develop a detailed self assessment against the entry and success criteria
      • Subsystem Criteria will be evaluated at the SSDRs
      • System & Module CDR Criteria will be evaluated first at the Pre-Board and final at the CDR Board.
    PIT will mature CDR Objectives and Criteria, coordinate with all impacted stakeholders, and document in the CxP 72422 Orion CDR Plan.
  • CxP 72422 Orion CDR Plan
    • A comprehensive CDR plan will be documented in CxP 72422 and placed under configuration control
      • All significant stakeholders will participate in review of document prior to release
      • Formalizes the overall CDR process
    • Documents the requirements and associated documentation baseline for CDR
    • Incorporates NPR 7123.1A criteria into plan (includes any tailoring)
      • Serves as the primary method of documenting all agreements regarding method to meet the SSDR and CDR criteria
    • Used as a reference throughout the execution to ensure the process is adhered to
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • Majority of Component CDRs will be executed prior to CDR
      • For component CDRs which occur post CDR an Interim Design Review should be conducted prior to CDR
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • CDR Kickoff
      • Purpose: kick off the Orion Critical Design Review and to provide an overview of the DAC-4 close-out (607B) vehicle configuration and system architecture
      • Goals:
        • Common understanding of the system architecture and CDR technical baseline (Requirements, Environments, etc.)
        • Common understanding of the Design Reference Missions
        • Common understanding of flight operations by mission phase
        • Status on significant design issues and their LM/NASA agreed to mitigations/closure plans
        • Clarification of what design documents are available at CDR and what design reviews the DRDs trace to
        • Action Item/RID criteria and ground rules
      • 1 Day Review
        • Available to all CDR participants (e.g. NESC, SRB, NASA Crew Office, NASA MOD, NASA Engineering, NASA S&MA, NASA Health & Medical, Constellation, Orion Project, & LM Central Engineering) but tailored to external stakeholders
      • A better educated reviewer will provide improved feedback and support to the review
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • Systems Analysis Review
      • Prior to the SSDRs, a System Analysis Review (1 to 2 days) will be held to educate the reviewers on the integrated analysis products that apply to multiple subsystems (Loads, Environments, Mass, etc.)
        • Begin taking Action Items/RIDs at the beginning of the System Analysis Review
      • Topics:
        • Environments
          • Aerosciences
        • Loads and Dynamics
          • Thermal Analysis
          • MMOD
        • Specialty
          • Radiation
          • EMI/EMC
    • Special SSDR-like reviews will address T&V and S&MA during this period
      • T&V Review will cover the integrated T&V activities (module-level, external interfaces, facilities, and common Subsystem T&V)
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • SSDRs and Interactive Reading Rooms (IRRs)
      • CDR SSDRs will be conducted similarly to PDR SSDRs
      • SSDR data packages are dropped 5 business days prior to the beginning of the SSDRs
      • Action Items/RIDs will be taken during the review
      • A dedicated IRR for each subsystem is available to all reviewers during each SSDR and contain the following
        • Hard copies of significant design products and/or computers with all pertinent review data (Requirements, Compliance Data, Drawings, Analyses, Test Results)
        • IRRs are staffed at all times to answer questions or show the design work products; runners available to get SMEs
        • Test or prototype HW/SW if available
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • SSDR Caucuses
      • Purpose: disposition Action Items/RIDs identified during the SSDRs, review forward action plans, and complete the Subsystem criteria assessment
      • Membership: SSDR Co-Chairs, SS Leads, Module Rep, SEIT rep, ITA reps as appropriate (8-10 people in Caucus)
      • Action Items/RIDs are dispositioned daily
      • Dispositions are provided to reviewers the next day
      • Reviewers reclama the disposition with the Co-Chairs at the SSDR if they disagree (5-10 minutes given after lunch for people to let co-chairs know if the disagree with the disposition)
      • If not resolved prior to the caucus, the reviewer is invited to the caucus to discuss issue
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • Management Review Team (MRT)
      • Purpose: to review all major design issues, any reclamas/disagreements, cross-team integration issues, and cost and schedule issues (> $500,000 (TBR))
      • Membership:
        • Chairs: Mark Geyer and Cleon Lacefield
        • IPT, Chief Engineers, Crew, MOD, and other ITA rep as needed
      • “ Hot” issues will be elevated to the MRT after the caucuses to discuss the issues and provide integration at the IPT level
        • Detailed Forward Plans are discussed and modified/approved for each major issue
        • Any reclama or disagreement is resolved in the MRT
        • Pre-Board topics are identified by the MRT
      • Allows SSDR caucuses to disposition all Action Items/RIDs and the IPTs to view all the major issues and provide feedback
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Orion CDR Process CDR Kickoff CDR DRDs Review Interactive Reading Room (DRDs, Drawings, Etc.) SSDRs SSDR Caucuses System Analysis Review / T&V / S&MA (1 to 2 days) Reviews Begin (Interactive Reading Rooms available) Pre-Board Board Integrated Vehicle Design Review IVDR SSDR Material Dropped Tool Closed for all SSDR DRDs Only Major Issues and Disagreements are brought Post-CDR DRDs Component reviews Executed or a Interim Design Review conducted for those reviews not scheduled prior to CDR Management Review Team (as needed) IVDR Caucus Product Owners/SS Leads provide action plan and feedback to Caucuses Pre-CDR DRDs Tool Closes for SSDR Products as SSDR concludes Tool Closed for all DRDs
    • Integrated Vehicle Design Review (IVDR)
      • A 5-day IVDR follows the SSDRs by one week
      • The IVDR is treated similar to the SSDRs
      • All design DRDs that did not trace to an SSDR will trace to the IVDR
      • Candidate Topics:
        • Ground/Flight Operations
        • Integrated Vehicle Performance and Margins (operational mitigations to hazards)
        • Integrated Aborts Performance and Margins
        • Critical Events as a function of Orion 1 and Orion 2
        • External Interfaces
        • Logistics (Either covered here or in AI&P Review)
        • Module Interfaces
        • Integrated Module and System Test and Verification Summary
        • Module and System Assembly, Integration, and Production
        • Vehicle level hazards and reliability
      • Chairs: Chuck Dingell and Bill Johns
        • The Management Review Team (MRT) will convene to handle issues from the IVDR Caucus
    Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11
  • Verifier Rejects Verifier Closes Action Item/RID/Comment Closure Process Rework / Update LM IDE Assignee Works Issue & Submits Result to Initiator Process Detail
    • Action Item/RIDs are dispositioned during the SSDR Caucus
    - Approved Actions/RIDs/Comments are entered into the LM IDE Action tracking tool - Estimated closure dates are established with rationale
    • Assignee resolves issue and documents in IDE
    • IDE notifies initiator of “issue answered”
    • Initiator either accepts or rejects closure
    • The verifier is identified by the NASA SSDR Co-Chair
    • IDE notifies verifier of issue status
    • - Verifier reviews and either closes issue or rejects closure
    - If initiator, verifier, and assignee don’t agree that Action/RID/Comment is closed, the issue will be elevated to VICB/CPCB as required - Verifier dispositions as “Reject” or “Close” in the LM IDE Approved Action/RID/Comments Verifier Reviews Closure Rationale VICB (if necessary)
  • CDR Action/RID Process Project Orion Initiator Action/RID Flow SSDR CDR Board Caucus MRT Withdrawn Actions/RIDs Approved Actions/RIDs CDR Disposition Process Reduce Number of DRDs Structured Improvement Structured Improvement PDR Disposition Process PDR Process Elevated RIDs
  • CDR Streamlining Benefits
    • SSDR sponsoring of Action Items/RIDs will greatly improve the Sponsoring step and decrease the amount of instances issues are discussed throughout the process
    • Having all pertinent reviewers in the room will increase the effectiveness of the review and ensure all stakeholders are aware of issues that are discussed
    • Reviewing only design DRDs during the CDR Process will decrease the amount of Action Items/RIDs and spread out the time and effort needed to review and accept all the documents
    • The design will become the focus of the reviews during CDR
  • Closing Remarks
  • CDR Supporting Documentation
    • CDR Plan
      • CxP 72422 CxP Orion PDR Plan will be available on the CDR Wiki and CDR Portal/Dashboard
    • PDR Resources
      • CDR Wiki
        • Used as a resource to all Orion personnel and CDR stakeholders to communicate the current planning status
        • Once created, users can find the link from the Orion Vehicle Integration Wiki
          • https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=12793039
      • CDR Portal
        • Used to formally provide all SSDR and CDR details, data package, and review results
        • Federal record of review
        • Once created, users can find the link from the Orion Vehicle Integration Wiki
          • https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=12793039
          • Or the ICE Portal CDR Top-level Site
          • https://ice.exploration.nasa.gov/ice/site/cx/review_cdr/
    Throughout CDR planning and execution activities, frequent communication and easy access to CDR related data is essential to ensure success.
  • CDR Integration Team As the program nears the more active CDR planning phase, additional CDR Primary Integration Team (PIT) members will be identified.
  • Questions?
    • Backup Charts
    • OFT1 PTRs
    • Orion CDR DRDs Comparison
    • PDR Results Summary
  • OFT-1 Periodic Technical Review (PTR) Scope & Objectives
    • PTR Series (1, 2, & 3) Scope & Assumptions
      • The purpose of PTR process is to:
        • Establish the OFT-1 requirements baseline and the associated DRM
        • Disclose the complete system design in full detail
        • Show that the analysis validates the current design
        • Ensure the system design meets the requirements and DRM
        • Ascertain that technical problems and design anomalies have been resolved
        • Ensure that the design maturity justifies the decision to initiate fabrication/manufacturing, integration, and verification of hardware and software
      • As a result of successful completion of the three PTRs, the OFT-1 “build-to” baseline, production, and verification plans will be approved.
      • PTRs will establish compliance to LM’s CDR tailored criteria for OFT-1
        • Where possible the PTRs will establish compliance to the CDR criteria for Orion-2
        • The Orion-2 CDR and associate SSDRs will complete the Orion CDR process
      • PTRs can be viewed as analogous to traditional program reviews with reduced data-package expectations
        • Data-packages consist of design and analysis products (not DRDs)
        • DRD deliveries are not aligned to PTRs
          • DRDs aligned to need dates
  • Orion PTR Board Members
    • PTR Chairs are responsible for:
      • Presiding over PTR and for making decisions based on inputs from Board Members
      • Judging adequacy of the PTR
      • Making final decisions on Action Item dispositions
      • Presenting results of the PTR to next controlling auth.
    • PTR Board Members are responsible for:
      • Participating in technical interchange
      • Supporting the Chairs on issues or actions
      • Supporting the reporting of findings and recommendations made by the Chairs
      • Voting following the completion of the PTR
    PTR Board Members PTRs Project Management NASA & LM Tech Auth – Chief Engineer NASA & LM (Chair) Tech Auth – S&MA NASA Tech Auth – AI&P NASA VIO/SEI NASA & LM CSM NASA & LM LAS NASA & LM T&V NASA & LM S&MA NASA & LM Production Ops NASA & LM COTR NASA Mission Ops NASA
  • OFT-1 Data Flow
    • DRDs delivered to NASA and reviewed by NASA
    • TCM held with LM SMEs to discuss comments
    • Comments are provided back to LM contractually
    • Type 2 documents require NASA approval
    OFT-1 DRDs OFT-1 Specification OFT-1 Verification Plan OFT-1 Post-Flight Data Package OFT-1 Data OFT-1 Integrated Data Plan OFT-1 Data Per the Integrated Data Plan Delivered to NASA Uploaded to ProjectLink Managed by LM
    • Data is uploaded to ProjectLink per the Integrated Data Plan
    • Data is managed on ProjectLink by LM
    • NASA has access to all of the OFT-1 Data once uploaded (per the Integrated Data Plan)
    • Technical Baseline data is accessible per the Integrated Data Plan
    Data Type 2 Data Type 2 Data Type 3 OFT-1 Integrated Data Schedule
    • Optimization team will develop the plan
      • SEIT owned after release
    • Planning will develop the schedule
      • SEIT & C&DM will support
      • Planning owns integrated schedule
    • IPTs develop data/documentation
      • C&DM responsible for maintaining
      • & populating data repository
    DRAFT
  • OFT-1 Data/Work Product Scheduling CSM Scheduling Managing LAS AVP Systems Analysis SEIT ATLO IPT/Lower Level Schedules Integrated Data Schedule Data Products not captured in Lower Level Schedules Lower Level Schedules are networked with the Integrated Data Schedule
    • IDS Weekly Report:
    • Soon to come due
    • Changes to dates
    • Metrics
    • Status of data/work products (G/Y/R)
    PIT Review - SEIT, C&DM, Planning d - IPTs as needed Issue? Yes No Morning Tagup Briefing Review the IDS status and highlight any overdue data/work products Ensures changes to schedules align with project needs Developed based on the most current IDS Requires additional coordination with the lower-level scheduling team Stop DRAFT
  • Incremental DRD Sets for Orion CDR Timing Pre-CDR CDR Post-CDR Quantity 73 104 21 Types of DRDs Con Ops Ground Sys Reqt’s Safety Docs Fault Tree Analysis Lunar Upgrade Plan Workmanship Stds Model & Sims Plans Sw Dev Plan IRDs CAD Models Drawings MVP ICDs ADD Mass Properties Data & Cmd Dictionary Avionics DDB Ops Data Book Test Procedures Time to Review 2 weeks 2 weeks 2 weeks Review Team IPTs SSDR Caucus IPTs Re-delivery CDR + 90 days (Delivered DRD + Approved Comments are reference) CDR + 90 days TBD Comment Mechanism Spreadsheet or ConCERT* IDE Action Tool Spreadsheet or ConCERT*
  • PDR Assessment
    • Design Issues
      • A detailed pre-assessment was performed by each of the module prior to entering PDR
      • All known issues were presented at each SSDR and the SMR
        • No new significant issues surfaced during the reviews
      • Significant focus was applied to ensure known issues had agreed-to plans to proceed
      • Top issues contributing to non-green assessments
        • Mechanisms – FBC R&R – FBC Recontact, CM/SM R&R Maturity
        • LRS – FBC Recontact / Parachute Architecture & Testability
        • TPS & Structures – Parachute Architecture Impacts to Baseline
        • EPS – 120 V Incorporation
        • AV&SW – C&T Incorporation, BFCS Maturity
        • ECLS – ARS Suit Loop, Procurement Schedule
        • CM Prop – Procurement Schedule
        • Pyro – Devices Dependent on Mechanism Maturity
        • LAS ACM – Production Motor Maturity
    • 18 Subsystem Design Reviews (SSDRs) executed successfully
      • Each SSDR assessed against a common set of entry and success criteria plus a joint set of NASA/LM agreed to objective evidence customized for each Subsystem
      • 59 Total Review Days & approximately 500 review participants
    • PDR executed successfully
      • Each module assessed against a common set of entry and success criteria; ITAs assessment integrated
      • 56 days of DRD review, 7 day System & Module review, 1 week pre-Board and a Board
      • Approximately 1200 reviewers
    Yellow / Green
    • Project Management assessed PDR a success and authorized the program to proceed to the detailed design phase
    • Assessment based on a Red, Yellow, Green scale
    Parachute Architecture Study was underway prior to entering PDR. Assessment concluded in October 2009. Driven by significant post SDR requirements changes Procurement schedules were being updated to align with project funding profile
  • PDR Data Package Delivery
    • PDR Data Package
      • NASA/LM jointly assessed PDR deliverables to determine which DRDs were necessary to meet PDR expectations and satisfy criteria
        • PDR –
          • 146 Deliverable Documents
        • SSDR & PDR –
          • 26 Design & Data Books
        • Data Package Total
          • 172 Total PDR Documents
          • 53 “Drafts Available” DRDs provided to NASA for comment
    • Waterfalled Review and Delivery
      • Managed via the PDR Integration Team
        • Captured in a 2000+ line MS Project Schedule
        • Metrics generated & reported on a weekly basis
        • Deliveries culminated on 7/7/09, PDR-45 Days
    As of July 8, 2009 PDR -45 days
  • Action Item & RID Closure
    • Detailed Action and RID screening occurred throughout the SSDR & PDR process
    • All actions and RIDs dispositioned and closure aligned with project post PDR work plan
    • All RIDs approved by PDR Board
    • Project approved closure plan implemented
    • Approximately 100,000 pages of design, process, and plan documentation reviewed
    • Included requirements, interface definition, verification details, general plans, etc.
    PDR RIDs LAS 150 CM 241 SM 90 Av&Pwr&Wire 444 Software 38 T&V, Fac, GSE 411 AI&P 18 SR&QA 91 Systems 566 Total 2049 SSDR Actions AV&SW 140 Crew Systems 39 ECLSS 49 EPS 97 GN&C 26 LAS AM 60 LAS ACM 96 LAS JM 60 LAS SE&I AVP 12 LRS 42 Mechanisms 56 Propulsion 150 PTC 33 Pyro 26 Structures 141 TPS 20 Wiring 14 Total 1061