Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Ares I-X Requirements and Verification Lessons LearnedPM Challenge 2011<br />Kevin Vipavetz, Ares I-X Requirements and Verification Manager<br />Systems Engineering and Integration Office/Engineering Directorate <br />November 30, 2010 <br />1<br />
  2. 2. Agenda<br />Ares I-X Requirement and Verification Successes<br />Independent Reviews<br />Success factors<br />Lessons Learned<br />2<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  3. 3. Ares I-X Requirements and Verification Successes <br />Multiple NASA Centers successfully implemented system engineering best practices and used the same formats and processes for requirement and verification development.<br />Requirements and verifications had complete accountability (possessed ownership).<br />Requirements were tracked and linked between all requirement levels.<br /> All verifications had a one-to-one traceability to requirements. <br />Tracked and closed 100% of all system and element waivers prior to launch.<br />Verified 100% of all system and element level requirements prior to launch.<br />All verification artifacts configuration controlled and stored on Windchill including supporting documentation. <br />3<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  4. 4. Independent Reviews<br />Positive Lesson –<br />Independent reviews contributed extensively to the success of the mission.<br />Independent reviews provided a special oversight of processes and helped ensure mission processes are adhered to. <br />Independent reviews provided an additional check that requirements are complete, well communicated and reduce the possibility of missing requirements. <br />Independent reviews helped enforce issues to closure. <br />Independent reviews are interested in your success and provide in-depth detailed reports in areas you may not have thought of. <br />Ares I-X used independent reviews extensively for requirements and verification development. <br />4<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  5. 5. 5<br />Success Factors:<br />Ares I-X Requirement Owners<br /><ul><li>Requirement owners were key to getting the right requirements developed and verified.
  6. 6. Ares I-X had requirement owners at the system, element and contract levels.
  7. 7. All verification activities had to be collaborated with the system requirement owners.
  8. 8. This provided insight to the verification processes two levels down for the system requirement owners.
  9. 9. Ensured that verification activities would meet system verification requirements. </li></ul>Ares I-X Requirements and Verifications Lessons Learned<br />5<br />11/30/2010<br />
  10. 10. Success Factors:Good System Requirements<br />Ares I-X modeled system requirements development after the CxP Requirements Engineering Management Plan (REMP) and followed the NASA System Engineering Handbook. <br />Commitment to SE&I processes. <br />Communicated and negotiated through work groups, summits, technical interchange meetings.<br />Requirement Managers took requirement training classes. <br />Had SE&I representatives at each NASA center and contract sites.<br />Used Integrated Design Centers to develop requirements. <br />Had independent reviews and audits and worked closely with the Ares I-X Chief Engineers Office. <br />Requirements were product oriented and well communicated. <br />Had rationales – which provided requirement background and justification for each requirement.<br />Had traceability – links to parent requirements.<br />Had allocations – links to lower level child requirements.<br />Had verification method attribute – how are we going to measure this?<br />Were configuration controlled.  <br />6<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  11. 11. Success Factors: A Different Interface Development Approach<br />Ares I-X did not use the typical role of Interface Requirement Documents (IRDs) to define requirements for interfaces or use Interface Control Documents (ICDs) as the solutions to the IRDs. <br />IRDs were considered developing ICDs. ICDs were considered established interfaces under configuration control. <br />Interface requirements were defined in the system and element requirement documents. This eliminated redundant interface documentation and enhanced the interface verification process. <br /> The ICD sections became the verification success criteria for the requirement owners on each side of the interface. <br />This process allowed: <br />Each interface section of an ICD tobe verified independently of the other side. <br />Requirement owners toclose out a baselined interface requirements one at a time. <br />Key success factor was including rationales (justification) for each interface definition and any associated agreements negotiated by requirement owners (helped determine roles and responsibilities during interface development). <br />7<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  12. 12. Success Factors: Verification Implementation Process<br />Verification is essential. Projects often write requirements and then later decide on how to verify them. This is significantly less effective as the time between the two activities increases. <br />Ares I-X verifications were developed upfront as the requirements were being developed and then assigned requirement owners. <br />Requirement owners developed separate Verification Requirement Definition Sheets (VRDS) for each requirement. <br />VRDSs contained the following: <br />Verification requirement.<br />Verification requirement trace to the associated requirement.<br />Verification measurement activity description.<br />Verification success criteria.<br />Verification artifacts with Windchill links identified.<br />Associated Waivers.<br />Compliance statement.<br />Sign off section (included Requirement Owner signature).<br />8<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  13. 13. FTV-014 VRDS Example<br />9<br />11/30/2010<br />
  14. 14. FTV-014 Compliance Statement Example<br />10<br />11/30/2010<br />
  15. 15. VRDS FTV-014 Example cont…<br />11<br />11/30/2010<br />
  16. 16. Success Factors: Configuration Management<br />Processes of approval and acquiring document baselines were well defined. <br />All system and element level requirements, interfaces, verifications, drawings, procedures, waivers and analysis went to the Ares I-X Control Board (XCB) for approval.<br />The XCB used board member voting instead of document signatures for approval. <br />This reduced turn around time and allowed for rapid decision making. <br />Votes were recorded and accepted as signatures. <br />SE&I was well represented and was a member of the XCB. <br />Minutes were kept on all XCB meetings and published on Windchill. <br />Key to the Ares I-X success was the tracking, configuration control and closure of all verifications and waivers. <br />12<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  17. 17. Success Factors:Mission Management and SE&I Co-location<br />The Ares I-X Mission Manager co-located with SE&I during the system engineering development phase.<br />The Ares I-X Mission Manager and SE&I Chief co-located with ground operations and the launch vehicle integration team during the system assembly, integration and launch phase. <br />This led to a well informed management, improved communications and team direction, and enhanced the capability to make decisions rapidly. <br />13<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  18. 18. Lessons Learned – Systems Engineering & Integration (SE&I) <br />Lesson<br />It is important to have SE&I buy-in early in mission formulation. Otherwise you can lose control over the system processes. Ares I-X SE&I was set up after IPTs and contracts were in place which made it difficult early on to get complete collaboration over system processes. <br />Recommendation<br />Establish SE&I’s role at the beginning of the mission.<br /> Key: Need to link IPT contracts (prime contractors) to address both the IPTs and the SE&I roles, processes and requirements – this will be difficult.<br />If contracts are already in place then the contract must be set up to allow incorporation of SE&I’s roles, processes and requirements.<br />Co-locate of SE&I managers or leads as soon as possible. <br />14<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  19. 19. Lessons Learned – Product versus Design Verifications<br />Lesson <br />Need to identify where product or design verification apply.<br />Affects when verification paper closure occurs.<br />Affects quality of engineering documentation at CDR.<br />Affects acceptance reviews. <br />Drawings are not a substitute for “as built” documentation. <br />Recommendation <br />Better communication and agreement on what product verification is versus design verification. <br />Be open to other methods of verification practiced by industry.<br />Recognize the impact of verification plans on contracts and pre-engage with contractors and other centers on how verification processes will be supported. <br />Get understanding and buy-in for your verification plan. <br />Baseline agreements by PDR. <br />15<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  20. 20. Lessons – Verification Requirements Closure <br />Lesson<br />Required heroic effort at the end to close all Verification Definition Requirement Sheets (VRDSs) at the element and system levels. This was due misunderstanding the verification implementation process and culture differences. <br />Recommendation<br />Baseline a system level verification implementation plan by PDR.<br />This is not a part of current NASA PDR Exit Success Criteria.<br />Take advantage of “Lean Event” activities between SE&I and IPTs to develop and agree to verification implementation.<br />Get culture buy-in. <br />Provide detailed examples of the verification implementation process.<br />Insist on closing out all verifications at acceptance reviews. <br />16<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  21. 21. Lessons Learned – System Requirement Owners <br />Lesson<br />Insufficient SROresource loading early on. <br />SRO availability for verification reviews often had time conflicts with other competing tasks. <br />Recommendation<br />Have SROs committed starting from CDR. <br />Select SROs that can provide the time for verification support.<br />Avoid selecting SROs that are involved at the high management levels. <br />Tap IPT opportunities to act as SROs.<br />Increase monitoring of SRO workload and off load as soon as backlog is identified. <br />Ensure that newly delegated SROs have the appropriate engineering discipline and background knowledge to hit the ground running. <br />17<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  22. 22. Lessons Learned – Waivers<br />Lesson<br />Verification database needs to be able to retrieve all waivers associated with a requirement when a verification compliance is called up for requirements. Ares I-X had great difficulty in collecting all of the waivers needed to close out compliance statements. This was due to improper set up of database tools. <br />Recommendation<br />Know your database limitations. <br />Conduct a configuration and data management review of your verification process. <br />Ensure that there are data entries and fields set up for waivers to trace back to a requirements.<br />Run test scenarios for the waiver process. <br />Set up a specific field in the verification database to link waivers and requirements.<br />Bring Configuration Management process under SE&I. <br />18<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  23. 23. Lessons Learned – Tools<br />Lesson<br /><ul><li>Cradle was used as a project management tool to track all activities, however it was found that it was not helpful for requirements management. </li></ul>Could not provide link analysis<br />Not localized and therefore turn around for reports were slow. <br /><ul><li>Windchill system, used for the project document and data repository, was slow in retrieving files in addition to poor directory structure prevented fast file searches. </li></ul>Recommendation<br />SE&I should select what requirements management tool should be used. <br />Ares I-X SE&I switched to a Microsoft Access database in the final year of the mission. <br />Data management is a technical process and should be moved under the responsibility of a trained SE&I.<br />Data Manager familiar with Windchill or other central data repositories being used. <br /> In addition there is a need to develop better directory structures that are easily searchable. <br /> A study of other successful product data management directories should be conducted, a directory model selected and incorporated into the next mission. <br />19<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  24. 24. Summary <br />Ares I-X was a highly successful mission. <br />Using best practice requirements and verification processes played a large part in the success of Ares I-X.<br />Ares I-X Requirements and Verification Lessons Learned along with all the Mission Lessons Learned have been reported, shared, and archived. <br />Ares I-X Systems Engineering and Integration team won the prestigious “NASA Systems Engineering Excellence” award. <br />And the engineering and management success, complexity and innovation of Ares I-X led Time Magazine to select Ares I-X as the “Invention of the Year” for 2009. <br />20<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />
  25. 25. Acronyms <br />CDR – Critical Design Review<br />IPT – Integrated Product Team<br />PDR – Preliminary Design Review<br />SE&I – System Engineering and Integration (under Ares I-X)<br />SRO – System Requirement Owner<br />VRDS – Verification Requirement Definition Sheet<br />21<br />11/30/2010<br />Ares I-X Requirements and Verifications Lessons Learned<br />