Your SlideShare is downloading. ×
Wheatcraft hill.lou terry
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Wheatcraft hill.lou terry


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Developing Requirements forConstellation’s Next GenerationSpace Suit Presented at PM Challenge 2010 Presenters: Terry Hill, NASA/JSC Lou Wheatcraft, Compliance Automation February 2010 Used with Permission
  • 2. Overview  Part 1: The Basics  Risk and Requirements  Requirement Validation  Part 2: Application Case Study  CxP Suit Requirement Development Process  Results  CxP Suit Continuing Requirement Management Process  What we could have done betterNational Aeronautics and Space Administration 2
  • 3. Risk and Requirements WHAT’S COMING NEXTNational Aeronautics and Space Administration 3
  • 4. NASA OIG  “NASA must be vigilant in its process of establishing and validating project requirements.”  “Program risks increase when NASA awards contracts before developing a sound business case and clearly defining requirements;”  Placing “the project at risk of significant cost overruns, schedule delays, and performance shortfalls.”  “Effective risk management, safety, and mission assurance controls are key to supporting robust and reliable operations in the context of very challenging launch and mission schedules.” NASA’s Most Serious Management and Performance Challenges – Nov 2008National Aeronautics and Space Administration 4
  • 5. GAO  “The start of product development represents the point at which program managers make a commitment to provide a product that will perform as required and be delivered on time and within estimated costs.”  “Programs are more likely to succeed if program managers are able to achieve a match between user needs, which eventually become requirements, and resources (technology, design and production knowledge, money, and time) at the start of product development.”  “Conversely, if they do not match requirements with resources, cost overruns and schedule delays are likely to occur, reducing the organization’s buying power in other areas.” GAO-03-598National Aeronautics and Space Administration 5
  • 6. Effect Of Requirements Definition Investment On Program Costs 200 OMV 180 GRO 78 160 GALL Why are you running so fast TDRSS when you don’t know where Actual – Target Cost 140 CEN IRAS HST you are going? 120 Target Cost 100 GOES I-M TETH German proverb MARS LAND 76 MAG 80 ACT STS LAND 78 ERB 77 COBE 60 ERB 80 40 SEASAT UARS GRO 82 HEA VOYAGER EUVE/EP ISEE 20 DE SMM ULYSSES IUE PION/VEN 5 10 15 20 25 30 Requirements Definition and Preliminary Design Target Total CostNational Aeronautics and Space Administration 6
  • 7. Cost to fix requirement defects1,000- 1000 70 60- 50- 40- 40 40 30- 30 20- 15 10 10- 6 1 3 0- Requirements Design Coding Development Acceptance Operations Phase Phase Phase Testing TestingNational Aeronautics and Space Administration 7
  • 8. Importance of Best Requirement Practices onProject Success “The companies using best requirements practices will estimate a project at $3 million and better than half the time will spend $3 million on that project. Including all failures, scope creep, and mistakes across the entire portfolio of projects, this group will spend, on average, $3.63 million per project.” “The companies using poor requirements practices will estimate a project at $3 million and will be on budget less than 20% of the time. 50% of time, the overrun on the project both in time and budget will be massive. Across the entire portfolio of successes and failures, this company with poor requirements practices will (on average) pay $5.87 million per project.” 2008 Study by Keith Ellis, IAG Consulting of 100 companies with projects in excess of $250,000
  • 9. Setting Yourself Up for Failure Project success is “improbable” for 68% of the companies Ellis studied “Projects might succeed – but not by design. Based on the competencies present, these companies are statistically unlikely to have a successful project.” While these companies indicated they recognized that requirements are important to project success, they still failed to take effective actions to insure a good set of requirements. By doing so, they tripled their chances of project failure 2008 Study by Keith Ellis, IAG Consulting of 100 companies with projects in excess of $250,000
  • 10. What Needs to be Done “Organizations understand conceptually that requirements are important, but do not internalize this understanding and change their behavior as a result. The most successful of companies do not view requirements as a document which either existed or didn’t at the beginning of a project, they view it as a process of requirements discovery. Only companies that focus on both the process and the deliverables are consistently successful at changing project success rates.” 2008 Study by Keith Ellis, IAG Consulting of 100 companies with projects in excess of $250,000
  • 11. No Surprises People who write bad requirements should not be surprised when they get bad products… but they always are. Ivy HooksNational Aeronautics and Space Administration 11
  • 12. Words of Wisdom “Putting forth the same effort, or using the same approach, then expecting different results is...insanity” - Charles Bolden, NASA Administrator, July 2009National Aeronautics and Space Administration 12
  • 13. A Winning Product  Delivers what’s needed  Within budget  Within schedule  With desired quality Risk: Anything that can prevent you from delivering a winning product!National Aeronautics and Space Administration 13
  • 14. Requirement Validation WHAT’S COMING NEXTNational Aeronautics and Space Administration 14
  • 15. Requirement Validation vs. Verification  Requirement validation  Verification confirms that confirms the the designed and completeness and built product meets the correctness of the requirements requirements  Done after design and build  Starts with first requirement and continues through life cycle Requirement validation makes sure you are Verification makes sure building the right thing. you built it right.National Aeronautics and Space Administration 15
  • 16. Requirement Validation  Helps ensure our requirements are:  Needed, verifiable, achievable  Clear, concise  Correct, consistent, and complete  Two types  Continuous  DiscreteNational Aeronautics and Space Administration 16
  • 17. Continuous Validation Process  Requirement Writer:  Write requirement and attributes  Check requirements against standards  Submit to gatekeeper for review in “chunks”  Gatekeeper (person or inspection points)  One or more experts  Reviews requirements against standards  Accepts defect-free requirements for input to database or document  Return requirements to author if defects foundNational Aeronautics and Space Administration 17
  • 18. Who Does Requirement Validation? Writers Managers Everyone is accountable Developers ReviewersNational Aeronautics and Space Administration 18
  • 19. Continuous Validation Process  Continuous validation holds everyone responsible  Requires standards and checklists  Requires training  Management has to enforce discipline and accountability  Benefits  Stops the creation of BIG bad documents  Most effective way to realize process improvement  Reduces time for milestone reviews  Prevents lost time due to reworkNational Aeronautics and Space Administration 19
  • 20. Discrete Validation Process  Key milestone that requires  Effective IF time and resources  The right people are involved  Formal process  The products are ready for review  Complete document  The participants know what to do  Involves a wide range of  Management ensures stakeholders compliance  Requires standards and feedback mechanisms  Requires training  Management has to ensure responsiveness  SRR results in a requirement baselineNational Aeronautics and Space Administration 20
  • 21. System Requirement Review (SRR) Scope Requirements Design MCR Manufacture or Code Verification SRR Operations SDR Upgrade/ PDR Maintain CDR TRR MCR: Mission Concept Review  Baseline requirements SRR: System Requirements Review  Assess feasibility SDR: System Definition Review PDR: Preliminary Design Review  Set expectations CDR: Critical Design Review TRR: Test Readiness ReviewNational Aeronautics and Space Administration 21
  • 22. CxP Suit Element Requirement Development Process WHAT’S COMING NEXTNational Aeronautics and Space Administration 22
  • 23. Background: Case Study  Shuttle / ISS Extra Mobility Unit (EMU)  Shuttle was designed to be a shirt sleeve environment not requiring space suits of any kind.  EMU developed after Shuttle was designed to address a late in design possible failure scenario with the Shuttle – risk that payload bad doors would not close and would have to do it manually.  Suit dimensions were driven by Shuttle hatch sizes.  Many, many requirements and later functionality of the suit were not addressed or defined early in the process.  The Shuttle EMU was later augmented and recertified to work at and around the ISS to perform station assembly and repair.National Aeronautics and Space Administration 23
  • 24. Shuttle/ISS EMU Case Study – Design and Development Cycles Pre-declared Cert. Initial Cert. Re-Certification and Ops Con & Requirements evolvedNational Aeronautics and Space Administration 24
  • 25. Background: EMU Case Study  Lessons Learned  Any little design change will take you into some level of re-certification.  Always larger and more expensive and takes longer than ever planned.  The better the ConOps is thought through in the beginning allows the writing of better and more comprehensive requirements eliminating many re-certification cycles later and thus saving Life Cycle dollars.National Aeronautics and Space Administration 25
  • 26. Background: CxP Suit Element Requirement Development Challenge  The Task  The team was asked to develop a CxP Initial Capability (and Lunar requirements when common hardware would be used) requirement set for the Space Suit Element in time to support the CxP Suit Element SRR and for release with the RFP for a prime contractor in mid-fall.  Schedule  Very Short Schedule – 3 Months from initiation of requirements generation to Major Project Milestone review and 5 months to baseline set of functional requirements.National Aeronautics and Space Administration 26
  • 27. Background: CxP Suit Element Requirement Development Challenge  The Philosophy  Learn from past projects mistakes in how and when requirement are written  Clean sheet approach to developing a space suit and writing the requirements  Exercise the text book methodology of Systems Engineering and Requirement generation in a NASA project  Produce quality requirements that are verifiable in a cost effective manner that address the functionality defined in the CxP EVA System Operations Concept and EVA Systems Architecture documents.National Aeronautics and Space Administration 27
  • 28. Background: CxP Suit Requirement Development Schedule  Planned Schedule for 2007:  May 31-Jun 5 – Requirement Training and Kick-off  Jun 1-22 – Suit Element Requirements Generation Activities  June 25-29 – Suit Element SRR Doc(s) review  July 2-6 – Suit Element SRR Doc(s) update & SRR final prep.  July 10 – Suit & Vehicle Interface Elements SRR Kick-off  July 10-20 – Suit/VI Element SRR Doc review & RID submittal  July 22-Aug. 7 – Suit/VI Element SRR Panels and Boards.  Aug. 9-Oct 20 – Close SRR Actions and update ERD  Oct. 23 – Suit ERD for Baselining at EVA PCB.  Oct. 29 – Suit ERD rev. A draft submitted to EVA CM for pre-blackout CSSS Tech. Library drop for Prime contract RFP release * Final outcome, Element SRR slipped one week to ensure quality of products where ready for review. Review revealed products were ready and of the appropriate fidelity by EVA project management.National Aeronautics and Space Administration 28
  • 29. Background: CxP Suit Requirement Development Approach: Ground rules  Co-located the team off-site in a conference  Requirements meet the criteria for good room facility to enable a concentrated effort requirements in the Checklist for Good – this reduces the risk of day-to-day Requirements, distractions which is a risk to product quality  Rationale for each requirement, no copying and schedule. parent requirement with a noun change,  Provided task training in requirement  All requirements verifiable, clear, concise development and writing processes – this and if it can be interpreted in more than one reduces the risk to quality. way, it is not ready for acceptance.  Reduce risk of requirement development by contracting out the training and using consultation which provided standard training as provided to CxP, and an independent “fresh set of eyes” on how requirements could be interpreted and implemented – this reduces the risk to quality.  Used CRADLE tool to develop requirements prior to baseline. Post baseline, the requirements were managed out of CRADLE, but draft revisions were handled outside of CRADLE until change approval.National Aeronautics and Space Administration 29
  • 30. Putting Requirement Risk in the Proper Perspective  Not to put too much pressure on you….  The Requirements Document is probably the single most influential piece of paper that we have control over in the entire Constellation Program.  This is our chance to make sure that we are asking for what we really want. Let’s get it right.  This is a big, fat, hairy deal. If we don’t get this right, folks 20 years from now will be shaking their heads and saying, “What were those yahoos thinking?”  I’ll be around and don’t want to go to that meeting. CxP EVA Suit PGS Team Requirement Kickoff Meeting 5/2007National Aeronautics and Space Administration 30
  • 31. Suit Requirement Development ProcessNational Aeronautics and Space Administration 31
  • 32. Subsystem Teams  Responsible for the drafting & modifying requirements  Four independent teams  Suit Element-level (integrated across subsystems)  Subsystems  Pressure Garment, Crew Survival, Power-Comm & Avionics, Life Support, (Ground Support Equipment team was after the initial requirement generation effort)  Each team consisted of:  Functional Lead  NASA Subject Matter Experts (SMEs)  “Core” Scrub team  Teams given training during kick off meeting  Briefed in the Goal, Schedule and Process  Checklist for Good RequirementsNational Aeronautics and Space Administration 32
  • 33. Scrub Team  “Core” Scrub Team:  SE&I representative  Compliance Automation representative – independent “goodness assessment”, and guidance in requirement development.  Tech Writer  CRADLE Operator  Review requirements from the Subsystem Teams  Edit & “clean” Requirements based on Checklist for Good Requirements  If they could “fix” requirement & verify the intent was understood, would do so & pass onto the CRADLE Board  If intent not understood and the requirement needed more work, representatives from appropriate Requirement Team were called in for clarification.  Kept list of common defects and briefed Requirement Teams dailyNational Aeronautics and Space Administration 33
  • 34. CRADLE Board  Had three functions:  Determined if requirements where technically traceable and appropriate  Determined if requirements where technically appropriate and the correct mission phases applied  And determine any outstanding issues were resolved as the CRADLE Board was used as a dry-run for final approval.  Base Team:  SE&I representative  Subsystem representative or subject matter expert  Secondary Team:  Project Management  Subsystem Leads  Focus Meetings a.k.a. Offline  Resolves outstanding issues  Membership as requiredNational Aeronautics and Space Administration 34
  • 35. Approval Board  Function: Team Consensus/Final Approval for requirement to be included in ERD  Base Team:  SE&I representative  Project Management  Subsystem Leads  CRADLE operation  Subject matter expert  If discussions proceeded for longer than 15 minutes, the requirement and discussion were sent to a Focus Meeting for resolution. The idea was that if a requirement was not clear enough to convey the intent and importance in 15 minutes of discussion, then it was not written correctly.  Focus Meetings a.k.a. Offline  Resolves outstanding issues  Membership as requiredNational Aeronautics and Space Administration 35
  • 36. Results WHAT’S COMING NEXTNational Aeronautics and Space Administration 36
  • 37. Results in Project Reviews  For the Suit ERD SRR, a ratio of 0.38 Review Item Descriptions (RIDs) were received per requirement.  In comparison, the parent document had a 2.94 RID/requirement ratio at its SRR. Potential bidders for the “… the most comprehensive and of the highest development of the Suit Element stated that the ERD was: quality they ever remember seeing.” “I cant say enough about how amazed I am by The JSC Engineering Directorate this set of requirements documents. As far as I Crew and Thermal Systems Division Chief was also very know, no other Cx project has allocated and impressed with the quality of the decomposed anywhere near to this level of Suit ERD, saying: depth. You are the first. I have also never seen anything like these from previous programs.”National Aeronautics and Space Administration 37
  • 38. CxP Suit Continuing Requirement Management Process WHAT’S COMING NEXTNational Aeronautics and Space Administration 38
  • 39. Post-baseline Requirement Development and Validation  As a result of the extremely compressed requirement development schedule, there remained several areas that needed more work and issues that needed to be resolved prior to preliminary design taking place.  These open areas included:  resolving TBDs/TBRs in the requirement set,  finishing the verification requirements,  defining the internal interfaces and maturing applicable interface requirements  and adding Ground Support Equipment (GSE) requirements to the ERD.  Therefore there was requirement development and maturation process needed to continue this process.National Aeronautics and Space Administration 39
  • 40. Requirement Review Process Post- ERD BaselineNational Aeronautics and Space Administration 40
  • 41. Requirement Review Process Post- ERD Baseline: SEIWG  The SEIWG (Systems Engineering and Integration Working Group)  Meets once a week to discuss new and existing requirements and/or requirements issues, such as TBDs/TBRs, action items, allocations, etc.,  Serves as the SE&I pre-coordination for the Suit Control Board.  SEIWG Procedure  Step 1:  All potential changes must be submitted via a change request form.  Step 2:  Scheduled SEIWG meeting to present/discuss change and any final modification of the draft FROM/TO language.National Aeronautics and Space Administration 41
  • 42. Suit Control Board Functions:  Detailed review of proposed FROM/TO language to go into the ERD.  A minimum of a 2 business day review by board members prior to the Suit Control Board meeting.  Functions as the official forum of stakeholder review, concurrence or rejection  Functions as the final approval gate for base-lined requirements. Note: Since Baselineing of ERD the SCB closed approximately 180 TBD/Rs prior to levying the document on the prime contractor.
  • 43. Wrap up WHAT’S COMING NEXTNational Aeronautics and Space Administration 43
  • 44. What we could have done better  Challenging schedule resulted in a lot of open work post baselining of ERD.  Large number of TBDs/TBRs  Interfaces identification and definition incomplete  Some of the external interfaces just didn’t exist due to sister projects were evolving in parallel and at times without the same rigor demonstrated in the Suit Element effort.  Traceability incomplete  Verification requirements  GSE requirements  Difficult to keep requirements at the right level  Some requirements may not have been needed  Some requirements reflected or assumed a design where NASA was clear there was only one desirable functional solution.National Aeronautics and Space Administration 44
  • 45. Parting Thoughts  Address Requirement Risk at the beginning of your project  Develop a formal requirement development process that includes continuous requirement validation  Include continuous requirement validation into your requirement management process  Train your team  Enforce the process  Allocate the time and resources needed to do the job right – the first time!!National Aeronautics and Space Administration 45
  • 46. Presenter BiographiesNational Aeronautics and Space Administration 46
  • 47. Terry Hill NASA/JSC Constellation Space Suit System Engineering Project Manager Terry Hill is NASA’s Johnson Space Center’s Engineering Project Manager and deputy CxP EVA Suit Lead for the CxP Suit Element, responsible for the development of the functional, performance, and quality requirements and preliminary design of NASA’s next generation space suit system. Terry has a BS in Aerospace Engineering and a MS in Guidance, Navigation & Control Theory with a minor in Orbital Mechanics and Mathematics from the University of Texas at Austin. He began his career at NASA while working on his masters thesis project in developing banks of simplified Kalman filters integrated into an artificial neural network to obtain an optimal state solution for precision landing on Mars. While at NASA ,Terry has worked on projects and programs spanning verification of ISS navigation software, Shuttle Design Test Objectives (DTO) and back room mission support, X-38 Crew Return Vehicle navigation algorithm development, Space Launch Initiative technology development, Orbital Space Plane project office ISS-Prime integration, STS-107 Return to Flight Tile Repair capability development, to Constellation Program Space Suit System leadership. In leading the CxP Suit Element engineering team, Terry has facilitated the development of system requirements for space suit development and a clean-sheet design approach that has widely recognized within and outside of NASA.National Aeronautics and Space Administration 47
  • 48. Lou Wheatcraft Compliance Automation – Senior Consultant Trainer Lou Wheatcraft is a senior consultant/instructor for Compliance Automation, who has over 40 years experience in the aerospace industry, including 22 years in the United States Air Force. Lou has taught over 120 seminars in requirement development and management for NASA’s APPEL Program and industry over the past nine years. He has worked with, and provided intact team training and consultation to multiple NASA project teams at many of the NASA Centers. Lou has had articles published in the International Council of Systems Engineering (INCOSE) INSIGHT magazine and in DoD’s magazine, CrossTalk. Lou has made presentations at NASA’s PM Challenge, INCOSE’s International Symposium, and at the local Project Management Institute (PMI) Chapter Meetings. Lou has a BS degree in Electrical Engineering, an MA degree in Computer Information Systems, an MS degree in Environmental Management, and has completed the course work for an MS degree in Studies of the Future. Lou is a member of INCOSE, co-chair of the INCOSE Requirements Working Group, a member of PMI, the Software Engineering Institute, the World Futures Society, and the National Honor Society of Pi Alpha Alpha. Lou is the recipient of NASA’s Silver Snoopy Award and Public Service Medal and was nominated for the Rotary Stellar Award for his significant contributions to the Nation’s Space Program.National Aeronautics and Space Administration 48
  • 49. National Aeronautics and Space Administration 49