Strassner retherford


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Strassner retherford

  1. 1. Spacecraft Software Engineering Branch/ER6 The CMMI Models in Oversight of Space Flight Software Development Liz Strassner Lead, Software Process & Process Management Group David Retherford Sr. Software and Systems Engineer/ERC Spacecraft Software Engineering Branch NASA/JSC Used with permission
  2. 2. AgendaSpacecraft Software Engineering Branch/ER6  Introduction  Selection of CMMI Models for Spacecraft Software Engineering Branch  NASA Surveillance Strategies  Project Orion Surveillance Strategies  Orion Software Oversight  CMMI Maturity Level 2 Rating on Oversight  Who are we  What we did to earn CMMI Level 2  Trials & Tribulations of Software Oversight  What we are doing now 2/22/2010 2
  3. 3. Spacecraft Software Engineering & CMMI ModelsSpacecraft Software Engineering Branch/ER6  We evaluated the three CMMI models (CMMI-ACQ, CMMI-SVC, & CMMI- DEV) for applicability to our work  CMMI-SVC was discovered to be completely non-applicable  IT support help desk; Pizza delivery service  Oversight of a contract doesn’t meet the intent of “service” in this model  CMMI-ACQ was reviewed for applicability in a SCAMPI B  NASA/JSC organizational structure caused some goals to be completely out of scope of the organization  We were fully compliant with goals and practices that were in scope of the organization  The model does not allow goals to be out of scope  CMMI-DEV  Software development work was obviously in scope  Software oversight is also in scope, but needs to be looked at from a systems engineering perspective  Software oversight on a 30 year project is never-ending so needs to be broken into smaller projects 2/22/2010 3
  4. 4. Selection of CMMI Model & ProjectsSpacecraft Software Engineering Branch/ER6  Based on our experience with the SCAMPI B, we chose to go for a CMMI Maturity Level 2 rating under the CMMI-DEV model  Selecting projects was an interesting experience  Software Development Projects  Many active software development projects to select from  A long history of flight software development projects from the former Flight Software Branch that was merged into Spacecraft Software Engineering Branch  Software Oversight  Role is significantly different than in the Shuttle or ISS Programs  Lead appraiser insisted that we have at least one project in this area because of the large focus in the organization on oversight  Chose to create sub-projects out of milestone reviews  Software Requirements Review for SCAMPI B  “System PDR & Software Spiral Review" for SCAMPI A 2/22/2010 4
  5. 5. NASA Surveillance Strategies (From COTR Refresher Training 2-18- 2008)  Basic strategies: insight, oversight, or hybridSpacecraft Software Engineering Branch/ER6  Insight: relies on performance requirements & metrics. Use a minimum set of product/process data to give adequate visibility into the integrity of the product/process.  Contractor assumes more responsibility & accountability  Government steps back, evaluates deliverables & existing contractor processes—not day-to-day close monitoring  Appropriate strategy for contracts with little cost risk, clearly-defined products/services, & confidence in proven contractor performance  Oversight: rely on NASA-imposed product specification’s & process controls, MIL standards, mandatory inspection points, etc. Intrusive monitoring at contractor’s plant, & in-line NASA involvement in contract work.  Appropriate strategy when NASA is assuming the liability for performance or quality; when we determine oversight is necessary to mitigate performance risk; or when the contractor is unproven 2/22/2010 5
  6. 6. Project Orion SurveillanceSpacecraft Software Engineering Branch/ER6  The Orion Contract as a whole uses the hybrid approach to surveillance with pre-declared oversight in high risk areas  Building a space vehicle is high risk so the contract is Cost Plus Award Fee (CPAF) type  Very extensive DRD requirements for insight  Software was pre-declared as a major risk item by the Agency, so oversight of software was planned and scheduled  Joint NASA-Contractor Risk Board  Insight for Software  All of the NPR 7150.2 chapter 5 requirements exist as individual DRDs  CMMI Maturity Level 3 was levied on all software development organizations 2/22/2010 6
  7. 7. Software Oversight of OrionSpacecraft Software Engineering Branch/ER6  Oversight for Software  NASA co-chairs all boards and panels  Every Contractor functional area lead has a NASA counterpart  NASA participates in ALL software working groups  NASA has signature authority on all major software deliveries including the Software Development Plan  NASA has an internal software test environment  NPR 7150.2 NASA Software Engineering levied on contract in its entirety  NASA Standard 7009(I) Models & Simulations Standard levied on contract in its entirety  NASA Software Assurance and Safety Standards levied on contract in a “cut and paste” method into the contract (DRD CEV-S-001)  Constellation Software documents also levied by the “cut and paste” method or by reference to specific sections or paragraphs of the CxP document 2/22/2010 7
  8. 8. Spacecraft Software Engineering Branch/ER6 Spacecraft Software Engineering Branch & CMMI Level 2 2/22/2010 8
  9. 9. Spacecraft Software Engineering BranchSpacecraft Software Engineering Branch/ER6 Spacecraft Software Engineering Branch Orion Flight Software System Manager Orion Recon System Manager Orion Vehicle Systems Management Manager Altair Lead Process & Process Management Test & Verification Group GFE Group Technology Group Systems Engineering Group Group 2/22/2010 9
  10. 10. Orion Software Team InterfacesSpacecraft Software Engineering Branch/ER6 Customer - Provider Orion Constellation Project Office Provider - Customer Orion Stakeholder – Provider Rep Software Team Customer Rep - Provider - Customer Provider Prime Contractor Subcontractor Subcontractor Subcontractor Subcontractor Subcontractor Subcontractor 2/22/2010 10
  11. 11. NASA Orion Software Team OrganizationSpacecraft Software Engineering Branch/ER6  Orion Software Manger is part of the Orion Project Office  Everyone below that level is matrixed out of Engineering organizations at JSC, Ames, GRC, and LaRC  Frank Delgado/ER6 is the Flight Software System Manager and leads the multicenter flight software team  Neil Townsend/ER6 is the Reconfiguration System Manager and leads the multicenter software process and process management team  The Spacecraft Software Engineering Branch and the Orion Software Team are approximately the same size and have a large overlap. Orion Software Spacecraft Team Software Engineering Branch 2/22/2010 11
  12. 12. Year One – Baby StepsSpacecraft Software Engineering Branch/ER6  Organization was very small  Only function was Orion oversight  Responsible for Orion Vehicle System Management (Class A)  Responsible for in-house flight software development environment for all Orion software oversight (Class D & E)  Interviewed all employees to find out how they did their work  Wrote processes based on the “how they did their work”  Assumed no higher level organizational processes affected oversight  At a conceptual level, we did not have any problem mapping the CMMI Specific Goals and Practices to processes  Late in the year, Orion Software Manager function moved here in addition to Vehicle System Manager and the branch grew 2/22/2010 12
  13. 13. Year Two – Chaos Reigns  Former Flight Software Branch merged into the branch as one ofSpacecraft Software Engineering Branch/ER6 many steps JSC Engineering took to focus software  Brought in several GFE Flight Software Development projects, most in maintenance and operations phase already CMMI Level 2 compliant  Consolidated all JSC Orion software oversight personnel  Brought in methodology for meeting Generic Goals and Practices useful for all projects  GFE Projects must follow extensive higher level organizational processes  New Branch Chief assigned  Perfect storm  Policies and processes for oversight crashed into GFE flight software policies & processes  Previous branch-only view had to be extended to reflect division and directorate infrastructure  New branch chief had new goals and expectations 2/22/2010 13
  14. 14. Year Two – Coming out of ChaosSpacecraft Software Engineering Branch/ER6  CMMI Consultation  Bill Pierce provided advise on how to structure projects for oversight and methods to apply CMMI practices to an oversight project  Selected projects: 1) Class A development, 2) Class D development/acquisition, 3) Class A-E oversight  The former Intelligent Systems Branch was merged into Spacecraft Software Engineering Branch  Primarily Class E research and technology software  SCAMPI B  Newly acquired projects from the recent merger were excluded  Uncovered a problem with PPQA  Discovered that we still had disconnects in applicability of division and directorate policies and processes  Determined that our SwRR Oversight project was mis-scoped  Appraiser direction was to do better for PDR 2/22/2010 14
  15. 15. Year 3 – Success!Spacecraft Software Engineering Branch/ER6  Solved PPQA problem temporarily by returning to original resources  Orion PDR slipped significantly, so best we could do was “plan” for PDR without any action past that  Appraiser decided to merge our SwRR Project and our start of the PDR Project into a single “Software Oversight” project for the purpose of the SCAMPI A  Allowed us to show that we learned from SwRR and incorporated those lessons into the PDR planning activities  Since all our development projects were new, we had to pull in an old (CMMI L2 rated) project that was in maintenance to ensure end of lifecycle coverage  This end of lifecycle coverage would be a problem for any new organization 2/22/2010 15
  16. 16. Spacecraft Software Engineering Branch/ER6 Trials & Tribulations of planning Software Oversight
  17. 17. Orion Software Spiral Review – OverviewSpacecraft Software Engineering Branch/ER6  Meet diverse set of goals & requirements  Orion project  SW Management Plan (CxP 72099)  Need to plan component-level PDR activities for LM SW Spiral delivery  Follow general flow of Orion system PDR (CxP 72212)  Spacecraft Software Engineering Branch  Demonstrate an oversight project meeting CMMI ML 2  Agency  Software engineering requirements (NPR 7150.2)  Systems/project requirements (NPR 7123.1)  CMMI Dev 1.2  Demonstrate that oversight project can meet ML 2 SPs & GPs 2/22/2010 17
  18. 18. Software Oversight – Changes in AttitudeSpacecraft Software Engineering Branch/ER6  Requires an orthogonal change of view of  Requirements  Configuration Management  Evaluation & assessment vs. technical development  Requirements  Technical requirements of contractor product (e.g. spacecraft) differ from oversight  As part of project tasks may influence or evaluate technical requirements  Actual oversight project requirement is to review products (e.g. SRS) , generate comments  Separation of concerns  Technical requirement development vs. oversight activities 2/22/2010 18
  19. 19. SW PDR Project Plan Impact SourcesSpacecraft Software Engineering Branch/ER6 2/22/2010 19
  20. 20. Orion PDR Flow – High Level Overview PDR Plan & KickoffSpacecraft Software Engineering Branch/ER6 • PDR Ground Rules & POD • PDR Review Process • PDR RID Process Subsystem Design Reviews (SSDR) • Subsystem Specifications & IRDs • Subsystem Design Data Books • Subsystem Specific Design & Requirements Review Presentations System & Module Review • Subsystem Rollups • System Specifications and Design Review Presentations System PDR • RID Process • RID Screening, Review, & Dispositions • PDR Pre-board & Board Software PDR • Specification & Design Tech/Peer Review Process • SW Artifact Release and RID Process • SW RID Screening, Review, & Disposition • SW PDR Pre-board & Board 2/22/2010 20
  21. 21. Oversight Project – CMMI ModelSpacecraft Software Engineering Branch/ER6  CMMI Maturity Level 2 process areas for oversight  7 PA’s – PP, PMC, REQM, CM, PPQA, MA, SAM  CMMI Maturity Level 2 process area emphasis on managing project  Planning project, process, and products  Monitor & control project 2/22/2010 21
  22. 22. Spiral Review Project – CMMI Process AreasSpacecraft Software Engineering Branch/ER6  Project Planning (PP) compliance  Develop project life cycle and  Evaluate team assessment process products  Document in project plan  Configuration Management  Project Monitoring and (CM) Control (PMC)  NASA ICE/Windchill area for  Track team participation in document storage contractor reviews  Measurement and Analysis  Team assessment of reviews (MA)  Requirements Management  Metrics for effort and effort per (REQM) review/document  Screen/review requests from  Supplier Agreement Project Office or Branch Management (SAM) Management  Contractors part of Orion SW  Product & Process Quality team Assurance (PPQA)  Use division process for  Evaluate team process engineering service support 2/22/2010 22
  23. 23. Basic Software Oversight RequirementsSpacecraft Software Engineering Branch/ER6  Not the same as the product technical requirements  What ‘we’ must do to demonstrate  Contractor development of software technical baseline  Accomplished proper oversight of contractor  Assure adequacy of contractor efforts and products  As an oversight team we must  Review contractor technical efforts (requirements & design)  Generate comments/RIDs  Attend technical and peer reviews  Evaluate software product quality and maturity  Within known (agreed to) constraints  Basically – our requirements are:  To provide a set of software engineering services to Orion project  Engineering involvement and assessment 2/22/2010 23
  24. 24. Software PDR Oversight LifecycleSpacecraft Software Engineering Branch/ER6  Not traditional SW lifecycle – no actual SW being produced  Formulate and initiate the project/process (Inception)  Execute the plan/process (Execution)  Finish the project & baseline (Closeout)  Product/artifact data  Assessment/evaluation vs. SRS/IRD  Configuration Management  Oversight project artifacts, not contractor technical artifacts 2/22/2010 24
  25. 25. Oversight PhasesSpacecraft Software Engineering Branch/ER6  Inception  Closeout  Define project lifecycle and  Formal review release organization  RID inputs  Define review process(es) and  RID review/screening boards  RID dispositions  Define NASA technical and  SW Spiral Pre-board management roles  SW Spiral board  Generate project plan  Artifact update/re-release  And away we go…  Execution  Monitor contractor software development efforts  Review artifacts, generate comments, attend reviews, attend disposition and solution meetings 2/22/2010 25
  26. 26. Process Area ConsiderationsSpacecraft Software Engineering Branch/ER6  PPQA  REQM  No software artifacts (e.g. SRS, SDD,  Oversight requirements NOT the source code, etc.) same as technical product  Process and assessment report  Activity and evaluation/assessment focused based  Use outside organization – provide  “The software shall do this …..” vs. independent assessment “The project shall participate in technical reviews.”  M&A  Control/manage between Orion SW  No technical measures (e.g. Mgr & Av&SW Project Office SLOCs, etc.)  PP  Effort/product based  Project “within a project”  Hrs/review  Predetermined schedule  No. comments  Orion project/contractor level  Assessments  Completion  Quality 2/22/2010 26
  27. 27. SW Spiral Review – CMMI Appraisal NuggetsSpacecraft Software Engineering Branch/ER6  Double vision syndrome  KISS  Easy to get cross-eyed over  Keep oversight plan, processes, apparent identical areas between and tasks simple LM and NASA oversight project  Simple data/configuration  CMMI model vs. NASA management method engineering focus  Only controlling documents  Ex.: CM – whose CM? LM  Reasonable/simple plan and Project Link vs. NASA ICE process description results in  Ex.: Requirements management  Effective CMMI Maturity Level – Oversight project requirements 2 compliance vs. contractor project technical requirements  Project within a project  Scope focus very tight for oversight project – milestone based  Other work did not stand still 2/22/2010 27
  28. 28. Typical Review ProcessSpacecraft Software Engineering Branch/ER6 Perform Tech Review Product Owner SFM/SWE Perform Peer Review Disposition Product Product Comments Comments  Initial attempt to use SE process to describe & define process oversight  Ex.: Use case diagram to represent project process for technical and peer reviews  Each use case may breakdown and be described in project plan for each specific process  Value of approach is still being evaluated (may refine for next version & evaluate for improvement) 2/22/2010 28
  29. 29. Spacecraft Software Engineering Branch/ER6 2/22/2010 Next Steps29
  30. 30. Next StepsSpacecraft Software Engineering Branch/ER6  New Project – Software Process Improvement Project  Among other things, earn CMMI Maturity Level 3 by April 2011  Requires major culture change to implement OT, OPF, & OPD  Orion Software PDR Project  Continues to evolve  Creating checklists to further refine measures from reviews  Expect to learn enough to provide guidance to development projects on  Planning and scheduling peer reviews  Planning and scheduling milestone reviews  Expect confusion in implementing PI, VAL, VER, and TS because our products are brain-power related rather than code 2/22/2010 30