Your SlideShare is downloading. ×
Strassner retherford
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Strassner retherford


Published on

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Spacecraft Software Engineering Branch/ER6 The CMMI Models in Oversight of Space Flight Software Development Liz Strassner Lead, Software Process & Process Management Group David Retherford Sr. Software and Systems Engineer/ERC Spacecraft Software Engineering Branch NASA/JSC Used with permission
  • 2. AgendaSpacecraft Software Engineering Branch/ER6  Introduction  Selection of CMMI Models for Spacecraft Software Engineering Branch  NASA Surveillance Strategies  Project Orion Surveillance Strategies  Orion Software Oversight  CMMI Maturity Level 2 Rating on Oversight  Who are we  What we did to earn CMMI Level 2  Trials & Tribulations of Software Oversight  What we are doing now 2/22/2010 2
  • 3. Spacecraft Software Engineering & CMMI ModelsSpacecraft Software Engineering Branch/ER6  We evaluated the three CMMI models (CMMI-ACQ, CMMI-SVC, & CMMI- DEV) for applicability to our work  CMMI-SVC was discovered to be completely non-applicable  IT support help desk; Pizza delivery service  Oversight of a contract doesn’t meet the intent of “service” in this model  CMMI-ACQ was reviewed for applicability in a SCAMPI B  NASA/JSC organizational structure caused some goals to be completely out of scope of the organization  We were fully compliant with goals and practices that were in scope of the organization  The model does not allow goals to be out of scope  CMMI-DEV  Software development work was obviously in scope  Software oversight is also in scope, but needs to be looked at from a systems engineering perspective  Software oversight on a 30 year project is never-ending so needs to be broken into smaller projects 2/22/2010 3
  • 4. Selection of CMMI Model & ProjectsSpacecraft Software Engineering Branch/ER6  Based on our experience with the SCAMPI B, we chose to go for a CMMI Maturity Level 2 rating under the CMMI-DEV model  Selecting projects was an interesting experience  Software Development Projects  Many active software development projects to select from  A long history of flight software development projects from the former Flight Software Branch that was merged into Spacecraft Software Engineering Branch  Software Oversight  Role is significantly different than in the Shuttle or ISS Programs  Lead appraiser insisted that we have at least one project in this area because of the large focus in the organization on oversight  Chose to create sub-projects out of milestone reviews  Software Requirements Review for SCAMPI B  “System PDR & Software Spiral Review" for SCAMPI A 2/22/2010 4
  • 5. NASA Surveillance Strategies (From COTR Refresher Training 2-18- 2008)  Basic strategies: insight, oversight, or hybridSpacecraft Software Engineering Branch/ER6  Insight: relies on performance requirements & metrics. Use a minimum set of product/process data to give adequate visibility into the integrity of the product/process.  Contractor assumes more responsibility & accountability  Government steps back, evaluates deliverables & existing contractor processes—not day-to-day close monitoring  Appropriate strategy for contracts with little cost risk, clearly-defined products/services, & confidence in proven contractor performance  Oversight: rely on NASA-imposed product specification’s & process controls, MIL standards, mandatory inspection points, etc. Intrusive monitoring at contractor’s plant, & in-line NASA involvement in contract work.  Appropriate strategy when NASA is assuming the liability for performance or quality; when we determine oversight is necessary to mitigate performance risk; or when the contractor is unproven 2/22/2010 5
  • 6. Project Orion SurveillanceSpacecraft Software Engineering Branch/ER6  The Orion Contract as a whole uses the hybrid approach to surveillance with pre-declared oversight in high risk areas  Building a space vehicle is high risk so the contract is Cost Plus Award Fee (CPAF) type  Very extensive DRD requirements for insight  Software was pre-declared as a major risk item by the Agency, so oversight of software was planned and scheduled  Joint NASA-Contractor Risk Board  Insight for Software  All of the NPR 7150.2 chapter 5 requirements exist as individual DRDs  CMMI Maturity Level 3 was levied on all software development organizations 2/22/2010 6
  • 7. Software Oversight of OrionSpacecraft Software Engineering Branch/ER6  Oversight for Software  NASA co-chairs all boards and panels  Every Contractor functional area lead has a NASA counterpart  NASA participates in ALL software working groups  NASA has signature authority on all major software deliveries including the Software Development Plan  NASA has an internal software test environment  NPR 7150.2 NASA Software Engineering levied on contract in its entirety  NASA Standard 7009(I) Models & Simulations Standard levied on contract in its entirety  NASA Software Assurance and Safety Standards levied on contract in a “cut and paste” method into the contract (DRD CEV-S-001)  Constellation Software documents also levied by the “cut and paste” method or by reference to specific sections or paragraphs of the CxP document 2/22/2010 7
  • 8. Spacecraft Software Engineering Branch/ER6 Spacecraft Software Engineering Branch & CMMI Level 2 2/22/2010 8
  • 9. Spacecraft Software Engineering BranchSpacecraft Software Engineering Branch/ER6 Spacecraft Software Engineering Branch Orion Flight Software System Manager Orion Recon System Manager Orion Vehicle Systems Management Manager Altair Lead Process & Process Management Test & Verification Group GFE Group Technology Group Systems Engineering Group Group 2/22/2010 9
  • 10. Orion Software Team InterfacesSpacecraft Software Engineering Branch/ER6 Customer - Provider Orion Constellation Project Office Provider - Customer Orion Stakeholder – Provider Rep Software Team Customer Rep - Provider - Customer Provider Prime Contractor Subcontractor Subcontractor Subcontractor Subcontractor Subcontractor Subcontractor 2/22/2010 10
  • 11. NASA Orion Software Team OrganizationSpacecraft Software Engineering Branch/ER6  Orion Software Manger is part of the Orion Project Office  Everyone below that level is matrixed out of Engineering organizations at JSC, Ames, GRC, and LaRC  Frank Delgado/ER6 is the Flight Software System Manager and leads the multicenter flight software team  Neil Townsend/ER6 is the Reconfiguration System Manager and leads the multicenter software process and process management team  The Spacecraft Software Engineering Branch and the Orion Software Team are approximately the same size and have a large overlap. Orion Software Spacecraft Team Software Engineering Branch 2/22/2010 11
  • 12. Year One – Baby StepsSpacecraft Software Engineering Branch/ER6  Organization was very small  Only function was Orion oversight  Responsible for Orion Vehicle System Management (Class A)  Responsible for in-house flight software development environment for all Orion software oversight (Class D & E)  Interviewed all employees to find out how they did their work  Wrote processes based on the “how they did their work”  Assumed no higher level organizational processes affected oversight  At a conceptual level, we did not have any problem mapping the CMMI Specific Goals and Practices to processes  Late in the year, Orion Software Manager function moved here in addition to Vehicle System Manager and the branch grew 2/22/2010 12
  • 13. Year Two – Chaos Reigns  Former Flight Software Branch merged into the branch as one ofSpacecraft Software Engineering Branch/ER6 many steps JSC Engineering took to focus software  Brought in several GFE Flight Software Development projects, most in maintenance and operations phase already CMMI Level 2 compliant  Consolidated all JSC Orion software oversight personnel  Brought in methodology for meeting Generic Goals and Practices useful for all projects  GFE Projects must follow extensive higher level organizational processes  New Branch Chief assigned  Perfect storm  Policies and processes for oversight crashed into GFE flight software policies & processes  Previous branch-only view had to be extended to reflect division and directorate infrastructure  New branch chief had new goals and expectations 2/22/2010 13
  • 14. Year Two – Coming out of ChaosSpacecraft Software Engineering Branch/ER6  CMMI Consultation  Bill Pierce provided advise on how to structure projects for oversight and methods to apply CMMI practices to an oversight project  Selected projects: 1) Class A development, 2) Class D development/acquisition, 3) Class A-E oversight  The former Intelligent Systems Branch was merged into Spacecraft Software Engineering Branch  Primarily Class E research and technology software  SCAMPI B  Newly acquired projects from the recent merger were excluded  Uncovered a problem with PPQA  Discovered that we still had disconnects in applicability of division and directorate policies and processes  Determined that our SwRR Oversight project was mis-scoped  Appraiser direction was to do better for PDR 2/22/2010 14
  • 15. Year 3 – Success!Spacecraft Software Engineering Branch/ER6  Solved PPQA problem temporarily by returning to original resources  Orion PDR slipped significantly, so best we could do was “plan” for PDR without any action past that  Appraiser decided to merge our SwRR Project and our start of the PDR Project into a single “Software Oversight” project for the purpose of the SCAMPI A  Allowed us to show that we learned from SwRR and incorporated those lessons into the PDR planning activities  Since all our development projects were new, we had to pull in an old (CMMI L2 rated) project that was in maintenance to ensure end of lifecycle coverage  This end of lifecycle coverage would be a problem for any new organization 2/22/2010 15
  • 16. Spacecraft Software Engineering Branch/ER6 Trials & Tribulations of planning Software Oversight
  • 17. Orion Software Spiral Review – OverviewSpacecraft Software Engineering Branch/ER6  Meet diverse set of goals & requirements  Orion project  SW Management Plan (CxP 72099)  Need to plan component-level PDR activities for LM SW Spiral delivery  Follow general flow of Orion system PDR (CxP 72212)  Spacecraft Software Engineering Branch  Demonstrate an oversight project meeting CMMI ML 2  Agency  Software engineering requirements (NPR 7150.2)  Systems/project requirements (NPR 7123.1)  CMMI Dev 1.2  Demonstrate that oversight project can meet ML 2 SPs & GPs 2/22/2010 17
  • 18. Software Oversight – Changes in AttitudeSpacecraft Software Engineering Branch/ER6  Requires an orthogonal change of view of  Requirements  Configuration Management  Evaluation & assessment vs. technical development  Requirements  Technical requirements of contractor product (e.g. spacecraft) differ from oversight  As part of project tasks may influence or evaluate technical requirements  Actual oversight project requirement is to review products (e.g. SRS) , generate comments  Separation of concerns  Technical requirement development vs. oversight activities 2/22/2010 18
  • 19. SW PDR Project Plan Impact SourcesSpacecraft Software Engineering Branch/ER6 2/22/2010 19
  • 20. Orion PDR Flow – High Level Overview PDR Plan & KickoffSpacecraft Software Engineering Branch/ER6 • PDR Ground Rules & POD • PDR Review Process • PDR RID Process Subsystem Design Reviews (SSDR) • Subsystem Specifications & IRDs • Subsystem Design Data Books • Subsystem Specific Design & Requirements Review Presentations System & Module Review • Subsystem Rollups • System Specifications and Design Review Presentations System PDR • RID Process • RID Screening, Review, & Dispositions • PDR Pre-board & Board Software PDR • Specification & Design Tech/Peer Review Process • SW Artifact Release and RID Process • SW RID Screening, Review, & Disposition • SW PDR Pre-board & Board 2/22/2010 20
  • 21. Oversight Project – CMMI ModelSpacecraft Software Engineering Branch/ER6  CMMI Maturity Level 2 process areas for oversight  7 PA’s – PP, PMC, REQM, CM, PPQA, MA, SAM  CMMI Maturity Level 2 process area emphasis on managing project  Planning project, process, and products  Monitor & control project 2/22/2010 21
  • 22. Spiral Review Project – CMMI Process AreasSpacecraft Software Engineering Branch/ER6  Project Planning (PP) compliance  Develop project life cycle and  Evaluate team assessment process products  Document in project plan  Configuration Management  Project Monitoring and (CM) Control (PMC)  NASA ICE/Windchill area for  Track team participation in document storage contractor reviews  Measurement and Analysis  Team assessment of reviews (MA)  Requirements Management  Metrics for effort and effort per (REQM) review/document  Screen/review requests from  Supplier Agreement Project Office or Branch Management (SAM) Management  Contractors part of Orion SW  Product & Process Quality team Assurance (PPQA)  Use division process for  Evaluate team process engineering service support 2/22/2010 22
  • 23. Basic Software Oversight RequirementsSpacecraft Software Engineering Branch/ER6  Not the same as the product technical requirements  What ‘we’ must do to demonstrate  Contractor development of software technical baseline  Accomplished proper oversight of contractor  Assure adequacy of contractor efforts and products  As an oversight team we must  Review contractor technical efforts (requirements & design)  Generate comments/RIDs  Attend technical and peer reviews  Evaluate software product quality and maturity  Within known (agreed to) constraints  Basically – our requirements are:  To provide a set of software engineering services to Orion project  Engineering involvement and assessment 2/22/2010 23
  • 24. Software PDR Oversight LifecycleSpacecraft Software Engineering Branch/ER6  Not traditional SW lifecycle – no actual SW being produced  Formulate and initiate the project/process (Inception)  Execute the plan/process (Execution)  Finish the project & baseline (Closeout)  Product/artifact data  Assessment/evaluation vs. SRS/IRD  Configuration Management  Oversight project artifacts, not contractor technical artifacts 2/22/2010 24
  • 25. Oversight PhasesSpacecraft Software Engineering Branch/ER6  Inception  Closeout  Define project lifecycle and  Formal review release organization  RID inputs  Define review process(es) and  RID review/screening boards  RID dispositions  Define NASA technical and  SW Spiral Pre-board management roles  SW Spiral board  Generate project plan  Artifact update/re-release  And away we go…  Execution  Monitor contractor software development efforts  Review artifacts, generate comments, attend reviews, attend disposition and solution meetings 2/22/2010 25
  • 26. Process Area ConsiderationsSpacecraft Software Engineering Branch/ER6  PPQA  REQM  No software artifacts (e.g. SRS, SDD,  Oversight requirements NOT the source code, etc.) same as technical product  Process and assessment report  Activity and evaluation/assessment focused based  Use outside organization – provide  “The software shall do this …..” vs. independent assessment “The project shall participate in technical reviews.”  M&A  Control/manage between Orion SW  No technical measures (e.g. Mgr & Av&SW Project Office SLOCs, etc.)  PP  Effort/product based  Project “within a project”  Hrs/review  Predetermined schedule  No. comments  Orion project/contractor level  Assessments  Completion  Quality 2/22/2010 26
  • 27. SW Spiral Review – CMMI Appraisal NuggetsSpacecraft Software Engineering Branch/ER6  Double vision syndrome  KISS  Easy to get cross-eyed over  Keep oversight plan, processes, apparent identical areas between and tasks simple LM and NASA oversight project  Simple data/configuration  CMMI model vs. NASA management method engineering focus  Only controlling documents  Ex.: CM – whose CM? LM  Reasonable/simple plan and Project Link vs. NASA ICE process description results in  Ex.: Requirements management  Effective CMMI Maturity Level – Oversight project requirements 2 compliance vs. contractor project technical requirements  Project within a project  Scope focus very tight for oversight project – milestone based  Other work did not stand still 2/22/2010 27
  • 28. Typical Review ProcessSpacecraft Software Engineering Branch/ER6 Perform Tech Review Product Owner SFM/SWE Perform Peer Review Disposition Product Product Comments Comments  Initial attempt to use SE process to describe & define process oversight  Ex.: Use case diagram to represent project process for technical and peer reviews  Each use case may breakdown and be described in project plan for each specific process  Value of approach is still being evaluated (may refine for next version & evaluate for improvement) 2/22/2010 28
  • 29. Spacecraft Software Engineering Branch/ER6 2/22/2010 Next Steps29
  • 30. Next StepsSpacecraft Software Engineering Branch/ER6  New Project – Software Process Improvement Project  Among other things, earn CMMI Maturity Level 3 by April 2011  Requires major culture change to implement OT, OPF, & OPD  Orion Software PDR Project  Continues to evolve  Creating checklists to further refine measures from reviews  Expect to learn enough to provide guidance to development projects on  Planning and scheduling peer reviews  Planning and scheduling milestone reviews  Expect confusion in implementing PI, VAL, VER, and TS because our products are brain-power related rather than code 2/22/2010 30