Strassner retherford
Upcoming SlideShare
Loading in...5
×
 

Strassner retherford

on

  • 12,227 views

 

Statistics

Views

Total Views
12,227
Views on SlideShare
12,227
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Strassner retherford Strassner retherford Presentation Transcript

    • Spacecraft Software Engineering Branch/ER6 The CMMI Models in Oversight of Space Flight Software Development Liz Strassner Lead, Software Process & Process Management Group David Retherford Sr. Software and Systems Engineer/ERC Spacecraft Software Engineering Branch NASA/JSC Used with permission
    • AgendaSpacecraft Software Engineering Branch/ER6  Introduction  Selection of CMMI Models for Spacecraft Software Engineering Branch  NASA Surveillance Strategies  Project Orion Surveillance Strategies  Orion Software Oversight  CMMI Maturity Level 2 Rating on Oversight  Who are we  What we did to earn CMMI Level 2  Trials & Tribulations of Software Oversight  What we are doing now 2/22/2010 2
    • Spacecraft Software Engineering & CMMI ModelsSpacecraft Software Engineering Branch/ER6  We evaluated the three CMMI models (CMMI-ACQ, CMMI-SVC, & CMMI- DEV) for applicability to our work  CMMI-SVC was discovered to be completely non-applicable  IT support help desk; Pizza delivery service  Oversight of a contract doesn’t meet the intent of “service” in this model  CMMI-ACQ was reviewed for applicability in a SCAMPI B  NASA/JSC organizational structure caused some goals to be completely out of scope of the organization  We were fully compliant with goals and practices that were in scope of the organization  The model does not allow goals to be out of scope  CMMI-DEV  Software development work was obviously in scope  Software oversight is also in scope, but needs to be looked at from a systems engineering perspective  Software oversight on a 30 year project is never-ending so needs to be broken into smaller projects 2/22/2010 3
    • Selection of CMMI Model & ProjectsSpacecraft Software Engineering Branch/ER6  Based on our experience with the SCAMPI B, we chose to go for a CMMI Maturity Level 2 rating under the CMMI-DEV model  Selecting projects was an interesting experience  Software Development Projects  Many active software development projects to select from  A long history of flight software development projects from the former Flight Software Branch that was merged into Spacecraft Software Engineering Branch  Software Oversight  Role is significantly different than in the Shuttle or ISS Programs  Lead appraiser insisted that we have at least one project in this area because of the large focus in the organization on oversight  Chose to create sub-projects out of milestone reviews  Software Requirements Review for SCAMPI B  “System PDR & Software Spiral Review" for SCAMPI A 2/22/2010 4
    • NASA Surveillance Strategies (From COTR Refresher Training 2-18- 2008)  Basic strategies: insight, oversight, or hybridSpacecraft Software Engineering Branch/ER6  Insight: relies on performance requirements & metrics. Use a minimum set of product/process data to give adequate visibility into the integrity of the product/process.  Contractor assumes more responsibility & accountability  Government steps back, evaluates deliverables & existing contractor processes—not day-to-day close monitoring  Appropriate strategy for contracts with little cost risk, clearly-defined products/services, & confidence in proven contractor performance  Oversight: rely on NASA-imposed product specification’s & process controls, MIL standards, mandatory inspection points, etc. Intrusive monitoring at contractor’s plant, & in-line NASA involvement in contract work.  Appropriate strategy when NASA is assuming the liability for performance or quality; when we determine oversight is necessary to mitigate performance risk; or when the contractor is unproven 2/22/2010 5
    • Project Orion SurveillanceSpacecraft Software Engineering Branch/ER6  The Orion Contract as a whole uses the hybrid approach to surveillance with pre-declared oversight in high risk areas  Building a space vehicle is high risk so the contract is Cost Plus Award Fee (CPAF) type  Very extensive DRD requirements for insight  Software was pre-declared as a major risk item by the Agency, so oversight of software was planned and scheduled  Joint NASA-Contractor Risk Board  Insight for Software  All of the NPR 7150.2 chapter 5 requirements exist as individual DRDs  CMMI Maturity Level 3 was levied on all software development organizations 2/22/2010 6
    • Software Oversight of OrionSpacecraft Software Engineering Branch/ER6  Oversight for Software  NASA co-chairs all boards and panels  Every Contractor functional area lead has a NASA counterpart  NASA participates in ALL software working groups  NASA has signature authority on all major software deliveries including the Software Development Plan  NASA has an internal software test environment  NPR 7150.2 NASA Software Engineering levied on contract in its entirety  NASA Standard 7009(I) Models & Simulations Standard levied on contract in its entirety  NASA Software Assurance and Safety Standards levied on contract in a “cut and paste” method into the contract (DRD CEV-S-001)  Constellation Software documents also levied by the “cut and paste” method or by reference to specific sections or paragraphs of the CxP document 2/22/2010 7
    • Spacecraft Software Engineering Branch/ER6 Spacecraft Software Engineering Branch & CMMI Level 2 2/22/2010 8
    • Spacecraft Software Engineering BranchSpacecraft Software Engineering Branch/ER6 Spacecraft Software Engineering Branch Orion Flight Software System Manager Orion Recon System Manager Orion Vehicle Systems Management Manager Altair Lead Process & Process Management Test & Verification Group GFE Group Technology Group Systems Engineering Group Group 2/22/2010 9
    • Orion Software Team InterfacesSpacecraft Software Engineering Branch/ER6 Customer - Provider Orion Constellation Project Office Provider - Customer Orion Stakeholder – Provider Rep Software Team Customer Rep - Provider - Customer Provider Prime Contractor Subcontractor Subcontractor Subcontractor Subcontractor Subcontractor Subcontractor 2/22/2010 10
    • NASA Orion Software Team OrganizationSpacecraft Software Engineering Branch/ER6  Orion Software Manger is part of the Orion Project Office  Everyone below that level is matrixed out of Engineering organizations at JSC, Ames, GRC, and LaRC  Frank Delgado/ER6 is the Flight Software System Manager and leads the multicenter flight software team  Neil Townsend/ER6 is the Reconfiguration System Manager and leads the multicenter software process and process management team  The Spacecraft Software Engineering Branch and the Orion Software Team are approximately the same size and have a large overlap. Orion Software Spacecraft Team Software Engineering Branch 2/22/2010 11
    • Year One – Baby StepsSpacecraft Software Engineering Branch/ER6  Organization was very small  Only function was Orion oversight  Responsible for Orion Vehicle System Management (Class A)  Responsible for in-house flight software development environment for all Orion software oversight (Class D & E)  Interviewed all employees to find out how they did their work  Wrote processes based on the “how they did their work”  Assumed no higher level organizational processes affected oversight  At a conceptual level, we did not have any problem mapping the CMMI Specific Goals and Practices to processes  Late in the year, Orion Software Manager function moved here in addition to Vehicle System Manager and the branch grew 2/22/2010 12
    • Year Two – Chaos Reigns  Former Flight Software Branch merged into the branch as one ofSpacecraft Software Engineering Branch/ER6 many steps JSC Engineering took to focus software  Brought in several GFE Flight Software Development projects, most in maintenance and operations phase already CMMI Level 2 compliant  Consolidated all JSC Orion software oversight personnel  Brought in methodology for meeting Generic Goals and Practices useful for all projects  GFE Projects must follow extensive higher level organizational processes  New Branch Chief assigned  Perfect storm  Policies and processes for oversight crashed into GFE flight software policies & processes  Previous branch-only view had to be extended to reflect division and directorate infrastructure  New branch chief had new goals and expectations 2/22/2010 13
    • Year Two – Coming out of ChaosSpacecraft Software Engineering Branch/ER6  CMMI Consultation  Bill Pierce provided advise on how to structure projects for oversight and methods to apply CMMI practices to an oversight project  Selected projects: 1) Class A development, 2) Class D development/acquisition, 3) Class A-E oversight  The former Intelligent Systems Branch was merged into Spacecraft Software Engineering Branch  Primarily Class E research and technology software  SCAMPI B  Newly acquired projects from the recent merger were excluded  Uncovered a problem with PPQA  Discovered that we still had disconnects in applicability of division and directorate policies and processes  Determined that our SwRR Oversight project was mis-scoped  Appraiser direction was to do better for PDR 2/22/2010 14
    • Year 3 – Success!Spacecraft Software Engineering Branch/ER6  Solved PPQA problem temporarily by returning to original resources  Orion PDR slipped significantly, so best we could do was “plan” for PDR without any action past that  Appraiser decided to merge our SwRR Project and our start of the PDR Project into a single “Software Oversight” project for the purpose of the SCAMPI A  Allowed us to show that we learned from SwRR and incorporated those lessons into the PDR planning activities  Since all our development projects were new, we had to pull in an old (CMMI L2 rated) project that was in maintenance to ensure end of lifecycle coverage  This end of lifecycle coverage would be a problem for any new organization 2/22/2010 15
    • Spacecraft Software Engineering Branch/ER6 Trials & Tribulations of planning Software Oversight
    • Orion Software Spiral Review – OverviewSpacecraft Software Engineering Branch/ER6  Meet diverse set of goals & requirements  Orion project  SW Management Plan (CxP 72099)  Need to plan component-level PDR activities for LM SW Spiral delivery  Follow general flow of Orion system PDR (CxP 72212)  Spacecraft Software Engineering Branch  Demonstrate an oversight project meeting CMMI ML 2  Agency  Software engineering requirements (NPR 7150.2)  Systems/project requirements (NPR 7123.1)  CMMI Dev 1.2  Demonstrate that oversight project can meet ML 2 SPs & GPs 2/22/2010 17
    • Software Oversight – Changes in AttitudeSpacecraft Software Engineering Branch/ER6  Requires an orthogonal change of view of  Requirements  Configuration Management  Evaluation & assessment vs. technical development  Requirements  Technical requirements of contractor product (e.g. spacecraft) differ from oversight  As part of project tasks may influence or evaluate technical requirements  Actual oversight project requirement is to review products (e.g. SRS) , generate comments  Separation of concerns  Technical requirement development vs. oversight activities 2/22/2010 18
    • SW PDR Project Plan Impact SourcesSpacecraft Software Engineering Branch/ER6 2/22/2010 19
    • Orion PDR Flow – High Level Overview PDR Plan & KickoffSpacecraft Software Engineering Branch/ER6 • PDR Ground Rules & POD • PDR Review Process • PDR RID Process Subsystem Design Reviews (SSDR) • Subsystem Specifications & IRDs • Subsystem Design Data Books • Subsystem Specific Design & Requirements Review Presentations System & Module Review • Subsystem Rollups • System Specifications and Design Review Presentations System PDR • RID Process • RID Screening, Review, & Dispositions • PDR Pre-board & Board Software PDR • Specification & Design Tech/Peer Review Process • SW Artifact Release and RID Process • SW RID Screening, Review, & Disposition • SW PDR Pre-board & Board 2/22/2010 20
    • Oversight Project – CMMI ModelSpacecraft Software Engineering Branch/ER6  CMMI Maturity Level 2 process areas for oversight  7 PA’s – PP, PMC, REQM, CM, PPQA, MA, SAM  CMMI Maturity Level 2 process area emphasis on managing project  Planning project, process, and products  Monitor & control project 2/22/2010 21
    • Spiral Review Project – CMMI Process AreasSpacecraft Software Engineering Branch/ER6  Project Planning (PP) compliance  Develop project life cycle and  Evaluate team assessment process products  Document in project plan  Configuration Management  Project Monitoring and (CM) Control (PMC)  NASA ICE/Windchill area for  Track team participation in document storage contractor reviews  Measurement and Analysis  Team assessment of reviews (MA)  Requirements Management  Metrics for effort and effort per (REQM) review/document  Screen/review requests from  Supplier Agreement Project Office or Branch Management (SAM) Management  Contractors part of Orion SW  Product & Process Quality team Assurance (PPQA)  Use division process for  Evaluate team process engineering service support 2/22/2010 22
    • Basic Software Oversight RequirementsSpacecraft Software Engineering Branch/ER6  Not the same as the product technical requirements  What ‘we’ must do to demonstrate  Contractor development of software technical baseline  Accomplished proper oversight of contractor  Assure adequacy of contractor efforts and products  As an oversight team we must  Review contractor technical efforts (requirements & design)  Generate comments/RIDs  Attend technical and peer reviews  Evaluate software product quality and maturity  Within known (agreed to) constraints  Basically – our requirements are:  To provide a set of software engineering services to Orion project  Engineering involvement and assessment 2/22/2010 23
    • Software PDR Oversight LifecycleSpacecraft Software Engineering Branch/ER6  Not traditional SW lifecycle – no actual SW being produced  Formulate and initiate the project/process (Inception)  Execute the plan/process (Execution)  Finish the project & baseline (Closeout)  Product/artifact data  Assessment/evaluation vs. SRS/IRD  Configuration Management  Oversight project artifacts, not contractor technical artifacts 2/22/2010 24
    • Oversight PhasesSpacecraft Software Engineering Branch/ER6  Inception  Closeout  Define project lifecycle and  Formal review release organization  RID inputs  Define review process(es) and  RID review/screening boards  RID dispositions  Define NASA technical and  SW Spiral Pre-board management roles  SW Spiral board  Generate project plan  Artifact update/re-release  And away we go…  Execution  Monitor contractor software development efforts  Review artifacts, generate comments, attend reviews, attend disposition and solution meetings 2/22/2010 25
    • Process Area ConsiderationsSpacecraft Software Engineering Branch/ER6  PPQA  REQM  No software artifacts (e.g. SRS, SDD,  Oversight requirements NOT the source code, etc.) same as technical product  Process and assessment report  Activity and evaluation/assessment focused based  Use outside organization – provide  “The software shall do this …..” vs. independent assessment “The project shall participate in technical reviews.”  M&A  Control/manage between Orion SW  No technical measures (e.g. Mgr & Av&SW Project Office SLOCs, etc.)  PP  Effort/product based  Project “within a project”  Hrs/review  Predetermined schedule  No. comments  Orion project/contractor level  Assessments  Completion  Quality 2/22/2010 26
    • SW Spiral Review – CMMI Appraisal NuggetsSpacecraft Software Engineering Branch/ER6  Double vision syndrome  KISS  Easy to get cross-eyed over  Keep oversight plan, processes, apparent identical areas between and tasks simple LM and NASA oversight project  Simple data/configuration  CMMI model vs. NASA management method engineering focus  Only controlling documents  Ex.: CM – whose CM? LM  Reasonable/simple plan and Project Link vs. NASA ICE process description results in  Ex.: Requirements management  Effective CMMI Maturity Level – Oversight project requirements 2 compliance vs. contractor project technical requirements  Project within a project  Scope focus very tight for oversight project – milestone based  Other work did not stand still 2/22/2010 27
    • Typical Review ProcessSpacecraft Software Engineering Branch/ER6 Perform Tech Review Product Owner SFM/SWE Perform Peer Review Disposition Product Product Comments Comments  Initial attempt to use SE process to describe & define process oversight  Ex.: Use case diagram to represent project process for technical and peer reviews  Each use case may breakdown and be described in project plan for each specific process  Value of approach is still being evaluated (may refine for next version & evaluate for improvement) 2/22/2010 28
    • Spacecraft Software Engineering Branch/ER6 2/22/2010 Next Steps29
    • Next StepsSpacecraft Software Engineering Branch/ER6  New Project – Software Process Improvement Project  Among other things, earn CMMI Maturity Level 3 by April 2011  Requires major culture change to implement OT, OPF, & OPD  Orion Software PDR Project  Continues to evolve  Creating checklists to further refine measures from reviews  Expect to learn enough to provide guidance to development projects on  Planning and scheduling peer reviews  Planning and scheduling milestone reviews  Expect confusion in implementing PI, VAL, VER, and TS because our products are brain-power related rather than code 2/22/2010 30