• Like
Upcoming SlideShare
Loading in...5
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. OCEOffice of the Chief Engineer Cross-Cutting Look at OCE Policy Compliance withinOCE NASAOffice of the Chief EngineerOCEOffice of the Chief Engineer (February 2010)OCEOffice of the Chief Engineer Presented By: Beth Keer NASA HEADQUARTERS OFFICE of the CHIEF ENGINEEROCEOffice of the Chief Engineer Used with Permission
  • 2. OCENASA Engineering & Survey Team MembersProgram/ProjectManagement Philosophy: • Involve personnel that are knowledgeable, experienced and able to take lessons back to home Center to institute change. • Take advantage of symmetry with other activities as much as possible. • Some consistency with some positions held for POC and unique skills specific to the Center. Team Members: • Beth Keer – OCE Survey Team Manager • Jim Lawrence – OCE (PSGS/Dell) • John Kelly – OCE Software Manager • Dr. David Liskowsky – OCHMO (Part time) • Participant from Center to be Surveyed • Participant from Center with interest in chosen Projects • * Patti Stockman – NASA HQ CIO (Part time) • * EVMWG Champion: Mike Ryschkewitsch Chief Engineer Sponsor: Sandra Smalley 2
  • 3. OCENASA Engineering & Survey ObjectivesProgram/ProjectManagement • Review Center processes and infrastructure for compliance with OCE requirements, policy, procedures, processes, statutes, and regulations • Review at least two Projects/Programs’ documents and processes for compliance with OCE requirements, policy, procedures, processes, statutes, and regulations • Identify systemic problems or deficiencies • Recognize areas of excellence/best practices • Receive Center feedback regarding areas where Agency policy and requirements should be modified • Results are used in Agency response to OMB/GAO/ IG issue on oversight of implementation of requirements 3
  • 4. OCENASA Engineering & Survey BasisProgram/ProjectManagement The basis for the survey: • NPR 7120.5D, NASA Space Flight Program and Project Management Requirements; • NPD 2820.1, NASA Software Policy; • NPR 7150.2, NASA Software Engineering Requirements; • NPR 7120.6, Lessons Learned; • NPD 8070.6, Technical Standards; • NPR 7120.8, NASA Research and Technology Program and Project Management Requirements (Planning Perspective); • NPR 7123.1, NASA Systems Engineering Processes and Requirements (beginning with DFRC in 2010). 4
  • 5. OCENASA Engineering & Findings Definition GuidelinesProgram/ProjectManagement Strength: A finding of OCE Policy implementation that results in reduced risk to the Program or Project. Weakness: A finding of OCE Policy implementation that results in risk to the Program or Project. Observation: A finding that is a potential weakness or potential risk to the Program or Project. Opportunity: A finding of OCE Policy Implementation that is potentially a strength. Non-Conformance: A finding of OCE Policy not being implemented and with no waiver in place. 5
  • 6. OCENASA Engineering & Survey Core ElementsProgram/ProjectManagement • Common framework for unified program and project life cycle • Program and project review structure • Technical Authority implementation • Dissenting Opinions/ Waiver Processes • Lessons Learned • Technical Standards • Software Engineering Management • Research and Technology (NPR 7120.8) 6
  • 7. OCENASA Engineering & ScheduleProgram/ProjectManagement GSFC – April 2008 KSC – July 2008 MSFC – December 2008** JSC – February 2009** LaRC – March 2009** JPL – August 2009** ARC – October 2009** (Current Status) GRC – November 2009 DFRC – *February 2010 (Round 1 complete) SSC - March 2010 HQ MD or PSO – May 2010 GSFC – July 2010 KSC – September 2010 * Planning underway ** Dates consistent with IPS 7
  • 8. OCENASA Engineering & Common FrameworkProgram/ProjectManagement Management Reporting: • Various degrees of strength to the CMC as far as Management Reporting of Projects. – Meet commitments of Center vs Roads and camodes • Integrated CMCs – Strong implementation for instances with robotic missions – Originally had good reports on CxP, (JSC, MSFC, LaRC) – ARC response not as strong (not clear if external environment change for CxP or other factors). Excellent: GSFC, MSFC, LaRC, JPL 8
  • 9. OCENASA Engineering & Common FrameworkProgram/ProjectManagement Configuration Management: • CM/DM/RM practices are inconsistent across the Centers. – Implementation is strong on projects. • Most Centers have one or two tools that they “bless” for use but do not require those tools to be used. – Program driven tool (Windchill) viewed as not user-friendly. • Some use as repository, not an improvement over working systems. • Potential for confusion/risk by using separate systems. – Records Management awareness is improving. • Records retention schedules are not consistently being completed. • Records Management planning is not clearly considered when setting up CM systems. • Records Management responsibility is consistently being assigned as the CM Officer or CM lead on the projects. 9
  • 10. OCENASA Engineering & Common FrameworkProgram/ProjectManagement Information Management: • Directives Management Systems not up to date to reflect Policy (update processes not perfect). – HQ requirements are too many and at different levels of detailed description (what vs how). Excellent: JPL has contract with definite applicability of policy AND flowdown of requirements into JPL Rules! and JPL FPP. Earned Value Management: • Centers do not have validated EVMS (JPL is exception). • Implementation within the Projects not being institutionalized within the Centers. • Value of the data is inconsistent to the Managers. 10
  • 11. OCENASA Engineering & Common FrameworkProgram/ProjectManagement Risk Management (RM): • Inconsistent use of RM as part of the decision making process across the Centers. – Some use to allocate reserve for risk mitigation. – Strongest CMCs include Risk reporting and have strong discussions at monthly CMCs. (GSFC, MSFC, JPL, LaRC) • Use of IRMA on CxP is a strength. – Possible false confidence that risks are being reported. – Focus is on identifying and tracking, rather than managing the risks by working levels. – Perspective is different from working level to Project level (i.e. rating). – Separate risk systems maintained to use the risks to manage. 11
  • 12. OCENASA Engineering & Program and Project ReviewProgram/ProjectManagement Implementation Issues: • Direction vs Recommendation • Loss of understanding that SRB is meeting objectives of TA (OCE/Centers), MD, and AA (PA&E). Acceptance of SRB requirements. 12
  • 13. OCENASA Engineering & Technical AuthorityProgram/ProjectManagement Two Implementations on the Projects: 1) Mission/Lead System Engineer is the TA-Eng and independently funded. (GSFC & MSFC) • Performs TA as a collaborative effort with the Systems Engineers on the Project 2) Chief Engineer is the TA-Eng and independently funded. • Focus of TA is strongly on the independent funding and requirements ownership (technical voice not prevalent) • Performs special studies • Independent of Systems Engineering functions of the Project/Program • Defines specific roles between TA and SE (potential to have a gap) 13
  • 14. OCENASA Engineering & Technical AuthorityProgram/ProjectManagement • TA-Software responsibilities are not widely understood. • TA Implementation Plans require updating – Most Centers addressed only TA-Eng – Surveys prompting updates to include SMA and HMTA • HMTA Infrastructure is limited to JSC. – Infrastructure that can be leveraged does exist – Training/awareness needs to be increased • Non-Conformance: JPL is submitting a waiver to HQ for independently funded TA-SMA. 14
  • 15. OCENASA Engineering & Waivers/Dissenting OpinionsProgram/ProjectManagement • Waiver Process at the Centers is not clearly understood by Projects and Programs – Weaknesses exist in content for waivers within Center documentation (inclusion of rationale) • Dissenting Opinions Process – Some Centers have a documented process within specific areas (i.e. Engineering) – No Center has a documented process that covers all possible dissenting opinions (SMA, Eng, procurement, programmatic, etc.) 15
  • 16. OCENASA Engineering & Technical Standards/Lessons LearnedProgram/ProjectManagement • Tech standards: – Overall implementation is good – Documentation of the working process • Lessons Learned: – Overall the requirements of the NPR are being met. – Personnel get the lessons they need for their next Program/Project through informal means. 16
  • 17. OCENASA Engineering & Software Engineering ManagementProgram/ProjectManagement • Overall implementation strengths exist at the Centers. • Policy is being taken seriously. • NASA workforce is working to assure requirements are flowed down to Projects. 17
  • 18. OCENASA Engineering & Software Engineering ManagementProgram/ProjectManagement Non-Compliances: • KSC: Risk Analysis and Waiver approved for Ares 1-X GS • JSC: SW Engineering Improvement Plan is out of date-working. Class A s/w being developed by organizations not having CMMI level 2 rating or higher - (in-house CMMI corrected, contractor levels are being tracked and reported). • LaRC: Required 7150.2 Compliance matrix not completed (corrected) Firmware development is proceeding without applicable s/w requirements. • HQ/JPL: NPR 7150.2 requirements are not flowed to the Contract Meet the intent and have the ability to apply at the task level. • ARC: SW Engineering Improvement Plan does not exist. • GRC: NPR 7150.2 requirements are not flowed to some contracts and procurement efforts. Class A software development with no current CMMI rating in the required 18 process areas performed by the Center.
  • 19. OCENASA Engineering & 7120.5E SuggestionsProgram/ProjectManagement • Address all requirements, but allow “Tailoring” of the approach to implementation of the requirements consistent with Program/Project characteristics such as scope, complexity, visibility, cost, safety, and acceptable risk of a project. • Address how requirements are determined to be n/a for work at a Center (elements within the Project). • Training Requirement on Project Management teams to have annual training. • Collecting Lessons learned as go through the life cycle does not take priority. • Inclusion of context within policy is viewed as positive. 19
  • 20. OCENASA Engineering & 7120.5E SuggestionsProgram/ProjectManagement • TA-Eng for Software ( including TA for Software as described in NPR 7150.2) • TA implementation focus is on the independent funding of the technical authority and waiving the technical requirements at the Project level (good). – Add language that an additional role of the independently funded TA is to be the voice of working level engineers that have not been designated as TA. • Include table to describe and point to different classifications (7120.5, 8705.4, 7150.2, etc) and context for the need of the classifications. 20
  • 21. OCENASA Engineering & 7120.5E SuggestionsProgram/ProjectManagement SRB Implementation Issues: • Implementation of SRBs is not clear to Projects that all convening authorities objectives are being met . • SRBs act as a decisional body rather than as a recommendation body. (Identified as an implementation issue, not a policy issue) • Inconsistent implementation of SRB process (personality dependent). • Timeframe to complete the report-out process is too long. – Policy change to allow parallel activities – Describe process to authorize continuation into the next phase while the road to the KDP continues. 21
  • 22. OCENASA Engineering & Additional OCE ActionsProgram/ProjectManagement • Policy-related letters/correspondence to be attached to the affected NPR/NPD(s) within NODIS • Use of SMART Id’s on OCE Policy to assist Centers with requirements flowdown • Blanket waiver for Class D missions 22
  • 23. OCENASA Engineering & Topic for DiscussionProgram/ProjectManagement ISSUE: Class D Payload classification and implementation of Class B Software requirements in a resource constrained environment. 1. Comply 2. Ctr level waivers project by project for delegated requirements (80+% of NPR 7150.2) and implement remaining non- delegated requirements 3. Ctr level waivers project by project for delegated requirements (80+% of NPR 7150.2) and submit waiver to HQ for relief on remaining requirements. 4. Submit blanket waiver to HQ to implement alternate Class B Software requirements that do not meet or exceed the requirements of NPR 7150.2. • Describe the Center process to identify, mitigate, communicate and accept project risk associated with the implementation approach proposed by the Center. 23
  • 24. OCE Generic/Blanket Tech Authority Waiver capabilityNASA Engineering & in NPR 7150.2A*, Chapter 6Program/ProjectManagement“6.1.1 For those cases in which a Center or project desires a general exclusion from requirement(s) in this NPR or desires to generically apply specific alternate requirements that do not meet or exceed the requirements of this NPR, the requester shall submit a waiver for those exclusions or alternate requirements for approval by the NASA Headquarters Chief Engineer with appropriate justification. [SWE-120]” “Note: This type of waiver (which is approved by the NASA Headquarters Chief Engineer) is for generic/blanket relief from a requirement for a Center, Center organization, or multiple projects over an extended time. Generic/blanket waivers are not to be confused with normal waivers that address relief from a requirement on a single project or in a specific instance (which can be approved at the Center level if so specified in last column of Appendix D).” 24* This document has passed NODIS review an is in the Administrator’s suite for signature
  • 25. OCENASA Engineering & Research and TechnologyProgram/ProjectManagement • Definition of roles and responsibilities for implementation of Technical Authority in R&T Programs and Projects is needed by HQ. (ARMD TA, Program TA, Project TA, PI, and Center TAs) • Roles and responsibilities of the Center TAs need to be clearly defined in Center documentation. • R&T Project Management Structure is complex from a communication perspective. • Checks and Balances developing within the organization and Project specific to the R&T management structure. 25
  • 26. OCENASA Engineering & Topic for DiscussionProgram/ProjectManagement • Transition from R&T project (NPR 7120.8) into a flight project (NPR 7120.5). – Clarity required – Implementation of NPR 7123.1 requirements – Implementation of SMA requirements • ARC and GRC have working processes for projects that transition from R&T to flight. • OCE/MDs evaluation of NPR 7120.8 and NPR 7123.1 for potential clarifications 26
  • 27. OCENASA Engineering &Program/ProjectManagement Back Up Charts 27
  • 28. OCENASA Engineering & Survey/Audit DifferencesProgram/ProjectManagement • Higher level of documentation review • Focus on implementation of OCE requirements • Fewer interviews conducted • Response to findings is different • Actively looking for Center feedback on areas where Agency policy and requirements should be modified 28
  • 29. OCENASA Engineering & Survey Ground RulesProgram/ProjectManagement 1. The basis for the OCE Survey is NPR 7120.5D - NASA Space Flight Program and Project Management Requirements, NPD 2820.1 - NASA Software Policy, NPD 8070.6 – Technical Standards, NPR 7120.6 - Lessons Learned Process and NPR 7150.2, NASA Software Engineering Requirements. NPR 7120.8 – NASA Research and Technology Program and Project Management Requirements. 2. The Survey will be conducted using the OCE Requirements Survey Questions checklist developed for the Center. 3. The Center interface point of contact assists the Survey Team in obtaining documentation and establishing interview schedules for the areas being surveyed. 29
  • 30. OCENASA Engineering & Survey Ground Rules (cont.)Program/ProjectManagement 4. The OCE Survey is being conducted in coordination with the CxP SA Level 2 Audit. Some Survey areas (such as TA-SMA, Risk Management, Design documentation, etc.) also have attributes which are within the scope of the CxP Audit Team. In these areas, the Audit teams will also review the OCE Survey attributes. The OCIO Records Management Assessment is also coordinated with the Survey activities. The integrated effort is taken to ensure complete coverage of all OCE requirements without duplication of reviews by multiple teams. 5. Interviews will be conducted. Designated Program/Project managers, technical authorities, records managers, configuration managers, institutional directives manager, procurement acquisition managers and software managers. Additional interviews may be conducted based on issues found during the Survey and will be arranged by the Center interface point of contact. 30
  • 31. OCENASA Engineering & Survey Ground Rules (cont.)Program/ProjectManagement 6. The Survey Team (in conjunction with the Audit/Assessment Teams) will conduct a brief meeting on Tuesday and Wednesday afternoon. The team members will discuss their findings from that day, their planned activities for the following day and any support needed from the Center. Survey efforts may be refocused or additional areas may be added at the daily meeting based on findings generated by the team members. The Center interface point of contact is requested to attend the daily team meeting to maintain close coordination with the Survey Team. 7. The Survey Team will conduct a meeting as early in the day on Thursday to discuss the findings with the Center interface point of contact and any other Center personnel invited by the POC. 31
  • 32. OCENASA Engineering & Survey Ground Rules (cont.)Program/ProjectManagement 8. OCE Survey Manager will concur with all survey findings to ensure consistent interpretation of requirements. 9. An Out-brief will be held on Friday for OCE, SMA, OCHMO and Center personnel marking the close of the on-site Survey. 10. The Center POC is provided access to the detailed dBase (SAARIS) of findings from the Survey. 11. OCE will work with the Center POC to ensure understanding of the findings and the responses. 32