Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
FacilitatingNPR7150.2 Compliance and    CMMI L2/L3 audit            NASA PM Challenge 2012Michel Izygon, Scott Hetheringto...
NASA & ProcessIncreased Process Focus                          Strong Call for                          Process Institutio...
Process Evolution  Chief Engineer increase awareness and consistency across the Agency and         advance the practice of...
SDA Roots - Flight Software Problem•       How can we make it easier for engineers to follow the rigorous        process w...
SDA Concepts• Flexible Process Customization – Multiple Levels   – Project, program, organization   – Workflow Engine + Ru...
SDA Core Features• Flexibility   – Designed to support any process       • Select existing one       • Create new one• Rob...
Evolution of SDA             TPA 1.0                                                      TPA 2.0              2003       ...
Tietronix Process Architecture 3.0                                                    TPA 3.0                             ...
NASA Class A – Software Process                                            Example Process                                ...
Iterative Process       10
SDA - Process ManagementAny SW Process           Processes Enactment Tool                                                 ...
SDA - Project ManagementMultiple Project         Task AssignmentsDashboard View                                           ...
SDA – Perform Work        13
NASA-JSC benefits from SDA• Supports multiple processes and lifecycles for Mission  Operations and Engineering Directorate...
NASA-JSC benefits from SDA (cont’d)• Spacecraft Software Engineering Branch/ER6 (SSEB) is currently  integrating all softw...
CMMI Support• Facilitate CMMI Implementation and Audit   – Provides painless enforcement of CMMI compliant process   – Org...
Latest Features Additions•   Iterative Processes:                                  •   Process Editor     –   Multiple ite...
Latest Features Additions•   Features:                                                 •   Infrastructure:     – Additiona...
SDP Lite (ER WI-035)        19
SDP Lite: simplified process for            small projects• Implement Design Elements• Conduct Formal Code Inspection• Per...
SDP1 version 2 (ER WI-025)Instructions for 3.4.4.1 “Implement Design Elements” activity:The Development Team implements th...
SDA Benefits for Project• Ease of scaling from large to small and growing from  small to large are the key.• Having differ...
SDA tracks compliance                    1.7.9 CMMI Mapping                    This process activity:                    -...
CMMI Compliance Report                             For Process Step 1.7.9:                             “Complete Project P...
SCAMPI Audit Process observations                 from SEI Method Description Document – SCAMPI A1.4 – Obtain & analyze in...
26
Compliance Report Sample                                            No        Not       Partially   Largely      Fully    ...
Process Institutionalization                                             From SEI - CMMIGeneric Goals Generic PracticesGG1...
How to Achieve Institutionalization• Organization wide Culture Adjustment   –   Visible, Persistent & Genuine Display of M...
SARP Infusion of SDA into the                Orion Software Oversight Process•   Process workflow development     – Techni...
Oversight Process Enactment•   Oversight Process Flows Implemented for Tech Reviews•   Based on NPR 7150.2 requirements an...
Infusion Study Conclusions•   Software Development Oversight (SDO) process enactment via SDA    will rely on a process “bu...
SDA Use• 9 NASA Projects – Engineering Directorate   –   ARI   –   LIDS Software   –   Morpheus   –   JEOD   –   SAFER   –...
SDA Observations & Results• SDA Concept well received   – Managers especially enthusiastic on automatic status capture• Lo...
Value to NASA – observations          Compliance Governance                               Compliance Governance           ...
SDA Roadmap & Status               Roadmap                                                            Status•   Recent Add...
Upcoming SlideShare
Loading in …5
×

Michel izygon

15,312 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Michel izygon

  1. 1. FacilitatingNPR7150.2 Compliance and CMMI L2/L3 audit NASA PM Challenge 2012Michel Izygon, Scott Hetherington, Svetlana Hanson
  2. 2. NASA & ProcessIncreased Process Focus Strong Call for Process Institutionalization 2
  3. 3. Process Evolution Chief Engineer increase awareness and consistency across the Agency and advance the practice of engineering …The engineering of NASA systems requires a systematic and disciplined set of processes … for the design development, operation, maintenance, and closeout of systems throughout the lifecycle of the programs and projects. ISO 9001 CMMI NPR7150.2 ITIL You IEEE 12207 are Six Sigma ... here Best Practices Process Improvement Institutionalization COMPLIANCE IS MANDATORY Resistance is FutileSoftware Productivity Consortium “The Quagmire”2000 3www.software.org
  4. 4. SDA Roots - Flight Software Problem• How can we make it easier for engineers to follow the rigorous process with minimal overhead? • Software Process Automation• Survey of workflow tools (2002) found very limited support for dealing with unanticipated events – Need a workflow tool to automate the flight software process but also allow for the following process exceptions: • Go back in the process to redo a task • Go back in the process to redo an entire group of tasks • To jump ahead • To Tailor the process • SBIR Phase I: Jan 04 – July 04 • SBIR Phase II: Nov 04 – Nov 06 Implemented EA-WI-025 • SBIR Phase III: Sep 07 – Mar 09 – SDA for MOD & Engineering • ER6 funded improvements: Apr 09 – Sept 11 • SARP Funded Infusion: 2009-2011 4
  5. 5. SDA Concepts• Flexible Process Customization – Multiple Levels – Project, program, organization – Workflow Engine + Rules – key to FLEXIBILITY• Framework to model/automate any software process – Currently implement NASA/JSC Engineering Flight Software and MOD Ground Software – Can be customized to any other NASA process• Easy to capture and share NASA “Best Practices” – Lessons learned, Process Asset Library (PAL), Best toolset• Facilitate software process execution by supporting: – Developers: worklist, guidelines, instructions, templates, examples • “Everything is at your fingertips” – Teams: distribution of tasks, collaboration tools, notification – Managers: visibility, reports, metrics 5
  6. 6. SDA Core Features• Flexibility – Designed to support any process • Select existing one • Create new one• Robust workflow engine – Process clarity – Workflow Logic• Tool Integration – Currently: Version control, MS Project, Calendar, Message Board, Wiki,…• Rich Instrumentation – Real time status of projects – Dashboard – Reporting• Mapping to any policy or regulations – NPR 7150.2A – CMMI 6
  7. 7. Evolution of SDA TPA 1.0 TPA 2.0 2003 2006 Project Project Reports Activity/task Portlet Dashboard Summary … News Calendar Web Interface Business Layer TieFlow Process TieFlow Process Automation Engine V1 Rules Portal Automation Engine V2 Engine V1 Framework Database Repository Software Process Automation (SDA)– SBIR Phase II and III NASA Infusion Awards – SDA 2007-2012 SDA used by Tietronix on most software projectsCCB Automation – 600 Users – 3 NASA SitesDCA – Budget App – Widely used at JSC 7
  8. 8. Tietronix Process Architecture 3.0 TPA 3.0 2008 Project Project Activity/task Reports Summary Notify Dashboards News Calendar SOA/Web Services Business Layer TieFlow Process Automation Engine PortalMS Project Rules Engine Graphical Workflow FrameworkCM tools EditorSVN/CVSTieTracker 3rd Party Tool Repository Services Integration Process PAL Artifacts CM Rules Harness Library 8
  9. 9. NASA Class A – Software Process Example Process EA-WI-025 – GFE Flight SW/Firmware 230 ActivitiesSDA Guides/Leads team through the process 9
  10. 10. Iterative Process 10
  11. 11. SDA - Process ManagementAny SW Process Processes Enactment Tool Document Templates Examples & References Individual ToDo List Task related Instructions Each ActivityEach Team Member 11
  12. 12. SDA - Project ManagementMultiple Project Task AssignmentsDashboard View Reports Project Status & Health Audit Trail, etc. 12
  13. 13. SDA – Perform Work 13
  14. 14. NASA-JSC benefits from SDA• Supports multiple processes and lifecycles for Mission Operations and Engineering Directorates: ‒ MOD SMP processes based on the NPR 7150 class, safety criticality, software type,… ‒ Waterfall, iterative and agile lifecycles ‒ EA-WI025: GFE Flight Project Software and Firmware Development• Provides a wizard to assist team members and new project owners in selecting Process adapted to project characteristics ‒ Reduces time for decision making• Support SCAMPI Appraisals by providing a streamlined way to gather all required information ‒ reduced the cost of man hours necessary in the preparation of SCAMPI B and Appraisals significantly 14
  15. 15. NASA-JSC benefits from SDA (cont’d)• Spacecraft Software Engineering Branch/ER6 (SSEB) is currently integrating all software projects into SDA• Assist the projects in meeting all applicable project planning, monitoring, and control requirements imposed by SMPs• Provide greater visibility into projects• Help in training novice developers• Make it easier to coordinate large, complex, geographically dispersed project• Promote higher quality/less defects at each stage through fill-the- blank templates• Earlier notice of issues, problems and schedule deviations 15
  16. 16. CMMI Support• Facilitate CMMI Implementation and Audit – Provides painless enforcement of CMMI compliant process – Organization wide consistent process execution – Traceability of all work• Reduced Effort to Achieve CMMI compliance – Accelerate the compliance schedule• Generation & Auto capture of CMMI specific metrics• Reports – Detailing CMMI compliance• Supports Continuous Process Improvement – Project History and Metrics stored in DB – Enable comparison with other projects – Share Best Practices with other projects/departments – Easily customize existing processes 16
  17. 17. Latest Features Additions• Iterative Processes: • Process Editor – Multiple iterations and decomposition of tasks – Creation and modification of processes• Relationships between projects : – Browsing of SDA processes – Parent-Child relationships – Publication of processes to SDA – Sharing of documents between related – Simulation of processes in the Process projects Editor – Parent project have read-only access to child • PAL (Process Asset Library) project – Project observers Improvements: • read-only access to the project – PAL portlet outside of project scope – User-requested addition and deletion of• External project artifacts documents to and from the PAL – Documents external to SDA may be added as – PAL Administrators control PAL content project artifacts – Definition of categories and association• Multiple process standards with documents – Processes mapped to multiple process • Usability and bug fixes standards – Compliance reports available for multiple process standards• MS Project integration improvements – Import/export of non-SDA tasks 17
  18. 18. Latest Features Additions• Features: • Infrastructure: – Additional PAL Capabilities – e-Auth support for authentication – Improve Process Editor – Upgrades of SDA third-party software • Process Library for Editor – Section 508 Accessibility Compliance – Formal document review mechanism – SDA Lite • Document states • Document relationships • Tools: • Subscription/Notification for document changes – BugFlow improvements – Additional Reporting – Tool content integration – Backload project data for existing – Relate RIDs, Bugs, and Action Items projects migrating to SDA • Processes: – Process Selection Wizard improvements – WI-023, WI-035 (Engineering) – Editable activity instructions and – Scrum / Agile background using Wiki – Improve/add MOD Processes – Improve integration with MS-Project – Improve Project relationships 18
  19. 19. SDP Lite (ER WI-035) 19
  20. 20. SDP Lite: simplified process for small projects• Implement Design Elements• Conduct Formal Code Inspection• Perform Unit Testing Completion of a single activity requires performance of multiple tasks 20
  21. 21. SDP1 version 2 (ER WI-025)Instructions for 3.4.4.1 “Implement Design Elements” activity:The Development Team implements the project’s products, based on the requirementsand design. The implementation should include: 1. Implementing all of software safety design features and methods. 2. Following all applicable standards and recommendations defined in the Project Plan. 3. Following the organization polices and standards. If the activities are broken down it is easier to assign, easier to close and easier to monitor compliance and project progress. 21
  22. 22. SDA Benefits for Project• Ease of scaling from large to small and growing from small to large are the key.• Having different templates provides flexibility to serve current needs of the organization• Provides a jump start in your project: – it has project organization that comes with the process, – predefined set of activities, – templates, – CM and document repositories, – web based – easy to collaborate from different locations.• The ability to keep all project artifacts in one place. 22
  23. 23. SDA tracks compliance 1.7.9 CMMI Mapping This process activity: - 2 Practice Areas (CM, PP) - 5 Specific Practices 23
  24. 24. CMMI Compliance Report For Process Step 1.7.9: “Complete Project Plan” CMMI Artifacts/Objective Evidence + embedded history + comments For SP 1.1 – Identify Configuration Items - six (6) artifacts containing evidence: Direct Evidence & possibly Indirect Evidence 24
  25. 25. SCAMPI Audit Process observations from SEI Method Description Document – SCAMPI A1.4 – Obtain & analyze initial objective evidence SDA use 1.4.1 – Prepare participants 1.4.2 – Administer Instruments 1.4.3 – Obtain initial objective evidence Large Productivity Gain 1.4.4 – Inventory objective evidence1.5 – Prepare for Collection of Evidence 1.5.1 - Perform readiness review 1.5.2 – Prepare Data Collection Plan Organizationally Simplified 1.5.3 – Replan Data Collection (if needed) Productivity Gain2.1 – Examine Objective Evidence 2.1.1 – examine OE from instruments 2.1.2 – examine OE from Presentations 2.1.3 – examine OE from Docs Operationally Simplified 2.1.4 – examine OE from Interviews Productivity Gain2.2 – Verify and Validate Objective Evidence 2.2.1 – Verify Objective Evidence Simplified 2.2.2 – Characterize Implementation of Model Practices 2.2.3 – Validate Practice Implementation2.3 – Document Objective Evidence Partially Automated 2.3.1 – take/review/tag notes Productivity Gain 2.3.2 – Record presence/absence of OE 2.3.3 – Document Practice Implementation 2.3.4 – Review and Update the data collection plan2.4 – Generate Appraisal Results 25
  26. 26. 26
  27. 27. Compliance Report Sample No Not Partially Largely Fully Standard Requirements Details Mapping Implemented Implemented Implemented Implemented SWE-040 Access to software [Compliance : 0%] products SWE-041 Open source [Compliance : 0%] notification SWE-042 Electronic access to Source code SWE-027 COTS, GOTS, [Compliance : 100%] MOTS, etc. SWE-034 Acceptance Criteria [Compliance : 50%] PartiallyLegend: Fully Implemented Largely Implemented Not Implemented No Mapping N/A Implemented 27
  28. 28. Process Institutionalization From SEI - CMMIGeneric Goals Generic PracticesGG1: Achieve GP 1.1: Perform Base PracticesSpecific Goals Institutionalization implies thatGG2: GP 2.1: Establish an Organizational PolicyInstitutionalize a GP 2.2: Plan the Process the process is ingrained in theManaged Process GP 2.3: Provide Resources way the work is performed and GP 2.4: Assign Responsibility there is commitment and GP 2.5: Train People GP 2.6: Manage Configurations consistency to GP 2.7: Identify and Involve Relevant Stakeholders performing the process. An GP 2.8: Monitor and Control the Process institutionalized process is GP 2.9: Objectively Evaluate Adherence more likely to be retained GP 2.10: Review Status with Higher Level Management during times of stressGG3: Institutionalize GP 3.1: Establish a Defined Processa Defined Process GP 3.2: Collect Improvement InformationGG4: Institutionalize GP 4.1: Establish Quantitative Objectives for thea Quantitatively ProcessManaged Process GP 4.2: Stabilize Subprocess PerformanceGG5: Institutionalize GP 5.1: Ensure Continuous Process Improvementan Optimizing GP 5.2: Correct Root Causes of ProblemsProcessInstitutionalization a cornerstone of CMMI 28
  29. 29. How to Achieve Institutionalization• Organization wide Culture Adjustment – Visible, Persistent & Genuine Display of Management buy-in – Key people with strong Conceptual & Operational Process Skills – Commitment to a process-centric lifestyle – Continuous Improvement Expected – Everyone is involved – Process Mentality Woven into Organizational Fabric – Survives all Storms• Organization wide Support – Tools and Infrastructure supporting the methods, practices and procedures – Strong multi-level training, references, examples, and general support – Internal Marketing – Process Models, Info/Results sharing, Community – Diligent Feedback, Monitoring and Improvement Response• Infusion into multiple Centers/Projects – SARP funded Infusion Projects 29
  30. 30. SARP Infusion of SDA into the Orion Software Oversight Process• Process workflow development – Technical Review Process in place • Supports major review events • Compliant with NASA standards – Orion Software Development Oversight (SDO) Process assessed • Some process elements have been identified • Rapid customization will be required by a software System Manager (SM)• Orion Tool Chain integration assessment – Results were mixed; integration challenges remain – Complete tool suite integration will be required to ensure SDA effectiveness – All engineering information sources must be integrated• SDA Oversight Process assessment – Evaluated SM/SFM behaviors against SDA oversight capabilities – Identified key integration concepts 30
  31. 31. Oversight Process Enactment• Oversight Process Flows Implemented for Tech Reviews• Based on NPR 7150.2 requirements and Oversight Review process. 31
  32. 32. Infusion Study Conclusions• Software Development Oversight (SDO) process enactment via SDA will rely on a process “building block” approach rather than on the tailoring of existing processes. – Stable SDA oversight processes will exist, e.g. major milestone reviews – Majority of the insight/oversight work however is very fluid – Emphasis is on applying scarce resources & skills to pre-empt issues – SM’s will construct short-duration, high-intensity studies and sub-projects – SDA “Process Editor” capability is now available; supports agile processes• Tool and information integration is a key to SDA institutionalization – Objective is “process transparency” • SDA serves the needed tools and information according to the task at hand • Push the process to the background – Web-based tools can be integrated easily using I-Frames – Integration of server-based or “installation” resident tools will be a challenge – Access to engineering data located on other internal/external networks will require additional investigation
  33. 33. SDA Use• 9 NASA Projects – Engineering Directorate – ARI – LIDS Software – Morpheus – JEOD – SAFER – SPIP (software process improvement project)• 5 NASA Projects – Mission Operations Directorate (MOD) – PLATO – Timeliner – Several Bundles – RegenSafing – ConFRM – CWSyncR8toR9ConfigTest• Non-NASA Projects – Medical Device Industry – AlfaVue Server – Micro Transponder – FDA Medical Device process – DocControl (document management system) – AlfaThermo Tablet – All Tietronix internal projects 33
  34. 34. SDA Observations & Results• SDA Concept well received – Managers especially enthusiastic on automatic status capture• Lots of feedback – Usability – Enhancement Suggestions – Eerily upbeat• Most Believe they are more Productive – With good opportunity for improvement• Management, Engineers – same side of fence – Unusual lack of resistance• Everyone a Process Critic, Knowledgeable Citizen – Process awareness & understanding higher across the board – Parallels a trend in the Business Intelligence (BI) market • Next Generation BI products allow everyone to be an analyst 34
  35. 35. Value to NASA – observations Compliance Governance Compliance Governance Without process technology using process technology - SDA(more) Manual coordination & effort:• Executing project processes ….. correctly• Determining & locating Objective Evidence SDA tool :• Handling Objective Evidence Coordinating & recording the interactions of people,• PIID creation & maintenance information and process execution in a project context.• More human interviews vs. audit trail & history • guiding project/process work per SOP• Figuring who, what, when, where • on demand reports verifying compliance• Collecting metrics• Monitoring the process• Improving the process 35
  36. 36. SDA Roadmap & Status Roadmap Status• Recent Additions • Core Lifecycle Processes – Process Editor – End users – Waterfall, Iterative & Agile – Tailored for projects• Near Term • Process Wizard (for MOD) – Multiple Standards Mappings – 3rd party upgrades – User Q & A – Improved Process Mgt. – Lifecycle? – NPR Class – A, B, …, H?• Future – Safety? … – Globalization – time zones, languages – Auto generate/tailor Process – Auto – Artifact Semantic Analysis – Tool Integration Harness • Deployments – Requirements Level Traceability – 19 NASA Projects – 3rd party integrations – 10 Internal Tietronix Projects • Application Lifecycle Management – 4 Medical Projects – Other Technologies – EA, BI • Infusion • Enterprise Architecture, Business Intelligence – SARP Infusion to Multiple NASA• Processes Centers – SCAMPI, Process Improvement 36

×