Michel izygon


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Implement Design ElementsDevelop the project’s hardware and software products from the design. Integrate the project’s products in preparation for certification, acceptance and delivery, updating and adding any drawings, as required. Produce a Version Description Document (VDD) for software and firmware. Part of this development may include the use of OTS software and any modifications to open source, reuse, legacy, and heritage code. The requirements and design must be used as the basis for the software implementation. This includes: Implementing all of software safety design features and methods. Following the design standards. Following any other standards and recommendations. Follow the system specific polices and standards. Updating the software design documents as the software implementation evolvesConduct Formal Code InspectionThe Software Inspection Report shall include: Identification information (including item being inspected, inspection type (e.g., requirements inspection, code inspection, etc) and inspection time and date). Summary on total time expended on each inspection/peer review (including total hour summary and time participants spent reviewing the product individually). Participant information (including total number of participants and participant's area of expertise). Total number of defects found (including the total number of major defects, total number of minor defects, and the number of defects in each type (such as accuracy, consistency, completeness, etc.)). Inspection results summary (i.e., pass, re-inspection required). Listing of all inspection defects" The Lead assigns the roles and responsibilities for the Formal Code Inspection to the appropriate individuals The software is distributed to all the relevant stakeholders for inspection The SQA Representative conducts formal code inspections on all CSCIs. The findings and associated closure information are stored according to the SDP.Follow the inspection process below:1. The project must ensure the review products have completed the readiness criteria.2. The review lead will schedule the review and insure required attendees can participate.a. Define what is to be reviewedb. Define who is required to attend and who will fulfill each of the roles and responsibilities.c. Decide on whether to proceed if the required stakeholders can not participate or provide a substitute. Also document any additional risk.3. The authors will send out all material to be reviewed in advance of the meeting.a. Reviewers should familiarize themselves with the material prior to the meeting.b. Record pre-review defects/issues and bring to the meeting.4. Inspection Meetingsa. Moderator explains what is to be inspected and the inspection procedure.b. Review material and discuss defects/issues found during pre-review.c. Scribe records defects in defect log.d. Avoid excessive debate and discussion on defect resolution. Schedule off-line resolution meetings if necessary.e. Keep the inspection professional.f. Record whether each inspected item has passed, requires an update but not a re-inspection, or needs to be re-inspected.g. Code inspections should not last more than 3 hours based on best practices.5. Virtual Reviews a. Same as the above inspections except the reviewers can review on their own time and provide inputs to the moderator/lead.6. The project can document and report the results in the Software Inspection Report.The form is stored in the SDF. The report can include:a. What is being inspected and versionb. Type of inspection (code inspection, requirements review, etc.)c. Inspection date and time d. Summary of the total time spend preparing for and in the reviewe. List of participants and their area of expertise f. List of defects found and category (major or minor)g. Total number of defects found broken out by category h. Inspection result (pass, re-inspection required, etc.)i. Actions generated at the review and the results7. Post-reviewa. The project can track issues and actions identified in the reviews/inspections until they are resolved.b. For documentation reviews, RIDs may be submitted to the project manager if an item is unclear, wrong, missing, etc. RIDs follow the same process as CRs.c. Schedule re-review/inspection, if necessary.8. The project must have completed the post-review process in order for the project phase to be completed"For issues found during the review process, submit RIDs to track the outcome of the changes to the requirements document(s).1. Identify the source of the inconsistency and the rationale.2. Identify changes that need to be made to the plans and work products resulting from changes to the requirements baseline.3. Initiate corrective actions4. Include any deviations in project requirements in the status reports to upper managementPerform Unit TestingImplement one or more tests that enable the validation of the individual software components through physical execution and develop tests that can be executed in conjunction with other tests as part of a larger test infrastructure.1. Refine the Scope and Identify the Tests2. Select Appropriate Implementation Technique3. Implement the Test4. Establish External Data Sets5. Verify the Test Implementation6. Maintain Traceability Relationships.Unit testing is performed to ensure the software code accurately reflects the documented design, properly handles errors, and identifies off nominal behavior. Unit testing is performed by the Software Development Team to verify the functionality of the lowest level source code routines possible. For example, all routines that perform arithmetic functions should each be exercised individually using every available option. The Software Development Team performs unit testing on all CSUs. The results are documented and stored according to the SDP. The Software Development Team creates the unit tests and develops any supporting software necessary to exercise the code. The Software Development Team members perform unit testing on the software modules to ensure the software accurately reflects the documented design, properly handles errors and conditions, and to identify any off-nominal behavior that needs to be investigated or documented in a User's Guide. The Software Development Team investigates and researches resolutions for any unexpected or off-nominal conditions according to the CM defined process and procedure.The Software Development Team documents any identified condition or off-nominal behavior of the software that the customer / user / stakeholder should be aware of in the User's Guide.
  • Benefits of SDP Lite (abbreviated version): small project, new to SDA project manager. Makes easier to understand activities required to comply with 7150.2. Here is a high level project for you to get familiarized with SDA features. In general termsThese is the forest (SDP Lite), if you are going to do any logging , you need to get to the trees (SDP 1 version 2).
  • Here’s a Graphical View of a project’s Audit report. This project is mapped to CMMI ML2. The report provides which requirement is being addressed, and how much of the requirement is compliant with the standard to which it was mapped.(Green is Fully Implemented, Yellow is Largely, Red is Partially, Orange is Not and Brick is “No mapping.”)
  • Michel izygon

    1. 1. FacilitatingNPR7150.2 Compliance and CMMI L2/L3 audit NASA PM Challenge 2012Michel Izygon, Scott Hetherington, Svetlana Hanson
    2. 2. NASA & ProcessIncreased Process Focus Strong Call for Process Institutionalization 2
    3. 3. Process Evolution Chief Engineer increase awareness and consistency across the Agency and advance the practice of engineering …The engineering of NASA systems requires a systematic and disciplined set of processes … for the design development, operation, maintenance, and closeout of systems throughout the lifecycle of the programs and projects. ISO 9001 CMMI NPR7150.2 ITIL You IEEE 12207 are Six Sigma ... here Best Practices Process Improvement Institutionalization COMPLIANCE IS MANDATORY Resistance is FutileSoftware Productivity Consortium “The Quagmire”2000 3www.software.org
    4. 4. SDA Roots - Flight Software Problem• How can we make it easier for engineers to follow the rigorous process with minimal overhead? • Software Process Automation• Survey of workflow tools (2002) found very limited support for dealing with unanticipated events – Need a workflow tool to automate the flight software process but also allow for the following process exceptions: • Go back in the process to redo a task • Go back in the process to redo an entire group of tasks • To jump ahead • To Tailor the process • SBIR Phase I: Jan 04 – July 04 • SBIR Phase II: Nov 04 – Nov 06 Implemented EA-WI-025 • SBIR Phase III: Sep 07 – Mar 09 – SDA for MOD & Engineering • ER6 funded improvements: Apr 09 – Sept 11 • SARP Funded Infusion: 2009-2011 4
    5. 5. SDA Concepts• Flexible Process Customization – Multiple Levels – Project, program, organization – Workflow Engine + Rules – key to FLEXIBILITY• Framework to model/automate any software process – Currently implement NASA/JSC Engineering Flight Software and MOD Ground Software – Can be customized to any other NASA process• Easy to capture and share NASA “Best Practices” – Lessons learned, Process Asset Library (PAL), Best toolset• Facilitate software process execution by supporting: – Developers: worklist, guidelines, instructions, templates, examples • “Everything is at your fingertips” – Teams: distribution of tasks, collaboration tools, notification – Managers: visibility, reports, metrics 5
    6. 6. SDA Core Features• Flexibility – Designed to support any process • Select existing one • Create new one• Robust workflow engine – Process clarity – Workflow Logic• Tool Integration – Currently: Version control, MS Project, Calendar, Message Board, Wiki,…• Rich Instrumentation – Real time status of projects – Dashboard – Reporting• Mapping to any policy or regulations – NPR 7150.2A – CMMI 6
    7. 7. Evolution of SDA TPA 1.0 TPA 2.0 2003 2006 Project Project Reports Activity/task Portlet Dashboard Summary … News Calendar Web Interface Business Layer TieFlow Process TieFlow Process Automation Engine V1 Rules Portal Automation Engine V2 Engine V1 Framework Database Repository Software Process Automation (SDA)– SBIR Phase II and III NASA Infusion Awards – SDA 2007-2012 SDA used by Tietronix on most software projectsCCB Automation – 600 Users – 3 NASA SitesDCA – Budget App – Widely used at JSC 7
    8. 8. Tietronix Process Architecture 3.0 TPA 3.0 2008 Project Project Activity/task Reports Summary Notify Dashboards News Calendar SOA/Web Services Business Layer TieFlow Process Automation Engine PortalMS Project Rules Engine Graphical Workflow FrameworkCM tools EditorSVN/CVSTieTracker 3rd Party Tool Repository Services Integration Process PAL Artifacts CM Rules Harness Library 8
    9. 9. NASA Class A – Software Process Example Process EA-WI-025 – GFE Flight SW/Firmware 230 ActivitiesSDA Guides/Leads team through the process 9
    10. 10. Iterative Process 10
    11. 11. SDA - Process ManagementAny SW Process Processes Enactment Tool Document Templates Examples & References Individual ToDo List Task related Instructions Each ActivityEach Team Member 11
    12. 12. SDA - Project ManagementMultiple Project Task AssignmentsDashboard View Reports Project Status & Health Audit Trail, etc. 12
    13. 13. SDA – Perform Work 13
    14. 14. NASA-JSC benefits from SDA• Supports multiple processes and lifecycles for Mission Operations and Engineering Directorates: ‒ MOD SMP processes based on the NPR 7150 class, safety criticality, software type,… ‒ Waterfall, iterative and agile lifecycles ‒ EA-WI025: GFE Flight Project Software and Firmware Development• Provides a wizard to assist team members and new project owners in selecting Process adapted to project characteristics ‒ Reduces time for decision making• Support SCAMPI Appraisals by providing a streamlined way to gather all required information ‒ reduced the cost of man hours necessary in the preparation of SCAMPI B and Appraisals significantly 14
    15. 15. NASA-JSC benefits from SDA (cont’d)• Spacecraft Software Engineering Branch/ER6 (SSEB) is currently integrating all software projects into SDA• Assist the projects in meeting all applicable project planning, monitoring, and control requirements imposed by SMPs• Provide greater visibility into projects• Help in training novice developers• Make it easier to coordinate large, complex, geographically dispersed project• Promote higher quality/less defects at each stage through fill-the- blank templates• Earlier notice of issues, problems and schedule deviations 15
    16. 16. CMMI Support• Facilitate CMMI Implementation and Audit – Provides painless enforcement of CMMI compliant process – Organization wide consistent process execution – Traceability of all work• Reduced Effort to Achieve CMMI compliance – Accelerate the compliance schedule• Generation & Auto capture of CMMI specific metrics• Reports – Detailing CMMI compliance• Supports Continuous Process Improvement – Project History and Metrics stored in DB – Enable comparison with other projects – Share Best Practices with other projects/departments – Easily customize existing processes 16
    17. 17. Latest Features Additions• Iterative Processes: • Process Editor – Multiple iterations and decomposition of tasks – Creation and modification of processes• Relationships between projects : – Browsing of SDA processes – Parent-Child relationships – Publication of processes to SDA – Sharing of documents between related – Simulation of processes in the Process projects Editor – Parent project have read-only access to child • PAL (Process Asset Library) project – Project observers Improvements: • read-only access to the project – PAL portlet outside of project scope – User-requested addition and deletion of• External project artifacts documents to and from the PAL – Documents external to SDA may be added as – PAL Administrators control PAL content project artifacts – Definition of categories and association• Multiple process standards with documents – Processes mapped to multiple process • Usability and bug fixes standards – Compliance reports available for multiple process standards• MS Project integration improvements – Import/export of non-SDA tasks 17
    18. 18. Latest Features Additions• Features: • Infrastructure: – Additional PAL Capabilities – e-Auth support for authentication – Improve Process Editor – Upgrades of SDA third-party software • Process Library for Editor – Section 508 Accessibility Compliance – Formal document review mechanism – SDA Lite • Document states • Document relationships • Tools: • Subscription/Notification for document changes – BugFlow improvements – Additional Reporting – Tool content integration – Backload project data for existing – Relate RIDs, Bugs, and Action Items projects migrating to SDA • Processes: – Process Selection Wizard improvements – WI-023, WI-035 (Engineering) – Editable activity instructions and – Scrum / Agile background using Wiki – Improve/add MOD Processes – Improve integration with MS-Project – Improve Project relationships 18
    19. 19. SDP Lite (ER WI-035) 19
    20. 20. SDP Lite: simplified process for small projects• Implement Design Elements• Conduct Formal Code Inspection• Perform Unit Testing Completion of a single activity requires performance of multiple tasks 20
    21. 21. SDP1 version 2 (ER WI-025)Instructions for “Implement Design Elements” activity:The Development Team implements the project’s products, based on the requirementsand design. The implementation should include: 1. Implementing all of software safety design features and methods. 2. Following all applicable standards and recommendations defined in the Project Plan. 3. Following the organization polices and standards. If the activities are broken down it is easier to assign, easier to close and easier to monitor compliance and project progress. 21
    22. 22. SDA Benefits for Project• Ease of scaling from large to small and growing from small to large are the key.• Having different templates provides flexibility to serve current needs of the organization• Provides a jump start in your project: – it has project organization that comes with the process, – predefined set of activities, – templates, – CM and document repositories, – web based – easy to collaborate from different locations.• The ability to keep all project artifacts in one place. 22
    23. 23. SDA tracks compliance 1.7.9 CMMI Mapping This process activity: - 2 Practice Areas (CM, PP) - 5 Specific Practices 23
    24. 24. CMMI Compliance Report For Process Step 1.7.9: “Complete Project Plan” CMMI Artifacts/Objective Evidence + embedded history + comments For SP 1.1 – Identify Configuration Items - six (6) artifacts containing evidence: Direct Evidence & possibly Indirect Evidence 24
    25. 25. SCAMPI Audit Process observations from SEI Method Description Document – SCAMPI A1.4 – Obtain & analyze initial objective evidence SDA use 1.4.1 – Prepare participants 1.4.2 – Administer Instruments 1.4.3 – Obtain initial objective evidence Large Productivity Gain 1.4.4 – Inventory objective evidence1.5 – Prepare for Collection of Evidence 1.5.1 - Perform readiness review 1.5.2 – Prepare Data Collection Plan Organizationally Simplified 1.5.3 – Replan Data Collection (if needed) Productivity Gain2.1 – Examine Objective Evidence 2.1.1 – examine OE from instruments 2.1.2 – examine OE from Presentations 2.1.3 – examine OE from Docs Operationally Simplified 2.1.4 – examine OE from Interviews Productivity Gain2.2 – Verify and Validate Objective Evidence 2.2.1 – Verify Objective Evidence Simplified 2.2.2 – Characterize Implementation of Model Practices 2.2.3 – Validate Practice Implementation2.3 – Document Objective Evidence Partially Automated 2.3.1 – take/review/tag notes Productivity Gain 2.3.2 – Record presence/absence of OE 2.3.3 – Document Practice Implementation 2.3.4 – Review and Update the data collection plan2.4 – Generate Appraisal Results 25
    26. 26. 26
    27. 27. Compliance Report Sample No Not Partially Largely Fully Standard Requirements Details Mapping Implemented Implemented Implemented Implemented SWE-040 Access to software [Compliance : 0%] products SWE-041 Open source [Compliance : 0%] notification SWE-042 Electronic access to Source code SWE-027 COTS, GOTS, [Compliance : 100%] MOTS, etc. SWE-034 Acceptance Criteria [Compliance : 50%] PartiallyLegend: Fully Implemented Largely Implemented Not Implemented No Mapping N/A Implemented 27
    28. 28. Process Institutionalization From SEI - CMMIGeneric Goals Generic PracticesGG1: Achieve GP 1.1: Perform Base PracticesSpecific Goals Institutionalization implies thatGG2: GP 2.1: Establish an Organizational PolicyInstitutionalize a GP 2.2: Plan the Process the process is ingrained in theManaged Process GP 2.3: Provide Resources way the work is performed and GP 2.4: Assign Responsibility there is commitment and GP 2.5: Train People GP 2.6: Manage Configurations consistency to GP 2.7: Identify and Involve Relevant Stakeholders performing the process. An GP 2.8: Monitor and Control the Process institutionalized process is GP 2.9: Objectively Evaluate Adherence more likely to be retained GP 2.10: Review Status with Higher Level Management during times of stressGG3: Institutionalize GP 3.1: Establish a Defined Processa Defined Process GP 3.2: Collect Improvement InformationGG4: Institutionalize GP 4.1: Establish Quantitative Objectives for thea Quantitatively ProcessManaged Process GP 4.2: Stabilize Subprocess PerformanceGG5: Institutionalize GP 5.1: Ensure Continuous Process Improvementan Optimizing GP 5.2: Correct Root Causes of ProblemsProcessInstitutionalization a cornerstone of CMMI 28
    29. 29. How to Achieve Institutionalization• Organization wide Culture Adjustment – Visible, Persistent & Genuine Display of Management buy-in – Key people with strong Conceptual & Operational Process Skills – Commitment to a process-centric lifestyle – Continuous Improvement Expected – Everyone is involved – Process Mentality Woven into Organizational Fabric – Survives all Storms• Organization wide Support – Tools and Infrastructure supporting the methods, practices and procedures – Strong multi-level training, references, examples, and general support – Internal Marketing – Process Models, Info/Results sharing, Community – Diligent Feedback, Monitoring and Improvement Response• Infusion into multiple Centers/Projects – SARP funded Infusion Projects 29
    30. 30. SARP Infusion of SDA into the Orion Software Oversight Process• Process workflow development – Technical Review Process in place • Supports major review events • Compliant with NASA standards – Orion Software Development Oversight (SDO) Process assessed • Some process elements have been identified • Rapid customization will be required by a software System Manager (SM)• Orion Tool Chain integration assessment – Results were mixed; integration challenges remain – Complete tool suite integration will be required to ensure SDA effectiveness – All engineering information sources must be integrated• SDA Oversight Process assessment – Evaluated SM/SFM behaviors against SDA oversight capabilities – Identified key integration concepts 30
    31. 31. Oversight Process Enactment• Oversight Process Flows Implemented for Tech Reviews• Based on NPR 7150.2 requirements and Oversight Review process. 31
    32. 32. Infusion Study Conclusions• Software Development Oversight (SDO) process enactment via SDA will rely on a process “building block” approach rather than on the tailoring of existing processes. – Stable SDA oversight processes will exist, e.g. major milestone reviews – Majority of the insight/oversight work however is very fluid – Emphasis is on applying scarce resources & skills to pre-empt issues – SM’s will construct short-duration, high-intensity studies and sub-projects – SDA “Process Editor” capability is now available; supports agile processes• Tool and information integration is a key to SDA institutionalization – Objective is “process transparency” • SDA serves the needed tools and information according to the task at hand • Push the process to the background – Web-based tools can be integrated easily using I-Frames – Integration of server-based or “installation” resident tools will be a challenge – Access to engineering data located on other internal/external networks will require additional investigation
    33. 33. SDA Use• 9 NASA Projects – Engineering Directorate – ARI – LIDS Software – Morpheus – JEOD – SAFER – SPIP (software process improvement project)• 5 NASA Projects – Mission Operations Directorate (MOD) – PLATO – Timeliner – Several Bundles – RegenSafing – ConFRM – CWSyncR8toR9ConfigTest• Non-NASA Projects – Medical Device Industry – AlfaVue Server – Micro Transponder – FDA Medical Device process – DocControl (document management system) – AlfaThermo Tablet – All Tietronix internal projects 33
    34. 34. SDA Observations & Results• SDA Concept well received – Managers especially enthusiastic on automatic status capture• Lots of feedback – Usability – Enhancement Suggestions – Eerily upbeat• Most Believe they are more Productive – With good opportunity for improvement• Management, Engineers – same side of fence – Unusual lack of resistance• Everyone a Process Critic, Knowledgeable Citizen – Process awareness & understanding higher across the board – Parallels a trend in the Business Intelligence (BI) market • Next Generation BI products allow everyone to be an analyst 34
    35. 35. Value to NASA – observations Compliance Governance Compliance Governance Without process technology using process technology - SDA(more) Manual coordination & effort:• Executing project processes ….. correctly• Determining & locating Objective Evidence SDA tool :• Handling Objective Evidence Coordinating & recording the interactions of people,• PIID creation & maintenance information and process execution in a project context.• More human interviews vs. audit trail & history • guiding project/process work per SOP• Figuring who, what, when, where • on demand reports verifying compliance• Collecting metrics• Monitoring the process• Improving the process 35
    36. 36. SDA Roadmap & Status Roadmap Status• Recent Additions • Core Lifecycle Processes – Process Editor – End users – Waterfall, Iterative & Agile – Tailored for projects• Near Term • Process Wizard (for MOD) – Multiple Standards Mappings – 3rd party upgrades – User Q & A – Improved Process Mgt. – Lifecycle? – NPR Class – A, B, …, H?• Future – Safety? … – Globalization – time zones, languages – Auto generate/tailor Process – Auto – Artifact Semantic Analysis – Tool Integration Harness • Deployments – Requirements Level Traceability – 19 NASA Projects – 3rd party integrations – 10 Internal Tietronix Projects • Application Lifecycle Management – 4 Medical Projects – Other Technologies – EA, BI • Infusion • Enterprise Architecture, Business Intelligence – SARP Infusion to Multiple NASA• Processes Centers – SCAMPI, Process Improvement 36