Schedule assessment is used to determine whether the program/project’s planned schedule is adequate to accomplish the proposed technical work. Independent schedule risk analysis helps identify any potential risks not already identified by the project and the potential impact to the baseline schedule. The IPAO Programmatic Analysis Group (PAG) is growing its schedule assessment capability through hiring of dedicated schedule analysts and through evolution of fully integrated cost and schedule assessment approaches. This presentation discusses positive steps taken in IPAO to address the changing climate for programmatic assessment in the Agency. Lessons learned and future plans will be shared.
Documented in forensics analysis based on IPAO schedule analysis results (IPA Reports – schedule health checks)
GAO audits many of the programs and projects that IPAO assesses. It is important to understand and adhere to their criteria.
Brooks's law is a principle in software development which says that "adding manpower to a late software project makes it later". It was coined by Fred Brooks in his 1975 book The Mythical Man-Month. The corollary of Brooks's Law is that there is an incremental person who, when added to a project, makes it take more, not less time. Wikipedia.
Projects do not always understand the timeline and tend to want to shorten it. Having a credible schedule is imperative to performing the risk assessment.
MK: Added a space between each line.
Schedule Assessment and Risk Analysis Richard M. Greathouse Lead Programmatic Analysis Group Heidemarie S. Borchardt, PMP Lead Schedule AnalystContributors: Kelley Cyr, Michele King, Kelly Moses, Robin Smith Independent Program Assessment Office (IPAO)
Overview• The schedule is a powerful tool for project management. Schedule risk analysis: Identifies potential risks Identifies impacts of risks to project schedule• IPAO has the opportunity to meet with many programs and projects: Lessons Learned Future Plans 2
IPAO Forensics Analysis ExcerptStrengths:• Risk management, budget, and schedule tend to gravitate to very few (typically one) overall strengths for each program.• Resources Other Than Budget received the fewest strengths.Issues:• Schedule issues were the most frequent issues identified and may be an opportunity for improvement. – Schedule issues may be caused by poor planning, poor schedule (duration) estimates, or an overall lack of emphasis on good schedule practices. – Insufficient time may have been allowed to achieve Level I milestones, perhaps complicated by insufficient funding. IPAO Forensics Analysis performed by Keith Burleson, 2008-2009; 2009-2010 3
IPAO Schedule Analysis Results• IPAO schedule analysts typically have to amend the schedules in order to be able to assess the schedule.• Schedules received for assessment are typically not of good quality due to poor scheduling practices. Missing predecessors and successors Containing constrained data Risks are not mapped to the schedule Lack of a true critical path Inappropriate use of lags/leads hides actual work being done which does not allow the schedule to be properly loaded with resources• NASA Schedule Management Handbook and NPR 7120.5D (NID), NM 7120-81, and NPR 7123.1• Schedule management may be the area that presents the best opportunity for improvement -- There are immediate actions we can take to improve! 4
Schedule Health/Quality Check DCMA 14 Point Assessment An Integrated Baseline Review (IBR) will assess the metrics outlined in the DCMA 14 point assessment. The biggest areas for improvement are in logic and constraints.
GAO Criteria Government Accounting Office (GAO) Scheduling Best GAO audits many of Practices the programs and Metric Definition projects that IPAOScope of Work Capturing all Activities - WBS assesses. It isLogic Network Sequencing all activities – All activities have a predecessor important to and successor excepting the start and finish activity. understand andResource Loaded Schedule Assigning resources to all activities – labor, material, and adhere to their overhead criteria.Durations Establishing the duration of all activities – Basis of Estimate to include resourcesSchedule Integration Integrating schedule activities horizontally and vertically GAO is in the processCritical Path Method Establishing the critical path for all activities – Project of developing a master schedule and critical path are defined Schedule EstimatingFloat/Slack Identifying reasonable float between activities – calculated Best by schedule analysis Practices Guidebook –Schedule Risk Conducting a schedule risk analysis Draft Release MarchStatusing Updating the schedule using logic and durations to 2011 determine the dates – not constraints; Schedule analyzed continuously to determine changes in critical path and GAO Cost Estimating and variances Assessment Guide, GAO-09-3SPBaseline Schedule (new metric) Creating and managing a baseline schedule 7
Schedule Health/Quality Check NASA STAT Tool STAT is a NASA-developed schedule integrity check tool for MS Project-based schedules that aids in the development, assessment, analysis and reporting of project schedules. Many NASA schedulers do not seem to be aware of the STAT tool. This analysis should be performed as part of the update cycle. To get your copy of the STAT software and a registration form, please contact Ken Poole, MSFC, at Kenneth.W.Poole@nasa.gov 256.544.2419 8
Schedule AnalysisThe schedule is an informative management tool. Itshould be used for analysis as well as reporting actualstatus.• Critical Path Analysis• Schedule Performance Trend Analysis• Baseline vs. Current Comparison Analysis – Variance• Resource Analysis• What-if Analysis• Earned Value Management (EVM) Analysis• Schedule Margin Assessment• Schedule Risk Analysis• Joint Confidence Level (JCL) Analysis Reference: NASA/SP-2010-33403 NASA Schedule Management Handbook 9
Critical Path AnalysisMany projects will report what the critical path is; however,the reported path is not reflected in the project and in somecases is a misrepresentation of the true critical path. It isimperative that the critical path come from the IMS and thatit and the other merging paths are properly managed.• Critical Path – “the sequence of schedule activities that determines the duration of the project.” PMI• Importance of C/P Analysis – helps determine which activities can impact the project completion date (excludes LOE tasks). Understand schedule drivers.• The critical path needs to be monitored as it can change. Monitoring the critical path will show the project manager where resources are required to prevent schedule delays. Activities with higher slack values can be postponed in order to work activities that must be completed. 10
Schedule Performance Trend AnalysisTypes of Analysis:• Schedule Performance Work-off – compares baseline dates against actual and forecasted dates• Schedule Baseline vs. Actual Finish• Schedule Performance Efficiency (Baseline Execution Index)• Projection of Actuals Based upon Schedule Performance• Total Slack Trends• Baseline vs. Current Comparison Analysis Schedule Milestones Studies show that once a Schedule Variance Reports schedule slips more than 15%,• Earned Value Management it cannot get back on track. 11
Resource AnalysisResource loading of schedules is a tool to manageresources effectively.• Are the right people, material, and facilities available to do the job when needed?• Workforce planning is dependent on the project’s schedule - Brooks’s Law.• Most NASA schedules are resource and time constrained.• Cost and schedule analysts review how the resources are loaded or allocated in the IMS that supports the JCL Time dependent/variable (i.e. program management staffing) Time independent/fixed (i.e. material costs, resources) 12
What-if AnalysisThe schedule is a powerful management tool,as well as, a reporting vehicle.• Tool for assessing potential schedule problems and mitigations.• Used to plan for corrective actions. 13
Schedule Margin AssessmentMargin needs to be applied to critical path and high risk activities.• Margin – “a provision in the project management plan to mitigate cost and/or schedule risk.” PMI• Margin is not float/slack. Float/slack only appears on non-critical paths; critical paths for NASA projects tend have 0; cost reserves can be applied to both float/slack on a given path as well as schedule margin on the critical path and it is important to know the difference.• Margin can be moved around the schedule as needed for high risk activities – if this is not the case, for instance, if the time is allocated to working a specific task/risk mitigation, it is no longer considered margin.• A probabilistic risk assessment is recommended as a basis for determining required margins. 14
Schedule Risk AnalysisThe schedule is a powerful tool for assessingand mitigating risks. Project Manager, RiskManager and Scheduler work together to closelymonitor risks and schedule impacts.• Map risk ID to schedule task that is affected by the risk.• Determine if mitigation is required (mitigation tasks, additional resources, use of schedule margin if on the critical path) 15
Schedule Risk Analysis (SRA) and Simulation - GAO1. Does the schedule reflect all work to be completed?2. Are the P/p critical dates used to plan the schedule?3. Are the activities sequenced logically?4. Are activity interdependencies identified and logical?5. If there are constraints, lags, and lead times, are they required, and is documentation available to justify the amounts?6. How realistic are the schedule duration estimates?7. How were resource estimates developed for each activity and will be the resources be available when needed?8. How accurate is the critical path and was it developed with scheduling software?9. How reasonable are float estimates?10. Can the schedule determine current status and provide reasonable completion date forecasts?11. What level of confidence is associated with the P/p schedule completion date? Does it reflect a schedule risk analysis and the organization’s or stakeholder’s risk tolerance? GAO-09-3SP Appendix X
Project InputsProjects deliver products at SR-100. SRB members understand thatprogrammatic documents are works in progress.• Documents (Requested of Project): Project Management Plan, Risk Management Plan, Schedule Management Plan Work Breakdown Structure (WBS) and WBS Dictionary Resources Plan (staffing, workforce, facilities) Project management status reports and EVM Data• Best Practice Master Schedule / Integrated Master Schedule (IMS) Project runs NASA STAT health check and schedule should be green for delivery Identification of Critical Path and Schedule Margin/Reserve with basis Removal of constraints from schedule for purposes of performing SRA Provide basis for any constraints not removed from schedule Identification of risk mitigation activities in the schedule Qualitative uncertainty ratings by WBS (Ref. schedule risk consequence chart)• Risk List Likelihood and Consequence Probabilities for Risks Mapping of Risks to Specific Schedule Activities• Joint Confidence Level (JCL) and assumptions (if required)
Kickoff Initial PAG Site Meeting Review Peer Review with with Review (SR) SRB PAG Receive Initial Receive Programmatic Data Receive Updated Data Receive Final JCL Final Risk Drop from P/p Drop form P/p (Final Model* and Updates (Master Schedule & IMS Delivery and Initial Programmatic from P/p Integrated Master JCL Model Delivery*) Updates from P/p Schedule (IMS)) Due at SR-100 days Due at SR-60 days Due at SR-20 days Due at SR Collect Initial SRB Risk Collect Updated SRB Collect Final SRB Risk and Uncertainty Risk and Uncertainty and Uncertainty Assessment Assessment Assessment Apply Update Brief SRB on Updated Draft IPA Plan and Perform Cost/Schedule Create Cost/Schedule Brief SRB on Initial Updated Analysis Analysis Methodology; Review Analysis and Work with Risk Model Analysis SRB Risks per Peer with PAG Lead; Get Project to Resolve and Review Run Final SRB Risk signatures from IPAO Issues, Identify Apply P/p Risks & Initial Begin Drafting IPA Uncertainty Updates and P/p Risk Director, Review Uncertainty, and Map SRB Risks and Report Values to Updates in Model and Manager, and PAG Risks Uncertainty Values to Continue Model Provide Chair with Lead Model Updating Results IPA Report Communicate with the Create SRB P/p Regarding IPA /IPA Inform PAG Lead of Any Process The project timeline provides an Briefing and Updates to the Model Review Programmatic Internal Results Documents opportunity for the project to deliver Peer Review Charts a best-practices schedule to be Present Updated IPA Review IMS and Work Continue Findings to the SRB with P/P to resolve any used in performing the schedule Updating quality issues IPA Report Complete IPA Report risk assessment. * Required if the P/p has a JointConfidence Level (JCL) Requirement
Schedule Risk Assessment OutputThe schedule risk assessment is a tool to mitigateproject risks: Schedule Health/Quality Assessment Critical Path Analysis Tornado chart depicting the impact of risks occurring S-Curve depicting the impact of risks on the schedule confidence level Risk Assessed Stochastic Critical Path Frontier Curve showing the schedule or schedule and cost confidence level (or JCL if required)* Sensitivity Analysis (if needed) implemented to analyze the various risks to the project by looking at all their potential impact on the overall goal
Future Plans• Schedule Analysis Working Group• Connect and Discover the NASA Planning & Scheduling Community of Practice: To obtain access to the NASA P&S CoP contact: Jennifer Poston, Jennifer.L.Poston@nasa.gov (301) 286-7492• Continuous improvement of IPAO Standard Operating Procedure Instruction (SOPI) which contains cost, schedule, and risk analysis processes.• Supporting the development of 7120.5E• Visiting NASA centers and providing overview of programmatic assessment process. Ames – November 2010• Consistency between reviews and across centers 20
References• NASA Schedule Management Handbook, NASA/SP- 2010-3403, January 2010• DOD Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide Version 0.9, October 21, 2005• GAO Cost Estimating and Assessment Guide, GAO-09- 3SP 21