The WBS is the touchstone of all work activities, cost, schedule, and technical performance on the program.
It describes technical, process, and programmatic deliverables over the life of the program
It describes how these deliverables are related through a well formed tree structure – parents and children – defined by MIL-STD-881C
It describes how costs are assigned to this work and these costs roll up to their parents in Control Account and CLINS to the Performance Measurement Baseline (PMB)
Integrated master plan (imp) - the foundation of the program successGlen Alleman
Technical Performance Measures provide assessment of Physical Percent Complete in Earned Value Management Systems in ways not possible with simple measures of progress to plan stated by Control Account Managers
Starting with the development of a Rough Order of Magnitude (ROM) estimate of work and duration, creating the Product Roadmap and Release Plan, the Product and Sprint Backlogs, executing and statusing the Sprint, and informing the Earned Value Management Systems, using Physical Percent Complete of progress to plan.
The integrated master plan and integrated master scheduleGlen Alleman
The Integrated Master Plan (IMP) and Integrated Master Schedule( (IMS) provide a strategy for the incremental delivery of program outcomes through increasing maturity assessments with Measures of Effectiveness, Measures of Performance, Technical Performance Measures, and Key Performance Parameters.
These assessment assure the needed capabilities of the project are met at each assessment point to confirm physical percent complete as planned in the Integrated Master Plan
Building the Integrated Master Plan (and its Integrated Master Schedule) is a critical success factor in any project domain. It describes the increasing maturity of all deliverables in units of measure meaningful to the decision makers.
The IMP contains the Measures of Effectiveness and Measures of Performance. The IMS contains the Technical Performance Measures (as exit criteria for the Work Packages).
Risk and estimates are applied at all levels of the IMP and IMS, then definitized in the Performance Measurement Baseline on contract
Integrated master plan (imp) - the foundation of the program successGlen Alleman
Technical Performance Measures provide assessment of Physical Percent Complete in Earned Value Management Systems in ways not possible with simple measures of progress to plan stated by Control Account Managers
Starting with the development of a Rough Order of Magnitude (ROM) estimate of work and duration, creating the Product Roadmap and Release Plan, the Product and Sprint Backlogs, executing and statusing the Sprint, and informing the Earned Value Management Systems, using Physical Percent Complete of progress to plan.
The integrated master plan and integrated master scheduleGlen Alleman
The Integrated Master Plan (IMP) and Integrated Master Schedule( (IMS) provide a strategy for the incremental delivery of program outcomes through increasing maturity assessments with Measures of Effectiveness, Measures of Performance, Technical Performance Measures, and Key Performance Parameters.
These assessment assure the needed capabilities of the project are met at each assessment point to confirm physical percent complete as planned in the Integrated Master Plan
Building the Integrated Master Plan (and its Integrated Master Schedule) is a critical success factor in any project domain. It describes the increasing maturity of all deliverables in units of measure meaningful to the decision makers.
The IMP contains the Measures of Effectiveness and Measures of Performance. The IMS contains the Technical Performance Measures (as exit criteria for the Work Packages).
Risk and estimates are applied at all levels of the IMP and IMS, then definitized in the Performance Measurement Baseline on contract
SOLVING PROJECT ALLOCATION RESOURCE PROBLEMS WITH AEROSPACE ERPKevin West
Defence manufacturing is all about project manufacturing and project accounting. And that means enterprise resource planning (ERP) software for defence manufacturing must include robust functionality for project management and specifically project cost allocation.
Understanding the five immutable principles to project success will help project managers deliver on time, on budget when talking projects of any size, in any domain
This briefing is an overview of the probabilistic risk analysis processes that can be applied to our program. Although it may not appear to be a “simple” overview, this material is the tip of the iceberg of this complex topic.
Just schedule analysis has been addressed in detail here. The cost aspects of forecasting and simulation must be addressed as well to complete the connections between schedule and cost.
Probabilistic cost will be surveyed here, but an in depth review is for a later time.
From WBS to Integrated Master ScheduleGlen Alleman
A step by step guide to increasing the Probability of Program success starting with the WBS, developing the Integrated Master Plan and Integrated Master Schedule, risk adjusting the IMS, and measuring progress to plan in units of measure meaningful to the decision makers.
Establishing schedule margin using monte carlo simulation Glen Alleman
The first order goal is to develop a resource loaded, risk tolerant, Integrated Master Schedule, derived from the Integrated Master Plan that clearly shows the increasing maturity of the program's deliverables, through vertical and horizontal traceability to the program's requirements.
Design Patterns in Electronic Data ManagementGlen Alleman
The concept of a design pattern is based on the seminal work of Christopher Alexander
This White Paper describes the design patterns that are applicable to the Electronic Document Management domain. These include Management of Change, Hypermedia Navigation, Aggregated Relationships, Object to Relational Database Mapping, Compound Documents, and Data Entry Management.
Establishing the Performance Measurement BaselineGlen Alleman
The Performance Measurement Baseline is a time-phased schedule of all work to be performed, the budgeted cost for this work, and the organizational elements that produce the deliverables from this work.
Planning projects usually starts with tasks and milestones. The planner gathers this information from the participants – customers, engineers, subject matter experts. This information is usually arranged in the form of activities and milestones. PMBOK defines “project time management” in this manner. The activities are then sequenced according to the projects needs and mandatory dependencies.
The Role of the Architect in ERP and PDM System DeploymentGlen Alleman
The architect’s role in the development of an ERP or PDM system is to maintain the integrity of the vision statement produced by the owners, users, and funders of the system.
Integrated Program Performance ManagementGlen Alleman
Integrated Program Performance Management elements are organization planning schedule applied to increase the probability of program success on traditional and agile programs.
Resource Paper of Enterprise-Wide Deployment of EDMGlen Alleman
The acquisition of an Enterprise–wide software system requires careful planning and execution of a multitude of activities unrelated to the actual software systems being deployed.
Calculating Physical Percent Complete on Agile ProjectsGlen Alleman
Developing software using Scrum still needs the ability to determine the Estimate to Complete (ETC), the Estimate at Completion (EAC), and the Estimated Completion Data (ECD) when there is a fixed deliverable date, a fixed budget for that software, and a minimal set of Features and Capabilities on that date for that budget
IMP & WBS - Getting Both Right is ParamountGlen Alleman
The WBS is the starting point for program success. It tells us what DONE looks like in terms of deliverables.
The Integrated Master Plan (IMP) tells us how the increasing maturity of the deliverables will be assessed at each Program Event.
Integrated Master Schedule (IMS) tells us the order of the Work Packages needed to produce this increasing maturity.
SOLVING PROJECT ALLOCATION RESOURCE PROBLEMS WITH AEROSPACE ERPKevin West
Defence manufacturing is all about project manufacturing and project accounting. And that means enterprise resource planning (ERP) software for defence manufacturing must include robust functionality for project management and specifically project cost allocation.
Understanding the five immutable principles to project success will help project managers deliver on time, on budget when talking projects of any size, in any domain
This briefing is an overview of the probabilistic risk analysis processes that can be applied to our program. Although it may not appear to be a “simple” overview, this material is the tip of the iceberg of this complex topic.
Just schedule analysis has been addressed in detail here. The cost aspects of forecasting and simulation must be addressed as well to complete the connections between schedule and cost.
Probabilistic cost will be surveyed here, but an in depth review is for a later time.
From WBS to Integrated Master ScheduleGlen Alleman
A step by step guide to increasing the Probability of Program success starting with the WBS, developing the Integrated Master Plan and Integrated Master Schedule, risk adjusting the IMS, and measuring progress to plan in units of measure meaningful to the decision makers.
Establishing schedule margin using monte carlo simulation Glen Alleman
The first order goal is to develop a resource loaded, risk tolerant, Integrated Master Schedule, derived from the Integrated Master Plan that clearly shows the increasing maturity of the program's deliverables, through vertical and horizontal traceability to the program's requirements.
Design Patterns in Electronic Data ManagementGlen Alleman
The concept of a design pattern is based on the seminal work of Christopher Alexander
This White Paper describes the design patterns that are applicable to the Electronic Document Management domain. These include Management of Change, Hypermedia Navigation, Aggregated Relationships, Object to Relational Database Mapping, Compound Documents, and Data Entry Management.
Establishing the Performance Measurement BaselineGlen Alleman
The Performance Measurement Baseline is a time-phased schedule of all work to be performed, the budgeted cost for this work, and the organizational elements that produce the deliverables from this work.
Planning projects usually starts with tasks and milestones. The planner gathers this information from the participants – customers, engineers, subject matter experts. This information is usually arranged in the form of activities and milestones. PMBOK defines “project time management” in this manner. The activities are then sequenced according to the projects needs and mandatory dependencies.
The Role of the Architect in ERP and PDM System DeploymentGlen Alleman
The architect’s role in the development of an ERP or PDM system is to maintain the integrity of the vision statement produced by the owners, users, and funders of the system.
Integrated Program Performance ManagementGlen Alleman
Integrated Program Performance Management elements are organization planning schedule applied to increase the probability of program success on traditional and agile programs.
Resource Paper of Enterprise-Wide Deployment of EDMGlen Alleman
The acquisition of an Enterprise–wide software system requires careful planning and execution of a multitude of activities unrelated to the actual software systems being deployed.
Calculating Physical Percent Complete on Agile ProjectsGlen Alleman
Developing software using Scrum still needs the ability to determine the Estimate to Complete (ETC), the Estimate at Completion (EAC), and the Estimated Completion Data (ECD) when there is a fixed deliverable date, a fixed budget for that software, and a minimal set of Features and Capabilities on that date for that budget
IMP & WBS - Getting Both Right is ParamountGlen Alleman
The WBS is the starting point for program success. It tells us what DONE looks like in terms of deliverables.
The Integrated Master Plan (IMP) tells us how the increasing maturity of the deliverables will be assessed at each Program Event.
Integrated Master Schedule (IMS) tells us the order of the Work Packages needed to produce this increasing maturity.
Understanding DO-178: Importance and How It Affects Your CompanyAversan Inc.
Learn DO-178: Why is it important? How can it affect your company? These questions and more will be answered in this presentation, including information regarding the purpose of DO-178, Design Assurance Levels (DAL), objectives, and integral processes.
Questions? Email bd@aversan.com for more information.
To achieve success in any project domains, measures of progress to plan are needed in units meaningful to the decision-makers. These include cost, schedule, and technical performance
Informing Program Performance with Technical Performance Measures (TPMs)Glen Alleman
This Handbook provides guidance to Government Program Managers and Program Performance Analysts for identifying, integrating, managing and assessing the impacts on program performance from Technical Performance Measures (TPMs).
Recent College of Performance Management Webinar on using Technical Performance to inform Earned Value Management. Six steps to building a credible Performance Measurement Baseline to connect the dots between all the elements of the program
• Overall 5 years of IT Experience which include experience in the field of RBT in HSIT (Hardware software integration testing), SWI (Software integration testing), UT (Unit testing) Experienced in the Software Verification activities for safety critical systems in compliance with DO-178B standards
EVMS with Technical Performance MeasuresGlen Alleman
Assessing the general health of the program to establish early warning system for unexpected problems starts by integrating Earned Value metrics and Technical Performance Measures for key program deliverables.
Increasing the Probability of Project SuccessGlen Alleman
Risk Management is essential for development and production programs. Information about key cost, performance and schedule attributes are often uncertain or unknown until late in the program.
Risk issues that can be identified early in the program, which may potentially impact the program, termed Known Unknowns, can be alleviated with good risk management. -- Effective Risk Management 2nd Edition, Page 1, Edmund Conrow, American Institute of Aeronautics and Astronautics, 2003
Cost and schedule growth for complex projects is created when unrealistic technical performance expectations, unrealistic cost and schedule estimates, inadequate risk assessments, unanticipated technical issues, and poorly performed and ineffective risk management, contribute to project technical and programmatic shortfalls
From Principles to Strategies for Systems EngineeringGlen Alleman
From Principles to Strategies How to apply Principles, Practices, and Processes of Systems Engineering to solve complex technical, operational,
and organizational problems
Building a Credible Performance Measurement BaselineGlen Alleman
Establishing a credible Performance Measurement Baseline, with a risk adjusted Integrated Master Plan and Integrated Master Schedule, starts with the WBS and connects Technical Measures of progress to Earned Value
Capabilities‒Based Planning the capabilities needed to accomplish a mission or fulfill a business strategy
Only when capabilities are defined can we start with requirements elicitation
Program Management Office Lean Software Development and Six SigmaGlen Alleman
Successfully combining a PMO, Agile, and Lean / 6 starts with understanding what benefit each paradigm brings to the table. Architecting a solution for the enterprise requires assembling a “Systems” with processes, people, and principles – all sharing the goal of business improvement.
This resource document describes the Program Governance Road map for product development, deployment, and sustainment of products and services in compliance with CMS guidance, ITIL IT management, CMMI best practices, and other guidance to assure high quality software is deployed for sustained operational success in mission critical domains.
Increasing the Probability of Project Success with Five Principles and PracticesGlen Alleman
There are many approaches to managing projects in every domain.
This seminar lays the foundations for increasing the probability of project success, no matter the domain, what technology, what approach to delivering the outcomes of the project.
The principles of this approach are immutable.
The practices for implementing the principles are universally applicable.
Each chart in this presentation, contains guidance that can be applied to your project, no matter the domain.
In our short hour here, we’re going to cover a lot of material.
The bibliography contains the supporting materials we can tailor to your individual project
Seven Habits of a Highly Effective agile project managerGlen Alleman
Recent neurological studies indicate that the role of emotion in human cognition is essential; emotions are not a luxury. Instead, emotions play a critical role in rational decision–making, in perception, in human interaction, and in human intelligence. Habits are the intersection of knowledge, skill, and desire.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
JMeter webinar - integration with InfluxDB and Grafana
WBS is Paramount
1. 0Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk & Risk Assessment
- Level of Detail
- Program Oversight
level of detail
Technical
- TPMs
- Specifications
- Design docs
- Performance char.
Earned Value
Cost and Schedule Status
WBS
Schedule
IMP / IMS
Getting the WBS Right is Paramount
Focus of most programmatic and technical status discussions
Glen B Alleman
Institute for Defense Analyses
Gordon Kranz
DOD(OSD) PARCA
2. 1Office of Performance Assessments and Root Cause Analyses (PARCA)
WBS is Paramount
The WBS is the touchstone of all work activities, cost,
schedule, and technical performance on the program
It describes technical, process, and programmatic
deliverables over the life of the program
It describes how these deliverables are related
through a well formed tree structure – parents and
children – defined by MIL-STD-881C
It describes how costs are assigned to this work and
these costs roll up to their parents in Control Account
and CLINS to the Performance Measurement
Baseline (PMB)
3. 2Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk & Risk Assessment
- Level of Detail
- Program Oversight
level of detail
Schedule
IMP / IMS
Technical
- TPMs
- Specifications
- Design docs
- Performance char.
Earned Value
Cost and Schedule Status
Getting the WBS Right is Paramount
Focus of most programmatic and technical status discussions
WBS
4. 3Office of Performance Assessments and Root Cause Analyses (PARCA)
Technical Performance Measures (TPM)
Attributes that determine how well a system or
system element is satisfying or expected to satisfy
a technical requirement or goal
Assess design progress,
Track compliance to performance requirements,
Include projected performance.
Are allocated across elements of the system (WBS
components)
5. 4Office of Performance Assessments and Root Cause Analyses (PARCA)
TPMs Allocated and Tracked by Relevant
WBS Element
All TSAS equipment fits in a single C-130J aircraft
Weight budget allocated to WBS element(s)
Lvl 1 Lvl 2 Lvl 4 Transportability
Size Weight (T)
TSAS 19
1.1 Unmanned Air Vehicle Airframe, Propulsion, Avionics 7
1.2 Unmanned Ground Vehicle Vehicle, Sensors 4
1.3 UAV Payload Electronics, support, calibration 1.5
1.4 Ground System(s) Power generation, huts, support 2.5
1.10 Special Support Equipment 0.5
1.11 Standard Support facilities Shelters, hangers, towing, storage 0.5
1.14 Spares, Repair Parts Electronics, propulsion, airframe 1
1.15 Surface sensor(s) Housings, power, communications 1.5
6. 5Office of Performance Assessments and Root Cause Analyses (PARCA)
TSAS Program Performance Measures
TSAS
Element
Measures Answers the Question Example
SOW,
SOO,
ConOps
Measures of
Effectiveness
(MoE)
Are We Accomplishing The Mission?
We need the capability to: Increase
IED Placement search capabilities
by 50%
WBS
Technical
Performance
Measure (TPM)
What are we building and how do we
know it meets the specifications?
Systems, subsystems, and
supporting processes for each
deliverable
IMP – PE
and SA
MoE from CBA,
ICD, and CDD
How can we measure the increasing
maturity of the deliverables in the
narratives in the CBA, ICD, and CDD
Sensor payloads capable of IR and
UV detectors within the avionics bay
IMP – AC MoP
Technical Performance of the
deliverables derived from the MoEs
100 square miles per hours search
capabilities
IMS TPM
How does the work increase the
maturity of the deliverables?
Sensor platform TPMs inside the
bounds, on-time, on-schedule?
Tasks CPI, SPI, TCPI
What work is needed to increase the
maturity of the deliverables?
Cost and schedule matching TPM
progress?
Risk
Register
Identified risks, with
handling strategies
What are the Epistemic risks and how
are they represented in the IMS?
All variance risks included in
duration and cost. Event based risk
retirement handled in IMS, others
contained in MR
7. 6Office of Performance Assessments and Root Cause Analyses (PARCA)
TPM Trends and Responses at each
Program Event for total TSAS weight target
EV Taken, planned values met, tolerances kept, etc.
19T
16T
28T
23T
PDRSRRSFRCA TRRCDR
ROM in
Proposal
Design Model
Mock Up Scale Model Measurement
Detailed System Model
Prototype Measurement
1st Article
TechnicalPerformanceMeasure
TSASSoSWeight
8. 7Office of Performance Assessments and Root Cause Analyses (PARCA)
TPM Adjusted Program Performance
TPMs are correlated to WBS elements and their cost
and schedule performance
TPM performance impacts CPI and SPI with
burdened cost and schedule to get back to green.
9. 8Office of Performance Assessments and Root Cause Analyses (PARCA)
Merging TPMs and EV Data
EV measures quantity
TPMs measures quality
Both are needed
10. 9Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk & Risk Assessment
- Level of Detail
- Program Oversight
level of detail
Schedule
IMP / IMS
Technical
- TPMs
- Specifications
- Design docs
- Performance char.
Earned Value
Cost and Schedule Status
Getting the WBS Right is Paramount
Focus of most programmatic and technical status discussions
WBS
11. 10Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk Processes start with WBS
WBS level of detail is driven by technical and
program risk, needed to identify
– WBS provides framework for joint situational awareness
between Government and contractor
– MIL-STD-881C used as starting point and individual WBS
elements driven to level of detail required for program oversight
Work Packages from the WBS, linked to form the Tier
3 and Tier 2 IMS (per DI-MGMT-81861) must have
risk assigned and margin and mitigation strategies
– Natural variance of cost and schedule handled by margin
– Event based risk handled through mitigations or Management
Reserve
12. 11Office of Performance Assessments and Root Cause Analyses (PARCA)
Generic UAV WBS guides the elicitation
of risks in the Risk Register
11
Event based risks for each
subsystem with impacts
on cost, schedule, and
technical performance.
Each risk held in the Risk
Register, marked with
WBS.
WBS number used to
trace the risk to the IMS,
PE, SA, AC, CA, WP.
Risk retirement plans can
then be shown by PE or
Milestone in a risk
waterfall
Variance based
uncertainties handled in
cost and schedule margin
13. 12Office of Performance Assessments and Root Cause Analyses (PARCA)
DAG Definition of Risk
Risk is a measure of future uncertainties in
achieving program performance goals and
objectives within defined cost, schedule, and
performance constraints.
Risk can be associated with all aspects of a
program (e.g., threat environment, hardware,
software, human interface, technology maturity,
supplier capability, design maturation, performance
against plan,) as these aspects relate across the
work breakdown structure and Integrated Master
Schedule.
14. 13Office of Performance Assessments and Root Cause Analyses (PARCA)
Both risk and uncertainty must be
addressed on the program
Uncertainty
Naturally occurring variance in
the work efforts or cost
Like the weather, these variances
are always there and are always
changing
Uncertainty can be modeled with
a Monte Carlo Simulation tool
and Reference Class Forecasting
based on past performance
Event Based Risk
Probability of an event occurring
in the future that results in an
unfavorable outcome
When this event occurs the
consequential may be
probabilistic as well.
Probability of occurrence and
impact are used to model the
cost and schedule. Also uses
Monte Carlo Simulation tool
The natural variation of the project
activities. Variance and impacts need
cost and schedule margin
The probability that something will
happen to impact cost, schedule, and
technical performance of deliverables
15. 14Office of Performance Assessments and Root Cause Analyses (PARCA)
Risks Identified with WBS elements
Each risk identified in the elicitation process
WBS contained deliverables assigned to risk
retirement processes
Risk water fall defined by Program Event
ID Risk Title
Initial
Risk
Risk at
IBR
Risk at
PDR Risk Type WBS
038 Center-of-Gravity Limits 16 15 10 Technical 2.1.5
006 Gross Liftoff Weight 16 15 10 Technical 2.1.5
090 Flight & Mission-Critical Software Development Effort 16 11 10 Schedule 2.1.4
101 Unattended launch system design 16 12 8 Schedule 6.2.14
082 Achieving Component, Subsystem- & System Quals 15 14 11 Schedule 2.1.7
244 Vehicle Production timing 12 12 10 Schedule 6.5
095 Autonomous Rendezvous flight pattern design 12 10 9 Schedule 6.2.12
017 EMI Anti-Jam Protection System Development 12 10 7 Technical 6.2.5
243 Landing and Impact Attenuation 12 12 6 Technical 6.2.11
098 Recover/Landing System (RLS) Rigging Complexity 12 12 6 Technical 6.2.11
088 Qualification of EEE Parts 12 10 4 Schedule 2.1.9.3
091 Uncertain To Achieve Payload Mounting Limits 12 8 3 Schedule 2.1.8
16. 15Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk chart for TPM variance improvement
as the program matures
Target launch mass is 27 Metric Tons for the WBS
element 3.0 – launch ready spacecraft
Baseline
Mass plan
80%
Mean
Too heavy
to launch
Target Mass
of spacecraft
23T
24T
25T
25T
26T
27T
28T
29T
30T
Margin
Risk
Margin
Current plan with
probabilistic margin
model
CDR
PDR
SRR
FRR
ATLO
20%
Aug 05 Jan 06 Aug 06 Mar 07 Dec 07 Feb 08
Current mass plan with
deterministic margin
Plan
Title
Probability distribution
varies as time passes
17. 16Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk Management Processes for Program
Management
An approach to programmatic and technical risk
18. 17Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk Mitigation Activities in IMS
“Buying down” risk is
planned in the IMS.
MoE, MoP, and KPP
defined in the work
package for the critical
measure – weight.
If we can’t verify we’ve
succeeded, then the
risk did not get reduced.
The risk may have
gotten worse
Risk: CEV-037 - Loss of Critical Functions During Descent
Planned Risk Level Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete)
RiskScore
24
22
20
18
16
14
12
10
8
6
4
2
0
Conduct Force and Moment Wind
Develop analytical model to de
Conduct focus splinter review
Conduct Block 1 w ind tunnel te
Correlate the analytical model
Conduct w ind tunnel testing of
Conduct w ind tunnel testing of
Flight Application of Spacecra
CEV block 5 w ind tunnel testin
In-Flight development tests of
Damaged TPS flight test
31.Mar.05
5.Oct.05
3.Apr.06
3.Jul.06
15.Sep.06
1.Jun.07
1.Apr.08
1.Aug.08
1.Apr.09
1.Jan.10
16.Dec.10
1.Jul.11
Weight risk
reduced from
RED to Yellow
Weight confirmed
ready to fly – it’s
GREEN at this point
19. 18Office of Performance Assessments and Root Cause Analyses (PARCA)
Risks in Risk Register connected to WBS
elements provide cost impact analysis
Risk ID traceable to and from IMS with schedule
impacts defined by probabilistic impact
WBS elements collect cost impact of risk from the
IMS
Risk handling strategies connected to IMP, IMS,
WBS, SOW, and TPM measures
20. 19Office of Performance Assessments and Root Cause Analyses (PARCA)
Management Reserve Log (MRL) provides
the integrity for all changes to the PMB
All changes authorized through the BCR process
All impacts recorded with BCR and in Management
Reserve impacts (ups and downs) captured in the
same meeting
21. 20Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk & Risk Assessment
- Level of Detail
- Program Oversight
level of detail
Schedule
IMP / IMS
Technical
- TPMs
- Specifications
- Design docs
- Performance char.
Earned Value
Cost and Schedule Status
Getting the WBS Right is Paramount
Focus of most programmatic and technical status discussions
WBS
22. 21Office of Performance Assessments and Root Cause Analyses (PARCA)
Earned Value Management is the ‘best
tool’ for managing large, complex
acquisition programs.
– Ashton Carter (USD, AT&L) 26
November 2009
23. 22Office of Performance Assessments and Root Cause Analyses (PARCA)
Standard ANSI-748B Earned Value
Management
Measures program performance against plan by
comparing the:
– Planned Value (Budgeted Cost of Work Scheduled) – the
budget for the planned work and period of performance for that
budget
– Earned Value (Budgeted Cost of Work Performed) – the
measure physical percent complete x Planned Value
BCWP = BCWS x Physical Percent Complete
With the Actual Cost collected, Earned Value can
determine both Cost Variance and Schedule
Variance
24. 23Office of Performance Assessments and Root Cause Analyses (PARCA)
Earned Value Management
23
25. 24Office of Performance Assessments and Root Cause Analyses (PARCA)
Control Accounts are formed at the
Intersection of the WBS and OBS
26. 25Office of Performance Assessments and Root Cause Analyses (PARCA)
Format 1 shows program performance by
WBS element against planned cost
27. 26Office of Performance Assessments and Root Cause Analyses (PARCA)
Connecting the EVM Elements with
Technical Performance Measures
Integrating Cost, Schedulele, and Technical Performance
Assures Program Management has the needed performance information to
deliver on‒time, on‒budget, and on‒specification
Technical Performance Measures
Cost Schedule
Conventional Earned Value
+
=
Master Schedule is used to
derive Basis of Estimate
(BOE) not the other way
around.
Probabilistic cost
estimating uses past
performance and cost risk
modeling.
Labor, Materiel, and other
direct costs accounted for
in Work Packages.
Risk adjustments for all
elements of cost.
Cost Baseline
Earned Value is diluted by
missing technical
performance.
Earned Value is diluted by
postponed features.
Earned Value is diluted by
non compliant quality.
All these dilutions require
adjustments to the
Estimate at Complete
(EAC) and the To Complete
Performance Index (TCPI).
Technical Performance
Requirements are
decomposed into physical
deliverables.
Deliverables are produced
through Work Packages.
Work Packages are
assigned to accountable
manager.
Work Packages are
sequenced to form the
highest value stream with
the lowest technical and
programmatic risk.
Schedule Baseline
29. 28Office of Performance Assessments and Root Cause Analyses (PARCA)
Control Account Plan
Confirms the
agreement between
CAM and PM for
accomplishing scope,
budget, and schedule
Processes
– Development and
assignment – identify key
control points between
WBS and OBS
– Work Authorization –
develop detail cost and
schedules for Work
Packages and Planning
Packages
30. 29Office of Performance Assessments and Root Cause Analyses (PARCA)
Control Account Plan Work Sheet
31. 30Office of Performance Assessments and Root Cause Analyses (PARCA)
Risk & Risk Assessment
- Level of Detail
- Program Oversight
level of detail
Schedule
IMP / IMS
Technical
- TPMs
- Specifications
- Design docs
- Performance char.
Earned Value
Cost and Schedule Status
Getting the WBS Right is Paramount
Focus of most programmatic and technical status discussions
WBS
32. 31Office of Performance Assessments and Root Cause Analyses (PARCA)
IMP/IMS from RFP to Execution starts with
the WBS
SOW
WBS
SOO
ConOps
Objective status and views to
support proactive management
33. 32Office of Performance Assessments and Root Cause Analyses (PARCA)
IMP/IMS Structure
IMS
IMP
Describes how program
capabilities will be
delivered and
how these
capabilities will
be recognized
as ready for
delivery
Supplemental Schedules
Work Packages and Tasks
Criteria
Accomplishment
Events
or
Milestones
34. 33Office of Performance Assessments and Root Cause Analyses (PARCA)
Horizontal and Vertical Traceability of the
IMP/IMS
Integrated Master Schedule
Work sequenced to
produce outcomes
for each WP.
Vertical traceability AC SA PE
Horizontal traceability WP WPAC
Program Events
Define the maturity
of a Capability at a point in
time.
Significant Accomplishments
Represent requirements
that enable Capabilities.
Accomplishment Criteria
Exit Criteria for the Work
Packages that fulfill Requirements.
Work
Package
Work
Package
Work
Package
Work
Package
Work
Package
Work
package
Work
Package
Work
Package
35. 34Office of Performance Assessments and Root Cause Analyses (PARCA)
IMP/IMS Role During Execution
Program ExecutionPMB for IBRProposal SubmittalDRFP & RFP
Performance Measurement Baseline
Tasks (T)
BOE
% Complete
Statement of Work
Program Deliverables
IMP
Accomplishments (A)
Criteria (C)
EVMS
Events (E)
Budget Spreads by CA & PackagesCAIV
Capabilities Based Requirements
X BCWS =
Probabilistic Risk Analysis
=
Time keeping and ODC =
Technical Performance Measure
BCWP
ACWP
Cost & Schedule Risk Model
BCWS
D ecreasing techni cal and programmatic ri sk usi ng R i sk Management Methods
IMS
Physical % Complete
Continuity and consistency from DRFP through Program Execution
WBS CWBS
36. 35Office of Performance Assessments and Root Cause Analyses (PARCA)
IMP Narrative for TSAS PDR
Program
Event
Program Event Description
Maturity Assessment
measured by Significant
Accomplishments
PDR PDR establishes the TSAS “design- to” allocated
baseline to the subsystem level, ensures this design
meets the functional baseline, and system
requirements have been properly allocated to the
proper subsystem.
PDR establishes the feasibility of the TSAS design
approach to meet the technical requirements and
provide acceptable interface relationships between the
hardware and other interfacing items. Any changes to
the requirements that have occurred since the System
Requirements Review (SRR2) will be verified at the
PDR.
PDR assures the design is verifiable, does not pose
major IMS or cost risk, and is mature enough to
advance to the detailed design phase.
Subsystem level operational
concepts defined
System level interfaces
baselined
Supportability plans
established
Software requirements
finalized
Subsystem requirements
finalized & allocated
System verification,
validation & certification
plans updated
PDR subsystem design
completed
Build IMP
37. 36Office of Performance Assessments and Root Cause Analyses (PARCA)
WBS collects costs and assures mapping
to SOW and maturity measures in the IMP
The product and process WBS collects costs at the
deliverables level
The IMP shows measures of increasing maturity
against those costs
38. 37Office of Performance Assessments and Root Cause Analyses (PARCA)
Work Packages, Planning Packages, and
Rolling Waves form the IMS
The planning horizon should not be beyond a
distance where reliable estimated cannot be formed
Rolling Waves and Planning provide this capability
X3_1158_043_F
Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Mon10
Time Now
WP #4
Plan and Input
Next RW Period
WP #5
WP #6
WP #7
WP #9
WP #8
Minimum of 1 month
advance detail planning
Detail planning of next
rolling wave
30 Days
RW #3RW #3
Rolling Wave Period #2
Rolling Wave Period #1
WP #2
WP #3
PP #9
PP #8
PP #7
PP #6
PP #5
39. 38Office of Performance Assessments and Root Cause Analyses (PARCA)
INTEGRATED PROGRAM
MANAGEMENT SYSTEM
Each element of the program, anchored through the Work
Breakdown Structure, contributes to increased visibility to Program
Performance using all performance measures.
40. 39Office of Performance Assessments and Root Cause Analyses (PARCA)
Integrated Program Management System
42. 41Office of Performance Assessments and Root Cause Analyses (PARCA)
Capabilities Based Assessment
DOTMLPF
Analysis
CBA ICD
DCR
MDD
MSA
MS A
Development
MS B
CDD
EMD
MS C
Production and
Deployment
CPD
Doctrine, Organization, Training,
Materiel, Leadership, Personnel,
Facilities
DOTMLPF Change Recommendation
Capabilities Based Assessment
Initial Capabilities Document
Materiel Development Decision
Materiel Solution Analysis
Capability Development Document
Engineering, Manufacturing, Deployment
Capability Development Documenthttp://www.acqnotes.com/Acquisitions/JCIDS%20Overview.html
43. 42Office of Performance Assessments and Root Cause Analyses (PARCA)
Drivers of Technical Performance
Measures Start with Mission Need
MoE
KPP
MoP TPM
Mission
Need
Acquirer Defines the Needs and Capabilities
in terms of Operational Scenarios
Supplier Defines Physical Solutions that
meet the needs of the Stakeholders
Operational
measures of success
related to the
achievement of the
mission or
operational
objective being
evaluated.
Measures that
characterize physical
or functional
attributes relating
to the system
operation.
Measures used to
assess design
progress,
compliance to
performance
requirements, and
technical risks.
Government Contractor
JROC KPP’s or
program specific KPP’s
Loiter on station ≥ 2
hours
44. 43Office of Performance Assessments and Root Cause Analyses (PARCA)
TPM Work Products
Mandatory Key Performance Parameters (KPP)
– JROC – (1) Force Protection, (2) Survivability, (3) Sustainment,
(4) Net-Ready, (5) Training, (6) Energy
– Program specific KPP’s
Technical Performance Measures
– Weight, thermal load, loiter time, serviceability
Earned Value Management
– CPI
– SPI
45. 44Office of Performance Assessments and Root Cause Analyses (PARCA)
Key elements to Connecting EV with TPM†
Traceability – Requirements to WBS to TPMs to EV
control accounts.
Impact – How much WBS work, & therefore EV
money, is covered by the TPM(s)? What is effect?
TPM Banding/Sensitivity – What banding (R/Y/G)
and sensitivity (EV impact) should be used for each
TPM?
Technical Readiness Level – What’s the state of the
technology supporting the requirement(s) for which
TPM is a metric?
† Implementing Technical Performance Measurement, Mike Ferraro, PEO/SYSCOM Conference, 20-22 Nov 2002
46. 45Office of Performance Assessments and Root Cause Analyses (PARCA)
Some Candidates for Technical
Performance Measures
TPM Concept Description of TPM
Physical Size and Stability
Useful Life
Weight 25kg
Volumetric capacity
Functional Correctness
Accuracy
Power performance
All the “ilities”
Supportability
Maintainability
Dependability
Reliability = Mean Time Failure
Efficiency
Utilization
Response time
Throughput
Suitability for Purpose Readiness
Weight
47. 46Office of Performance Assessments and Root Cause Analyses (PARCA)
1.1 Air Vehicle
1.1.1 Sensor Platform
1.1.2 Airframe
Airframe Weight TPM
The WBS for a UAV
1.1.2 Airframe
CA SFR SRR PDR CDR TRR
Planned Value 28.0kg 27.0kg 26.0kg 25.0kg 24.0kg 23.0kg
Actual Value 30.4kg 29.0kg 27.5kg 25.5kg
Assessed Risk
to TRR
Moderate
>2.0kg off
target
Low
1–2 kg off
target
Low
1–2 kg off
target
Very Low (less
than 1.0 kg
off target)
Planned
Method
“Similar to”
Estimate
ROM
Program–
unique design
model
Program–
unique design
model with
validated data
Actual
measurement
of bench–test
components
Actual
measurement
of prototype
airframe
Actual
Method
“Similar to”
Estimate
ROM ROM ROM
The planned weight is
25kg. The actual weight is
25.5kg.
Close to plan! So we are
doing okay, right?
Here’s the Problem
48. 47Office of Performance Assessments and Root Cause Analyses (PARCA)
1.1 Air Vehicle
1.1.1 Sensor Platform
1.1.2 Airframe
1.1.3 Propulsion
1.1.4 On Board Comm
1.1.5 Auxiliary Equipment
1.1.6 Survivability
Modules
1.1.7 Electronic Warfare
Module
1.1.8 On Board
Application &
System SW
1.3 Mission Control /
Ground Station SW
1.3.1 Signal Processing
SW
1.3.2 Station Display
1.3.3 Operating System
1.3.4 ROE Simulations
1.3.5 Mission Commands
The WBS for a UAV
1.1.2 Airframe
TPMs start with the WBS Dictionary
Editor's Notes
The target weight for the UAV is 25kg. At contract award that target was the “awarded goal.”
At System Functional Review (SFR) the CATIA 3D model showed that the “all in” weight of the vehicle is around 28kgs. Systems engineering, sensors, and propulsion design efforts focused on scrubbing the vehicle to keep under the upper bound of allowable weight.
At Systems Requirements Review (SRR) they had had not succeeded with the stringent requirements for the sensor array, loiter time for fuel on board and the newly discovered reduction of thrust in the ducted fan assembly requiring addition vanes. By PDR those issues had been addressed, and alternative materials had been identified from the build from “same as” model used for SFR and SRR.
With the new materials, reduction of sensor standby power, and other “lightening” effort, the vehicle came in under the max weight allowable for the thrust available. Changes to the control system to account for addition momentum induced guidance loops allowed this new weight to meet the flight performance KPPs.