Creating	Credible	Plans,
Integrated	Reporting,
and	Control	Systems
Part	1
Glen	B.	Alleman
Niwot	Ridge	L.L.C.
glen.alleman@niwotridge.com
+1	303	241	9633
IPM	Workshop	 2015
Thomas	Coonce
Cost	Analysis	and	Research	Division
Institute	for	Defense	Analyses
tcoonce@ida.org
+1	703	575	6634
Rick	Price
Project	Management	 &	Planning	
Operations	Principal
Lockheed	 Martin	Space	Systems	Company
Rick.a.price@lmco.com
+1	303	971	1826
2
Learning Objectives
• TLO #1 – the student will understand how to apply the 5 core
processes needed to produce a credible Performance
Measurement Baseline.
• TLO #2 – the student will understand how to objectively
measure progress and use the performance information to
control the program.
GA
3
Building a credible PMB for Wilbur and
Orville’s flying machine.
• Our talk today uses the Wright Brothers
Flyer as an example for developing a
Credible PMB
• The Urban Legend of Orville and
Wilbur's efforts to invent manned flight
has been revealed in McCullough’s
book.
• Their work has all the attributes of the
Five Core Processes.
GA
4
There is a Method to our Madness
• If you’re over 60 you’ll recognize
this chart template for HIPO, used
to describe Inputs, Processes,
Outputs we’re using to create a
credible Performance
Measurement Baseline, measure
progress, and control the program
• These are the minimum HIPO
elements, there can be and should
be more.
GA
5
The data and processes needed to create a
Credible Performance Measurement Baseline
GA
6
A Readable Version
The Five processes needed to build a credible PMB
❶ Define Done in Units of Measure Meaningful to the
Decision Makers.
❷ Develop the Plan and Schedule to reach Done as planned.
❸ Define the Resources Needed to Reach Done as Planned.
❹ Adjust Plan for Reducible and Irreducible Risks.
❺ Finalize the Risk Adjusted Measures of Progress Toward
Done.
GA
7
Inputs and Outputs for Building the Credible
Performance Measurement Baseline
• Customer Capabilities
• Time phased budget
• Desired completion date
• Available technologies
• Time phase resources
• Reference Class Data
• Programmatic and technical risks with
probabilities and impacts
• Historic uncertainties
• Measures of Effectiveness
• Measures of Performance
• Work Breakdown Structure
• Integrated Master Plan
• Technical Performance Management
Plan
• Initial Integrated Master Schedule
• Time Phased Staffing Plan
• Schedule Reserve
• Management Reserve
• Risk Register and Mitigation Plans
• Adjusted Integrated Master Schedule
• Risk Management Plan
Inputs Outputs
GA
8
❶ DEFINE DONE
Define Done in units of measure meaningful to the decision makers
for each deliverable in the WBS.
Measures of Effectiveness (MOE), Measures of Performance
(MOP), Technical Performance Measures (TPM), and …illities are
defined before the work starts, for each needed capability that
accomplishes the mission or fulfills the Business Case.
TC
9
Systems Engineering Concepts Used By
The Wright Brothers [1]
SE Concept Wright BrothersActivities
Operational
Concept
Vision: clear concept of the system they were developing
MissionRequirements: powered flight for substantial timeand distance
Scenarios: recognitionof a key scenarioinvolvinglateral control during flight
Define the System
Boundary
Understoodthe need for the pilot to be part of the system
Objectives Definitionof distanceand time objectives
Derived
Requirements
Horsepower and weight of the engine
Thrust of the propellers
Requirements
Management
Managed weight requirement of the engine carefully
Functional
Analysis
Understood the functions of the airplane, which drove their design process
Further understood the functions of the propulsionsubsystem
Physical
Architecture&
Interfaces
Matched the physical architecture tothe functions of the airplane
Interfaces between the pilot and the control surfaces
Interfacebetween the engine and propellers TC
10
Systems Engineering Concepts Used By
The Wright Brothers [1]
SE Concept Wright BrothersActivities
Prototypes and
Testing
Built a successfullymorecomplex set of gliders before building the airplane
First airplane was successful,due to the series of gliders and kites
Employed sophisticatedtestingwitheach prototype, atypical of others at the
time
Trade-Off
Decisions
Design favoredcontrol of the airplane over stability
Test System
Extensiveevenduring glider tests
Allowed problem identificationand solvingduring all test phases
Created wind tunnel to address discrepancy between their test data and the
literature
Included the careful selectionof the test site
Included in the first airplane
Verification
Best illustratedby testingof the engine to determine whether it met its
requirements
Validation The final test that proved flight was possible
Team Activities Two brothers formed a design-build team TC
11
Measure of Effectiveness (MoE)
Measures of Effectiveness …
• Are stated in units meaningful to the buyer,
• Focus on capabilities independent of any technical
implementation,
• Are connected to the mission success.
Operational measures of success that are closely related to
the achievements of the mission or operational objectives
evaluated in the operational environment, under a specific set
of conditions.
MoE’s Belong to the End User
“Technical	Measurement,”	INCOSE–TP–2003–020–01
11 TC
12
Measure of Performance (MoP)
Measures of Performance are …
• Attributes that assure the system has the capability and capacity to
perform,
• Assessment of the system to assure it meets design requirements to
satisfy the MoE.
Measures that characterize physical or functional attributes
relating to the system operation, measured or estimated
under specific conditions.
“Technical	Measurement,”	INCOSE–TP–2003–020–01
MoP’s belong to the Program – Developed by the Systems
Engineer, Measured By CAMs, and Analyzed by PP&C
12 TC
13
Technical Performance Measures (TPM)
Technical Performance Measures …
• Assess design progress,
• Define compliance to performance requirements,
• Identify technical risk,
• Are limited to critical thresholds,
• Include projected performance.
Attributes that determine how well a system or system
element is satisfying or expected to satisfy a technical
requirement or goal
“Technical	Measurement,”	INCOSE–TP–2003–020–01
13 TC
14
What Done Looks Like for the Wright Brothers’
Military Flyer
• Propose $25,000 to build and
deliver a flyer by 1st week in
August 1908
• Conduct Acceptance testing for
30 days thereafter
Contract
Specification
Value/Goal
Passenger # Two persons
Payload 350 lbs
Range 125 miles
Air Speed 40 mph
Duration Aloft One hour
Maneuverability
Be steered in all directions
w/o difficulty and in perfect
control at all times
Ease of Use
An intelligentman become
proficientwithin a
reasonable length oftime
Logistics
Assembled and
disassembled within one
hour and packed into an
Army wagon
TC
15
❷ DEVELOP A PLAN FOR
TECHNICAL SUCCESS
The Technical Plan shows the sequence of work needed to
produce products and services that move the program toward
Done.
The Technical Plan can take many forms. From a formal Integrated
Master Plan / Integrated Master Schedule to a Kanban work plan or
Scrum Release/Sprint/Task Plan in Agile Software Development
TC
16
Develop the Plan and Schedule
• Define the technical approach to building the product that will
allow it to meet the performance and effectiveness goals
– Identify all the variables that will impact the solution to the
technical and operational requirements
– Orville and Wilbur used the spiral development approach for
the Wright Flyer
• Sequence the work to assure the increasing maturity of the
deliverables supports the plan to reach Done.
TC
17
The Wright Brothers’ Technical Plan
• Modify aircraft to reduce weight from 845 to 800 pounds
• Modify aircraft and controls for two persons
• Increase HP from 21 to 31 horsepower
• Apply a Modify – Test – Modify approach
• Spend lots of time practicing (and not kill yourself)
• Dry run assembly and disassembly procedures
TC
18
❸ DEFINE THE
PROGRAMMATIC PLAN
The Programmatic Plan describes all the activities that enable the
Technical Plan to function properly. What work do we need to do in
what order and what’s the cost of that planned work.
What facilities, resources, equipment, and materials are needed to
reach done as planned.
RP
19
Define the Programmatic Plan
• Assure the programmatic plan aligns with the technical plan
• Qualified Staffing Plan to produce the needed outcomes at the
needed time
• Facilities and equipment needed to produce the products
• Material availability and readiness Plan
• Phased funding available to sustain the planned progress of the
Technical Plan
RP
20
Wright Brothers’ Flyer Resources Needed
• Facilities and equipment for modifications
– Fabrication hangar @ Kitty Hawk
– Launching track
– Catapult
• Materials (propellers, wood, fabric, sewing supplies, chain, etc.)
• Test areas
– Kitty Hawk at Kill Devil Hills
• Requisite labor skills
– Orville & Wilbur Wright
– Charlie Furnas (Dayton mechanic)
– Charlie Taylor (engine design/builder)
– Carpenters at Kitty Hawk (to rebuild the hangar)
• Money to support the development/build effort
• GFE
– Site availability at Ft Myer, VA
– Hangar shed at Ft Myer, VA
RP
21
Initial Wright Brothers’ Summary Level IMS
RP
22
❹ ADJUST FOR REDUCIBLE
AND IRREDUCIBLE RISK
Any impediment reduces the probability of success. Each
impediment must be identified, a plan to address the impediment
developed, each handling strategy applied, monitored, and
measured to assure there is an increasing probability of success.
GA
2323
Uncertainties are
things we can not
be certain about.
Uncertainty is
created by our
incomplete
knowledge; not
by our ignorance
GA
24
2 Types of Uncertainty – Both Create
Programmatic and Technical Risk
Uncertainty
Irreducible
(Aleatory)
Reducible
(Epistemic)
Natural	Variability
Ambiguity
Ontological	
Uncertainty
Probabilistic	Events
Probabilistic	
Impacts
Periods	of	Exposure
GA
25
Assemble a credible WBS and Integrated Master Plan / Integrated
Master Schedule (IMP/IMS)
–WBS Dictionary says what will be built
–IMP/IMS says how, where, when, and what will be built and the
processes to build it.
Identify Reducible – Event Based – uncertainties and the
resulting risks from WBS Dictionary and IMP Narratives in the
Risk Register
Put these risks in the Risk Register, with probability and impacts
Develop risk reduction plans and place them in the IMS using
CBB funding.
How to Build a Risk Adjusted IMS in 8
Steps
0
1
2
3
RP
26
Assess the Irreducible – naturally occurring –
uncertainties and the resulting schedule and cost
risks in the WBS and IMS.
Use Monte Carlo Simulation to determine schedule
margin and budget impacts from work activity
durations to handle these irreducible uncertainties.
Formulate schedule and cost margin to protect key
item deliverables in the IMS.
How to Build a Risk Adjusted IMS in 8
Steps
4
5
6
RP
27
Determine cost and schedule impacts of unmitigated
risks in the risk register with their exposure and impact
and place them in Management Reserve and Schedule
Margin
Assemble mitigated Irreducible (Aleatory) and
Reducible (Epistemic) risks with the unmitigated event–
based risks into the Total Allocated Budget
– Risk handling plans included in baseline for reducible
and irreducible in the PMB.
– Risk Handling in Management Reserve.
Building a Risk Adjusted IMS in 8 Steps
(Concluded)
8
7
Adjust	the	IMS		with	rearranging	parallel	or	serial	work,		changing	scope,	or	add	
resources.	
RP
28
Wright Brothers’ Risk Register and Risk
Strategy
1. Control. Buy $1.6K of key spare parts and send to Kitty Hawk
2. Accept. Add schedule margin in schedule
3. Accept. Add schedule margin in schedule
4. Accept. Add schedule margin in schedule
TC
29
Wright Brothers’ 1908 Flyer Management
Reserve Calculation
Element Cost CL Comments
P80 Cost $19,696 80%
This is the cost at the
80% CL assuming
only non-reducible
risks in the RR
Deterministic
BACf
$16,355 < 1%
Final BAC based on
the resource-loaded
schedule
MR (P80
Cost - BACf)
3,341 Calculated MR
MR as a
percent of
BACf
20.4%
Caculated MR % of
BACf
TC
30
Wright Brothers’ 1908 Flyer Schedule
Margin Calculation
Element
Finish
Date CL
Months
Work
Days
IMS
Deterministic Pf
Duration and
Date:= 7.25 152 9/15/1908 5%
Need Duration
and Date:= 7.75 163 9/30/1908 80%
Schedule Margin:= 1 11
Duration
TC
31
Schedule Margin in Final WB’s Flyer IMS
TC
32
❺ DEVELOP OBJECTIVE
MEASURES OF PROGRESS
For each deliverable develop an objective measure of progress to
plan using physical percent complete supported by Quantifiable
Backup Data (Objective Criteria)
Use these measures to assess progress and identify variances
requiring corrective actions to Keep the Program Green.
GA
33
Objective Measures of Progress to Plan
• Measures of Effectiveness (MOE)
• Measures of Performance (MOP)
• Technical Performance Measures (TPM)
– Weight TPM Plan
– Risk Burn Down Plan
– Schedule Margin Burn Down Plan
• All the other …ilities
– Availability (assemble within one hour)
– Usability (simple in construction and operation)
GA
34
Weight TPM for Wright Brothers 1908 Flyer
770
780
790
800
810
820
830
840
850
860
870
15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08
Pounds
15-
Feb-08
28-
Feb-08
15-
Mar-08
31-
Mar-08
15-
Apr-08
30-
Apr-08
15-
May-
08
31-
May-
08
15-Jun-
08
30-Jun-
08
15-Jul-
08
Upper	Limit 860 855 845 835 825 815 800 800 800 800 800
Planned 845 840 830 820 810 805 800 800 800 800 800
Target 800 800 800 800 800 800 800 800 800 800 800
GA
35
What’s Next
• TLO #1 – the student will learn how to produce an integrated report
of the Technicalprogress consistentwith the Programmatic
progress, detect variances,and suggestcorrective actions.
• TLO #2 – the student will understand how to objectively measure
progress and use the performance to manage the program.
• Earned Value Managementis part of a Technicaland
Programmatic control system
– Integrated risk, performance, effectiveness,technicaland other
measures provide leading indicators of program performance.
36
References
1. “The Concepts of Systems Engineering as Practiced by the
Wright Brothers,” Dennis Buede, Stevens Institute of
Technology School of Engineering, Hoboken, NJ 07030
Executing	the	Credible	Plan,
Using	an	Integrated	Reporting
and	Control	System
Part	2
Glen	B.	Alleman
Niwot	Ridge	L.L.C.
glen.alleman@niwotridge.com
+1	303	241	9633
IPM	Workshop	 2015
Thomas	Coonce
Cost	Analysis	and	Research	Division
Institute	for	Defense	Analyses
tcoonce@ida.org
+1	703	575	6634
Rick	Price
Project	Management	 &	Planning	
Operations	Principal
Lockheed	 Martin	Space	Systems	Company
Rick.a.proce@lmco.com
+1	303	971	1826
38
Learning Objectives
• TLO #1 – the student will learn how to produce an integrated
report of the Technical progress consistent with the
Programmatic progress, detect variances, and suggest
corrective actions.
• TLO #2 – the student will understand how to objectively
measure progress and use the performance to manage the
program.
• Earned Value Management is part of a Technical and
Programmatic control system
– Integrated risk, performance, effectiveness, technical and
other measures provide leading indicators of program
performance.
GA
39
In Part 1, we talked about …
• Addressed the five core processes needed to produce a
credible Performance Measurement Baseline (PMB).
• Showed how to use objective measures of progress and how to
inform program performance with these measures
• In Part 2, you’ll see how to execute the program with this
information.
GA
40
Executing the Performance Measurement
Baseline
GA
41
A Readable Version
The Four processes needed to execute the credible PMB
❶ Compare Technical and Programmatic Actuals to Plan
❷ Compare BCWP and Cost and Schedule Performance
Data with Management Reserve (MR) and Schedule
Margin (SM) balances
❸ Revise Integrated Master Schedule (IMS) and Risk
Register
❹ Run or Rerun Monte Carlo Simulation
GA
42
Inputs and Outputs for Building the Credible
Performance Measurement Baseline
• Customer Capabilities
• Time phased budget
• Desired completion date
• Available technologies
• Time phase resources
• Reference Class Data
• Programmatic and technical risks with
probabilities and impacts
• Historic uncertainties
• Measures of Effectiveness
• Measures of Performance
• Work Breakdown Structure
• Integrated Master Plan
• Technical Performance Management
Plan
• Initial Integrated Master Schedule
• Time Phased Staffing Plan
• Schedule Reserve
• Management Reserve
• Risk Register and Mitigation Plans
• Adjusted Integrated Master Schedule
• Risk Management Plan
Inputs Outputs
GA
43
❶ COMPARE PLANNED TO
ACTUAL PERFORMANCE
Compare technical and programmatic performance to the planned
performance. Identify variances needed to make decisions about
corrective actions to Keep the Program Green.
TC
44
Compare Planned to Actual Performance
• Physical Percent Complete
– Planned
– Actual
• Do this at the Work Package level
– Quantifiable Backup Data
– Avoid averaging out the variances at the Control Account
level
Objectively assess accomplishments at the Work Performance Level
EIA-748-C, page 1
TC
45
Wright Brothers’ Technical, Cost and
Schedule Progress
0.80
0.85
0.90
0.95
1.00
1.05
1.10
770
780
790
800
810
820
830
840
850
860
870
15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08
CPI/SPI
Pounds
WBs'	1908	Flyer	Integrated	Progress
(2/28/1908)
Upper	Limit Planned Target Achieved SPI CPI
Compare side-by-side the Technical Performance (Weight) with
the Programmatic performance CPI and SPI
TC
46
❷ COMPARE MR AND SM
BALANCES
Using the performance data and the variances, the computed
BCWP from measures of Physical Percent Complete.
Assess planned Management Reserve and Schedule Margin with
the current MR and SM and inform EAC and ETC from past
performance.
TC
47
Compare BCWP with MR and SM
• With actual BCWP assess impact on Estimate at Completion
and Estimate To Complete.
• Assess planned Management Reserve against actual
Management Reserve
• Assess planned Schedule Margin against actual Schedule
Margin
• Assess planned risk burn down against actual Risk Burn Down
TC
48
Wright Brothers’ MR Planned Use vs Actuals
$0
$500
$1,000
$1,500
$2,000
$2,500
$3,000
$3,500
$4,000
WB 1908 Flyer Planned vs Actual MR Use
(15 May 1908)
MR Planned Balance MR Actual Balance
TC
49
Wright Brothers’ Schedule Margin Burn
Planned Use vs Actuals
In May, when the schedule margin went below plan, immediate
corrective action is needed to get back on Schedule Margin Plan
0
2
4
6
8
10
12
15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08 15-Aug-08 15-Sep-08
Days
WB 1908 Flyer Planned vs Actual Schedule Margin Use
(15 May 1908)
Schedule Margin Plan Balance Schedule Margin Actual Balance
This dip
characterized as
negative slack
RP
50
❸ REVISE IMS AND RISK
REGISTER
With this data, update the Integrated Master Schedule and Risk
levels in the Risk Register for future periods of performance.
51
Revise IMS and Risk Register
• Every reporting period update the Risk Register with actual
assessment of
– Reduced risks
– Actual margin needed for unmitigated and irreducible risks
• Adjust Risk Register contents for current assessments
– New risks
– Retired risks
– Changes in risk attributes
• Update ETC, EAC, and ECD with the statused IMS and Risk
Register
TC
52
❹ RUN AND RERUN THE
SCHEDULE RISK ANALYSIS
Schedule Risk Analysis (per DI-MGMT-81861) at periodic intervals
to confirm the Estimate to Complete and the Estimate At
Completion are still within acceptable values to ensure the
probability of program success.
TC
53
Run and Rerun Schedule Risk Analysis
• With updated Risk Register – rerun the Schedule Risk Analysis
• Reassess EAC, ECD, and ETC
• Determine sensitivity indices (which elements are most likely to
cause the forward plan to go awry)
• If needed
– Replan IMS, possibly with more schedule margin
– Add resources
– Adjust technical solution
TC
54
Summary
• Cost and Schedule performance informed by Technical
Performance
– At IBR TechnicalPerformance plan defines the work activities
in the IMS
• Build a risk adjusted PMB
– Reducible and irreducible risks in the risk register, traceable to
the IMS.
• Budget active mitigation activities
• Protect unmitigated risks with ManagementReserve and
Schedule Margin
• Make sure programmatic data is aligned with Technical
Performance Data
• Perform these activities continually during execution to Keep the
Program Green
TC
55
Questions???
55

Credible Plans, Integrated Reporting, and Control Systems

  • 1.
  • 2.
    2 Learning Objectives • TLO#1 – the student will understand how to apply the 5 core processes needed to produce a credible Performance Measurement Baseline. • TLO #2 – the student will understand how to objectively measure progress and use the performance information to control the program. GA
  • 3.
    3 Building a crediblePMB for Wilbur and Orville’s flying machine. • Our talk today uses the Wright Brothers Flyer as an example for developing a Credible PMB • The Urban Legend of Orville and Wilbur's efforts to invent manned flight has been revealed in McCullough’s book. • Their work has all the attributes of the Five Core Processes. GA
  • 4.
    4 There is aMethod to our Madness • If you’re over 60 you’ll recognize this chart template for HIPO, used to describe Inputs, Processes, Outputs we’re using to create a credible Performance Measurement Baseline, measure progress, and control the program • These are the minimum HIPO elements, there can be and should be more. GA
  • 5.
    5 The data andprocesses needed to create a Credible Performance Measurement Baseline GA
  • 6.
    6 A Readable Version TheFive processes needed to build a credible PMB ❶ Define Done in Units of Measure Meaningful to the Decision Makers. ❷ Develop the Plan and Schedule to reach Done as planned. ❸ Define the Resources Needed to Reach Done as Planned. ❹ Adjust Plan for Reducible and Irreducible Risks. ❺ Finalize the Risk Adjusted Measures of Progress Toward Done. GA
  • 7.
    7 Inputs and Outputsfor Building the Credible Performance Measurement Baseline • Customer Capabilities • Time phased budget • Desired completion date • Available technologies • Time phase resources • Reference Class Data • Programmatic and technical risks with probabilities and impacts • Historic uncertainties • Measures of Effectiveness • Measures of Performance • Work Breakdown Structure • Integrated Master Plan • Technical Performance Management Plan • Initial Integrated Master Schedule • Time Phased Staffing Plan • Schedule Reserve • Management Reserve • Risk Register and Mitigation Plans • Adjusted Integrated Master Schedule • Risk Management Plan Inputs Outputs GA
  • 8.
    8 ❶ DEFINE DONE DefineDone in units of measure meaningful to the decision makers for each deliverable in the WBS. Measures of Effectiveness (MOE), Measures of Performance (MOP), Technical Performance Measures (TPM), and …illities are defined before the work starts, for each needed capability that accomplishes the mission or fulfills the Business Case. TC
  • 9.
    9 Systems Engineering ConceptsUsed By The Wright Brothers [1] SE Concept Wright BrothersActivities Operational Concept Vision: clear concept of the system they were developing MissionRequirements: powered flight for substantial timeand distance Scenarios: recognitionof a key scenarioinvolvinglateral control during flight Define the System Boundary Understoodthe need for the pilot to be part of the system Objectives Definitionof distanceand time objectives Derived Requirements Horsepower and weight of the engine Thrust of the propellers Requirements Management Managed weight requirement of the engine carefully Functional Analysis Understood the functions of the airplane, which drove their design process Further understood the functions of the propulsionsubsystem Physical Architecture& Interfaces Matched the physical architecture tothe functions of the airplane Interfaces between the pilot and the control surfaces Interfacebetween the engine and propellers TC
  • 10.
    10 Systems Engineering ConceptsUsed By The Wright Brothers [1] SE Concept Wright BrothersActivities Prototypes and Testing Built a successfullymorecomplex set of gliders before building the airplane First airplane was successful,due to the series of gliders and kites Employed sophisticatedtestingwitheach prototype, atypical of others at the time Trade-Off Decisions Design favoredcontrol of the airplane over stability Test System Extensiveevenduring glider tests Allowed problem identificationand solvingduring all test phases Created wind tunnel to address discrepancy between their test data and the literature Included the careful selectionof the test site Included in the first airplane Verification Best illustratedby testingof the engine to determine whether it met its requirements Validation The final test that proved flight was possible Team Activities Two brothers formed a design-build team TC
  • 11.
    11 Measure of Effectiveness(MoE) Measures of Effectiveness … • Are stated in units meaningful to the buyer, • Focus on capabilities independent of any technical implementation, • Are connected to the mission success. Operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions. MoE’s Belong to the End User “Technical Measurement,” INCOSE–TP–2003–020–01 11 TC
  • 12.
    12 Measure of Performance(MoP) Measures of Performance are … • Attributes that assure the system has the capability and capacity to perform, • Assessment of the system to assure it meets design requirements to satisfy the MoE. Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. “Technical Measurement,” INCOSE–TP–2003–020–01 MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C 12 TC
  • 13.
    13 Technical Performance Measures(TPM) Technical Performance Measures … • Assess design progress, • Define compliance to performance requirements, • Identify technical risk, • Are limited to critical thresholds, • Include projected performance. Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal “Technical Measurement,” INCOSE–TP–2003–020–01 13 TC
  • 14.
    14 What Done LooksLike for the Wright Brothers’ Military Flyer • Propose $25,000 to build and deliver a flyer by 1st week in August 1908 • Conduct Acceptance testing for 30 days thereafter Contract Specification Value/Goal Passenger # Two persons Payload 350 lbs Range 125 miles Air Speed 40 mph Duration Aloft One hour Maneuverability Be steered in all directions w/o difficulty and in perfect control at all times Ease of Use An intelligentman become proficientwithin a reasonable length oftime Logistics Assembled and disassembled within one hour and packed into an Army wagon TC
  • 15.
    15 ❷ DEVELOP APLAN FOR TECHNICAL SUCCESS The Technical Plan shows the sequence of work needed to produce products and services that move the program toward Done. The Technical Plan can take many forms. From a formal Integrated Master Plan / Integrated Master Schedule to a Kanban work plan or Scrum Release/Sprint/Task Plan in Agile Software Development TC
  • 16.
    16 Develop the Planand Schedule • Define the technical approach to building the product that will allow it to meet the performance and effectiveness goals – Identify all the variables that will impact the solution to the technical and operational requirements – Orville and Wilbur used the spiral development approach for the Wright Flyer • Sequence the work to assure the increasing maturity of the deliverables supports the plan to reach Done. TC
  • 17.
    17 The Wright Brothers’Technical Plan • Modify aircraft to reduce weight from 845 to 800 pounds • Modify aircraft and controls for two persons • Increase HP from 21 to 31 horsepower • Apply a Modify – Test – Modify approach • Spend lots of time practicing (and not kill yourself) • Dry run assembly and disassembly procedures TC
  • 18.
    18 ❸ DEFINE THE PROGRAMMATICPLAN The Programmatic Plan describes all the activities that enable the Technical Plan to function properly. What work do we need to do in what order and what’s the cost of that planned work. What facilities, resources, equipment, and materials are needed to reach done as planned. RP
  • 19.
    19 Define the ProgrammaticPlan • Assure the programmatic plan aligns with the technical plan • Qualified Staffing Plan to produce the needed outcomes at the needed time • Facilities and equipment needed to produce the products • Material availability and readiness Plan • Phased funding available to sustain the planned progress of the Technical Plan RP
  • 20.
    20 Wright Brothers’ FlyerResources Needed • Facilities and equipment for modifications – Fabrication hangar @ Kitty Hawk – Launching track – Catapult • Materials (propellers, wood, fabric, sewing supplies, chain, etc.) • Test areas – Kitty Hawk at Kill Devil Hills • Requisite labor skills – Orville & Wilbur Wright – Charlie Furnas (Dayton mechanic) – Charlie Taylor (engine design/builder) – Carpenters at Kitty Hawk (to rebuild the hangar) • Money to support the development/build effort • GFE – Site availability at Ft Myer, VA – Hangar shed at Ft Myer, VA RP
  • 21.
    21 Initial Wright Brothers’Summary Level IMS RP
  • 22.
    22 ❹ ADJUST FORREDUCIBLE AND IRREDUCIBLE RISK Any impediment reduces the probability of success. Each impediment must be identified, a plan to address the impediment developed, each handling strategy applied, monitored, and measured to assure there is an increasing probability of success. GA
  • 23.
    2323 Uncertainties are things wecan not be certain about. Uncertainty is created by our incomplete knowledge; not by our ignorance GA
  • 24.
    24 2 Types ofUncertainty – Both Create Programmatic and Technical Risk Uncertainty Irreducible (Aleatory) Reducible (Epistemic) Natural Variability Ambiguity Ontological Uncertainty Probabilistic Events Probabilistic Impacts Periods of Exposure GA
  • 25.
    25 Assemble a credibleWBS and Integrated Master Plan / Integrated Master Schedule (IMP/IMS) –WBS Dictionary says what will be built –IMP/IMS says how, where, when, and what will be built and the processes to build it. Identify Reducible – Event Based – uncertainties and the resulting risks from WBS Dictionary and IMP Narratives in the Risk Register Put these risks in the Risk Register, with probability and impacts Develop risk reduction plans and place them in the IMS using CBB funding. How to Build a Risk Adjusted IMS in 8 Steps 0 1 2 3 RP
  • 26.
    26 Assess the Irreducible– naturally occurring – uncertainties and the resulting schedule and cost risks in the WBS and IMS. Use Monte Carlo Simulation to determine schedule margin and budget impacts from work activity durations to handle these irreducible uncertainties. Formulate schedule and cost margin to protect key item deliverables in the IMS. How to Build a Risk Adjusted IMS in 8 Steps 4 5 6 RP
  • 27.
    27 Determine cost andschedule impacts of unmitigated risks in the risk register with their exposure and impact and place them in Management Reserve and Schedule Margin Assemble mitigated Irreducible (Aleatory) and Reducible (Epistemic) risks with the unmitigated event– based risks into the Total Allocated Budget – Risk handling plans included in baseline for reducible and irreducible in the PMB. – Risk Handling in Management Reserve. Building a Risk Adjusted IMS in 8 Steps (Concluded) 8 7 Adjust the IMS with rearranging parallel or serial work, changing scope, or add resources. RP
  • 28.
    28 Wright Brothers’ RiskRegister and Risk Strategy 1. Control. Buy $1.6K of key spare parts and send to Kitty Hawk 2. Accept. Add schedule margin in schedule 3. Accept. Add schedule margin in schedule 4. Accept. Add schedule margin in schedule TC
  • 29.
    29 Wright Brothers’ 1908Flyer Management Reserve Calculation Element Cost CL Comments P80 Cost $19,696 80% This is the cost at the 80% CL assuming only non-reducible risks in the RR Deterministic BACf $16,355 < 1% Final BAC based on the resource-loaded schedule MR (P80 Cost - BACf) 3,341 Calculated MR MR as a percent of BACf 20.4% Caculated MR % of BACf TC
  • 30.
    30 Wright Brothers’ 1908Flyer Schedule Margin Calculation Element Finish Date CL Months Work Days IMS Deterministic Pf Duration and Date:= 7.25 152 9/15/1908 5% Need Duration and Date:= 7.75 163 9/30/1908 80% Schedule Margin:= 1 11 Duration TC
  • 31.
    31 Schedule Margin inFinal WB’s Flyer IMS TC
  • 32.
    32 ❺ DEVELOP OBJECTIVE MEASURESOF PROGRESS For each deliverable develop an objective measure of progress to plan using physical percent complete supported by Quantifiable Backup Data (Objective Criteria) Use these measures to assess progress and identify variances requiring corrective actions to Keep the Program Green. GA
  • 33.
    33 Objective Measures ofProgress to Plan • Measures of Effectiveness (MOE) • Measures of Performance (MOP) • Technical Performance Measures (TPM) – Weight TPM Plan – Risk Burn Down Plan – Schedule Margin Burn Down Plan • All the other …ilities – Availability (assemble within one hour) – Usability (simple in construction and operation) GA
  • 34.
    34 Weight TPM forWright Brothers 1908 Flyer 770 780 790 800 810 820 830 840 850 860 870 15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08 Pounds 15- Feb-08 28- Feb-08 15- Mar-08 31- Mar-08 15- Apr-08 30- Apr-08 15- May- 08 31- May- 08 15-Jun- 08 30-Jun- 08 15-Jul- 08 Upper Limit 860 855 845 835 825 815 800 800 800 800 800 Planned 845 840 830 820 810 805 800 800 800 800 800 Target 800 800 800 800 800 800 800 800 800 800 800 GA
  • 35.
    35 What’s Next • TLO#1 – the student will learn how to produce an integrated report of the Technicalprogress consistentwith the Programmatic progress, detect variances,and suggestcorrective actions. • TLO #2 – the student will understand how to objectively measure progress and use the performance to manage the program. • Earned Value Managementis part of a Technicaland Programmatic control system – Integrated risk, performance, effectiveness,technicaland other measures provide leading indicators of program performance.
  • 36.
    36 References 1. “The Conceptsof Systems Engineering as Practiced by the Wright Brothers,” Dennis Buede, Stevens Institute of Technology School of Engineering, Hoboken, NJ 07030
  • 37.
  • 38.
    38 Learning Objectives • TLO#1 – the student will learn how to produce an integrated report of the Technical progress consistent with the Programmatic progress, detect variances, and suggest corrective actions. • TLO #2 – the student will understand how to objectively measure progress and use the performance to manage the program. • Earned Value Management is part of a Technical and Programmatic control system – Integrated risk, performance, effectiveness, technical and other measures provide leading indicators of program performance. GA
  • 39.
    39 In Part 1,we talked about … • Addressed the five core processes needed to produce a credible Performance Measurement Baseline (PMB). • Showed how to use objective measures of progress and how to inform program performance with these measures • In Part 2, you’ll see how to execute the program with this information. GA
  • 40.
    40 Executing the PerformanceMeasurement Baseline GA
  • 41.
    41 A Readable Version TheFour processes needed to execute the credible PMB ❶ Compare Technical and Programmatic Actuals to Plan ❷ Compare BCWP and Cost and Schedule Performance Data with Management Reserve (MR) and Schedule Margin (SM) balances ❸ Revise Integrated Master Schedule (IMS) and Risk Register ❹ Run or Rerun Monte Carlo Simulation GA
  • 42.
    42 Inputs and Outputsfor Building the Credible Performance Measurement Baseline • Customer Capabilities • Time phased budget • Desired completion date • Available technologies • Time phase resources • Reference Class Data • Programmatic and technical risks with probabilities and impacts • Historic uncertainties • Measures of Effectiveness • Measures of Performance • Work Breakdown Structure • Integrated Master Plan • Technical Performance Management Plan • Initial Integrated Master Schedule • Time Phased Staffing Plan • Schedule Reserve • Management Reserve • Risk Register and Mitigation Plans • Adjusted Integrated Master Schedule • Risk Management Plan Inputs Outputs GA
  • 43.
    43 ❶ COMPARE PLANNEDTO ACTUAL PERFORMANCE Compare technical and programmatic performance to the planned performance. Identify variances needed to make decisions about corrective actions to Keep the Program Green. TC
  • 44.
    44 Compare Planned toActual Performance • Physical Percent Complete – Planned – Actual • Do this at the Work Package level – Quantifiable Backup Data – Avoid averaging out the variances at the Control Account level Objectively assess accomplishments at the Work Performance Level EIA-748-C, page 1 TC
  • 45.
    45 Wright Brothers’ Technical,Cost and Schedule Progress 0.80 0.85 0.90 0.95 1.00 1.05 1.10 770 780 790 800 810 820 830 840 850 860 870 15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08 CPI/SPI Pounds WBs' 1908 Flyer Integrated Progress (2/28/1908) Upper Limit Planned Target Achieved SPI CPI Compare side-by-side the Technical Performance (Weight) with the Programmatic performance CPI and SPI TC
  • 46.
    46 ❷ COMPARE MRAND SM BALANCES Using the performance data and the variances, the computed BCWP from measures of Physical Percent Complete. Assess planned Management Reserve and Schedule Margin with the current MR and SM and inform EAC and ETC from past performance. TC
  • 47.
    47 Compare BCWP withMR and SM • With actual BCWP assess impact on Estimate at Completion and Estimate To Complete. • Assess planned Management Reserve against actual Management Reserve • Assess planned Schedule Margin against actual Schedule Margin • Assess planned risk burn down against actual Risk Burn Down TC
  • 48.
    48 Wright Brothers’ MRPlanned Use vs Actuals $0 $500 $1,000 $1,500 $2,000 $2,500 $3,000 $3,500 $4,000 WB 1908 Flyer Planned vs Actual MR Use (15 May 1908) MR Planned Balance MR Actual Balance TC
  • 49.
    49 Wright Brothers’ ScheduleMargin Burn Planned Use vs Actuals In May, when the schedule margin went below plan, immediate corrective action is needed to get back on Schedule Margin Plan 0 2 4 6 8 10 12 15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08 15-Aug-08 15-Sep-08 Days WB 1908 Flyer Planned vs Actual Schedule Margin Use (15 May 1908) Schedule Margin Plan Balance Schedule Margin Actual Balance This dip characterized as negative slack RP
  • 50.
    50 ❸ REVISE IMSAND RISK REGISTER With this data, update the Integrated Master Schedule and Risk levels in the Risk Register for future periods of performance.
  • 51.
    51 Revise IMS andRisk Register • Every reporting period update the Risk Register with actual assessment of – Reduced risks – Actual margin needed for unmitigated and irreducible risks • Adjust Risk Register contents for current assessments – New risks – Retired risks – Changes in risk attributes • Update ETC, EAC, and ECD with the statused IMS and Risk Register TC
  • 52.
    52 ❹ RUN ANDRERUN THE SCHEDULE RISK ANALYSIS Schedule Risk Analysis (per DI-MGMT-81861) at periodic intervals to confirm the Estimate to Complete and the Estimate At Completion are still within acceptable values to ensure the probability of program success. TC
  • 53.
    53 Run and RerunSchedule Risk Analysis • With updated Risk Register – rerun the Schedule Risk Analysis • Reassess EAC, ECD, and ETC • Determine sensitivity indices (which elements are most likely to cause the forward plan to go awry) • If needed – Replan IMS, possibly with more schedule margin – Add resources – Adjust technical solution TC
  • 54.
    54 Summary • Cost andSchedule performance informed by Technical Performance – At IBR TechnicalPerformance plan defines the work activities in the IMS • Build a risk adjusted PMB – Reducible and irreducible risks in the risk register, traceable to the IMS. • Budget active mitigation activities • Protect unmitigated risks with ManagementReserve and Schedule Margin • Make sure programmatic data is aligned with Technical Performance Data • Perform these activities continually during execution to Keep the Program Green TC
  • 55.