The document discusses the Federal Aviation Administration's (FAA) successful implementation of Earned Value Management (EVM). It provides background on the history and drivers that led the FAA to adopt EVM. The FAA formed an EVM Council and revised policies to require EVM planning and implementation. Through training, standardized processes, and integrating EVM with existing project management practices, the FAA improved program performance monitoring. As a result of its EVM efforts, the FAA was removed from the GAO's High Risk list. The document shares lessons learned and benefits identified by the FAA in using EVM to manage projects.

1. Federal Aviation
FAA’s Successful Administration
Implementation of
Earned Value
Management
Presented to :NASA PM Challenge
By: Robert Rovinsky
Date: February, 2010
Improving Program Performance at FAA Federal Aviation 1 1
Administration
Used with permission

2. Contents of Talk
• Why Talk about the FAA at the PM Challenge
• Quick FAA Facts
• History/Drivers for Earned Value Management
• What our Project Managers say about EVM
• What Products and Processes we follow
• Agency Accomplishments and GAO Audits
• Lessons Learned and the Way Forward
• Summary and Q&A
Improving Program Performance at FAA FAA's Successful Implementation of Federal Aviation 2 2
Administration
EVM

3. Why Talk about the FAA to Present at the PM Challenge?
• “NASA is currently on the GAO High Risk List and FAA just
came off that list – why and how did FAA make this happen?”
• “NASA faces challenges for EVM buy-in both internally (in-
house projects) and externally (contractors) – is it a culture
thing?”
• “There is no Agency mandated EVM system – how did the
FAA create one”
• “How did FAA make the changes and get to its leadership
position – was it an internal thing, was it a cultural thing, was
it a mandate?
• “Why are some folks intimidated or afraid of EVM? What is in
it for them? What benefits did the FAA get? How much did it
cost?”
Improving Program Performance at FAA Federal Aviation 3 3
Administration

4. FAA – Quick Facts
• 12/17/1903 –1st sustained & controlled flight (Orville &
Wilbur Wright)
• 07/01/1934 –Bureau of Air Commerce formed
• 08/23/1958 –Federal Aviation Act of ’58 signed
• TODAY:
• Safest System ever!
• ~ $16B budget(FY10)
• 45,757 FAA Employees(FY10)
• 19,930 U.S. Airports
• 811,593,000,000 –Passenger Miles Flown(‘08)
• 590,349 Active Pilots
Improving Program Performance at FAA Federal Aviation 4 4
Administration

5. FAA – Mission/Vision/Values
Our Mission Our Values
To provide the safest, most
SAFETY IS OUR PASSION.
efficient aerospace system in We are the worlds leaders in aerospace safety.
the world.
QUALITY IS OUR TRADEMARK.
We serve our country, our stakeholders, our
customers, and each other.
INTREGITY IS OUR CHARACTER.
Our Vision We do the right thing, even when no one is
We continue to improve the looking.
safety and efficiency of flight. PEOPLE ARE OUR STRENGTH.
We are responsive to our We treat people as we want to be treated.
customers and are
accountable to the taxpayer
and the flying public.
Improving Program Performance at FAA Federal Aviation 5 5
Administration

6. Background – History and Drivers for EVM
FY 03-04 FAA Business Case Failure and Recovery: The original
driver for the EVM initiative
2005:
• We assessed our major investments (80% of FAA capital
expenditures) against the industry EVM Standard and
produced our “EVM Flag”.
• We committed to OMB and GAO that FAA would implement
full EVM on our major acquisitions by the end of 2007.
• We established FAA’s EVM policy.
Improving Program Performance at FAA Federal Aviation 6 6
Administration

7. Improving Program Performance at FAA Federal Aviation 7 7
Administration

8. History and Drivers for EVM
2006:
• We formed a multi-functional EVM Council
• We met with our major prime and support contractors on our plan and
solicited industry best practices
• We conducted a “cost and benefits of EVM” study
• We revised FAAs Acquisition Policy to:
– Establish Standard Program Milestones
– Require Implementation Strategy and Planning (ISP) requirement for
EVM plan
• Result: Significant traction toward FAA compliance targets
• We produced a Lay Guide to EVM
• All FAA Acquisition Executives received EVM Executive Briefing
• We developed standard program milestones for EVM
Improving Program Performance at FAA Federal Aviation 8 8
Administration

9. FAA’s EVM Council
• The EVM Council, composed of representatives across the agency and all
disciplines, leads the timely development, implementation and operation of FAA
EVMS procedures, policies, training and tools to fully realize efficiencies and
savings arising from the adoption and continuous improvement of a common
EVMS employing best practices.
• It provides support to Agency wide program management with respect to
implementation of EVM on FAA programs as required by Acquisition Policy and
the FAA EVM Guide.
• The council reviews and evaluates draft EVM policy documents, including
pertinent background information and special instructions or requirements to
ensure all internal stakeholders are involved in the development and
implementation of the FAA EVMS.
• The EVM Council oversees the implementation and application of common EVM
methods throughout FAA.
Improving Program Performance at FAA Federal Aviation 9 9
Administration

10. History and Drivers for EVM
2007:
• Achieved Green EVM assessments for the 2005 legacy
programs
• Established Portfolio/Program Performance Metrics (P3M)
• Established requirement for product oriented WBS
• Conducted EVM Data and Tools Studies
• Conducted ANSI Accounting Guidelines EVM assessment
• Conducted more EVM industry days
• FAA Decides to try to get off GAO High Risk List; GAO
sees our EVM effort as a step in right direction.
Improving Program Performance at FAA Federal Aviation 10 10
Administration

11. Improving Program Performance at FAA Federal Aviation 11 11
Administration

12. History and Drivers for EVM
2008:
• Completed EVM Guides for Project Managers and
Contracting Officers
– Integrated Baseline Reviews
– Surveillance Guides
– EVM System Acceptance Guide
• Revised EVM assessment criteria – consistent with NDIA
Intent Guide
• Formulated product oriented WBS using standard
program milestones
• EVM progress reported to GAO as part of High Risk
Initiative
Improving Program Performance at FAA Federal Aviation 12 12
Administration

13. History and Drivers for EVM
2009:
• FAA Gets off the High Risk List!
• Acquisition Executive Board formed to institutionalize
best practices including EVM
• Developed methodology/criteria for “fee for service”
projects
• FAA one of only two federal agencies to certify
contractors not already DCMA certified
• FAA EVM approach adopted by Dept of
Transportation
Improving Program Performance at FAA Federal Aviation 13 13
Administration

14. Today’s FAA EVM “Flag”
Today we are: 82% Green, 17% Yellow and 1% Red
Improving Program Performance at FAA Federal Aviation 14 14
Administration

15. Industry Days have been conducted since 2005
to obtain our suppliers EVM Best Practices
Description / Contractor A B C D E
EVM Surveillance
Joint Contractor / Government EVM surveillance
Contractor Internal EVM Surveillance
Use of EVM surveillance template or checklist
IBRs
Internal IBRs
Joint Contractor / Government Reviews
Perf Meas B/L implemented prior to Definitization
Training
Executive level EVM orientation briefings
EVM Training Program
Other
EVM for Service contracts
Program Start Up process P
EVM User Guide
Use EVM on LOE.
Use EVM on FFP contracts.
Program performance metrics plan
Eliminating cultural organizational EVM barriers P
Executive EVM metrics reporting
= Current Practice
Contracting for EVM “Tips & Traps”
Tailored EVM
P = Planned
Improving Program Performance at FAA Federal Aviation 15 15
Administration

16. We are integrating EVM with our PM processes
• Many of the Program Management Metrics reported to senior executive
board use EVM data.
• Standard Program Milestones improve completion criteria and map to FAA
standard WBS.
• EVM defined Program Baseline WBS used for Final Investment Decisions
– We are piloting product oriented WBS using standard program
milestones
– Our OMB Exhibit 300 format uses the same Program Baseline WBS
• We use EVM for DOT program performance “Bubble Chart” reports to OMB
• We use Primavera for the Corporate Workplan – it is the enterprise
schedule management solution for EVM
• Our Implementation Strategy & Planning document is used to document the
program EVM plan.
Improving Program Performance at FAA Federal Aviation 16 16
Administration

17. From our program managers we are
getting affirmation of EVM
• NEXCOM program manager, Dieter Thigpen,
“earned value data clearly identified schedule performance variances and got
management attention quicker as compared to the traditional approach not
using EVM”.
• Kevin Sharrett, ASR-11 Special Projects,
“Implementing EVM provides early visibility into future schedule risks that we
did not have before. As a result of the schedule risk assessment the ASR-11
EVM plan, we accelerated 5 system deliveries from 2007 to 2006 to mitigate
our schedule risk for the last site Operational Readiness Demonstration which
is baselined for 9/30/2009. We also had the schedule analysis to justify
construction and systems acquisition funds in the first quarters of 2007 under
the CR that had become critical for Last ORD if delayed. Implementing EVM
has provided the ASR-11 team members more insight and involvement into the
overall program cost and schedules.”
Improving Program Performance at FAA Federal Aviation 17 17
Administration

18. From our program managers we are
getting affirmation of EVM
• Dan Watts, the ERAM program manager,
“Earned value data is the only true indicator I have to obtain insight for cost and
schedule performance for a large scale complex program like ERAM”.
• Former ECG program manager, Bill Boyer,
“EVM is one of the more powerful tools in my PM toolbox. EVM allows me to
see the variations in program execution that enable me to take action to
resolve issues that are often beyond the capability of the prime contractor to
handle. With EVM, I am better able to manage planning and execution of
program funds, better able to track program progress against a benchmark,
and better able to work with the prime contractor and the support contractors to
apply resources when needed...and where needed.”
Improving Program Performance at FAA Federal Aviation 18 18
Administration

19. EVM training is customized
• We make extensive use of FAA standard program milestones
– Program Baseline WBS – Program phases / useful segments
– Schedule completion criteria – Control account definitions
• We include FAA enterprise tools (SPIRE, WorkLenz, and Primavera) in EVM
training material
• EVM training is consistent with FAA Accounting Order 2500.8B
– FAA funding types tightly linked to EVM
– Training includes accounting system (Delphi) examples
• Acquisition program baseline definitions used
– EVM tightly linked to program life cycle
– Knowledge-Based Product Development Phases used
• EVM is consistent with and supports our OMB Exhibit 300 reporting requirements
• FAA risk management process used
• FAA program examples & best practices are used
Improving Program Performance at FAA Federal Aviation 19 19
Administration

20. FAA EVM Program examples and best practices
are being used for EVM training
Drop 2 ESI Progress
ERAM Successes:
• Able to load all processor types and roles.
ELOM functionality working well
• Can display Target, track, & strobe data
on glass
• Can display an active flight plan in the
Aircraft List. Can amend it and remove
strip.
• SAR data already being used for
debugging defects. Initial set of EOPD
reports working.
• ELOM onboard cache working
• Can load/unload and configure a single
OpSim session
Still Working
• Multiple OpSims & Adaptation variations
• Continue testing Surveillance & Wx
threads
• Release Management Threads
• Continue with EOPD report testing
• Continue Dynamic Library testing
EVM reporting integrated with monthly PMRs
Improving Program Performance at FAA Federal Aviation 20 20
Administration

21. FAA EVM Program examples and best practices
are being used for EVM training
Overall RoadRoad ECG A GA Site
Overall ECG to to Site Last
Government Acceptance
100.0
90.0
80.0
70.0
Per Cent Complete Plan
60.0
% Complete
50.0
GA outlook
40.0 5/31/05 vs
baseline
30.0 I&I 07/06/05
com plete
Fitup and 4/26/05 vs.
20.0 SAT
Install baseline
05/23/05 outlook
10.0 Com plete
5/23/05 vs
04/05/05
baseline
0.0 06/28/05
2/4/2005
2/11/2005
2/18/2005
2/25/2005
3/4/2005
3/11/2005
3/18/2005
3/25/2005
4/1/2005
4/8/2005
4/15/2005
4/22/2005
4/29/2005
5/6/2005
5/13/2005
5/20/2005
5/27/2005
6/3/2005
6/10/2005
6/17/2005
6/24/2005
7/1/2005
7/8/2005
7/15/2005
7/22/2005
7/29/2005
8/5/2005
= Plan
= Actual
FFP EVM reporting using performance based payments
Improving Program Performance at FAA Federal Aviation 21 21
Administration

22. FAA P3M is included in EVM training
Program Summary Metrics
Cost Schedule Performance Resources External Program
Interest
G G Y Y Y G
Supporting Metrics
Portfolio Metric Cost Schedule Portfolio Metric Performance Portfolio Metric Resources External
Definition Measure of Measure of Measure of Interest
cost schedule Definition Measure of Definition
performance performance technical metrics current Assessment
of work of work and milestones of funding and of External
performed performed work performed staffing Reviews
EVM Metrics Level 1 AMS Milestones Resource Metrics
- Cost Performance Index G - Prime Contractor Y
(CPI) - Negative Deviations G
- Program Cost Reserve G Technical Metrics - Support Contractors G
- FAA G
- To-Complete Performance G - Requirements Stability G
Index (TCPI) - Funding G
- Cost Variance- G
- System Defects Y
At-completion (CVAC) % - Obligations G
- Test Results n/a
- Schedule Performance G Flight Plan Acquisition
Index (SPI) - Deployment G
Program Goals
- Program Schedule G - Value of remaining High G**
Reserve - Cost G
Risks
- Schedule Variance-At- G External Reviews
Completion (SVAC) % - Technical Variance At- G
Completion (TVAC) - IG Y
Level 1 AMS Milestones
- Negative Deviations G
-GAO G
Flight Plan Acquisition - OMB Y
Program Goals
- Schedule G
Improving Program Performance at FAA Federal Aviation 22 22
Administration

23. FAA Programs that implemented higher quality EVM
have lower PM costs as a percent of total program cost
Cost of PM Using EVM (FY06)
FAA Cost of Program Management Using EVM
34.5%
27.5%
PM % Total Cost
PM % of of Total Cost
▪
22.2% Avg
20.5%
▪
16.6% Avg
▪
13.0% Avg
13.5%
6.5%
0.7 RED 1.2 1.7 YELLOW 2.2 2.7 GREEN 3.2
Quality of EVM Implementation
Quality of EVM Implementation ▪ = Wg’t Avg PM %
- Quality of EVM implementation based on EVM assessments (FAA EVM Flag)
- PM% of total cost based on FY06 Resource Planning Document (RPD)
Improving Program Performance at FAA Federal Aviation 23 23
Administration

24. FAA Program Performance Reporting
(DOT Monthly – JRC / OMB Quarterly)
Over Spent Under Spent
Ahead of Schedule Ahead of Schedule
10%
5%
ASR-9/1B
Schedule Variance %
ERAM ATOP
ITWS
STARS
ASR-11
SASO
ASDE-X
ASWON
SWIM
ATCBI-6 WAAS
IFPA ASR-11 TFM-M
TAMR
SBS
-5% ASR-11NEXCOM
-10%
Program Name
Color Code -
>-15% EVMS Quality:
Green: Meets
Yellow: Partially
Meets
Over Spent Red: Does Not Meet
Under Spent
Behind Schedule Black: Not Assessed
Behind Schedule
Not updated
-10% -5% 5% 10% >15%
Cost Variance %
Improving Program Performance at FAA Federal Aviation 24 24
Administration

25. FAA Program Performance Reporting
(JRC Quarterly)
April 2008 EVM Data Analysis (Trends) -
• ASR-11 Program: 88% Complete
ASR-11 Cost and Schedule Variance Percentages PPPM
Financial
2.00% Y
Schedule
1.00%
Y
0.00% Technical
CV%/SV%
-1.00% Y
CV%
Resources
-2.00% SV%
G
-3.00% External
Interest
-4.00% Y
-5.00% Program
Manager
7 7 7 7 7 7 07 v- 07 7 8 8 8 8
-0 -0 -0 l- 0 -0 -0 ct- -0 -0 -0 -0 -0
A pr ay J un Ju ug S ep O N o ec J an F eb Mar A pr Y
M A D
Improving Program Performance at FAA Federal Aviation 25 25
Administration

26. GAO EVM Review – July 2008 Report
“FAA has taken important steps to oversee program compliance with EVM
policies, but its oversight process lacks sufficient rigor.”
Seven Key Components of an Effective EVM Policy
Policy component Assessment of
FAA policy
Establish clear criteria for which programs are to use EVM Fully met
Require programs to comply with national standards Fully met
Require programs to use a standard structure for defining the work products that Partially met
enables managers to track cost and schedule by defined deliverables (e.g., hardware
or software component)
Require programs to conduct detailed reviews of expected costs, schedules, and Fully met
deliverables (called an integrated baseline review)
Require and enforce EVM training Partially met
Define when programs may revise cost and schedule baselines (called rebaselining) Partially met
Require system surveillance—routine validation checks to ensure that major Fully met
acquisitions continue to comply with agency policies and standards
Improving Program Performance at FAA Federal Aviation 26 26
Administration

27. GAO EVM Review – July 2008 Report
Recommendations for Executive Action
GAO Recommendation FAA Response
Modify acquisition policies governing EVM to:
- require the use of a product-oriented standard - FAA milestone based WBS implementation 8/08
work breakdown structure - EVM Council approval 9/08
- Acquisition Executive Board approval 10/08
- Update current acquisition management system 10/09
- enforce existing EVM training requirements and - Develop EVM training course for understanding FAA
expand these requirements to include senior EVM reports – complete by 9/08
executives responsible for investment oversight - Conduct executive EVM training 8/09
and program staff responsible for program
oversight, and
- define acceptable reasons for rebaselining and - Develop program rebaseline procedure:
require programs seeking to rebaseline to (1) - Rebaseline criteria 6/09
perform a root cause analysis to determine why - Root cause analysis methodology by 10/08
significant cost and schedule variances occurred - Lessons learned and mitigation process and Acquisition
and (2) establish mitigation plans to address the Executive Board approval by 2/09
root cause.
Improving Program Performance at FAA Federal Aviation 27 27
Administration

28. EVM challenges
• Programs EVM remaining “Green”
– Some programs are slipping
– Acceptance across agency continues to be “lumpy”
– July 2008 GAO finding highlighted lack of program performance
surveillance and data analysis - ”FAA assessments are not thorough
enough to identify anomalies in contractor data”
• Efficient EVM implementation
– Need more Standards
• 1) Updating our program baseline management
• 2) Need clear responsibility assignment matrix
• 3) Need program control log procedures
– Need to Optimize and standardize on toolkit
– Continue to integrate processes, procedures, systems, and reporting
Improving Program Performance at FAA Federal Aviation 28 28
Administration

29. EVM challenges
• Maintain “High Risk Initiative” commitments to GAO
– Program best practices
– Apply to smaller “IT” programs
– Program baseline management
– Original baseline performance reporting
• Fee for Service EVM
– Limited FAA/industry experience
– Industry has some service EVM experience
– Innovative solutions will be required
Improving Program Performance at FAA Federal Aviation 29 29
Administration

30. EVM Transformation
Concrete Accomplishments
• All major programs assessed in FY05 still in development have
implemented full EVM or are close to implementing full EVM.
• Financial systems have been assessed so all 32 ANSI elements
have been assessed for our major programs.
• Tools and data studies have been completed and we are getting
ready to procure some standard tools.
• All new major programs entering development have been
assessed against the ANSI standard and make progress towards
green
• The EVM Council has created and will measure and enforce
compliance with our EVM policies and standards via an internal
surveillance process
Improving Program Performance at FAA Federal Aviation 30 30
Administration

31. EVM Transformation
Concrete Accomplishments
• More than half of our program managers are using EVM as
an integral part of the way they manage their programs
• FAA Executives use EVM to support their critical investment
decisions including rebaselining, Air Traffic quarterly
reviews, Integrated Baseline and Service Level reviews, etc.
• OMB, other stakeholders continue to recognize FAAs use of
EVM and increase their confidence in our Program
management.
• FAA is starting to baseline and measure its progress
towards its new objective.
• We are making the required continuous process
improvement.
Improving Program Performance at FAA Federal Aviation 31 31
Administration

32. EVM Accomplishments - Implied
• EVM was part of moving the FAA ATC modernization
program off the GAO High Risk List – only program
removed during 2009-11 cycle
• OMB Budget Passback and President’s Budget have
supported FAA programs since 2006
• Program Managers using EVM have faced fewer audits
and none have been “painful”
• FAA’s leadership position has helped attract positive
attention and good people have come our way.
• Industry sees us as their partner and shares best
practices with us more and more.
Improving Program Performance at FAA Federal Aviation 32 32
Administration

33. Keys to FAA EVM Success
• We obtained and have kept Executive Management Support
• EVM protects FAA’s Capital Investment funding
• EVM helped remove FAA from the GAO high risk list
• Our EVM Leadership position helps convince our vendors, aviation
partners and stakeholders that FAA is a good steward of the taxpayers
and airline user fees money
• We obtained and have Project Management Support
• We tailored EVM implementation based on type of contract and scope
and avoided legacy EVM Problem areas
• We provided clear, and concise ANSI/EIA 748 criteria
• We adapted the NDIA EVM Intent Guide using FAA terms, processes,
and documents
• We reached out to our major PMs and our prime contractors
Improving Program Performance at FAA Federal Aviation 33 33
Administration

34. Keys to FAA EVM Success
We kept our stakeholders and the industry fully engaged.
• Contractors and Program Managers see this as a win-win.
• We did constant socialization with GAO, OMB, the Department of
Transportation and other agencies.
• We provided our materials – policies, guidelines, best practices,
training materials, guidebooks and analyses – to everyone.
We focused on improving project performance rather than EVM. It
was the transformation caused by EVM, not EVM itself.
We focused on fixing the problems with our projects, not fixing the
blame, and on continuous improvement.
We involved everyone, both inside and outside the agency.
We never declared victory, but we always demonstrated concrete
progress.
We were not afraid to show our costs and to be transparent about
what our problems and challenges were and are.
Improving Program Performance at FAA Federal Aviation 34 34
Administration

35. EVM Transformation – Lessons
Learned and the Way Forward
• The proper balance between oversight and engagement is critical
• The involvement of staff and contractors with program management
experience and empathy is key
• Constant pressure is needed to make change.
• EVM must evolve to broader program management excellence
• Constant education for and use by senior managers is required
• Program teams must be encouraged or forced to employ the right folks
• Standard tools/data/methods are critical and usually more acceptable to
program teams.
• Deciding when to do it for them vs teaching them to do it for themselves is an
art form.
• Metrics and measurement and transparency is critical
• The external reviewers can be your best friends
• You are never done and the “elephant walk” may always return.
We will share any and all of our documents and lessons learned.
Improving Program Performance at FAA Federal Aviation 35 35
Administration

36. Thank You for Inviting me!
Are there Any Questions?
Improving Program Performance at FAA Federal Aviation 36 36
Administration