Your SlideShare is downloading. ×
Rovinsky final
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Rovinsky final


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Federal AviationFAA’s Successful AdministrationImplementation ofEarned ValueManagementPresented to :NASA PM ChallengeBy: Robert RovinskyDate: February, 2010 Improving Program Performance at FAA Federal Aviation 1 1 AdministrationUsed with permission
  • 2. Contents of Talk• Why Talk about the FAA at the PM Challenge• Quick FAA Facts• History/Drivers for Earned Value Management• What our Project Managers say about EVM• What Products and Processes we follow• Agency Accomplishments and GAO Audits• Lessons Learned and the Way Forward• Summary and Q&AImproving Program Performance at FAA FAAs Successful Implementation of Federal Aviation 2 2 Administration EVM
  • 3. Why Talk about the FAA to Present at the PM Challenge?• “NASA is currently on the GAO High Risk List and FAA just came off that list – why and how did FAA make this happen?”• “NASA faces challenges for EVM buy-in both internally (in- house projects) and externally (contractors) – is it a culture thing?”• “There is no Agency mandated EVM system – how did the FAA create one”• “How did FAA make the changes and get to its leadership position – was it an internal thing, was it a cultural thing, was it a mandate?• “Why are some folks intimidated or afraid of EVM? What is in it for them? What benefits did the FAA get? How much did it cost?” Improving Program Performance at FAA Federal Aviation 3 3 Administration
  • 4. FAA – Quick Facts• 12/17/1903 –1st sustained & controlled flight (Orville & Wilbur Wright)• 07/01/1934 –Bureau of Air Commerce formed• 08/23/1958 –Federal Aviation Act of ’58 signed• TODAY:• Safest System ever!• ~ $16B budget(FY10)• 45,757 FAA Employees(FY10)• 19,930 U.S. Airports• 811,593,000,000 –Passenger Miles Flown(‘08)• 590,349 Active PilotsImproving Program Performance at FAA Federal Aviation 4 4 Administration
  • 5. FAA – Mission/Vision/Values Our Mission Our Values To provide the safest, most SAFETY IS OUR PASSION. efficient aerospace system in We are the worlds leaders in aerospace safety. the world. QUALITY IS OUR TRADEMARK. We serve our country, our stakeholders, our customers, and each other. INTREGITY IS OUR CHARACTER. Our Vision We do the right thing, even when no one is We continue to improve the and efficiency of flight. PEOPLE ARE OUR STRENGTH. We are responsive to our We treat people as we want to be treated. customers and are accountable to the taxpayer and the flying public.Improving Program Performance at FAA Federal Aviation 5 5 Administration
  • 6. Background – History and Drivers for EVMFY 03-04 FAA Business Case Failure and Recovery: The original driver for the EVM initiative2005:• We assessed our major investments (80% of FAA capital expenditures) against the industry EVM Standard and produced our “EVM Flag”.• We committed to OMB and GAO that FAA would implement full EVM on our major acquisitions by the end of 2007.• We established FAA’s EVM policy. Improving Program Performance at FAA Federal Aviation 6 6 Administration
  • 7. Improving Program Performance at FAA Federal Aviation 7 7 Administration
  • 8. History and Drivers for EVM2006:• We formed a multi-functional EVM Council• We met with our major prime and support contractors on our plan and solicited industry best practices• We conducted a “cost and benefits of EVM” study• We revised FAAs Acquisition Policy to: – Establish Standard Program Milestones – Require Implementation Strategy and Planning (ISP) requirement for EVM plan• Result: Significant traction toward FAA compliance targets• We produced a Lay Guide to EVM• All FAA Acquisition Executives received EVM Executive Briefing• We developed standard program milestones for EVMImproving Program Performance at FAA Federal Aviation 8 8 Administration
  • 9. FAA’s EVM Council• The EVM Council, composed of representatives across the agency and all disciplines, leads the timely development, implementation and operation of FAA EVMS procedures, policies, training and tools to fully realize efficiencies and savings arising from the adoption and continuous improvement of a common EVMS employing best practices.• It provides support to Agency wide program management with respect to implementation of EVM on FAA programs as required by Acquisition Policy and the FAA EVM Guide.• The council reviews and evaluates draft EVM policy documents, including pertinent background information and special instructions or requirements to ensure all internal stakeholders are involved in the development and implementation of the FAA EVMS.• The EVM Council oversees the implementation and application of common EVM methods throughout FAA. Improving Program Performance at FAA Federal Aviation 9 9 Administration
  • 10. History and Drivers for EVM2007:• Achieved Green EVM assessments for the 2005 legacy programs• Established Portfolio/Program Performance Metrics (P3M)• Established requirement for product oriented WBS• Conducted EVM Data and Tools Studies• Conducted ANSI Accounting Guidelines EVM assessment• Conducted more EVM industry days• FAA Decides to try to get off GAO High Risk List; GAO sees our EVM effort as a step in right direction.Improving Program Performance at FAA Federal Aviation 10 10 Administration
  • 11. Improving Program Performance at FAA Federal Aviation 11 11 Administration
  • 12. History and Drivers for EVM2008:• Completed EVM Guides for Project Managers and Contracting Officers – Integrated Baseline Reviews – Surveillance Guides – EVM System Acceptance Guide• Revised EVM assessment criteria – consistent with NDIA Intent Guide• Formulated product oriented WBS using standard program milestones• EVM progress reported to GAO as part of High Risk InitiativeImproving Program Performance at FAA Federal Aviation 12 12 Administration
  • 13. History and Drivers for EVM2009:• FAA Gets off the High Risk List!• Acquisition Executive Board formed to institutionalize best practices including EVM• Developed methodology/criteria for “fee for service” projects• FAA one of only two federal agencies to certify contractors not already DCMA certified• FAA EVM approach adopted by Dept of Transportation Improving Program Performance at FAA Federal Aviation 13 13 Administration
  • 14. Today’s FAA EVM “Flag” Today we are: 82% Green, 17% Yellow and 1% RedImproving Program Performance at FAA Federal Aviation 14 14 Administration
  • 15. Industry Days have been conducted since 2005 to obtain our suppliers EVM Best Practices Description / Contractor A B C D E EVM Surveillance Joint Contractor / Government EVM surveillance Contractor Internal EVM Surveillance Use of EVM surveillance template or checklist IBRs Internal IBRs Joint Contractor / Government Reviews Perf Meas B/L implemented prior to Definitization Training Executive level EVM orientation briefings EVM Training Program Other EVM for Service contracts Program Start Up process P EVM User Guide Use EVM on LOE. Use EVM on FFP contracts. Program performance metrics plan Eliminating cultural organizational EVM barriers P Executive EVM metrics reporting = Current Practice Contracting for EVM “Tips & Traps” Tailored EVM P = PlannedImproving Program Performance at FAA Federal Aviation 15 15 Administration
  • 16. We are integrating EVM with our PM processes• Many of the Program Management Metrics reported to senior executive board use EVM data.• Standard Program Milestones improve completion criteria and map to FAA standard WBS.• EVM defined Program Baseline WBS used for Final Investment Decisions – We are piloting product oriented WBS using standard program milestones – Our OMB Exhibit 300 format uses the same Program Baseline WBS• We use EVM for DOT program performance “Bubble Chart” reports to OMB• We use Primavera for the Corporate Workplan – it is the enterprise schedule management solution for EVM• Our Implementation Strategy & Planning document is used to document the program EVM plan. Improving Program Performance at FAA Federal Aviation 16 16 Administration
  • 17. From our program managers we are getting affirmation of EVM• NEXCOM program manager, Dieter Thigpen, “earned value data clearly identified schedule performance variances and got management attention quicker as compared to the traditional approach not using EVM”.• Kevin Sharrett, ASR-11 Special Projects, “Implementing EVM provides early visibility into future schedule risks that we did not have before. As a result of the schedule risk assessment the ASR-11 EVM plan, we accelerated 5 system deliveries from 2007 to 2006 to mitigate our schedule risk for the last site Operational Readiness Demonstration which is baselined for 9/30/2009. We also had the schedule analysis to justify construction and systems acquisition funds in the first quarters of 2007 under the CR that had become critical for Last ORD if delayed. Implementing EVM has provided the ASR-11 team members more insight and involvement into the overall program cost and schedules.” Improving Program Performance at FAA Federal Aviation 17 17 Administration
  • 18. From our program managers we are getting affirmation of EVM• Dan Watts, the ERAM program manager, “Earned value data is the only true indicator I have to obtain insight for cost and schedule performance for a large scale complex program like ERAM”.• Former ECG program manager, Bill Boyer, “EVM is one of the more powerful tools in my PM toolbox. EVM allows me to see the variations in program execution that enable me to take action to resolve issues that are often beyond the capability of the prime contractor to handle. With EVM, I am better able to manage planning and execution of program funds, better able to track program progress against a benchmark, and better able to work with the prime contractor and the support contractors to apply resources when needed...and where needed.” Improving Program Performance at FAA Federal Aviation 18 18 Administration
  • 19. EVM training is customized• We make extensive use of FAA standard program milestones – Program Baseline WBS – Program phases / useful segments – Schedule completion criteria – Control account definitions• We include FAA enterprise tools (SPIRE, WorkLenz, and Primavera) in EVM training material• EVM training is consistent with FAA Accounting Order 2500.8B – FAA funding types tightly linked to EVM – Training includes accounting system (Delphi) examples• Acquisition program baseline definitions used – EVM tightly linked to program life cycle – Knowledge-Based Product Development Phases used• EVM is consistent with and supports our OMB Exhibit 300 reporting requirements• FAA risk management process used• FAA program examples & best practices are used Improving Program Performance at FAA Federal Aviation 19 19 Administration
  • 20. FAA EVM Program examples and best practices are being used for EVM training Drop 2 ESI Progress ERAM Successes: • Able to load all processor types and roles. ELOM functionality working well • Can display Target, track, & strobe data on glass • Can display an active flight plan in the Aircraft List. Can amend it and remove strip. • SAR data already being used for debugging defects. Initial set of EOPD reports working. • ELOM onboard cache working • Can load/unload and configure a single OpSim session Still Working • Multiple OpSims & Adaptation variations • Continue testing Surveillance & Wx threads • Release Management Threads • Continue with EOPD report testing • Continue Dynamic Library testing EVM reporting integrated with monthly PMRsImproving Program Performance at FAA Federal Aviation 20 20 Administration
  • 21. FAA EVM Program examples and best practices are being used for EVM training Overall RoadRoad ECG A GA Site Overall ECG to to Site Last Government Acceptance 100.0 90.0 80.0 70.0 Per Cent Complete Plan 60.0 % Complete 50.0 GA outlook 40.0 5/31/05 vs baseline 30.0 I&I 07/06/05 com plete Fitup and 4/26/05 vs. 20.0 SAT Install baseline 05/23/05 outlook 10.0 Com plete 5/23/05 vs 04/05/05 baseline 0.0 06/28/05 2/4/2005 2/11/2005 2/18/2005 2/25/2005 3/4/2005 3/11/2005 3/18/2005 3/25/2005 4/1/2005 4/8/2005 4/15/2005 4/22/2005 4/29/2005 5/6/2005 5/13/2005 5/20/2005 5/27/2005 6/3/2005 6/10/2005 6/17/2005 6/24/2005 7/1/2005 7/8/2005 7/15/2005 7/22/2005 7/29/2005 8/5/2005 = Plan = Actual FFP EVM reporting using performance based paymentsImproving Program Performance at FAA Federal Aviation 21 21 Administration
  • 22. FAA P3M is included in EVM training Program Summary Metrics Cost Schedule Performance Resources External Program Interest G G Y Y Y G Supporting MetricsPortfolio Metric Cost Schedule Portfolio Metric Performance Portfolio Metric Resources ExternalDefinition Measure of Measure of Measure of Interest cost schedule Definition Measure of Definition performance performance technical metrics current Assessment of work of work and milestones of funding and of External performed performed work performed staffing ReviewsEVM Metrics Level 1 AMS Milestones Resource Metrics- Cost Performance Index G - Prime Contractor Y (CPI) - Negative Deviations G- Program Cost Reserve G Technical Metrics - Support Contractors G - FAA G- To-Complete Performance G - Requirements Stability GIndex (TCPI) - Funding G- Cost Variance- G - System Defects YAt-completion (CVAC) % - Obligations G - Test Results n/a- Schedule Performance G Flight Plan Acquisition Index (SPI) - Deployment G Program Goals- Program Schedule G - Value of remaining High G**Reserve - Cost G Risks- Schedule Variance-At- G External ReviewsCompletion (SVAC) % - Technical Variance At- G Completion (TVAC) - IG YLevel 1 AMS Milestones- Negative Deviations G -GAO GFlight Plan Acquisition - OMB YProgram Goals- Schedule G Improving Program Performance at FAA Federal Aviation 22 22 Administration
  • 23. FAA Programs that implemented higher quality EVMhave lower PM costs as a percent of total program cost Cost of PM Using EVM (FY06) FAA Cost of Program Management Using EVM 34.5% 27.5% PM % Total Cost PM % of of Total Cost ▪ 22.2% Avg 20.5% ▪ 16.6% Avg ▪ 13.0% Avg 13.5% 6.5% 0.7 RED 1.2 1.7 YELLOW 2.2 2.7 GREEN 3.2 Quality of EVM Implementation Quality of EVM Implementation ▪ = Wg’t Avg PM % - Quality of EVM implementation based on EVM assessments (FAA EVM Flag) - PM% of total cost based on FY06 Resource Planning Document (RPD) Improving Program Performance at FAA Federal Aviation 23 23 Administration
  • 24. FAA Program Performance Reporting (DOT Monthly – JRC / OMB Quarterly) Over Spent Under Spent Ahead of Schedule Ahead of Schedule 10% 5% ASR-9/1B Schedule Variance % ERAM ATOP ITWS STARS ASR-11 SASO ASDE-X ASWON SWIM ATCBI-6 WAAS IFPA ASR-11 TFM-M TAMR SBS -5% ASR-11NEXCOM -10% Program Name Color Code - >-15% EVMS Quality: Green: Meets Yellow: Partially Meets Over Spent Red: Does Not Meet Under Spent Behind Schedule Black: Not Assessed Behind Schedule Not updated -10% -5% 5% 10% >15% Cost Variance %Improving Program Performance at FAA Federal Aviation 24 24 Administration
  • 25. FAA Program Performance Reporting (JRC Quarterly) April 2008 EVM Data Analysis (Trends) - • ASR-11 Program: 88% Complete ASR-11 Cost and Schedule Variance Percentages PPPM Financial 2.00% Y Schedule 1.00% Y 0.00% TechnicalCV%/SV% -1.00% Y CV% Resources -2.00% SV% G -3.00% External Interest -4.00% Y -5.00% Program Manager 7 7 7 7 7 7 07 v- 07 7 8 8 8 8 -0 -0 -0 l- 0 -0 -0 ct- -0 -0 -0 -0 -0 A pr ay J un Ju ug S ep O N o ec J an F eb Mar A pr Y M A D Improving Program Performance at FAA Federal Aviation 25 25 Administration
  • 26. GAO EVM Review – July 2008 Report“FAA has taken important steps to oversee program compliance with EVM policies, but its oversight process lacks sufficient rigor.” Seven Key Components of an Effective EVM Policy Policy component Assessment of FAA policy Establish clear criteria for which programs are to use EVM Fully met Require programs to comply with national standards Fully met Require programs to use a standard structure for defining the work products that Partially met enables managers to track cost and schedule by defined deliverables (e.g., hardware or software component) Require programs to conduct detailed reviews of expected costs, schedules, and Fully met deliverables (called an integrated baseline review) Require and enforce EVM training Partially met Define when programs may revise cost and schedule baselines (called rebaselining) Partially met Require system surveillance—routine validation checks to ensure that major Fully met acquisitions continue to comply with agency policies and standards Improving Program Performance at FAA Federal Aviation 26 26 Administration
  • 27. GAO EVM Review – July 2008 ReportRecommendations for Executive Action GAO Recommendation FAA ResponseModify acquisition policies governing EVM to:- require the use of a product-oriented standard - FAA milestone based WBS implementation 8/08work breakdown structure - EVM Council approval 9/08 - Acquisition Executive Board approval 10/08 - Update current acquisition management system 10/09- enforce existing EVM training requirements and - Develop EVM training course for understanding FAAexpand these requirements to include senior EVM reports – complete by 9/08executives responsible for investment oversight - Conduct executive EVM training 8/09and program staff responsible for programoversight, and- define acceptable reasons for rebaselining and - Develop program rebaseline procedure:require programs seeking to rebaseline to (1) - Rebaseline criteria 6/09perform a root cause analysis to determine why - Root cause analysis methodology by 10/08significant cost and schedule variances occurred - Lessons learned and mitigation process and Acquisitionand (2) establish mitigation plans to address the Executive Board approval by 2/09root cause. Improving Program Performance at FAA Federal Aviation 27 27 Administration
  • 28. EVM challenges• Programs EVM remaining “Green” – Some programs are slipping – Acceptance across agency continues to be “lumpy” – July 2008 GAO finding highlighted lack of program performance surveillance and data analysis - ”FAA assessments are not thorough enough to identify anomalies in contractor data”• Efficient EVM implementation – Need more Standards • 1) Updating our program baseline management • 2) Need clear responsibility assignment matrix • 3) Need program control log procedures – Need to Optimize and standardize on toolkit – Continue to integrate processes, procedures, systems, and reportingImproving Program Performance at FAA Federal Aviation 28 28 Administration
  • 29. EVM challenges• Maintain “High Risk Initiative” commitments to GAO – Program best practices – Apply to smaller “IT” programs – Program baseline management – Original baseline performance reporting• Fee for Service EVM – Limited FAA/industry experience – Industry has some service EVM experience – Innovative solutions will be requiredImproving Program Performance at FAA Federal Aviation 29 29 Administration
  • 30. EVM Transformation Concrete Accomplishments• All major programs assessed in FY05 still in development have implemented full EVM or are close to implementing full EVM.• Financial systems have been assessed so all 32 ANSI elements have been assessed for our major programs.• Tools and data studies have been completed and we are getting ready to procure some standard tools.• All new major programs entering development have been assessed against the ANSI standard and make progress towards green• The EVM Council has created and will measure and enforce compliance with our EVM policies and standards via an internal surveillance process Improving Program Performance at FAA Federal Aviation 30 30 Administration
  • 31. EVM Transformation Concrete Accomplishments• More than half of our program managers are using EVM as an integral part of the way they manage their programs• FAA Executives use EVM to support their critical investment decisions including rebaselining, Air Traffic quarterly reviews, Integrated Baseline and Service Level reviews, etc.• OMB, other stakeholders continue to recognize FAAs use of EVM and increase their confidence in our Program management.• FAA is starting to baseline and measure its progress towards its new objective.• We are making the required continuous process improvement.Improving Program Performance at FAA Federal Aviation 31 31 Administration
  • 32. EVM Accomplishments - Implied• EVM was part of moving the FAA ATC modernization program off the GAO High Risk List – only program removed during 2009-11 cycle• OMB Budget Passback and President’s Budget have supported FAA programs since 2006• Program Managers using EVM have faced fewer audits and none have been “painful”• FAA’s leadership position has helped attract positive attention and good people have come our way.• Industry sees us as their partner and shares best practices with us more and more.Improving Program Performance at FAA Federal Aviation 32 32 Administration
  • 33. Keys to FAA EVM Success• We obtained and have kept Executive Management Support • EVM protects FAA’s Capital Investment funding • EVM helped remove FAA from the GAO high risk list • Our EVM Leadership position helps convince our vendors, aviation partners and stakeholders that FAA is a good steward of the taxpayers and airline user fees money• We obtained and have Project Management Support • We tailored EVM implementation based on type of contract and scope and avoided legacy EVM Problem areas • We provided clear, and concise ANSI/EIA 748 criteria • We adapted the NDIA EVM Intent Guide using FAA terms, processes, and documents • We reached out to our major PMs and our prime contractors Improving Program Performance at FAA Federal Aviation 33 33 Administration
  • 34. Keys to FAA EVM SuccessWe kept our stakeholders and the industry fully engaged.• Contractors and Program Managers see this as a win-win.• We did constant socialization with GAO, OMB, the Department of Transportation and other agencies.• We provided our materials – policies, guidelines, best practices, training materials, guidebooks and analyses – to everyone.We focused on improving project performance rather than EVM. It was the transformation caused by EVM, not EVM itself.We focused on fixing the problems with our projects, not fixing the blame, and on continuous improvement.We involved everyone, both inside and outside the agency.We never declared victory, but we always demonstrated concrete progress.We were not afraid to show our costs and to be transparent about what our problems and challenges were and are.Improving Program Performance at FAA Federal Aviation 34 34 Administration
  • 35. EVM Transformation – Lessons Learned and the Way Forward• The proper balance between oversight and engagement is critical• The involvement of staff and contractors with program management experience and empathy is key• Constant pressure is needed to make change.• EVM must evolve to broader program management excellence• Constant education for and use by senior managers is required• Program teams must be encouraged or forced to employ the right folks• Standard tools/data/methods are critical and usually more acceptable to program teams.• Deciding when to do it for them vs teaching them to do it for themselves is an art form.• Metrics and measurement and transparency is critical• The external reviewers can be your best friends• You are never done and the “elephant walk” may always return. We will share any and all of our documents and lessons learned.Improving Program Performance at FAA Federal Aviation 35 35 Administration
  • 36. Thank You for Inviting me! Are there Any Questions?Improving Program Performance at FAA Federal Aviation 36 36 Administration