Ellig Gpra And Use Of Performance Info April 2009

505 views
453 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
505
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ellig Gpra And Use Of Performance Info April 2009

  1. 1. Has the Government Performance and Results Act Improved the Availability and Use of Performance Information? Jerry Ellig Senior Research Fellow
  2. 2. Government Performance and Results Act of 1993 <ul><li>One motivation: “Federal managers are seriously disadvantaged in their efforts to improve program efficiency and effectiveness, because of insufficient articulation of program goals and inadequate information on program performance.” </li></ul><ul><li>Requirements: </li></ul><ul><ul><li>Strategic plans outlining goals (including outcomes) and planned program evaluations </li></ul></ul><ul><ul><li>Annual performance plans establishing quantitative goals </li></ul></ul><ul><ul><li>Annual performance reports beginning fiscal year 1999 </li></ul></ul>
  3. 3. Possible research questions <ul><li>Has GPRA increased the availability and use of performance information in agencies? </li></ul><ul><li>Has GPRA increased the use of performance information by policymakers? </li></ul><ul><li>Has increased use of performance information affected government’s efficiency, effectiveness, scope, or size? </li></ul><ul><li>Have GPRA-induced changes in government affected human welfare? </li></ul>
  4. 4. Why might GPRA improve availability or use of information? <ul><li>Tullock, Downs theories of bureaucracy: Individuals advance by doing what their superiors want. </li></ul><ul><li>How do agency officials know policymakers want performance management? </li></ul><ul><ul><li>GPRA is clearly the law </li></ul></ul><ul><ul><li>Presidents have shown consistent interest </li></ul></ul><ul><ul><li>Congress has shown less consistent interest </li></ul></ul>
  5. 5. Why might GPRA fail? <ul><li>GPRA skeptics </li></ul><ul><ul><li>Analytical and data difficulties </li></ul></ul><ul><ul><li>Outcomes harder to measure for some kinds of programs (eg, grants, R&D) </li></ul></ul><ul><ul><li>US political system hampers agreement on outcome goals </li></ul></ul><ul><li>Agency theory </li></ul><ul><ul><li>Align agency incentives with performance budgeting </li></ul></ul><ul><ul><li>Align managers’ incentives with performance contracts </li></ul></ul><ul><ul><li>Give mangers greater discretion over use of resources </li></ul></ul><ul><ul><li>Reform civil service </li></ul></ul>
  6. 6. The empirical analysis <ul><li>Dependent variable: Periodic Government Accountability Office (GAO) surveys ask federal managers in 24 agencies if they have various kinds of performance information and use it for various purposes </li></ul><ul><li>Independent variable: Annual Mercatus Center Scorecard measures quality of the same 24 agency GPRA performance reports </li></ul><ul><li>Are these 2 things correlated? </li></ul>
  7. 7. GAO surveys <ul><li>Asks federal managers if they have various kinds of performance measures for their programs or projects </li></ul><ul><li>Asks federal managers if they use performance information for various specified purposes in their programs or projects </li></ul><ul><li>2000 and 2007 had enough responses to calculate valid percentages for each agency </li></ul><ul><li>Use % of managers responding “to a great extent” or “to a very great extent” </li></ul>
  8. 8. Average of agency responses
  9. 9. Average of agency responses
  10. 10. The Mercatus Scorecard 1-5 rating scale <ul><li>3 Categories </li></ul><ul><li>Transparency </li></ul><ul><li>Public Benefits </li></ul><ul><li>Leadership </li></ul><ul><li>4 criteria in each category </li></ul><ul><li>Criteria tightened each year to reflect previous year’s best practices </li></ul><ul><li>Total score can range from 12 to 60 </li></ul>Fails to meet expectations 1 Partially complete 2 Satisfactory 3 Shows innovation and creativity 4 Sets a standard for best practice 5
  11. 12. 2 econometric specifications <ul><li>Pooled 2000 and 2007 data </li></ul><ul><ul><li>46 observations </li></ul></ul><ul><ul><li>Control variables </li></ul></ul><ul><ul><ul><li>Year, score*year </li></ul></ul></ul><ul><ul><ul><li>Leadership commitment (survey response) </li></ul></ul></ul><ul><ul><ul><li>% of budget spent on competitive grants, block grants, R&D, regulation </li></ul></ul></ul><ul><li>2007-2000 differences </li></ul><ul><ul><li>22 observations </li></ul></ul><ul><ul><li>Leadership control variable sometimes significant </li></ul></ul>
  12. 13. GPRA linked to availability of information (results significant at 90% level or higher) <ul><li>A one point change in an agency’s Scorecard score is associated with between one-third and one-half of a percentage point change in the number of managers stating that they have measures of </li></ul><ul><ul><li>Outcomes </li></ul></ul><ul><ul><li>Outputs </li></ul></ul><ul><ul><li>Efficiency </li></ul></ul><ul><li>An agency producing a GPRA report with an average Scorecard score of 34 would have about 10 percent more managers reporting that they have outcome, output, or efficiency measures. </li></ul><ul><li>40-60 percent of managers in various agencies said they had outcome, output, or efficiency measures. </li></ul>
  13. 14. GPRA and use of performance information (results significant at 90% level or greater) <ul><li>Score of agency GPRA reports is positively correlated with five different uses of performance information: </li></ul><ul><ul><li>allocating resources </li></ul></ul><ul><ul><li>setting priorities </li></ul></ul><ul><ul><li>coordinating with external agencies </li></ul></ul><ul><ul><li>establishing measures </li></ul></ul><ul><ul><li>setting goals </li></ul></ul>
  14. 15. How big is Scorecard score’s effect on uses of performance information? <ul><li>Coefficients range between 0.3 and 0.6. </li></ul><ul><li>An agency producing a GPRA report with an average Scorecard score of 34 would have 10-20 percent more managers reporting that they use performance information for these purposes. </li></ul><ul><li>25-50 percent of managers said they use performance information for various purposes. </li></ul>
  15. 16. Influence on use of information may be wider <ul><li>Scorecard score may affect other uses of information </li></ul><ul><ul><li>Existence of outcome measures is correlated with Scorecard scores </li></ul></ul><ul><ul><li>Every use of performance information is correlated with existence of outcome measures </li></ul></ul><ul><ul><li>Scorecard scores may have an indirect effect on every use of performance information </li></ul></ul><ul><li>All of these regressions only measure effects that are attributable to differences in quality of agencies’ GPRA initiatives </li></ul>
  16. 17. Other interesting results <ul><li>Leadership control variable often has a big positive influence </li></ul><ul><li>More competitive grants => more managers with outcome measures </li></ul><ul><li>More block grants => lower percentages of managers claiming they have efficiency measures </li></ul><ul><li>More regulation => lower percentages of managers that say they have outcome measures or use performance information to allocate resources </li></ul><ul><li>More R&D => lower percentages of managers reporting they have output measures, efficiency measures, or use performance information for many purposes </li></ul>
  17. 18. What can we conclude? <ul><li>GPRA had an effect: Quality of agencies’ GPRA initiatives has an effect on availability and use of performance information </li></ul><ul><li>Tullock was right: Passage of GPRA, executive branch guidance, and commitment of agency leadership motivated improvement without wholesale revision to bureaucratic incentive structure </li></ul><ul><li>Skeptics were right about some things: Some kinds of performance measurement and use of info seems less likely in agencies utilizing more block grants, regulation, or R&D programs </li></ul>

×