Ellig Mc Tigue Richardson Outcome Based Scrutiny 2000


Published on

Published in: Education, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ellig Mc Tigue Richardson Outcome Based Scrutiny 2000

  1. 1. Outcome-Based Scrutiny: Putting a Price on Performance <ul><li>Government Accountability Project </li></ul><ul><li>Mercatus Center </li></ul><ul><li>George Mason University </li></ul><ul><li>Arlington, VA </li></ul><ul><li>http://www.mercatus.org </li></ul>
  2. 2. What is Outcome-Based Scrutiny? <ul><li>Assessment of federal programs to determine: </li></ul><ul><li>What specific benefits were produced (or harms averted) for citizens? </li></ul><ul><li>2. How could resources be reallocated to make government more effective? </li></ul>
  3. 3. The 4 th Step in the GPRA Process <ul><li>Planning (Strategic Plans) </li></ul><ul><li>Implementation </li></ul><ul><li>Disclosure (Performance Reports) </li></ul><ul><li>Scrutiny </li></ul>
  4. 4. Why Outcome-Based Scrutiny? <ul><li>Improve quality of agency reporting </li></ul><ul><li>Improve congressional resource allocation decisions </li></ul><ul><li>Drive fundamental management change within federal agencies </li></ul>
  5. 5. A 7-Step Process
  6. 6. Mercatus demonstration study: Vocational Training Programs <ul><li>Note: This is a demonstration study only, not a full analysis that provides sufficient information for congressional decisions. </li></ul>
  7. 7. 1. Outcome: “What is the issue and the intended outcome?” <ul><li>Focus on programs whose primary purpose is to enhance employability </li></ul><ul><li>Outcome measure is the number of people who moved into work or became more employable as a result of the program </li></ul>
  8. 8. 2. Agencies: “Who is responsible?” <ul><li>Labor </li></ul><ul><li>Education </li></ul><ul><li>Health & Human Services </li></ul><ul><li>Interior </li></ul><ul><li>Housing & Urban Development </li></ul><ul><li>Justice </li></ul><ul><li>[Narrowed down from 12 agencies] </li></ul>
  9. 9. 3. Programs: “How do they do it?” 3 Justice 2 Housing & Urban Dev. 4 Interior 7 Health & Human Svcs. 11 Education 17 Labor # of Programs Department
  10. 10. Narrowing the list… <ul><li>105 programs mention vocational training </li></ul><ul><li>44 (grouped into 25 areas in APRs) have it </li></ul><ul><li>as a primary goal </li></ul><ul><li>34 (in 14 areas) have outcome data </li></ul>
  11. 11. How comprehensive is this? <ul><li>Our analysis covers </li></ul><ul><li>77 percent of vocational programs </li></ul><ul><li>56 percent of vocational program areas defined in performance reports </li></ul><ul><li>97 percent of spending we identified </li></ul>
  12. 12. Where’d we get this stuff? <ul><li>Programs: Agency Strategic Plans, Catalog of Federal Domestic Assistance (CFDA), agency web sites </li></ul><ul><li>Outcomes: GPRA Annual Performance Reports, agency analysts </li></ul><ul><li>Cost figures: CFDA, U.S. Budget, agency budget justifications </li></ul>
  13. 13. 4. Results: “Are they effective?”
  14. 14. 5: Cost/Benefit: “What is the best value?”
  15. 15. 6: Reallocation: “What changes give the greatest benefits?” <ul><li>Total expenditure: $8.4 billion </li></ul><ul><li>Total benefit: 2.8 million placements </li></ul><ul><li>Average cost: $3000/person </li></ul><ul><li>Hypothetical total benefit if most cost-effective program received all resources: 14.5 million placements </li></ul>
  16. 16. Reallocation within 4 youth programs with outcome info. <ul><li>Total cost: $3.6 billion </li></ul><ul><li>Total placements: 2 million </li></ul><ul><li>Hypothetical placements if 2 most cost-effective programs received all resources: 5.5 million </li></ul>
  17. 17. 7: Forgone Opportunity: “What do we lose under the status quo?” <ul><li>All programs example: </li></ul><ul><li>14.5 million hypothetical placements </li></ul><ul><li>- 2.8 million actual placements </li></ul><ul><li>11.7 million people not helped! </li></ul><ul><li>Youth example: </li></ul><ul><li>5.5 million hypothetical placements </li></ul><ul><li>- 2 million actual placements </li></ul><ul><li>3 million young people not helped! </li></ul>
  18. 18. Implementation Tips <ul><li>Alternative values define public benefit </li></ul><ul><li>Correlation is not causation </li></ul><ul><li>It can be OK to mix apples and oranges </li></ul><ul><li>Unit costs may not be constant </li></ul><ul><li>Match results with expenditures </li></ul>
  19. 19. Alternative values define public benefit <ul><li>Forgone opportunity may be 11.7 million people… </li></ul><ul><li>… or 281,178 additional trade-affected workers who might have been helped if all the resources went to that program. </li></ul><ul><li>If it only costs $578 per placement, do those students really need government help? </li></ul>
  20. 20. Correlation is not causation <ul><li>Observed placements </li></ul><ul><li>- Placements that would have occurred in the absence of the program </li></ul><ul><li>_____________________________________ </li></ul><ul><li>Outcome actually attributable to the program </li></ul>
  21. 21. Apples and Oranges <ul><li>Complaint: Comparisons “unfair” due to differences among target populations. </li></ul><ul><li>Judgment required: Compare all vocational programs or all youth vocational programs? </li></ul><ul><li>The forgone opportunities are still real. </li></ul>
  22. 22. Unit costs can change <ul><li>Easy results may already have been achieved. </li></ul><ul><li>Economies of scale may not be fully realized. </li></ul><ul><li>Forgone opportunity may be larger or smaller than our calculations suggest. </li></ul>
  23. 23. Marginal and average costs can vary with output
  24. 24. Match results with expenditures <ul><li>Cost is a single year’s appropriation, but placement figures are for a single year. </li></ul><ul><li>Ideal data would be time series, not snapshot. </li></ul><ul><li>Match this year’s outcomes with past expenditures contributing to the outcomes? </li></ul>
  25. 25. Outcome-Based Scrutiny improves Knowledge and Incentives <ul><li>Reveals what kind of information is needed from agencies </li></ul><ul><li>Reallocation improves govt. performance </li></ul><ul><li>Comparison promotes interagency learning </li></ul><ul><li>Prospect of reallocation motivates improvement </li></ul>