Mercatus Ellig Wray Presentation April


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mercatus Ellig Wray Presentation April

  1. 1. Pilots vs. PARs: Some Preliminary Results Jerry Ellig & Henry Wray
  2. 2. Project Design <ul><li>Team of 3 experts with experience in govt. and/or performance management evaluates reports </li></ul><ul><li>12 criteria based on GPRA requirements </li></ul><ul><li>Evaluate reports from 24 CFO Act agencies </li></ul><ul><li>Each evaluation reviewed by a member of advisory panel </li></ul><ul><li>Entire report reviewed by entire advisory panel </li></ul>
  3. 3. How we score the reports 1-5 rating scale <ul><li>3 Categories </li></ul><ul><li>Transparency </li></ul><ul><li>Public Benefits </li></ul><ul><li>Leadership </li></ul><ul><li>4 criteria in each category </li></ul><ul><li>Criteria tightened each year to reflect previous year’s best practices </li></ul><ul><li>Total score can range from 12 to 60 </li></ul>Fails to meet expectations 1 Partially complete 2 Satisfactory 3 Shows innovation and creativity 4 Sets a standard for best practice 5
  4. 4. Pilots vs. PARs, fiscal 2007 24 % Difference 7.33 Difference 30.00 Pilot average score 37.33 PAR average score
  5. 5. Pilot vs. PAR scores, fiscal 2006-07 -12 -1 % Change -4.11 -0.4 Change 30.00 37.33 2007 34.11 37.73 2006 Pilot PAR
  6. 6. Public Disclosure <ul><li>Whatever its impact on other users, the pilot was a step backward for public disclosure : </li></ul><ul><li>Access to performance information was delayed for several months. </li></ul><ul><li>Once released, the information was harder to find and use. </li></ul><ul><li>To the extent found, the information gave the public little if anything that was not available last November. </li></ul><ul><li>The one potential benefit for public disclosure: Some highlights documents (mainly those done voluntarily by non-pilot agencies) demonstrated the value they can add for lay readers, if done well . </li></ul>
  7. 7. Performance Information <ul><li>The pilot provided limited performance information that the general public could reasonably access and use: </li></ul><ul><li>We found and used all three relevant documents (highlights, performance report, and financial report) for only 2 of the 9 pilot agencies. </li></ul><ul><li>We used the highlights for all 9 agencies (although 2 were not timely posted on line). However, most failed to provide an insightful overview of agency performance and lacked user-friendly links to relevant information in other source documents. </li></ul><ul><li>We used only 3 of 9 performance reports. We could not find the reports for 3 agencies by the due date. The remaining 3 reports were embedded in budget justifications and thus beyond the practical reach of the general public. </li></ul>
  8. 8. Observations <ul><li>If the pilot continues, the highlights documents and their links to other source documents must improve substantially in order for the public to get any value from performance reporting. </li></ul><ul><li>Highlights documents can significantly enhance performance reporting for the general public. Therefore, they should be improved and expanded whether or not the pilot continues. </li></ul><ul><li>Our report will offer a number of specific suggestions for improving the highlights documents. </li></ul>