Ellig Perf Meas For U Service Catholic U March 28 2006


Published on

Published in: Business, Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ellig Perf Meas For U Service Catholic U March 28 2006

  1. 1. Good Intentions vs. Real Accomplishments: Effective Performance Measures for Universal Service Programs Jerry Ellig Senior Research Fellow
  2. 2. Universal Service 2004 Disbursements <ul><li>High-cost carriers ($3.5 billion) </li></ul><ul><li>Low-income households ($759 million) </li></ul><ul><li>Schools/libraries ($1.2 billion) </li></ul><ul><li>Rural health care ($15 million) </li></ul><ul><li>Source: FCC, Trends in Telephone Service (June 2005), Table 19.1. </li></ul>
  3. 3. Why discuss performance measures? <ul><li>Programs are intended to produce significant economic, social, educational, and health benefits </li></ul><ul><li>Programs widely criticized for lack of performance measures and other management problems </li></ul><ul><li>Congress may address universal service in upcoming telecom legislation </li></ul><ul><li>Discussion of expanding universal service to broadband </li></ul>
  4. 4. Ineffective programs are a bigger danger than waste/fraud/abuse <ul><li>Total schools/libraries commitments thru Oct 2005: </li></ul><ul><li>$15 billion </li></ul><ul><li>Waste/fraud/abuse recovery actions (as of 2004): </li></ul><ul><li>$36 million </li></ul><ul><li>“ Questionable” funding under investigation (2005): </li></ul><ul><li>$200 million </li></ul><ul><li>Each 1% of funding spent lawfully but ineffectively: </li></ul><ul><li>$150 million </li></ul>
  5. 5. 7 steps toward effective measures <ul><li>Articulate “logic model” </li></ul><ul><li>Identify specific, ultimate outcomes </li></ul><ul><li>Identify intermediate outcomes (if needed) </li></ul><ul><li>Develop measures for outcomes </li></ul><ul><li>Analyze causation </li></ul><ul><li>Measure cost-effectiveness, not just efficiency </li></ul><ul><li>Include hidden costs </li></ul>
  6. 6. 1. Implicit “Logic Model” <ul><li>Programs cause improvement in access/affordability </li></ul><ul><li>Improvement in access/affordability causes increase in subscribership or connectivity </li></ul><ul><li>Increased subscribership/connectivity causes economic, social, educational, health benefits </li></ul>
  7. 7. 2. Ultimate outcomes What are specific outcome(s) for broadband? Improved health outcomes Lower costs of maintaining healthy population Rural health Faster improvement in student achievement Schools/libraries Increased economic/social opportunities Low income Increased economic/social opportunities High cost
  8. 8. 3. Intermediate outcomes Similar for broadband? Service is available Service is “affordable” Connectivity increases Rural health Service is available Service is “affordable” Connectivity increases Schools/libraries Service is available Service is “affordable” Subscription increases Low income Service is available Service is “affordable” Subscription increases High cost
  9. 9. <ul><li>Intermediate Measures – Telephone We can easily imagine similar measures for broadband. </li></ul><ul><li># and % of target population with phone service available </li></ul><ul><li># and % of target population that spends less than a designated percentage of income on phone service </li></ul><ul><li># and % of target population subscribing to phone service who would not subscribe in the absence of the program </li></ul>
  10. 10. <ul><li>Intermediate Measures – Internet We can easily imagine similar measures for broadband. </li></ul><ul><li># and % of people served by facilities in locations where the service is available </li></ul><ul><li># and % of people served by facilities that spend less than a designated % of budget on service </li></ul><ul><li># and % of people served by facilities that would not have been connected to specified services in the absence of the program </li></ul>
  11. 11. Why subscribership is not a sufficient measure <ul><li>High subscribership = high availability </li></ul><ul><li>Low subscribership ≠ low availability </li></ul><ul><li>High subscribership ≠ affordability </li></ul><ul><li>Low subscribership ≠ lack of affordability </li></ul>
  12. 12. 5. Analyze causation <ul><li>How much of the observed change in the outcome can be attributed to the program after adjusting for other factors that affect the outcome? </li></ul><ul><li>-Simple calculations </li></ul><ul><li>-Control and treatment groups </li></ul><ul><li>-Econometric analysis </li></ul>
  13. 13. Simple calculations <ul><li>Select a benchmark for affordability </li></ul><ul><li>Calculate # and % of consumers paying less than the benchmark </li></ul><ul><li>Calculate the prices they would pay in the absence of the subsidy </li></ul><ul><li>Calculate # and % of consumers paying less than the benchmark if prices were not subsidized </li></ul><ul><li>Difference is the change in outcome attributable to subsidies </li></ul><ul><li>Note: Some subsidies inflate costs </li></ul>
  14. 14. Econometric analysis <ul><li>Subsidies for local telephone service have little effect on subscribership </li></ul><ul><li>High cost program costs $5-10k per additional subscriber </li></ul><ul><li>Low income programs cost $1500-2200 per additional subscriber </li></ul>
  15. 15. 6. Cost-effectiveness measures <ul><li>Cost of program </li></ul><ul><li>divided by </li></ul><ul><li># of successful outcomes </li></ul><ul><li>What does it cost to make service affordable for an additional household? </li></ul><ul><li>What does it cost to induce an additional household to subscribe? </li></ul><ul><li>What does it cost to achieve a specified amount of change in student performance? </li></ul><ul><li>What does it cost to improve a specified health outcome by a specified amount? </li></ul>
  16. 16. 7. Incorporate all economic costs <ul><li>Deadweight loss = Value of service that consumers forego, plus operating profits that producers forego, because increased price reduces use of the service </li></ul><ul><li>Universal service taxes services with elastic demand to subsidize services with inelastic demand </li></ul>
  17. 17. Explicit + Hidden Costs Source: Jerry Ellig, “Costs and Consequences of Federal Telecommunications Regulation,” 58 Federal Communications Law Journal 37 (Jan. 2006). $2.7 billion $978 million $1.76 billion Wireless $3.86 billion $1.16 billion $2.7 billion Long Distance Total Cost to Society DW loss U Service Expenditure
  18. 18. What do good performance measures do? <ul><li>Explicitly state desired outcomes </li></ul><ul><li>Use meaningful outcome measures that reflect causation </li></ul><ul><li>Examine cost-effectiveness in terms of outcomes </li></ul><ul><li>Include hidden costs when assessing cost-effectiveness </li></ul>
  19. 19. For more information … <ul><li>More detail on these concepts and examples can be found in a series of public interest comments filed with the FCC by Maurice McTigue and Jerry Ellig, available at </li></ul><ul><li>http://www.mercatus.org/regulatorystudies/article.php/1509.html </li></ul>