Decisions, Decisions: What Policymakers Need to Know About Cost-Benefit Analysis

994 views
883 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
994
On SlideShare
0
From Embeds
0
Number of Embeds
268
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Decisions, Decisions: What Policymakers Need to Know About Cost-Benefit Analysis

  1. 1. Decisions, Decisions: What PolicymakersNeed to Know About Cost-Benefit AnalysisOctober 25, 2012Craig Prins, Executive Director, Oregon Criminal Justice CommissionSarah Galgano, Policy Analyst, Vera Institute of Justice Slide 1
  2. 2. Decisions, Decisions: What PolicymakersNeed to Know About Cost-Benefit Analysis Craig Prins Sarah Galgano Executive Director Policy Analyst Oregon Criminal Justice Vera Institute of Justice Commission Slide 2
  3. 3. The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB)is a project of the Vera Institute of Justice funded by theU.S. Department of Justice’s Bureau of Justice Assistance. • Website (cbkb.org) • Cost-benefit analysis toolkit • Snapshots of CBA literature • Podcasts, videocasts, and webinars • Roundtable discussions • Community of practice • Technical assistance Slide 3
  4. 4. AgendaIntroductions & housekeeping 5 minutesCost-benefit analysis (CBA) basics 5 minutesCase study: The use of CBA in Oregon 10 minutesCBA and decision making 15 minutesHow CBA can answer questionsabout policy options 10 minutesQ&A 15 minutes Slide 4
  5. 5. Housekeeping items Questions: Use the Chat feature to send us questions during the webinar. Slide 5
  6. 6. Housekeeping items (continued)• If you need webinar support or for troubleshooting:  Type questions into the chat box.  Call 800-843-9166.  Send e-mail to help@readytalk.com.• This webinar is being recorded.• The recording and PowerPoint slides will be posted on cbkb.org. Slide 6
  7. 7. PreviewThis webinar will discuss:  The role of CBA in decision making, even when data is imperfect  Questions frequently asked about CBA results  The ways a CBA can address questions relevant to decision making Slide 7
  8. 8. CBA Basics Slide 8
  9. 9. What is cost-benefit analysis?A type of economic analysis thatcompares the costs and benefits ofpolicies and programs, and that: • includes the perspectives of multiple stakeholders; • presents a long-term picture; • uses dollars as a common measurement; and • allows you to compare programs and policies that have different outcomes. Slide 9
  10. 10. The CBA process1. Determine the impact of the initiative.2. Determine which perspectives to include.3. Measure costs and benefits (in dollars).4. Compare costs and benefits.5. Assess the reliability of the results.For a glossary of CBA terms, refer tocbkb.org/basics/glossary Slide 10
  11. 11. Case Study:The Use of CBA in Oregon Slide 11
  12. 12. Justice policy is multifaceted• Justice policy often has these goals:  Deter new criminal activity.  Reduce recidivism.  Improve public safety cost-effectively.• A new policy has resource costs, but can also save government resources and benefit society.• CBA monetizes savings, benefits, and resource costs. Slide 12
  13. 13. Building a capacity for CBA in Oregon• In 2006, Oregon Criminal Justice Commission began the development of statewide cost-benefit model for the criminal justice system.• Oregon leveraged the cost-benefit work of the Washington State Institute for Public Policy (WSIPP).• The Oregon model was designed to provide information to policymakers and the public about the relative costs and effectiveness of justice investments. Oregon’s cost-benefit methodology is available athttp://www.oregon.gov/cjc/docs/cost_benefit_methodology_090106.pdf Slide 13
  14. 14. Oregon invests in reentry programs• Oregon made an investment decision based on credible CBA results from another state.• WSIPP had estimates of reentry program success in Washington.• Would the results translate to Oregon?  The state has 36 diverse counties.  WSIPP cost estimates may not apply to rural areas.  The program’s effect in Oregon may differ from the WSIPP estimate. Slide 14
  15. 15. Example of WSIPP estimates Slide 15
  16. 16. Oregon’s reentry program• Continues prison treatment services for people once they are in the community• Coordinates ancillary, community-based services for recently released individuals:  Mental health services  Employment counseling  Educational attainment programs• Increases public safety and reduces recidivism Slide 16
  17. 17. CBA of Oregon’s reentry program• Researchers at the Oregon Criminal Justice Commission (CJC) conducted an outcome evaluation and a cost-benefit analysis.• Researchers used local data to:  Determine program-specific costs for the reentry center.  Track individuals who use services: Did they reoffend?  Measure the benefits specific to Oregon. Slide 17
  18. 18. CBA results: Oregon’s reentry programThe statewide reentry program:  Reduces recidivism by 27%  Results in an estimated 3.5 fewer felony convictions for every 10 participants over a 10-year period  Costs about $3,400 per participant  Generates benefits of $8,600 to taxpayers and $14,000 in avoided victimizations per participant  Results in benefit-cost ratio of $6.73 to every $1 spent Program evaluation and cost-benefit analysis available at: http://www.oregon.gov/CJC/docs/reentry_eval_final.pdf Slide 18
  19. 19. Lessons learned from Oregon CBA• Local data adds value when using a CBA to inform policy decisions. Statewide averages can be misleading.• Success is not a sure thing; program fidelity is key.• CBA adds important information to the policy dialogue. Slide 19
  20. 20. Questions?Please use the Chat feature to send us your questions. Craig Prins Sarah Galgano craig.prins@state.or.us sgalgano@vera.org Slide 20
  21. 21. CBA and Decision Making Slide 21
  22. 22. CBA is a decision-making tool• CBA is a way to consider the pros and cons systematically.• Justice policy is a long-term investment decision.• Beyond good intentions: Is a given policy the most effective way to improve public safety?• CBA is a business-like approach to increasing public safety and reducing cost as our “nonprofit’s” goal. Slide 22
  23. 23. Frequently asked questions about CBADecision makers (e.g., legislators, legislative staff,budget planners) often have questions about CBA:1. Can I believe the results?2. Are the results guaranteed?3. Who gets the benefits?4. What is the alternative? Slide 23
  24. 24. Can I believe the results?• Confidence in the results is related to the quality of the CBA’s inputs.  Are the predicted policy effects based on a rigorous analysis?  Are the program’s costs measured accurately?• Sensitivity analysis should be used to “kick the tires” on the results.• Transparency with the details will increase readers’ confidence in the study. Slide 24
  25. 25. Are the results guaranteed?• No. No one has a crystal ball.• “Past performance does not guarantee future results.”• The policy effect can vary from the CBA results for these reasons:  Measurement of the predicted policy effect  Fidelity of the program to the policy that was evaluated• The costs and benefits of the policy effect may be hard to monetize. Slide 25
  26. 26. Who gets the benefits?• The return on investment is both to the government budget (taxpayer benefits) and to society (such as victim benefits).• These benefits can be presented separately to provide clarity as to how they accrued.• Some benefits may accrue in the long term.• Budget savings are realized when there is less crime and therefore less spending on courts, policing, and corrections. Slide 26
  27. 27. Who gets the benefits? (continued)• CJC estimate: A 1% drop in recidivism results in $4.3 million avoided in annual victim and taxpayer costs due to crime.• Who gets these savings?  Savings may be reallocated to other sectors of corrections/state budgets.  Taxpayer resources may be used more efficiently. Slide 27
  28. 28. What is the alternative?• CBA is based on programmatic/initiative investments.• A single cost-benefit analysis reports the net benefit versus “business as usual.”• Several CBAs can provide a menu of policy choices (similar to the ratings in Consumer Reports). Slide 28
  29. 29. Questions?Please use the Chat feature to send us your questions. Craig Prins Sarah Galgano craig.prins@state.or.us sgalgano@vera.org Slide 29
  30. 30. How CBA Can AnswerQuestions About PolicyOptions Slide 30
  31. 31. Frequently asked questions about CBA1. Can I believe the results?2. Are the results guaranteed?3. Who gets the benefits?4. What is the alternative? Slide 31
  32. 32. How CBA can answer these questionsQuestion: How CBA can answer:1. Can I believe the results? Base-case scenario Partial sensitivity analysis2. Are the results guaranteed? Break-even analysis Best- and worse-case scenarios analysis Monte Carlo analysis3. Who gets the benefits? Analysis by perspective4. What is the alternative? Menu of CBA results Slide 32
  33. 33. Hypothetical prison education program• This example will be used to illustrate how CBA can address questions 1-3.• Hypothetical program:  Aims to reduce number of people who commit new crimes after their release  Uses estimates based on evaluations of similar programs because information on program impact is not available  Has reduced recidivism by 15 percent in other states  Costs an estimated $10,000 Slide 33
  34. 34. 1. Can I believe the results?A CBA can earn the trust of readers by:  Documenting why “base-case” scenario assumptions were chosen  Using partial sensitivity analysis to illustrate how the results vary if the assumptions are inaccurate Slide 34
  35. 35. Base-case scenarioTable 1: Hypothetical Prison Education Program: Base Case Total Cost Total Benefit Net BenefitPrison education $10,000 $17,000 $7,000 Note: The program is assumed to reduce recidivism by 15 percent over three years, based on studies of other similar programs. Slide 35
  36. 36. Base-case scenario• Provides information on the best estimate of a policy’s net benefits• Is based on direct evaluation of the initiative or evaluations of similar programs• Explains what assumptions were used Slide 36
  37. 37. Partial sensitivity analysis Table 2: Hypothetical Prison Education Program: Partial Sensitivity AnalysisChange in recidivism +5% 0% -5% -10% -15% -20%Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000Net benefits -$12,000 -$5,000 -$2,000 $0 $7,000 $12,000(Benefit minus cost) Slide 37
  38. 38. Partial sensitivity analysis• Provides information on how variation from the base-case scenario will affect the CBA results• Determines how sensitive the results are to changes in a single variable• Involves selecting a single variable in the analysis and changing its value (holding other variables constant) Slide 38
  39. 39. 2. Are the results guaranteed?• A CBA cannot provide guaranteed results, but can address uncertainty by:  Reporting the policy effect necessary for the costs and benefits to break even  Reporting the best-case and worse-case scenarios  Modeling the probability of potential net benefit using Monte Carlo analysis Slide 39
  40. 40. Break-even analysis Table 3: Hypothetical Prison Education Program: Break-Even AnalysisChange in recidivism +5% 0% -5% -10% -15% -20%Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000Net benefits -$12,000 -$5,000 -$2,000 $0 $7,000 $12,000(Benefit minus cost) Break-even point Slide 40
  41. 41. Break-even analysis• Provides information on the policy effect necessary for the program benefits to equal the program costs• Useful when information about program impact is unavailable Slide 41
  42. 42. Best-case/worse-case scenario analysis Table 4: Hypothetical Prison Education Program: Best-case and Worse-case ScenariosChange in recidivism +5% 0% -5% -10% -15% -20%Program costs $10,000 $10,000 $10,000 $10,000 $10,000 $10,000Program benefits -$2,000 $5,000 $8,000 $10,000 $17,000 $22,000Net benefits -$12,000 -$5,000 -$2,000 $0 $7,000 $12,000(Benefit minus cost) Worst Best case case Slide 42
  43. 43. Best-case/worse-case scenario analysis• Provides policymakers with information on how a broad range of a policy’s possible outcomes affect the bottom line.• Establish upper and lower boundaries of a cost-benefit study’s results.• Worst-case scenario analysis is based on using all the least favorable assumptions.• Best-case scenario analysis is based on all the most favorable assumptions. Slide 43
  44. 44. Example of Monte Carlo results Slide 44
  45. 45. Monte Carlo analysis• Provides information about the probability that outcomes will occur• Examines multiple variables simultaneously and simulates thousands of scenarios• Produces a range of possible results and their associated probabilitiesFor more on Monte Carlo analysis, watch the recording of the CBKBwebinar “Sensitivity Analysis for Cost-Benefit Studies of JusticePolicies” at cbkb.org. Slide 45
  46. 46. 3. Who gets the benefits?Table 5: Hypothetical Prison Education Program,by Perspective Taxpayer Victim Participant TotalCosts $10,000 $0 $0 $10,000Benefits $12,000 $2,000 $3,000 $17,000Net benefit $2,000 $2,000 $3,000 $7,000 Slide 46
  47. 47. Reporting costs and benefitsby perspective• CBA can answer this question by reporting the costs and benefits for each perspective (or stakeholder) in the analysis.• Outcomes related to perspectives that were not measured quantitatively should be discussed qualitatively.For more on perspectives, refer to the CBKB Perspectives tool atcbkb.org/toolkit/perspectives. Slide 47
  48. 48. 4. What is the alternative?CBA can be performed on more than one policy optionto provide a menu of options. Slide 48
  49. 49. Policy options in Washington State Slide 49
  50. 50. Wrap-upQuestion: How CBA can answer:1. Can I believe the results? Base-case scenario Partial sensitivity analysis2. Are the results guaranteed? Break-even analysis Best- and worse-case scenarios analysis Monte Carlo analysis3. Who gets the benefits? Analysis by perspective4. What is the alternative? Menu of CBA results Slide 50
  51. 51. Questions and Answers Slide 51
  52. 52. Questions?Please use the Chat feature to send us your questions. Craig Prins Sarah Galgano craig.prins@state.or.us sgalgano@vera.org Slide 52
  53. 53. Wrap-up Slide 53
  54. 54. Review• We covered: • How CBA adds to policy dialogue • What questions arise about CBA results • How CBA can address frequently asked questions Slide 54
  55. 55. Takeaways• CBA is a useful decision-making tool.• CBAs is not just about the results. The analysis adds important information to the policy dialogue. Slide 55
  56. 56. Follow-upBefore you log out of the webinar, please completethe evaluation form.To receive information and notifications about upcomingwebinars and other events: • Visit the Cost-Benefit Knowledge Bank for Criminal Justice at cbkb.org. • Subscribe to receive updates from CBKB. • Follow us on Twitter at twitter.com/CBKBank. Slide 56
  57. 57. Contact information Craig Prins Sarah Galgano Executive Director Policy Analyst Oregon Criminal Justice Vera Institute of Justice Commission craig.prins@state.or.us sgalgano@vera.org Slide 57
  58. 58. The Cost-Benefit Knowledge Bank for Criminal Justice (CBKB)is a project of the Vera Institute of Justice funded by theU.S. Department of Justice’s Bureau of Justice Assistance. • Website (cbkb.org) • Cost-benefit analysis toolkit • Snapshots of CBA literature • Podcasts, videocasts, and webinars • Roundtable discussions • Community of practice • Technical assistance Slide 58
  59. 59. Thank you! Slide 59
  60. 60. This project is supported by Grant No. 2009-MU-BX K029awarded by the Bureau of Justice Assistance. The Bureau ofJustice Assistance is a component of the Office of JusticePrograms, which also includes the Bureau of Justice Statistics,the National Institute of Justice, the Office of Juvenile Justiceand Delinquency Prevention, and the Office of Sex OffenderSentencing, Monitoring, Apprehending, Registering, andTracking. Points of view or opinions in this document are thoseof the author and do not represent the official position orpolicies of the United States Department of Justice. Slide 60

×