Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BBVT_Results Based Accountability_Presentation

981 views

Published on

Dr. Amy Carmola of the United Way of Chittenden County presents the basics of Results-Based Accountability™ to the Vermont Nonprofit Conference 2012: Benchmarks for a Better Vermont. Results-Based Accountability™ concepts utilized in this presentation are derived from the book "Trying Hard is Not Good Enough by Mark Friedman.”

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

BBVT_Results Based Accountability_Presentation

  1. 1. Results-Based AccountabilityVermont Nonprofit ConferenceMarch 20, 2012
  2. 2. Goals for Today(aka How I hope you will better off) Basic understanding of Results-Based Accountability (RBA) •Why bother? •What is it? •How it can help you plan and manage and maximize •How it can help you communicate •Where to get more information & resources Experience working with Performance Measures Be ready to take next steps Benchmarks for a Better VT March 2012 2
  3. 3. Sources & ResourcesResults-Based AccountabilityTM (RBA)Trying Hard Is Not Good Enough: How to ProduceMeasurable Improvements for Customers andCommunitiesMark Friedman (2005)Results and Performance Accountability, Decision-Making, and BudgetingFiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org Benchmarks for a Better VT March 2012 3
  4. 4. Wait a Second… Results Accountability Performance Measures Haven’t we heard about this before? Everyone wants to: •Improve our programs & maximize impact •Leverage support for our programsBenchmarks for a Better VT March 2012 4
  5. 5. What gets in the way?• “Too little time”• “Other priorities”• “Too complicated”• “Too many hard decisions”• “We don’t have expertise – or $$ to hire someone”• “They don’t get it”• “Doesn’t feel relevant”• Overthinking…Trying too hard Benchmarks for a Better VT March 2012 5
  6. 6. Why Bother? Don’t I have enough to worry about already??Benchmarks for a Better VT March 2012 6
  7. 7. Why Bother?Benchmarks for a Better VT March 2012 7
  8. 8. What is Results Accountability? Benchmarks for a Better VT March 2012 8
  9. 9.  Shifting our Thinking Results Accountability is Results Accountability IS a :NOT: •behavior discipline•a product •framework for planning, doing,•a year-end evaluation and evaluating•software •process for moving from Talk•(just) a reporting tool to Action •cycle for continuous improvement •tool for communicating Benchmarks for a Better VT March 2012 9
  10. 10.  Shifting our Thinking RBA is NOT: RBA is :•fancy •simple•new •the MAIN track•a side track RBA is : •asking an answering common sense questions (that we already think about all the time) Benchmarks for a Better VT March 2012 10
  11. 11. Common Sense Questions• What do we want?• How will we recognize it?• How do things look?• What will it take to get the results we want?• Who are the partners with a role to play?• What is our role in helping achieve the result?• How good of a job are we doing? Benchmarks for a Better VT March 2012 11
  12. 12. What’s the Big Deal? (1)Common Sense Questions•What do we want?  Start with ENDS•How will we recognize it?•How do things look now?•What will it take to get the resultswe want?•Who are the partners with a roleto play?•What is our role in helping achieve  Then talk aboutthe result? MEANS•How good of a job are we doing? Benchmarks for a Better VT March 2012 12
  13. 13. What’s the Big Deal? (2)Common Sense Questions•What do we want?  COMMUNITY FOCUS•How will we recognize it?•How do things look now? Can look at different levels…•What will it take to get the results …with appropriatewe want? accountability (more on that•Who are the partners with a role later)to play?•What is our role in helping achievethe result?  PROGRAM FOCUS•How good of a job are we doing? Benchmarks for a Better VT March 2012 13
  14. 14. What’s the Big Deal? (2continued)Common Sense Questions•What do we want? For each “thing” we do:•How will we recognize it?•How do things look now? •Who do we serve?•What will it take to get the results we •What do we do?want? •How good of a job are we•Who are the partners with a role to play? doing?•What is our role in helping •What works toachieve the result? improve/strengthen?•How good of a job are we doing? • What do we propose to do? Benchmarks for a Better VT March 2012 14
  15. 15. What’s the Big Deal? (3)Common Sense Questions•What do we want? Common Sense•How will we recognize it?•How do things look now? Clear Language•What will it take to get the resultswe want?•Who are the partners with a role No Jargonto play?•What is our role in helping achieve Tells a Storythe result?•How good of a job are we doing? Benchmarks for a Better VT March 2012 15
  16. 16. A few words about LANGUAGE• Outcome, Result, Benchmark, Goal, Target• It doesn’t matter what you call it, as long as people know what you mean.You can call me anything you like, but just don’t call me late for dinner (or do I mean supper?). Benchmarks for a Better VT March 2012 16
  17. 17. Agreeing on Clear Language• What do we want? RESULT A condition of well-being for children, adults, families or communities as a whole• How will we recognize it? INDICATOR How we measure this• How do things look now? condition• What will it take?• Who are the partners?• What is our role?• How good of a job are we doing? Benchmarks for a Better VT March 2012 17
  18. 18. EXAMPLESRESULTS for All Peopleof the State, County, City or Neighborhoodo A Prosperous EconomyA A Clean EnvironmentA Healthy and Safe CommunitiesSelf-Sufficient FamiliesS Children Ready for and Succeeding in SchoolC Parents and Other Adults Healthy and Self-SufficientP Elders Living with Dignity in Setting of Their Own Choice Benchmarks for a Better VT March 2012 18
  19. 19. Clear Language, continued • What do we want? • How will we recognize it? • How do things look now? • What will it take? • Who are the partners? • What is our role? What will STRATEGY we do to help achieve the An action taken to improve result? the RESULT • How good of a job are we PERFORMANCE doing? MEASURES How we know if strategiesBenchmarks for a Better VT March 2012 are working 19
  20. 20. Strategy Examples What we do to help achieve the desired result: •Program/Service Delivery •Collaboration •System Change •Service Integration •Funding •Resource Development/Leveraging Benchmarks for a Better VT March 2012 20
  21. 21. Different Levels of FocusRESULTS & INDICATORSare about the well being ofpopulations “All people are happy and healthy” Benchmarks for a Better VT March 2012 21
  22. 22. Different Levels of FocusSTRATEGIES &PERFORMANCE MEASURESare about the well being ofprogram participants / servicerecipients“Vaccinations for school children” Benchmarks for a Better VT March 2012 22
  23. 23. A quick test:RESULT, INDICATOR, or PERFORMANCEMEASURE? • Safe Community RESULT INDICATOR • Crime Rate PERF. MEASURE • Average Police Department response time RESULT • A community without graffiti INDICATOR • % of Montpelier buildings without graffiti RESULT • People have living wage jobs and income INDICATOR • % of people in Windsor County with living wage jobs and income PERF. MEASURE • % of participants in job training who get living wage jobs
  24. 24. Different Levels of Focus =Different Levels of Accountability1. Population Level - Focus = well being of population - Collective accountability: Many partners are responsible for achieving the desired result2. Program (Strategy) Level - Focus = well being of program participants/service recipients - Program accountability: Program is responsible for its performance Benchmarks for a Better VT March 2012 24
  25. 25. Who should be accountable?• Children in Brattleboro are ready for school• Children in grade 3 at St. Albans City School learn to read• Job training participants gaining new skills• Employment rate in Bennington County• Employment in Burlington’s New North End neighborhoods Benchmarks for a Better VT March 2012 25
  26. 26. What’s Appropriate Accountability?Who What Makes senseMany Partners All children are ready for school NO!!! Not appropriate!Program Makes sense Pre literacy skills in Head Start classroom Benchmarks for a Better VT March 2012 26
  27. 27. Bringing Two Levels TogetherSee Handout“Results Accountability: The WHOLE PICTURE” Benchmarks for a Better VT March 2012 27
  28. 28. Benchmarks for a Better VT March 2012 28
  29. 29. Benchmarks for a Better VT March 2012 29
  30. 30. Benchmarks for a Better VT March 2012 30
  31. 31. The Accountability Boundary_____________________________________________ ** ACCOUNTABILITY BOUNDARY **_____________________________________________•Why is this so important?•OK to cross –but have to be AWARE Benchmarks for a Better VT March 2012 31
  32. 32. How does RBA fit with:• Strategic planning• Outcomes evaluation• Logic Model• SSML• SWOT• PDCA Benchmarks for a Better VT March 2012 32
  33. 33. Break for Lunch Burning questions? Share them with the facilitators and fax to Amy See you again at 12:45 sharpAfter lunch we will focus in on:Program Accountability & Performance Measurement Benchmarks for a Better VT March 2012 33
  34. 34. REMINDER Different Levels of Focus =Different Levels of Accountability1. Population Level - Focus = well being of population - Collective accountability: Many partners are responsible for achieving the desired result2. Program (Strategy) Level - Focus = well being of program participants/service recipients - Program accountability: Program is responsible for its performance Benchmarks for a Better VT March 2012 34
  35. 35. REMINDER What’s the Big Deal?Common Sense Questions For each “thing” we do:•What do we want?•How will we recognize it?•How do things look now? •Who do we serve? •What do we do?•What will it take to get the results wewant? •How good of a job are we•Who are the partners with a role to play? doing? •What works to•What is our role in helping improve/strengthen?achieve the result? •What do we propose to do?•How good of a job are we doing? Benchmarks for a Better VT March 2012 35
  36. 36. Program Accountability PINK Handout• Who do we serve?• What do we do?• How good of a job are we doing? – How much do we do? – How well do we do it? – Is anyone better off? (Who? How so?)• What works to improve/strengthen?• What do we propose to do? Benchmarks for a Better VT March 2012 36
  37. 37. Program Accountability• Who do we serve?• What do we do?• How good of a job are we doing? 1. How much do we do? Performance 2. How well do we do it? Measures 3. Is anyone better off? (Who? How so?)• What works to improve/strengthen?• What do we propose to do? Benchmarks for a Better VT March 2012 37
  38. 38. Program Performance Measures COMMON EXAMPLESHow much?•# People served•# Hours service offered•# Activities (by type)How well?•Participant satisfaction•Use of best practice•Qualifications/training•External reviewAnyone better off?•Change in skills, knowledge, behavior, circumstance, well being•Possible sources: program records, participant survey, external data Benchmarks for a Better VT March 2012 38
  39. 39. Program Performance Measures EXAMPLEHigh SchoolHow much?•# studentsHow well?•Student teacher ratioAnyone better off?•NECAP scores•Graduation rates•% students entering higher education or employment aftergraduation Benchmarks for a Better VT March 2012 39
  40. 40. Program Performance Measures EXAMPLEWelfare to WorkHow much?•# people trainedHow well?•% child care and transportation needs metAnyone better off?•% employed 6 months after program completion Benchmarks for a Better VT March 2012 40
  41. 41. Program Performance Measures EXAMPLERaising ChildrenHow much?•# children, # hours spentHow well?•Parent training/education; use of best practice (child satisfaction??)Anyone better off?•Child health•Child learning achievement•Child happiness & well being Benchmarks for a Better VT March 2012 41
  42. 42. Performance MeasurementChallenges How do we know if anyone is better off? Required measures Lack of data Excess data Inappropriate data Inappropriate expectations Benchmarks for a Better VT March 2012 42
  43. 43. Choosing Performance Measures (1)• Answer all 3 questions• Choose what YOU want to know – What do we need to know to plan? – Evaluate our work? – Make it better?• Don’t think (too much) about others• This is about the program Benchmarks for a Better VT March 2012 43
  44. 44. Choosing Performance Measures (2)• Whole staff activity – Central part of planning, service delivery, budgeting, communications, reporting• Map out what you will measure & why• Look at what you already have• TRACK OVER TIME• TRY something. Did that work/help? Try something else. Benchmarks for a Better VT March 2012 44
  45. 45. GROUP ACTIVITYImaginary Program: Cavity Free RichmondWho do we serve?Residents of Richmond, all ages, who do not have access toaffordable dental care (e.g., underinsured, uninsured)What do we do?•Free dental care in clinic (preventive care & treatment)•Presentations in schools about good dental care habits•Outreach to build awareness of services Benchmarks for a Better VT March 2012 45
  46. 46. GROUP ACTIVITYImaginary Program: Cavity Free RichmondYour Task: Identify possible Performance MeasuresHow could this program measure:•How much it does?•How well it does it?•If anyone is better off?15 minutes for group workReport out from Newport, Rutland, St. Albans Benchmarks for a Better VT March 2012 46
  47. 47. More Examples Benchmarks for a Better VT March 2012 47
  48. 48. Example: Early Care & Education1. How much ?# children served; # receiving financial assistance2. How well?STARS ratingStaff training & qualifications, best practices3. Is anyone better off?Old measure: developmental statusNew measure: % needs addressed Benchmarks for a Better VT March 2012 48
  49. 49. Example: Youth Development1. How much ?# and characteristics of youth served2. How well?Licensing, external review, quality ratingStaff training & qualifications, best practices, standards3. Is anyone better off?Youth feel safe at the program.Youth feel connected to caring adults/role models.Youth for a Betterconnected to their community. Benchmarks feel VT 49March 2012
  50. 50. Example: Residential TreatmentIs anyone better off?How to define this?Complexity  SimplicityWhat do we really need to know?Benchmarks for a Better VT 50March 2012
  51. 51. WRAPPING UPPROGRAM ACCOUNTABILITY Benchmarks for a Better VT March 2012 51
  52. 52. POPULATION ACCOUNTABILITY YELLOW HandoutA few key points:•Indicators = data•Don’t get lost in the data. Focus on RESULT.•How to pick good indicators? There’s a chapter forthat. Let go of perfection.•When is Population Accountability useful? – Clarifying accountability – Showing relevance – Bringing together partners. Benchmarks for a Better VT March 2012 52
  53. 53. GROUP ACTIVITYPopulation Accountability ExerciseImaginary Problem:•Community upset about fires in Winooski•Fire Department budget questioned•Plan for community meeting Benchmarks for a Better VT March 2012 53
  54. 54. GROUP ACTIVITYPopulation Accountability ExerciseWhat we want? RESULTAll people in Winooski are safe from fireYour task today:Who are the partners with a role to play?10 minutesReport out from Montpelier & Bennington (if time) Benchmarks for a Better VT March 2012 54
  55. 55. Population Accountability: Examples Benchmarks for a Better VT March 2012 55
  56. 56. Population Accountability :Burlington Truancy ProjectStarted with a Problem:Burlington had highest drop out rate in VT.And it was going up.What will it take to turn this around?Who needs to be at the table? Benchmarks for a Better VT March 2012 56
  57. 57. Bur l i ngt on Tr uanc y Pr oj ec tShar ed ac c ount abi l i t y &par t ner s hi p• Desired Result All Burlington students graduate from high school• Indicator High school drop out rate• Baseline Not good! Highest dropout rate in VT Dropout Rate for Burlington School District Students 10.0% 9.0% 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% 96 98 99 01 03 04 06 07 08 95 97 00 02 05 19 19 20 20 20 20 20 19 19 19 20 20 20 20 Benchmarks for a Better VT March 2012 57
  58. 58. Bur l i ngt on Tr uanc y pr oj ec t ( c ont . )• What works? Address truancy to prevent drop out• Partners Non-profit service providers, school district, police department, school administrators, juvenile court judges, attorneys, child protection agencies, and United Way of Chittenden County• Strategies A coordinated response to truancy: - School District: new policies & procedures - Juvenile Court & SA : monthly court time - Non profits & state agencies: services that remove barriersBenchmarks for a Better VT 58March 2012
  59. 59. The Tr uanc y Pr oj ec t : KeyI ndi c at or Dropout Rate for Burlington School District Students10.0% 9.0% 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% 95 96 97 99 01 02 03 04 05 06 07 08 98 00 19 19 19 19 20 20 20 20 20 20 20 20 20 19Benchmarks for a Better VT 59March 2012
  60. 60. Example: Truancy Task Force &the EVEN BIGGER PictureWhat do we want? Youth succeed in schoolHow will we recognize it? School completion; CompetenciesWhat does it take? – Good educational opportunities – Youth attend school – Youth ready to learn Benchmarks for a Better VT March 2012 60
  61. 61. Example: Investing in Early Childhood(How UWCC fits in bigger picture) Result RESULT All children enter school ready to learn Indicator STRATEGY STRATEGY OTHER Kindergarten readiness 1 2 INFLUENCES UWCC Strategies (the roles we play) 1. Fund programs that provide parent support & education 2. Provide access to high quality early education 3. Volunteer programs supporting early literacy Benchmarks for a Better VT March 2012 61
  62. 62. RBA = ProcessWhat about Product? Benchmarks for a Better VT March 2012 62
  63. 63. 64
  64. 64. Reminder: Goals for Todayaka How I hope you will better off Brief Intro Results-Based Accountability (RBA) •Why bother? •What is it? ese? •How it can help you plan e co ver th Did w •How can it help you manage…and maximize…and communication •Where to get more information & resources If nothing else… 65
  65. 65. Key Features of RBA3 Performance Measure questions2 Levels of focus (& accountability)1 Simple framework to help focus Benchmarks for a Better VT March 2012 66
  66. 66. Remember:• Anyone can do this, at any time. Period.• This is a whole staff/whole organization process. All the time.• Don’t worry about perfection. Sketch it (don’t etch it).• Your program is not responsible for population level results.• You have a role to play in population level results. Know what that is.• Your program is responsible for helping its participants be better off. You have to know if and how.• Keep RBA questions in your pocket, bring them with you. Say, “How much? How well? Anyone better off?” Say, “Who is responsible for that? For this?”67
  67. 67. Feeling ready to give it a try?• Buy the book – “There’s a chapter for that!”• Check out the website• Apply for the Performance Institute• Start asking and answering the questions – Staff meeting – Board meeting – Coalition meeting• JUST DO IT. Benchmarks for a Better VT March 2012 68
  68. 68. Any questions?bbvt@marlboro.eduhttp://bbvt.marlboro.edu/ Benchmarks for a Better VT March 2012 69

×