Real world nonprofit outcomes measurement - Mass Nonprofit Network Oct 29 2013 10 2013

313 views
270 views

Published on

Workshop by Steve Backman and Ellen Bass at annual Mass Nonprofit Network summit

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
313
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • OM vs PM??
  • Outcomes are the profit of the nonprofit sector; results. Outcomes are our reason to exist, to be in business
    Just like a for-profit manages to maximize $ profit, a nonprofit can manage to maximize our profit: outcomes
    How can you maximize something if you can’t even define it? Or measure it?
    Define effective organization: One able to improve its outcomes over time
    Performance management is the term used nationally
    MNN uses outcomes management to distinguish between HR
    PM includes HR
  • We will not teach/train you how to do any of this.
  • This is the framework – reflect on it
    (If there are challenges, put them in parking lot until end)
    MEASURE AND ANALYZE OUTCOMES
    Have a culture that values the use of data in making decisions about the constituents
    Have a process for examining the data and understanding what it is saying
    Have a process for making necessary changes based on the data
    We are convinced that any nonprofit that wants to can define its most important outcomes clearly and can learn to manage to improve them.
    You can build the individual steps needed to get to these behaviors
    Can Small to Medium Sized Agencies Create this Culture?
  • Normally we work with 15 agencies over 2 years to build 17 practices through a combination of group sessions individualized consultation
    Self-assessment or outside consultant
    4-point scale with specific indicators
    High performing agencies say takes 3-10 years to incorporate these practices into your agency’s DNA
    Culture change
  • What did you learn? Take 2 comments
  • Who are the OM champions? How inspired are the leaders to adopt outcomes management?
    Board
    Executive Director
    Program staff
    Resource development staff
    What should characterize your agency related to outcomes management? Why? What should your agency look like?
    With which of the champions do you have the best relationship? Who are the early influencers?
    Takes 2 years
    Measure only the data you need to ensure success
  • If you want this:
    You will need outside help to articulate outcomes, select tools, hold yourselves accountable, envision and manage change
    If you want it, you can do it!
    Persuade your funders that this is important, get the help you need to build and sustain your system
    Replicate or scale effective practice: New Profit, Social Innovation Fund
    We hope many agencies in MA can take advantage of Pay for Success
  • Real world nonprofit outcomes measurement - Mass Nonprofit Network Oct 29 2013 10 2013

    1. 1. Real World Introduction to Outcomes Management Massachusetts Nonprofit Network Conference Tuesday, October 29, 2013 Steve Backman, Database Designs Ellen Bass, Capacity Institute Prepared by Ellen Bass and Steven Backman. You may use and adapt this under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    2. 2. Outcomes are Changes in People’s Lives  Initial outcomes (direct result of program activities)  new knowledge  increased skills  changed attitudes or values  Intermediate outcomes (significant milestone or graduation requirement)  modified behavior  Long-term outcomes  improved condition  altered status 10/29/2013 Real World Introduction to Outcomes Measurement 2
    3. 3. Nonprofit Bottom Line Our Their Outcomes = Profit 10/29/2013 Real World Introduction to Outcomes Measurement 3
    4. 4. Who’s In the Room? Ellen, Steve… and you! 10/29/2013 Real World Introduction to Outcomes Measurement 4
    5. 5. Workshop Agenda  Introductory (10 min)  Define performance management (15 min)  Role of mission statement in PM (10 min)  Role of logic model in PM, criteria (15 min)  Client data tracking systems (30 min)  Your next steps (5 min)  Total 85 min 10/29/2013 Real World Introduction to Outcomes Measurement 5
    6. 6. Workshop Goals  Understand elements of outcome management system  Understand your organization’s status and next steps in building outcome management system  Help all of us contribute to centering nonprofit work on improving participant outcomes 10/29/2013 Real World Introduction to Outcomes Measurement 6
    7. 7. Your Take-Aways Today  Assessment of your agency’s PM system  Assessment of your mission statement  Know how to assess a logic model  One piece of mission-critical measurable outcomes data for your agency  Assessment of your agency’s data system  Your action steps to strengthen PM in your agency 10/29/2013 Real World Introduction to Outcomes Measurement 7
    8. 8. Four Assessments  Benchmark Assessment Tool (PM)  Mission Statement Assessment  Logic Model Assessment  Data System Tools 10/29/2013 Real World Introduction to Outcomes Measurement 8
    9. 9. Nonprofit Performance Management System  Definition: What we do to ensure improved effectiveness, or measurable intermediate participant outcomes  Four big questions:  What participant outcomes are we aiming for?  How well are we doing?  With whom must we partner?  How can we improve? 10/29/2013 Real World Introduction to Outcomes Measurement 9
    10. 10. What Does an Effective Nonprofit Agency Look Like?  Improved participant outcomes  Use outcomes data to drive decision- making about programs, staff, partnerships and resources  Measure and analyze outcomes  Use learning to guide next cycle of work 10/29/2013 Real World Introduction to Outcomes Measurement 10
    11. 11. What does a useful Mission Statement look like?  Tells why agency is in business; definition of success  Clear  Concise  Compelling  Concrete  Should describe target population and outcome  Example: “Beat Rome” 10/29/2013 Real World Introduction to Outcomes Measurement 11
    12. 12. Role of Mission Statement in Outcomes Management  Focuses agency resources on priority activities  Guides staff to make strategic decisions  Tells if we are successful or not  How strong is your agency’s mission statement?  Take the Assessment for your agency  5 min 10/29/2013 Real World Introduction to Outcomes Measurement 12
    13. 13. Benchmark Assessment Tool  Purpose: help you build your performance management system from ground up  17 effectiveness practices  Pre- and post-test  Goal: level 4 practice  General or youth program 10/29/2013 Real World Introduction to Outcomes Measurement 13
    14. 14. Benchmark Assessment Tool  Pair up with a colleague  Take the Assessment for your agency  10 mins.  Questions? 10/29/2013 Real World Introduction to Outcomes Measurement 14
    15. 15. What is a Logic Model?  One page conceptual map showing flow of influence  Shows how program helps participants achieve outcomes  Target population, inputs, activities, outputs, outcomes  Outcome measurement plan (separate) to track only most important outcome 10/29/2013 Real World Introduction to Outcomes Measurement 15
    16. 16. Role of Logic Model in Performance Management  Identify specific target participant  Clarify logical expected outcomes progression  Clarify program strategy necessary to help participants reach outcomes  Provide some indication that program provides realistic dosage and duration to lead to outcomes  Help with program planning and/or improvement 10/29/2013 Real World Introduction to Outcomes Measurement 16
    17. 17. Logic Model Criteria 1. Logical flow activities through outcomes? 2. Outcomes clearly benefits to participants? 3. Program accountable for initial & intermediate outcomes? 4. Longest term outcomes meaningful, influenced by program? 5. Appropriate stakeholders included? 6. Inputs, activities, outputs, outcomes are clear, comprehensive, and detailed? 10/29/2013 Real World Introduction to Outcomes Measurement 17
    18. 18. Outcome Measurement Plan  Select most important outcome to measure  Identify indicators to signal outcome has been achieved  Select/create measurement tool and data collection process 10/29/2013 Real World Introduction to Outcomes Measurement 18
    19. 19. Effective Client Tracking Systems 10/29/2013 Real World Introduction to Outcomes Measurement 19
    20. 20. What does an effective Client Tracking System look like?  Intake: Know your community  Enroll: Demographics and cohort  Manage: Logic Model activities  Measure: Measure outcomes  Analyze: Up and down the organization 10/29/2013 Real World Introduction to Outcomes Measurement 20
    21. 21. Think in Cycles 10/29/2013 Real World Introduction to Outcomes Measurement 21
    22. 22. Think Holistically Community Partners Referral Sources Donors, Funders Leadership , Board Staff, Volunteers Referral resources Advocacy, Media 10/29/2013 Real World Introduction to Outcomes Measurement 22
    23. 23. What system do I have? What do I need?  Basic List Management (Excel, Google)  Custom database (Access, Filemaker)  Customizable off the shelf (Salesforce, CiviCRM)  General Human Services (ETO)  Specialized Human Services (KidTrax) 10/29/2013 Real World Introduction to Outcomes Measurement 23
    24. 24. Look for the Secret Spreadsheets Positives Negatives Easy, Quick Flattens the data Low Tech Hinders comparisons in time Self Managed Hinders holistic view Adapt to change easily Lots of hand counting 10/29/2013 Real World Introduction to Outcomes Measurement 24
    25. 25. Client Tracking System 10/29/2013 Real World Introduction to Outcomes Measurement 25 ParticipantParticipant Enroll 1Enroll 1 ClassClass CaseCase Program A SY13 Site 3 Stacey Referral 10/1/13 Marie Health Cntr Health Careers Oct-Nov 13 Peter Roosevelt HS
    26. 26. Participation Differences 10/29/2013 Real World Introduction to Outcomes Measurement 26 ParticipantParticipant Enroll 2Enroll 2 CourseCourse CaseCase Program A SY13 Site 3 Stacey Referral 10/1/13 Marie Health Cntr Health Careers Oct-Nov 13 Peter Roosevelt HS ClassesClasses Enroll 1Enroll 1 CourseCourse CaseCase StepsStepsActivitiesActivities
    27. 27. Where to record outcomes 10/29/2013 Real World Introduction to Outcomes Measurement 27 ParticipantParticipant Enroll 2Enroll 2 CourseCourse CaseCase Program A SY13 Site 3 Stacey Referral 10/1/13 Marie Health Cntr Health Careers Oct-Nov 13 Peter Roosevelt HS ClassesClasses Enroll 1Enroll 1 CourseCourse CaseCase StepsStepsActivitiesActivities
    28. 28. Outcomes And Processes  Find measures that indicate outcomes  “Successful Referral”  Define processes behind the measures  “Steps in making a Referral”  Empower All Staff to monitor  “My referrals over a week old  Empower leadership  “Which case types get stuck most often?” 10/29/2013 Real World Introduction to Outcomes Measurement 28
    29. 29. Steps to a New System  Assess Needs  Select Software  Discovery/Planning  Build/Configure  Data Import  Reporting and Analytics  Train/Administer/Support 10/29/2013 Real World Introduction to Outcomes Measurement 29
    30. 30. Current System Assessment  Pair up with a colleague  Take the Assessment for your agency  10 mins.  Questions? 10/29/2013 Real World Introduction to Outcomes Measurement 30
    31. 31. Software Selection Tool  Using the Worksheet 10/29/2013 Real World Introduction to Outcomes Measurement 31
    32. 32. Ready to start planning?  Imagine project leader  Imagine planning team  Imagine executive sponsorship  Imagine board, development funding  Imagine fitting to annual calendar  Imagine a data manager  Imagine when to go live 10/29/2013 Real World Introduction to Outcomes Measurement 32
    33. 33. Your Next Steps to Advocate for Outcomes Management  Identify champions  Draft a change statement  Identify costs and benefits  Develop a strategy to win key people to your vision  Invest in outside assistance, long term  Keep your measurement plan simple 10/29/2013 Real World Introduction to Outcomes Measurement 33
    34. 34. Are You Ready for Performance Management?  Get ready: Read, learn, assess, plan  Build your system:  Identify funded peer cohort of organizations (Capacity Institute, Bridgespan)  Build agency system solo (David Hunter, FSG Social Impact Consultants, Wellspring Consulting)  Performance contracts (Pay for Success bonds) 10/29/2013 Real World Introduction to Outcomes Measurement 34
    35. 35. Useful books and reports  Leap of Reason, Mario Morino http://www.vppartners.org/leapofreason/getit  Working Hard and Working Well, David E. K. Hunter http://www.vppartners.org/leapofreason/get-working- hard-and-working-well  Software for Program Evaluation, Idealware http://idealware.org/reports/understanding-software- program-evaluation?key=10035930 10/29/2013 Real World Introduction to Outcomes Measurement 35
    36. 36. About Us Ellen Bass, ebass@bmaboston.org Capacity Institute of the Black Ministerial Alliance. bmaboston.org/node/4 Steve Backman, sbackman@dbdes.com Database Designs, dbdes.com 10/29/2013 Real World Introduction to Outcomes Measurement 36

    ×