Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Determining and demonstrating value rebecca jones and moe hosseini

843 views

Published on

Vancouver symposium slides

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Determining and demonstrating value rebecca jones and moe hosseini

  1. 1. Determining & Demonstrating Value - with the Logic Model Moe Hosseini-Ara, City of Markham mhosse@markham.ca Rebecca Jones, Brampton Public Library rjones@bramlib.on.ca
  2. 2. TO WHOM ?
  3. 3. Ivanna U.R. Mony • Socially connected • Lobbying to have library budget reallocated to other projects • Has never used the library • Buys all her booksfrom Amazon • Why does the library need so many staff? And why are they paid so much?
  4. 4. Dr. Ural Doomed • Libraries days are numbered • – Everyonehas computers and internet – – Research happens in facultiesnot libraries • With eBooksand research online, believes facilities can be made smaller no need for print material
  5. 5. CFO Roi (Roy) Bottomline, MBA • All about the bottomline • Revenue and investments • How do your services support revenue generation? • revenue are a drain on the system • it!
  6. 6. Kee Paiper • Influential in many circles • Has deep seated beliefs in traditional library services and importance of paper for legal profession • Libraries are institutionsfor archives & research
  7. 7. Understand the Context
  8. 8. Know the Context & the Contexters How do they measuretheir progress? Know your Stakeholders who arethey? How is value defined, measured& communicated?
  9. 9. Symptoms Underlying causes
  10. 10. the water causing the symptoms? • • performance • Difficultand complex • decision-makers
  11. 11. • Identifying & illustrating value depends on conversations with your stakeholders •
  12. 12. & mental models
  13. 13. How should we frame this issue?
  14. 14. Know the language of your context MetricsValue Customer Constituent Student
  15. 15. Start with stakeholders Name Position How do they measure their progress? Goals & objectives What do we know about them? What type of stakeholder are they? What do they know about our services? Critical that you understand the context in which the measuresareconsidered. 1. In your organization,how is value defined, measuredand conveyed? 2. Who arethe stakeholderswho must understand the value of your services? EXERCISE
  16. 16. The Logic Model
  17. 17. The Logic Model goes beyond documenting whatyou did and measures whatdifference you made in the life of changed
  18. 18. Measuring Impact/Value Change in the human condition? • Behavior • Skill • Knowledge • Attitude • Circumstances • Awareness • Motivation • Condition • Status
  19. 19. INPUT resource perspective OUTPUT operational perspective OUTCOME user perspective IMPACT stakeholder perspective
  20. 20. IMPACT Equipment INPUT Resource Perspective  Equipment  Space  Supplies  Funding  Staff
  21. 21. IMPACT Equipment OUTPUT Operational Perspective  Training module  Program  Report  # of Program Attendees  # of Programs Held
  22. 22. IMPACT OUTCOME User Perspective  Increased skill  Know-how or know-that  Change in status  Change in behavior or attitude
  23. 23. IMPACT IMPACT Stakeholder Perspective  Long term results  Lower dropout rates  Increased employment levels  Increased recovery rates
  24. 24. Job Skills Training Input (resources) • 10 hours of staff time • $100 budget required to create the program (supplies and handouts) Output (operations) • 2 hour job skills program created for student job seekers • 4 programs offered over a period of a month • total of 107 participants in attendance Outcome (user) • Students gained resume and cover letter writing skills • Knowledge of where to look for jobs and submit applications • Ability to give interviews, proficiency in answering interview questions Impact (stakeholder) • 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements • University can claim that students are able to get jobs following graduation
  25. 25. Job Skills Training Input (resources) • 10 hours of staff time • $100 budget required to create the program (supplies and handouts) Output (operations) • 2 hour job skills program created for student job seekers • 4 programs offered over a period of a month • total of 107 participants in attendance Outcome (user) • Students gained resume and cover letter writing skills • Knowledge of where to look for jobs and submit applications • Ability to give interviews, proficiency in answering interview questions Impact (stakeholder) • 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements • University can claim that students are able to get jobs following graduation
  26. 26. Job Skills Training Input (resources) • 10 hours of staff time • $100 budget required to create the program (supplies and handouts) Output (operations) • 2 hour job skills program created for student job seekers • 4 programs offered over a period of a month • total of 107 participants in attendance Outcome (user) • Students gained resume and cover letter writing skills • Knowledge of where to look for jobs and submit applications • Ability to give interviews, proficiency in answering interview questions Impact (stakeholder) • 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements • University can claim that students are able to get jobs following graduation
  27. 27. Job Skills Training Input (resources) • 10 hours of staff time • $100 budget required to create the program (supplies and handouts) Output (operations) • 2 hour job skills program created for student job seekers • 4 programs offered over a period of a month • total of 107 participants in attendance Outcome (user) • Students gained resume and cover letter writing skills • Knowledge of where to look for jobs and submit applications • Ability to give interviews, proficiency in answering interview questions Impact (stakeholder) • 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements • University can claim that students are able to get jobs following graduation
  28. 28. Job Skills Training Input (resources) • 10 hours of staff time • $100 budget required to create the program (supplies and handouts) Output (operations) • 2 hour job skills program created for student job seekers • 4 programs offered over a period of a month • total of 107 participants in attendance Outcome (user) • Students gained resume and cover letter writing skills • Knowledge of where to look for jobs and submit applications • Ability to give interviews, proficiency in answering interview questions Impact (stakeholder) • 80% of the participants who attended the program reported that they were either able to get interviews or actual job placements • University can claim that students are able to get jobs following graduation
  29. 29. BEGINWITH THEIN MIND END Covey 1989
  30. 30. Public Library Example Input (resources) • Staffing; Program Design; Marketing Material, Facilities; Equipment • Speakers = 6 female professionals working in STEM careers scientists, astronauts, engineers, doctors, programmers • $1,500 to pay for honorariums and supplies Output (operations) • 6 week program offered twice a year • 25 girls ages 10-12 complete program each session (50 participants per year) • Pre and Post-session questionnaire; (permission to follow-up in 3 years) Outcome (user) • Girls ages 10-12 have a better understanding of STEM careers • Increased interest in sciences & maths as potential course options • Positive response to questionnaire that the girls want to be scientists, astronauts, engineers, programmers, etc. Impact (stakeholder) • Girls ages 10-12are interested in pursing STEM careers (based on questionnaire response (short-term) • These girls take maths, sciences, computers in high school and pursue degrees in STEM in university (mid-term) • Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long- term)
  31. 31. Public Library Example Input (resources) • Staffing; Program Design; Marketing Material, Facilities; Equipment • Speakers = 6 female professionals working in STEM careers scientists, astronauts, engineers, doctors, programmers • $1,500 to pay for honorariums and supplies Output (operations) • 6 week program offered twice a year • 25 girls ages 10-12 complete program each session (50 participants per year) • Pre and Post-session questionnaire; (permission to follow-up in 3 years) Outcome (user) • Girls ages 10-12 have a better understanding of STEM careers • Increased interest in sciences & maths as potential course options • Positive response to questionnaire about what the girls want to be - scientists, astronauts, engineers, programmers, etc. Impact (stakeholder) • Girls ages 10-12are interested in pursing STEM careers (based on questionnaire response (short-term) • These girls take maths, sciences, computers in high school and pursue degrees in STEM in university (mid-term) • Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long- term)
  32. 32. Public Library Example Input (resources) • Staffing; Program Design; Marketing Material, Facilities; Equipment • Speakers = 6 female professionals working in STEM careers scientists, astronauts, engineers, doctors, programmers • $1,500 to pay for honorariums and supplies Output (operations) • 6 week program offered twice a year • 25 girls ages 10-12 complete program each session (50 participants per year) • Pre and Post-session questionnaire; (permission to follow-up in 3 years) Outcome (user) • Girls ages 10-12 have a better understanding of STEM careers • Increased interest in sciences & maths as potential course options • Positive response to questionnaire that the girls want to be scientists, astronauts, engineers, programmers, etc. Impact (stakeholder) • Girls ages 10-12are interested in pursing STEM careers (based on questionnaire response (short-term) • These girls take maths, sciences, computers in high school and pursue degrees in STEM in university (mid-term) • Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long- term)
  33. 33. Public Library Example Input (resources) • Staffing; Program Design; Marketing Material, Facilities; Equipment • Speakers = 6 female professionals working in STEM careers scientists, astronauts, engineers, doctors, programmers • $1,500 to pay for honorariums and supplies Output (operations) • 6 week program offered twice a year • 25 girls ages 10-12 complete program each session (50 participants per year) • Pre and Post-session questionnaire; (permission to follow-up in 3 years) Outcome (user) • Girls ages 10-12 have a better understanding of STEM careers • Increased interest in sciences & maths as potential course options • Positive response to questionnaire that the girls want to be scientists, astronauts, engineers, programmers, etc. Impact (stakeholder) • Girls ages 10-12are interested in pursing STEM careers (based on questionnaire response (short-term) • These girls take maths, sciences, computers in high school and pursue degrees in STEM in university (mid-term) • Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long- term)
  34. 34. Academic Library Example Input (resources) • One liaison librarian Output (operations) • Faculty research support for a Grant Application • Completed application submitted on time Outcome (user) • Grant dollars received by faculty • Faculty member is able to hire 2 grad students to conduct research Impact (stakeholder) • Increased university reputation • A number of articles are submitted and accepted by academic journals, increased citations for faculty member • University increases visibility.
  35. 35. Government Library Example Input (resources) • $100K resource budget • Staff to advise on development of policies Output (operations) • 2500 resources utilized by 75% of department employees • Library has contributed to the development of 6 policies in past 12 months Outcome (user) • 6 policies approved and in place • Eliminate department-specific resource purchases saving the organization $35K Impact (stakeholder) • Credible policies • - by at least 25% (as reported during policy debriefings) • Reduced costs to run department
  36. 36. Corporate Library Example Input (resources) • $80K electronic content contract • departments applications • Skilled info pro to develop research portfolio for businessdevelopment department Output (operations) • Customized research portfolio in response to requestfor background on deal between XYZ company & ABC government agency, including specific data on terms how ABC likes deal structured • Library resources are integrated into businessdevelopment applications Outcome (user) • Client extrapolated how ABC agency likes to see deals structuredand structureddeal appropriately Impact (stakeholder) • Organization successfullycompleted a deal with ABC government agency for $10M, the 1stin 5 years
  37. 37. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs Define Measures Manage Measurement Data Translate Data into Outcomes & Impacts Communicate Results
  38. 38. Align Strategies & Objectives
  39. 39. Align Strategies & Objectives Your Goals & Objectives: How they Contribute Accelerate newcomers contribution to the community & economy Offer programs and services that connect newcomers with relevant agencies and services Increased research grants Support the research process and assist with submissionand completion of grants Newcomers contributing to community economically Welcome newcomers with links to language, literacy, employment & social supports
  40. 40. Align objectives • Articulate & document yourgoals & objectives Whatareyoutryingtoaccomplish? • Clarifyhow these contribute towards desired outcomes EXERCISE
  41. 41. Identify Services & Programs
  42. 42. Identify Services What services,programsor productscontributeto yourrealizing an objective? Stakeholder Goals & Objectives Your Goals & Objectives Your Services EXERCISE
  43. 43. Define Measures
  44. 44. What measures do you keep now?
  45. 45. • No one magic measure • Successful organizations: • clarity of purpose • understand their culture • performance measurement system that fits that culture • alternatives • Identifying outcomes & especially impact takes a long time & heavy duty analysis
  46. 46. Value SatisfactionOperational Three Types of Measures
  47. 47. Operational/Usage Measures • • We also need to look at differences: • Peaks? Dips? Switches? • What are these statistics really telling us? Or, not? • Who needs to see these statistics? • Are they telling a story that you want to tell? Operational
  48. 48. Customer Satisfaction Measures • Tends to focus on existing products and/or services • Usually measured through surveys • Are we also measuring importance? • Survey burnout, plus the expectation is to score all Satisfaction
  49. 49. • Are we adding value? • What difference are we making? • What is the impact of our services and/or programs? • How are we contributing to the Value
  50. 50. What are meaningful measures?  Matter to you AND your stakeholders  language  Focus attention on what is most important for the organization  Are critical for managing, planning & decision-making  Are organization-dependent
  51. 51. Measure: info about what is
  52. 52. Define success • What will success look like? • How will you track progress towardsmeeting goals? • What indicators or measures will you use? • Qualitative? Quantitative? • What data needs to be collected? • Keep it key • Keep it simple • Be practical • or overworkfrontlinestaff
  53. 53. Service Impact Inputs Person power, money, content, etc. Activity What you do Outputs What the activity produces Outcomes How the client uses the output Impact What changes for the client Outcome Measures Impact Measures EXERCISE
  54. 54. Service Impact Inputs Person power, money, content, etc. Activity What you do Outputs What the activity produces Outcomes How the client uses the output Impact What changes for the client Outcome Measures Impact Measures EXERCISE
  55. 55. Manage Measurement Data
  56. 56. Manage collection • Determine a collection schedule • Work out agreements with other departments that have needed data • Are you currently collecting data or indicatorsthat are no longer relevant? • How long do you need to keep data? • Who is responsible? EXERCISE
  57. 57. Translate Data into Outcomes & Impacts
  58. 58. Define outcomes & impact • What will success look like? • OUTCOME IMPACT INPUT OUTPUT
  59. 59. Define outcomes & impact OUTCOME User Perspective  Increased skill  Know-how or know-that  Change in status  Change in behaviour or attitude IMPACT Stakeholder Perspective  Longer term results  Lower dropout rates  Increased employment levels  Increased recovery rates
  60. 60. Measuring Impact/Value Change in the human condition? • Behavior • Skill • Knowledge • Attitude • Circumstances • Awareness • Motivation • Condition • Status
  61. 61. Interpret data • • • Examine it from various angles • What is the progress towards the goals & objectives? EXERCISE
  62. 62. Communicate the Results
  63. 63. Remember us ?
  64. 64. Communicate results • Actuallybegins when you are understanding the context • Your message: ehave contributedtowardsyourgoals • If your goals & objectivesare meaningful for your planning & decision-making, your measures will be indispensable EXERCISE
  65. 65. c Value SatisfactionOperational Sweet Spot
  66. 66. What do you need to stop doing?
  67. 67. Resources IMLS Webography http://www.imls.gov/applicants/webography.aspx Shaping Outcomes http://www.shapingoutcomes.org/index.htm Overcoming Our Habits and Learning to Measure Impact by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc. June 2013. http://www.infotoday.com/cilmag/jun13/Hosseini-Ara_Jones--Overcoming-Our-Habits-and-Learning-to- Measure-Impact.shtml Seven Easy Steps to Help You Measure Up by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc. July/August 2013. http://www.infotoday.com/cilmag/jul13/Hosseini-Ara_Jones--Seven-Easy-Steps-to-Help-You-Measure-Up.shtml
  68. 68. REBECCA JONES MOE HOSSEINI-ARA & thanks!rjones@bramlib.on.ca mhosse@markham.ca

×