Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using the Logic Model for Impact & Success; #SLA2017


Published on

Given at the SLA Conference in Phoenix 2017, an overview of the logic model to measure the outcomes of information services & programs from the decision-makers' perspectives.

Published in: Data & Analytics
  • Be the first to comment

Using the Logic Model for Impact & Success; #SLA2017

  1. 1. UsingtheLogicModel forImpact&Success SLA2017Monday@10:30 Rebecca Jones, Director, Branches & Neighbourhood Services @ Brampton Library 2017
  2. 2. My lens •must use evidence for decision-making • Healthy blend of evidence, emotion & risk •key questions: what, so what, now what •crucial to start with where you are, plan small progressive steps that won’t kill your Achilles’ heel •everyone’s lens looks through their mental models
  3. 3. Logically…...
  4. 4. Our logic isn’t everyone’s logic
  5. 5. The Logic Model Where we hope many miracles occur Goodwork, but we may needa few more details right here….
  6. 6. Impact & Success…...
  7. 7. Impact according to who ?
  8. 8. Board Chair Kim Kardforyour Mony • Socially connected • Lobbying to have library budget reallocated to other projects • Has never used the library • Buys all her books from Amazon • Why does the library need so many staff? And why are they paid so much?
  9. 9. VP Research Dr. Ural Doomed • Libraries days are numbered • Libraries aren’t relevant • Everyone has computers and internet • Need information? It’s all online • Research happens in faculties not libraries • With edocs, ebooks, open data and research online, believes facilities can be made smaller – no need for print material
  10. 10. CFO Roi Bottomline • All about the bottomline • Revenue and investments • How do your services support revenue generation? • Services that don’t generate revenue are a drain on the system • If it’s not efficient, outsource it!
  11. 11. Dean of Humanities Kee Paiper • Influential in many circles • Has deep seated beliefs in traditional library services and importance of paper • Libraries are institutions for archives & research • Needs to demonstrate to college that humanities still contributes to reputation
  12. 12. Logic model in action for Dr. Papier Input (resources) • One liaison librarian Output (operations) • Faculty research support for a Grant Application • Completed application submitted on time Outcome (user) • Grant dollars received by faculty • Faculty member is able to hire 2 grad students to conduct research Impact (stakeholder) • Increased university visibility in research area • Enhanced university reputation • A number of articles are submitted and accepted by academic journals, increased citations for faculty member
  13. 13. Project Outcomes: Jones & Raven 14 Resource Perspective What we put in Operational Perspective What we deliver User Perspective What comes out of it for customers Stakeholder Perspective What changes for customers input outcome output impact
  14. 14. Project Outcomes: Jones & Raven 15
  15. 15. Customer Outcome for Client
  16. 16. Stakeholders 17 Impact for Stakeholder
  17. 17. “ BEGINWITH THE IN MIND END Covey 1989
  18. 18. Stakeholders 19 Impact for Stakeholder
  19. 19. Customer Outcome for Client
  20. 20. Project Outcomes: Jones & Raven 21
  21. 21. start with where you are
  22. 22. Our “frames” & mental models Our “frames” & mental models
  23. 23. How should we frame this issue? How should we frame impact & success?
  24. 24. Outcomes Change Growth
  25. 25. It’s hard
  26. 26. What measures do you keep now?
  27. 27. • No one magic measure • “Impact” or value is dynamic, economic, psychological & relative to alternatives • Identifying outcomes & especially impact takes a long time & heavy duty analysis • Successful organizations: • clarity of purpose • understand their culture • performance measurement system that fits that culture
  28. 28. Operational/Usage Measures • We’ve long tracked “how much” • We also need to look at differences: • Peaks? Dips? Switches? • What are these statistics really telling us? Or, not? • Who needs to see these statistics? • Are they telling a story that you want to tell? Operational
  29. 29. Satisfaction Measures • Tends to focus on existing products and/or services • Usually measured through surveys • Are we also measuring importance? • Survey burnout, plus the expectation is to score all “excellent” or “10 out of 10” Satisfaction
  30. 30. Value Measures • What do you have? • Qualitative? • Testimonials, attributes • Quantitative? • Budget Value
  31. 31. Value Satisfaction Operational Focus on the sweet spot 72 % of First-year Students engaged with Library staff services in Fall 2017 semester 85% of these Students rated these services as very important in addressing their workload & assignment completion 70% of all Students used the Library in Fall 2017 83% of all these Students rated the Library as the 2nd most important service related to academic success on campus, 2nd only to their faculty
  32. 32. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs Define Measures Manage Measurement Data Translate Data into Outcomes & Impacts Convey Results
  33. 33. Understand the Context
  34. 34. Know the Context (& the Contexters) How do they measure their progress? Know your Stakeholders – who are they? How is value or impact defined, measured & communicated?
  35. 35. Know the language of your context Metrics Value Customer Growth Student Sustain Residents Constituent
  36. 36. We can't come to terms with new concepts until we learn new language Horseless Carriages Car Phones ATM’s or PTB
  37. 37. Start with stakeholders Name Position How do they measure their progress? Goals & objectives What do we know about them? What type of stakeholder are they? What do they know about our services? Critical that you understand the context in which the measures are considered. 1. In your organization, how is value defined, measured and conveyed? 2. Who are the stakeholders who must understand the value of your services?
  38. 38. The Logic Model …goes beyond documenting what you did and measures what difference you made in the life (work, reputation, etc. )of your target audience… What has been the positive change in their sphere… as a result of what you do?
  39. 39. Measurement Framework Understand the Context Align Strategies & Objectives
  40. 40. It’s logical that: INPUT resource perspective OUTPUT operational perspective OUTCOME user perspective IMPACT stakeholder perspective
  41. 41. IMPACT Equipment INPUT Resource Perspective • Equipment • Space • Supplies • Funding • Staff
  42. 42. IMPACT Equipment OUTPUT Operational Perspective • Training sessions completed • # of attendees • Grant application • Reports
  43. 43. IMPACT OUTCOME User Perspective • Increased skill to do • More or better know-how or know-that • Change in status • Change in behavior or attitude (confidence)
  44. 44. IMPACT IMPACT Stakeholder Perspective • Long term results • Lower dropout rates • Increased contract engagements • Increased recovery rates
  45. 45. Align Strategies & Objectives Stakeholder’s Goals & Objectives Your Goals & Objectives: How they Contribute Accelerate product to market Catalyze product development Improve market share in specific industry segments Customize segment dashboards for business development Increase research grants in xxx Underpin the research process and assist with submission and completion of grants
  46. 46. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs
  47. 47. Identify Services What services, programs or products contribute to your realizing an objective? Stakeholder Goals & Objectives Your Goals & Objectives Your Services
  48. 48. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs Define Measures
  49. 49. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs Define Measures Manage Measurement Data
  50. 50. Manage Measurement Data • What will success look like? • How will you track progress towards meeting goals? • What indicators or measures will you use? • Qualitative? Quantitative? • What data needs to be collected? • Keep it key • Keep it simple • Be practical • Don’t get mired – or overwork frontline staff
  51. 51. Manage collection • Determine a collection schedule • Work out agreements with other departments that have needed data • Are you currently collecting data or indicators that are no longer relevant? • How long do you need to keep data? • Who is responsible?
  52. 52. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs Define Measures Manage Measurement Data Translate Data into Outcomes & Impacts
  53. 53. Define outcomes & impact • What will success look like? • And how will you “get” there? OUTCOME IMPACT INPUT OUTPUT
  54. 54. Government Library Example Input (resources) • $100K resource budget • Staff to advise on development of policies Output (operations) • 2500 resources utilized by 75% of department employees • Library has contributed to the development of 6 policies in past 12 months Outcome (user) • 6 policies approved and in place • Eliminate department-specific resource purchases saving the organization $35K Impact (stakeholder) • Credible policies • Reduce policy duplication, “blind-sides” or contradictory policies by at least 25% (as reported during policy debriefings) • Reduced costs to run department
  55. 55. Corporate Library Example Input (resources) • $80K electronic content contract • Tech and applications to integrate library’s products and resources into the departments applications • Skilled info pro to develop research portfolio for business development department Output (operations) • Customized research portfolio in response to request for background on deal between XYZ company & ABC government agency, including specific data on terms how ABC likes deal structured • Library resources are integrated into business development applications Outcome (user) • Client extrapolated how ABC agency likes to see deals structured and structured deal appropriately Impact (stakeholder) • Organization successfully completed a deal with ABC government agency for $10M, the 1st in 5 years
  56. 56. Measurement Framework Understand the Context Align Strategies & Objectives Identify Services & Programs Define Measures Manage Measurement Data Translate Data into Outcomes & Impacts Convey Results
  57. 57. Communicate to… Remember us ?
  58. 58. Convey results • Actually begins when you are understanding the context • Your message: “wehave contributed towards your goals by……” • If your goals & objectives are meaningful for your planning & decision-making, your measures will be indispensable
  59. 59. c Value SatisfactionOperational Sweet Spot
  60. 60. Logically, we need to reallocate our time and attention
  61. 61. REBECCAJONES Thank you!