Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
COUNT YOUR IMPACT:
GETTING STARTED WITH
OUTCOME-BASED
EVALUATION
Nicolle Steffen & Linda Hofschire
www.lrs.org | @lrs_co |...
DATA TRIFECTA
Inputs Outputs Outcomes
INPUT MEASURES
Quantifying the effort
expended on a program
Inputs
funds expended
number of staff
staff hours
OUTPUT MEASURES
Quantifying the level of
services and materials
provided
Outputs
number of registered participants
number of events offered
total attendance at events
number of minutes spent reading
circulation
OUTCOME MEASURES
Quantifying the effect a
service has on the
program’s stated objectives
Outcomes
Children’s enjoyment
of reading increased
children maintained
or improved reading skills
children were more
likely to read by choice
library developed long-term
partnership with local school district
DATA TRIFECTA
Inputs Outputs Outcomes
ACTIVITY: WHAT IS YOUR DATA
TRIFECTA?
Inputs Outputs Outcomes
ACTIVITY: WHAT IS YOUR DATA
TRIFECTA?
Outcome-Based Evaluation Project
Project name: Senior Computer Class Series
Descript...
Inputs Outputs Outcomes
Funds expended:
• Copies of
handouts
• Marketing
campaigns at
local senior
centers
Staff hours:
• ...
Library A Library CLibrary B
Library A
• 100% of
participating
children
increased or
maintained
reading skills
• 85% of
participating
children
increase...
Library A
• 100% of
participating
children
increased or
maintained
reading skills
• 85% of
participating
children
increase...
Library A
• 100% of
participating
children
increased or
maintained
reading skills
• 85% of
participating
children
increase...
Library A
• 100% of
participating
children
increased or
maintained
reading skills
• 85% of
participating
children
increase...
Inputs Outputs Outcomes
ACTIVITY RECAP: WHAT IS YOUR
DATA TRIFECTA?
IntermediateImmediate Long-term
OUTCOMES
IntermediateImmediate Long-term
OUTCOMES
Learn how
to complete
online job
application
IntermediateImmediate Long-term
OUTCOMES
Learn how
to complete
online job
application
Get an
interview
IntermediateImmediate Long-term
OUTCOMES
Learn how
to complete
online job
application
Get an
interview
Get a job
IntermediateImmediate Long-term
OUTCOMES
Learn how
to complete
online job
application
Get an
interview
Get a job
LINKING IMMEDIATE OUTCOMES TO
LONG-TERM IMPACT
Reading for pleasure associated with:
 Increased empathy
 Improved relati...
LINKING OUTPUTS TO LONG-TERM
IMPACT
Reading more associated with…
 Improved reading skills
 Better development of vocabu...
MEASURING
OUTCOMES
INDICATORS
Outcome Indicator
Increased
enjoyment of
reading
INDICATORS
Outcome Indicator
Increased
enjoyment of
reading
Parent response
to survey
question
INDICATORS
Outcome Indicator
Increased
enjoyment of
reading
Parent response
to survey
question
Increased digital
literacy
INDICATORS
Outcome Indicator
Increased
enjoyment of
reading
Parent response
to survey
question
Increased digital
literacy
...
After participating in the Summer Reading
Program, did your child’s reading habits change?
Please indicate if the followin...
While I was on a computer in the computer center
today, I . . .
SURVEY
 Looked for employment
(job search, resume,
applic...
Please let us know in your own words
the difference the 2014 Summer
Reading Program has made for your
child(ren) and famil...
As a first time parent it's really
helpful to have tangible ideas
about ways to get my child
engaged in early literacy ski...
I loved being able to spend time with my
child and bond over my love for books.
My son's reading endurance has
really grow...
Please tell us how the computer center
helps you or your community:
SURVEY—OPEN ENDED
QUESTIONS
Probably the most valuable
resource, dollar for dollar,
available to community. I have
found jobs…researched…
located tax ...
The access to the internet
afforded by the public library is
most probably the only reason I
am not completely bereft of a...
PLA-PROJECT OUTCOME
PLA-PROJECT OUTCOME
 Early Childhood Literacy
 Education/Lifelong Learning
 Digital Inclusion
 Job Skills
 Economic D...
IMPACT SURVEY
CONTINUUM
CONTINUUM
CONTINUUM
POSTCARD
PRIZE DRAWING
SKILL DEMONSTRATION
COMMENT WALL
ACTIVITY: OUTCOME INDICATORS
AND METHODS FOR MEASUREMENT
Outcome Indicators:
Outcome=Digital literacy
Indicator #1=Class a...
ACTIVITY: OUTCOME INDICATORS
AND METHODS FOR MEASUREMENT
How will you measure outcomes?
Skill demonstration (indicators li...
I’ve collected some
outcome data, now
how do I use it?
 Strategic Planning
 Management
 Communication
DATA USE
Area of strategic focus:
Nurture children’s learning by providing
programs and services that instill a love
of reading and...
After participating in the Summer Reading
Program, did your child’s reading habits change?
Please indicate if the followin...
Implications of OBE results for:
 Budgeting
 Staffing
 Programs (focus, number offered,
size, timing, etc.)
MANAGEMENT
Library A
• 100% of
participating
children
increased or
maintained
reading skills
• 85% of
participating
children
increase...
 Internal stakeholders
 External stakeholders
COMMUNICATION
ACTIVITY: OUTCOME INDICATORS
AND METHODS FOR MEASUREMENT
Identify uses for data in…
1) Strategic planning:
2) Management:
...
ACTIVITY: OUTCOME INDICATORS
AND METHODS FOR MEASUREMENT
Identify uses for data in…
1) Strategic planning: One of my libra...
ACTIVITY: OUTCOME INDICATORS
AND METHODS FOR MEASUREMENT
Identify uses for data in…
2) Managing: Are we achieving our desi...
ACTIVITY: OUTCOME INDICATORS
AND METHODS FOR MEASUREMENT
Identify uses for data in…
3) Communicating: Our mayor recently
a...
THANK YOU!
Nicolle Steffen & Linda Hofschire
www.lrs.org | @lrs_co | ripl.lrs.org
Count Your Impact: Getting Started with Outcome Based Evaluation
Upcoming SlideShare
Loading in …5
×

Count Your Impact: Getting Started with Outcome Based Evaluation

300 views

Published on

Presented at the CoPLA Workshop, 2015

Published in: Education
  • Be the first to comment

  • Be the first to like this

Count Your Impact: Getting Started with Outcome Based Evaluation

  1. 1. COUNT YOUR IMPACT: GETTING STARTED WITH OUTCOME-BASED EVALUATION Nicolle Steffen & Linda Hofschire www.lrs.org | @lrs_co | ripl.lrs.org
  2. 2. DATA TRIFECTA Inputs Outputs Outcomes
  3. 3. INPUT MEASURES Quantifying the effort expended on a program Inputs
  4. 4. funds expended
  5. 5. number of staff
  6. 6. staff hours
  7. 7. OUTPUT MEASURES Quantifying the level of services and materials provided Outputs
  8. 8. number of registered participants
  9. 9. number of events offered
  10. 10. total attendance at events
  11. 11. number of minutes spent reading
  12. 12. circulation
  13. 13. OUTCOME MEASURES Quantifying the effect a service has on the program’s stated objectives Outcomes
  14. 14. Children’s enjoyment of reading increased
  15. 15. children maintained or improved reading skills
  16. 16. children were more likely to read by choice
  17. 17. library developed long-term partnership with local school district
  18. 18. DATA TRIFECTA Inputs Outputs Outcomes
  19. 19. ACTIVITY: WHAT IS YOUR DATA TRIFECTA? Inputs Outputs Outcomes
  20. 20. ACTIVITY: WHAT IS YOUR DATA TRIFECTA? Outcome-Based Evaluation Project Project name: Senior Computer Class Series Description: My library offers a class series for seniors that covers basic computer /technology skills
  21. 21. Inputs Outputs Outcomes Funds expended: • Copies of handouts • Marketing campaigns at local senior centers Staff hours: • Curriculum development • Instruction Number of events offered Number of registrants Number of attendees Digital literacy Confidence using technology Ongoing partnerships with local senior centers
  22. 22. Library A Library CLibrary B
  23. 23. Library A • 100% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills Library CLibrary B
  24. 24. Library A • 100% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills Library CLibrary B • 25 children participated • 100 children participated • 50 children participated
  25. 25. Library A • 100% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills Library CLibrary B • 25 children participated • 100 children participated • 50 children participated • $5,000 • $10,000 • $2,500
  26. 26. Library A • 100% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills Library CLibrary B • 25 children participated • 100 children participated • 50 children participated • $5,000 • $10,000 • $2,500 • $200/child • $100/child • $50/child
  27. 27. Inputs Outputs Outcomes ACTIVITY RECAP: WHAT IS YOUR DATA TRIFECTA?
  28. 28. IntermediateImmediate Long-term OUTCOMES
  29. 29. IntermediateImmediate Long-term OUTCOMES Learn how to complete online job application
  30. 30. IntermediateImmediate Long-term OUTCOMES Learn how to complete online job application Get an interview
  31. 31. IntermediateImmediate Long-term OUTCOMES Learn how to complete online job application Get an interview Get a job
  32. 32. IntermediateImmediate Long-term OUTCOMES Learn how to complete online job application Get an interview Get a job
  33. 33. LINKING IMMEDIATE OUTCOMES TO LONG-TERM IMPACT Reading for pleasure associated with:  Increased empathy  Improved relationships  Reduced depression symptoms  Reduced risk of dementia The Reading Agency, 2015
  34. 34. LINKING OUTPUTS TO LONG-TERM IMPACT Reading more associated with…  Improved reading skills  Better development of vocabulary, writing, and grammar Chin & Krashen (2007)
  35. 35. MEASURING OUTCOMES
  36. 36. INDICATORS Outcome Indicator Increased enjoyment of reading
  37. 37. INDICATORS Outcome Indicator Increased enjoyment of reading Parent response to survey question
  38. 38. INDICATORS Outcome Indicator Increased enjoyment of reading Parent response to survey question Increased digital literacy
  39. 39. INDICATORS Outcome Indicator Increased enjoyment of reading Parent response to survey question Increased digital literacy Demonstrate ability to text, send an email with an attachment, etc.
  40. 40. After participating in the Summer Reading Program, did your child’s reading habits change? Please indicate if the following things increased, stayed the same, or decreased for your child: Increased Stayed the Same Decreased Enjoyment of reading    Reading skills    Reading by his/her choice    SURVEY
  41. 41. While I was on a computer in the computer center today, I . . . SURVEY  Looked for employment (job search, resume, application, etc.).  Communicated with someone (email, chat, Facebook, etc.).  Used the computer for entertainment (ex: videos, music, games).  Used government resources (licenses, taxes, unemployment, disability, welfare, immigration, etc.).  Did schoolwork.  Found health information.
  42. 42. Please let us know in your own words the difference the 2014 Summer Reading Program has made for your child(ren) and family: SURVEY—OPEN ENDED QUESTIONS
  43. 43. As a first time parent it's really helpful to have tangible ideas about ways to get my child engaged in early literacy skills. There were some recommended activities I hadn't thought of trying!
  44. 44. I loved being able to spend time with my child and bond over my love for books. My son's reading endurance has really grown this summer, thanks to the reading program. He was very motivated to read for 40 minutes and sometimes he read for even longer stretches.
  45. 45. Please tell us how the computer center helps you or your community: SURVEY—OPEN ENDED QUESTIONS
  46. 46. Probably the most valuable resource, dollar for dollar, available to community. I have found jobs…researched… located tax information, and have done school work over the years. Thank you!
  47. 47. The access to the internet afforded by the public library is most probably the only reason I am not completely bereft of any and all computer skills. Were it not for this access, as well as the assistance rendered via classes offered, I would most likely be unemployed if not unemployable.
  48. 48. PLA-PROJECT OUTCOME
  49. 49. PLA-PROJECT OUTCOME  Early Childhood Literacy  Education/Lifelong Learning  Digital Inclusion  Job Skills  Economic Development  Civic/Community Engagement  Summer Reading
  50. 50. IMPACT SURVEY
  51. 51. CONTINUUM
  52. 52. CONTINUUM
  53. 53. CONTINUUM
  54. 54. POSTCARD
  55. 55. PRIZE DRAWING
  56. 56. SKILL DEMONSTRATION
  57. 57. COMMENT WALL
  58. 58. ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT Outcome Indicators: Outcome=Digital literacy Indicator #1=Class attendee is able to send an email with an attachment Indicator #2=Class attendee is able to like the library Facebook page
  59. 59. ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT How will you measure outcomes? Skill demonstration (indicators listed above) Pre-post continuum to assess confidence using technology
  60. 60. I’ve collected some outcome data, now how do I use it?
  61. 61.  Strategic Planning  Management  Communication DATA USE
  62. 62. Area of strategic focus: Nurture children’s learning by providing programs and services that instill a love of reading and advance reading skills. STRATEGIC PLANNING
  63. 63. After participating in the Summer Reading Program, did your child’s reading habits change? Please indicate if the following things increased, stayed the same, or decreased for your child: Increased Stayed the Same Decreased Enjoyment of reading    Reading skills    Reading by his/her choice    SURVEY
  64. 64. Implications of OBE results for:  Budgeting  Staffing  Programs (focus, number offered, size, timing, etc.) MANAGEMENT
  65. 65. Library A • 100% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills • 85% of participating children increased or maintained reading skills Library CLibrary B • 25 children participated • 100 children participated • 50 children participated • $5,000 • $10,000 • $2,500 • $200/child • $100/child • $50/child
  66. 66.  Internal stakeholders  External stakeholders COMMUNICATION
  67. 67. ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT Identify uses for data in… 1) Strategic planning: 2) Management: 3) Communication:
  68. 68. ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT Identify uses for data in… 1) Strategic planning: One of my library’s strategic priorities is to provide lifelong learning. An OBE of our senior tech classes will help us to determine whether we’re achieving this priority or if we need to make adjustments.
  69. 69. ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT Identify uses for data in… 2) Managing: Are we achieving our desired outcomes? If not, do we need to adjust staff hours, budget, etc.?
  70. 70. ACTIVITY: OUTCOME INDICATORS AND METHODS FOR MEASUREMENT Identify uses for data in… 3) Communicating: Our mayor recently announced an initiative to improve the lives of seniors in our community. This is a multi- faceted initiative that will address health, wellbeing, financial and digital literacy, etc. We can demonstrate the library’s unique role in supporting this initiative by sharing our OBE data with her office.
  71. 71. THANK YOU! Nicolle Steffen & Linda Hofschire www.lrs.org | @lrs_co | ripl.lrs.org

×