Your SlideShare is downloading. ×
0
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Ins and Outs of Program Evaluation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Ins and Outs of Program Evaluation

820

Published on

Published in: Self Improvement
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
820
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Program Evaluation<br />A Practical/Painless Approach<br />NLA/NEMA Conference – Oct. 14, 2010<br />
  • 2. About the presenter<br />Kathryn Brockmeier<br />Research Analyst/ Special Projects Associate, Nebraska Library Commission<br />Grant writer<br />Evaluator<br />kathryn.brockmeier@nebraska.gov<br />402-471-4002<br />
  • 3. About you<br />Library staff<br />Trustee<br />Friends/Foundation board member<br />
  • 4. About this presentation<br />Introduction to program evaluation: designs, methods, and techniques of analysis<br />Real-world examples<br />NLA/NLC &amp; IMLS Internship program<br />NLC/IMLS Scholarship program<br />NLC One Book for Nebraska Kids &amp; Teens<br />Q&amp;A<br />
  • 5. What is a program?<br />Stems from mission and goals<br />Goal becomes program<br />A collection of organizational resources geared to accomplish a certain major goal or set of goals<br />
  • 6. What is program evaluation?<br />Collecting information<br />Making decisions<br />“What do we need to know?”<br />For…<br />Needs assessment, return on investment study, grant application/report<br />
  • 7. Benefits of evaluation<br />Investigate what works/doesn’t work<br />Share results with stakeholders<br />Demonstrate effectiveness<br />Improve staff’s involvement<br />Model the program<br />
  • 8. Types of program evaluationto meet program objectives<br />Implementation evaluation (process evaluation)<br />Program implementation objectives<br />Outcome evaluation<br />Participant (stakeholder and individual) outcome objectives<br />
  • 9. Steps in the evaluation process<br />#1: Assemble an evaluation team<br />#2: Prepare for the evaluation<br />#3: Develop an evaluation plan<br />#4: Collect evaluation information<br />#5: Analyze your evaluation information<br />#6: Prepare the evaluation report<br />
  • 10. 1. Who will conduct the evaluation?<br />Outside evaluator supported by in-house staff<br />In-house evaluator supported by program staff and outside consultant<br />In-house evaluator supported by program staff<br />
  • 11. 2. How to prepare for the evaluation<br />a. Decide what to evaluate<br />b. Build a logic model of your program<br />c. State your program implementation and participant outcome objectives in measurable terms<br />d. Identify the context for your evaluation<br />
  • 12. 2a. Decide what to evaluate<br />Entire program, program component, service, activity<br />Depends on financial and staff resources<br />Grant-driven or funding-driven<br />At start-up of program<br />To check effectiveness<br />
  • 13. 2b. Logic models<br />Assumptions about your target audience<br />Program interventions<br />Implementation objectives<br />Immediate outcomes<br />Immediate participant outcome objectives<br />Intermediate outcomes<br />Long-term outcomes<br />
  • 14. 2c. SMART objectives<br />Specific<br />Measurable<br />Achievable<br />Relevant<br />Time-framed<br />By the end of ___________________ (enter timeframe), increase _______________ (what will change?) by _______ (enter a percentage) % or more for ___________________ (who will experience the change?) by _________________ (how will the change occur?).<br />http://www.marchofdimes.com/HI_SMART_objectives.pdf<br />
  • 15. 2d. Evaluation context<br />Assumptions about your target audience<br />Program interventions<br />Implementation objectives<br />Immediate outcomes<br />Immediate participant outcome objectives<br />Intermediate outcomes<br />Long-term outcomes<br />
  • 16. 3. Develop the plan<br />Discuss the evaluation framework<br />Evaluating implementation objectives<br />Types of info needed, Sources of info, Selection of info sources, Methods for collecting info, Confidentiality, Timeframe, Analysis methods<br />Evaluating participant outcome objectives<br />Do program participants demonstrate changes in knowledge, attitudes, behaviors, or awareness?<br />Were the changes the result of the program’s interventions?<br />Evaluation practices and procedures<br />
  • 17. 4. “Data” collection<br />When and where info will be collected<br />Who are the respondents?<br />Methods<br />Survey/questionnaire<br />Interview<br />Focus group<br />Existing records<br />Observation<br />
  • 18. 5. Analyze the information<br />Descriptive statistics<br />Frequency distributions<br />Summary statistics—measures of central tendency, range<br />Cross-tabulation<br />Filters<br />Content analysis<br />Case studies<br />
  • 19. 6. Write the report<br />Audience<br />Format<br />Writing tips<br />Oral presentations<br />Visual aids<br />Considerations<br />
  • 20. Resources<br />The Program Manager’s Guide to Evaluation, from the Administration for Children and Families<br />http://bit.ly/cNVlQ8 <br />Basic Guide to Program Evaluation, from the Free Management Library<br />http://www.managementhelp.org/evaluatn/fnl_eval.htm <br />

×