Ins and Outs of Program Evaluation
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Ins and Outs of Program Evaluation

on

  • 994 views

 

Statistics

Views

Total Views
994
Views on SlideShare
994
Embed Views
0

Actions

Likes
0
Downloads
39
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Ins and Outs of Program Evaluation Presentation Transcript

  • 1. Program Evaluation
    A Practical/Painless Approach
    NLA/NEMA Conference – Oct. 14, 2010
  • 2. About the presenter
    Kathryn Brockmeier
    Research Analyst/ Special Projects Associate, Nebraska Library Commission
    Grant writer
    Evaluator
    kathryn.brockmeier@nebraska.gov
    402-471-4002
  • 3. About you
    Library staff
    Trustee
    Friends/Foundation board member
  • 4. About this presentation
    Introduction to program evaluation: designs, methods, and techniques of analysis
    Real-world examples
    NLA/NLC & IMLS Internship program
    NLC/IMLS Scholarship program
    NLC One Book for Nebraska Kids & Teens
    Q&A
  • 5. What is a program?
    Stems from mission and goals
    Goal becomes program
    A collection of organizational resources geared to accomplish a certain major goal or set of goals
  • 6. What is program evaluation?
    Collecting information
    Making decisions
    “What do we need to know?”
    For…
    Needs assessment, return on investment study, grant application/report
  • 7. Benefits of evaluation
    Investigate what works/doesn’t work
    Share results with stakeholders
    Demonstrate effectiveness
    Improve staff’s involvement
    Model the program
  • 8. Types of program evaluationto meet program objectives
    Implementation evaluation (process evaluation)
    Program implementation objectives
    Outcome evaluation
    Participant (stakeholder and individual) outcome objectives
  • 9. Steps in the evaluation process
    #1: Assemble an evaluation team
    #2: Prepare for the evaluation
    #3: Develop an evaluation plan
    #4: Collect evaluation information
    #5: Analyze your evaluation information
    #6: Prepare the evaluation report
  • 10. 1. Who will conduct the evaluation?
    Outside evaluator supported by in-house staff
    In-house evaluator supported by program staff and outside consultant
    In-house evaluator supported by program staff
  • 11. 2. How to prepare for the evaluation
    a. Decide what to evaluate
    b. Build a logic model of your program
    c. State your program implementation and participant outcome objectives in measurable terms
    d. Identify the context for your evaluation
  • 12. 2a. Decide what to evaluate
    Entire program, program component, service, activity
    Depends on financial and staff resources
    Grant-driven or funding-driven
    At start-up of program
    To check effectiveness
  • 13. 2b. Logic models
    Assumptions about your target audience
    Program interventions
    Implementation objectives
    Immediate outcomes
    Immediate participant outcome objectives
    Intermediate outcomes
    Long-term outcomes
  • 14. 2c. SMART objectives
    Specific
    Measurable
    Achievable
    Relevant
    Time-framed
    By the end of ___________________ (enter timeframe), increase _______________ (what will change?) by _______ (enter a percentage) % or more for ___________________ (who will experience the change?) by _________________ (how will the change occur?).
    http://www.marchofdimes.com/HI_SMART_objectives.pdf
  • 15. 2d. Evaluation context
    Assumptions about your target audience
    Program interventions
    Implementation objectives
    Immediate outcomes
    Immediate participant outcome objectives
    Intermediate outcomes
    Long-term outcomes
  • 16. 3. Develop the plan
    Discuss the evaluation framework
    Evaluating implementation objectives
    Types of info needed, Sources of info, Selection of info sources, Methods for collecting info, Confidentiality, Timeframe, Analysis methods
    Evaluating participant outcome objectives
    Do program participants demonstrate changes in knowledge, attitudes, behaviors, or awareness?
    Were the changes the result of the program’s interventions?
    Evaluation practices and procedures
  • 17. 4. “Data” collection
    When and where info will be collected
    Who are the respondents?
    Methods
    Survey/questionnaire
    Interview
    Focus group
    Existing records
    Observation
  • 18. 5. Analyze the information
    Descriptive statistics
    Frequency distributions
    Summary statistics—measures of central tendency, range
    Cross-tabulation
    Filters
    Content analysis
    Case studies
  • 19. 6. Write the report
    Audience
    Format
    Writing tips
    Oral presentations
    Visual aids
    Considerations
  • 20. Resources
    The Program Manager’s Guide to Evaluation, from the Administration for Children and Families
    http://bit.ly/cNVlQ8
    Basic Guide to Program Evaluation, from the Free Management Library
    http://www.managementhelp.org/evaluatn/fnl_eval.htm