• Like

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Evaluation post mortems

  • 982 views
Uploaded on

 

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
982
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
14
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Amy A. Germuth, Ph.D.
    Evaluation 2010
    American Evaluation Association
    San Antonio, TX, November 13, 2010
    1
    Evaluation Post MortemsDissecting what went right or wrong and learning from it!
  • 2. What’s a Post Mortem?
    2
    A formal analysis of the success and failures of the project to-date.
    Findings are added to the general knowledge base for future use.
  • 3. Post Mortems in Practice
    3
    Conducting post-mortems is a common practice in business.
    Appears to be less common in evaluation.
    Very little written about evaluations that don’t work, including why they don’t work.
    - Fear of reporting failures?
  • 4. Post Mortems in Evaluation
    4
    What if we made post-mortems part of the evaluation process?
    - What would it look like?
    - Where would it fit?
    - Would it be beneficial?
    - If so, why and how?
  • 5. Post Mortems: The Process
    5
    Two-step process:
    Provide the persons involved a list of questions about the evaluation that they think about and respond to on their own.
    Bring all persons together in order to share what they thought and discuss lessons learned.
  • 6. Who to Involve
    6
    May depend upon where you are in the evaluation cycle.
    Ultimately want all stakeholders to weigh in.
    Remember – about evaluation process – NOT evaluation findings.
  • 7. General Questions to Consider
    7
    Are you/we proud of our finished deliverables (project work products)?
    - If yes, what's good about them?
    - If no, what's wrong with them?
    What was the single most frustrating part of the evaluation?
    How would you/we do things differently next time to avoid this frustration?
  • 8. General Questions Cont.
    8
    What was the most gratifying or professionally satisfying part of the evaluation?
    Which methods or processes worked particularly well?
    Which methods or processes were difficult or frustrating to use?
    If you could wave a magic wand and change anything about the evaluation, what would you change?
  • 9. Evaluation-focused Questions
    9
    Did our stakeholders, senior managers, customers, and sponsor(s) participate effectively? If not, how could we improve their participation?
    How accurate were our original estimates of the time, cost, and other resources required of the evaluation? What did we over- or under-estimate?
     Knowing what we know now, would we have chosen the same type of evaluation design as the one we used? If not, what could have pointed us to a design that would have been better suited for such a project?
  • 10. Evaluation-focused Questions Cont.
    10
    Were our evaluation questions the best ones, or were there other questions we did not fully explore with stakeholders, through our evaluation, etc. that needed addressing?
     How would we rate the quality of the data we gathered and what could we have done to have collected more convincing data for formative and summative purposes?
     Did our presentation of results highlight the data so that stakeholders could make their own interpretations or understand the ones we made?
     What did we do to help stakeholders understand and use the evaluation findings?
  • 11. Further Notes on Post Mortems
    11
    Separate from just following the program evaluation standards post-mortems have a very formal outcome, the identification of lessons learned.
    Separate from a meta-evaluation as meta-evaluations are themselves evaluations and not designed to identify lessons learned, as much as to identify the value in the evaluation that was conducted.
    Also, for meta-evaluation to be viewed as unbiased, they do need to be conducted by someone outside of the original evaluation, whereas post-mortems are specifically designed to engage the original evaluators.
  • 12. For More Information
    12
    http://www.evalthoughts.com/2009/10/evaluation-evaluation-post-mortems.html
    Amy A. Germuth, Ph.D.
    EvalWorks, LLC
    150 Solterra Way
    Durham, NC 27705
    (919) 401-5403
    agermuth@EvalWorks.com
    Blog: www.EvalThoughts.com