SlideShare a Scribd company logo
A M Y A . G E R M U T H , P H . D .
E v a l u a t i o n 2 0 1 0
A m e r i c a n E v a l u a t i o n A s s o c i a t i o n
S a n A n t o n i o , T X , N o v e m b e r 1 3 , 2 0 1 0
1
Evaluation Post Mortems
Dissecting what went right or wrong and learning from it!
What’s a Post Mortem?
2
 A formal analysis of the success and failures of the
project to-date.
 Findings are added to the general knowledge base for
future use.
Post Mortems in Practice
3
 Conducting post-mortems is a common practice in
business.
 Appears to be less common in evaluation.
 Very little written about evaluations that don’t work,
including why they don’t work.
- Fear of reporting failures?
Post Mortems in Evaluation
4
 What if we made post-mortems part of the
evaluation process?
- What would it look like?
- Where would it fit?
- Would it be beneficial?
- If so, why and how?
Post Mortems: The Process
5
Two-step process:
1. Provide the persons involved a list of questions
about the evaluation that they think about and
respond to on their own.
2. Bring all persons together in order to share what
they thought and discuss lessons learned.
Who to Involve
6
 May depend upon where you are in the evaluation
cycle.
 Ultimately want all stakeholders to weigh in.
 Remember – about evaluation process – NOT
evaluation findings.
General Questions to Consider
7
 Are you/we proud of our finished deliverables
(project work products)?
- If yes, what's good about them?
- If no, what's wrong with them?
 What was the single most frustrating part of the
evaluation?
 How would you/we do things differently next time
to avoid this frustration?
General Questions Cont.
8
 What was the most gratifying or professionally
satisfying part of the evaluation?
 Which methods or processes worked particularly
well?
 Which methods or processes were difficult or
frustrating to use?
 If you could wave a magic wand and change anything
about the evaluation, what would you change?
Evaluation-focused Questions
9
 Did our stakeholders, senior managers,
customers, and sponsor(s) participate
effectively? If not, how could we improve their
participation?
 How accurate were our original estimates of the
time, cost, and other resources required of the
evaluation? What did we over- or under-
estimate?
 Knowing what we know now, would we have
chosen the same type of evaluation design as the
one we used? If not, what could have pointed us
to a design that would have been better suited for
such a project?
Evaluation-focused Questions Cont.
10
 Were our evaluation questions the best ones, or
were there other questions we did not fully
explore with stakeholders, through our
evaluation, etc. that needed addressing?
 How would we rate the quality of the data we
gathered and what could we have done to have
collected more convincing data for formative and
summative purposes?
 Did our presentation of results highlight the data
so that stakeholders could make their own
interpretations or understand the ones we made?
 What did we do to help stakeholders understand
and use the evaluation findings?
Further Notes on Post Mortems
11
 Separate from just following the program evaluation
standards post-mortems have a very formal outcome, the
identification of lessons learned.
 Separate from a meta-evaluation as meta-evaluations are
themselves evaluations and not designed to identify
lessons learned, as much as to identify the value in the
evaluation that was conducted.
 Also, for meta-evaluation to be viewed as unbiased, they
do need to be conducted by someone outside of the
original evaluation, whereas post-mortems are
specifically designed to engage the original evaluators.
For More Information
12
http://www.evalthoughts.com/2009/10/evaluation-
evaluation-post-mortems.html
Amy A. Germuth, Ph.D.
EvalWorks, LLC
150 Solterra Way
Durham, NC 27705
(919) 401-5403
agermuth@EvalWorks.com
Blog: www.EvalThoughts.com

More Related Content

What's hot

Why Projects Fail: Obstacles and Solutions
Why Projects Fail: Obstacles and SolutionsWhy Projects Fail: Obstacles and Solutions
Why Projects Fail: Obstacles and Solutions
Michael Krigsman
 
Project Management Best Practices
Project Management Best PracticesProject Management Best Practices
Project Management Best Practices
Rohana K Amarakoon
 
Project management - a practical overview Sue Greener
Project management - a practical overview Sue GreenerProject management - a practical overview Sue Greener
Project management - a practical overview Sue Greener
Sue Greener
 
A3 Management
A3 ManagementA3 Management
Project Management To Project Governance , Knowledge Management
Project Management To Project Governance , Knowledge ManagementProject Management To Project Governance , Knowledge Management
Project Management To Project Governance , Knowledge Management
subramanian K
 
Technical Discovery: ... in a nutshell
Technical Discovery: ...  in a nutshellTechnical Discovery: ...  in a nutshell
Technical Discovery: ... in a nutshell
Vasileios Fasoulas
 
Project post-mortem analysis
Project post-mortem analysisProject post-mortem analysis
Project post-mortem analysis
Jaiveer Singh
 
Kepner Tregoe Developing Your Hr Project Management Skills
Kepner Tregoe  Developing Your Hr Project Management SkillsKepner Tregoe  Developing Your Hr Project Management Skills
Kepner Tregoe Developing Your Hr Project Management SkillsJessica Booth
 
Simple & Practical Project Management for Digital Marketing Teams
Simple & Practical Project Management for Digital Marketing TeamsSimple & Practical Project Management for Digital Marketing Teams
Simple & Practical Project Management for Digital Marketing Teams
Digitangle
 
Pragmatisk softwareinnovation, Ivan Aaen, AAU
Pragmatisk softwareinnovation, Ivan Aaen, AAUPragmatisk softwareinnovation, Ivan Aaen, AAU
Pragmatisk softwareinnovation, Ivan Aaen, AAU
InfinIT - Innovationsnetværket for it
 
Information Technology Project Management - part 05
Information Technology Project Management - part 05Information Technology Project Management - part 05
Information Technology Project Management - part 05Rizwan Khurram
 
Breaking the Project Failure Cycle
Breaking the Project Failure CycleBreaking the Project Failure Cycle
Breaking the Project Failure Cycle
Glen Alleman
 
Presentation by mangesh sardesai
Presentation by mangesh sardesaiPresentation by mangesh sardesai
Presentation by mangesh sardesaiPMI_IREP_TP
 
Effective logframes for international development
Effective logframes for international developmentEffective logframes for international development
Effective logframes for international developmentNIDOS
 
Symposium 2016 : Workshop 104 Brain and Leadership
Symposium 2016 : Workshop 104 Brain and LeadershipSymposium 2016 : Workshop 104 Brain and Leadership
Symposium 2016 : Workshop 104 Brain and Leadership
PMI-Montréal
 
Software Outsourcing: Pitfalls and Best Practices
Software Outsourcing: Pitfalls and Best PracticesSoftware Outsourcing: Pitfalls and Best Practices
Software Outsourcing: Pitfalls and Best PracticesAMC Bridge
 
Lean Projects with A3 Method
Lean Projects with A3 MethodLean Projects with A3 Method
Lean Projects with A3 Method
Nitor
 
Understanding the risks in enterprise project management
Understanding the risks in enterprise project managementUnderstanding the risks in enterprise project management
Understanding the risks in enterprise project management
Orangescrum
 
A Remnant Guide on Project Management
A Remnant Guide on Project ManagementA Remnant Guide on Project Management
A Remnant Guide on Project Management
orangescrum1
 
Managing IT Projects
Managing IT ProjectsManaging IT Projects

What's hot (20)

Why Projects Fail: Obstacles and Solutions
Why Projects Fail: Obstacles and SolutionsWhy Projects Fail: Obstacles and Solutions
Why Projects Fail: Obstacles and Solutions
 
Project Management Best Practices
Project Management Best PracticesProject Management Best Practices
Project Management Best Practices
 
Project management - a practical overview Sue Greener
Project management - a practical overview Sue GreenerProject management - a practical overview Sue Greener
Project management - a practical overview Sue Greener
 
A3 Management
A3 ManagementA3 Management
A3 Management
 
Project Management To Project Governance , Knowledge Management
Project Management To Project Governance , Knowledge ManagementProject Management To Project Governance , Knowledge Management
Project Management To Project Governance , Knowledge Management
 
Technical Discovery: ... in a nutshell
Technical Discovery: ...  in a nutshellTechnical Discovery: ...  in a nutshell
Technical Discovery: ... in a nutshell
 
Project post-mortem analysis
Project post-mortem analysisProject post-mortem analysis
Project post-mortem analysis
 
Kepner Tregoe Developing Your Hr Project Management Skills
Kepner Tregoe  Developing Your Hr Project Management SkillsKepner Tregoe  Developing Your Hr Project Management Skills
Kepner Tregoe Developing Your Hr Project Management Skills
 
Simple & Practical Project Management for Digital Marketing Teams
Simple & Practical Project Management for Digital Marketing TeamsSimple & Practical Project Management for Digital Marketing Teams
Simple & Practical Project Management for Digital Marketing Teams
 
Pragmatisk softwareinnovation, Ivan Aaen, AAU
Pragmatisk softwareinnovation, Ivan Aaen, AAUPragmatisk softwareinnovation, Ivan Aaen, AAU
Pragmatisk softwareinnovation, Ivan Aaen, AAU
 
Information Technology Project Management - part 05
Information Technology Project Management - part 05Information Technology Project Management - part 05
Information Technology Project Management - part 05
 
Breaking the Project Failure Cycle
Breaking the Project Failure CycleBreaking the Project Failure Cycle
Breaking the Project Failure Cycle
 
Presentation by mangesh sardesai
Presentation by mangesh sardesaiPresentation by mangesh sardesai
Presentation by mangesh sardesai
 
Effective logframes for international development
Effective logframes for international developmentEffective logframes for international development
Effective logframes for international development
 
Symposium 2016 : Workshop 104 Brain and Leadership
Symposium 2016 : Workshop 104 Brain and LeadershipSymposium 2016 : Workshop 104 Brain and Leadership
Symposium 2016 : Workshop 104 Brain and Leadership
 
Software Outsourcing: Pitfalls and Best Practices
Software Outsourcing: Pitfalls and Best PracticesSoftware Outsourcing: Pitfalls and Best Practices
Software Outsourcing: Pitfalls and Best Practices
 
Lean Projects with A3 Method
Lean Projects with A3 MethodLean Projects with A3 Method
Lean Projects with A3 Method
 
Understanding the risks in enterprise project management
Understanding the risks in enterprise project managementUnderstanding the risks in enterprise project management
Understanding the risks in enterprise project management
 
A Remnant Guide on Project Management
A Remnant Guide on Project ManagementA Remnant Guide on Project Management
A Remnant Guide on Project Management
 
Managing IT Projects
Managing IT ProjectsManaging IT Projects
Managing IT Projects
 

Similar to Evaluation post mortems

Training Program Evaluation
Training Program EvaluationTraining Program Evaluation
Training Program Evaluation
Laura Pasquini
 
Organizational Diagnosis
Organizational DiagnosisOrganizational Diagnosis
Organizational Diagnosis
jim
 
5 step approach to successful marketing research
5 step approach to successful marketing research5 step approach to successful marketing research
5 step approach to successful marketing research
babar mushtaq
 
Pick 2 topics and discusstalk about the topics. No plagiarism wi.docx
Pick 2 topics and discusstalk about the topics.  No plagiarism wi.docxPick 2 topics and discusstalk about the topics.  No plagiarism wi.docx
Pick 2 topics and discusstalk about the topics. No plagiarism wi.docx
randymartin91030
 
1 5InnovationEntrepreneurial Change Annotated .docx
1     5InnovationEntrepreneurial Change Annotated .docx1     5InnovationEntrepreneurial Change Annotated .docx
1 5InnovationEntrepreneurial Change Annotated .docx
honey725342
 
Active Problem Solving
Active Problem SolvingActive Problem Solving
Active Problem Solving
Michael Brown
 
Performance appraisal interviews
Performance appraisal interviewsPerformance appraisal interviews
Performance appraisal interviews
ritahenry316
 
Program Evaluation Studies TK Logan and David Royse .docx
Program Evaluation Studies TK Logan and David Royse .docxProgram Evaluation Studies TK Logan and David Royse .docx
Program Evaluation Studies TK Logan and David Royse .docx
stilliegeorgiana
 
10 things about_evaluation_final_with_hyperlinks
10 things about_evaluation_final_with_hyperlinks10 things about_evaluation_final_with_hyperlinks
10 things about_evaluation_final_with_hyperlinks
Research and Policy in Development (RAPID) Overseas Development Institute
 
Consulting framework bidhu
Consulting framework bidhuConsulting framework bidhu
Consulting framework bidhu
Bidhu Amant
 
Demystifying Evaluation
Demystifying EvaluationDemystifying Evaluation
Demystifying Evaluation
Skillsforhealth
 
Good practice in evaluation research
Good practice in evaluation researchGood practice in evaluation research
Good practice in evaluation research
QSR International
 
Nine Evaluation of TrainingLearning ObjectivesAfter readin.docx
Nine Evaluation of TrainingLearning ObjectivesAfter readin.docxNine Evaluation of TrainingLearning ObjectivesAfter readin.docx
Nine Evaluation of TrainingLearning ObjectivesAfter readin.docx
curwenmichaela
 
Workbook for Designing a Process Evaluation
 Workbook for Designing a Process Evaluation  Workbook for Designing a Process Evaluation
Workbook for Designing a Process Evaluation
MoseStaton39
 
Workbook for Designing a Process Evaluation .docx
Workbook for Designing a Process Evaluation .docxWorkbook for Designing a Process Evaluation .docx
Workbook for Designing a Process Evaluation .docx
AASTHA76
 
Workbook for Designing a Process Evaluation
 Workbook for Designing a Process Evaluation  Workbook for Designing a Process Evaluation
Workbook for Designing a Process Evaluation
MikeEly930
 
APF orlando diy survey workshop 071114 final
APF orlando diy survey workshop 071114 finalAPF orlando diy survey workshop 071114 final
APF orlando diy survey workshop 071114 final
Mike Courtney
 
Evaluating your public engagement activities (by Suzanne Spicer)
Evaluating your public engagement activities (by Suzanne Spicer)Evaluating your public engagement activities (by Suzanne Spicer)
Evaluating your public engagement activities (by Suzanne Spicer)
Emma Gillaspy
 
Performance appraisal survey
Performance appraisal surveyPerformance appraisal survey
Performance appraisal survey
dbell3034
 

Similar to Evaluation post mortems (20)

Training Program Evaluation
Training Program EvaluationTraining Program Evaluation
Training Program Evaluation
 
Organizational Diagnosis
Organizational DiagnosisOrganizational Diagnosis
Organizational Diagnosis
 
Basics of Extension Evaluations
Basics of Extension EvaluationsBasics of Extension Evaluations
Basics of Extension Evaluations
 
5 step approach to successful marketing research
5 step approach to successful marketing research5 step approach to successful marketing research
5 step approach to successful marketing research
 
Pick 2 topics and discusstalk about the topics. No plagiarism wi.docx
Pick 2 topics and discusstalk about the topics.  No plagiarism wi.docxPick 2 topics and discusstalk about the topics.  No plagiarism wi.docx
Pick 2 topics and discusstalk about the topics. No plagiarism wi.docx
 
1 5InnovationEntrepreneurial Change Annotated .docx
1     5InnovationEntrepreneurial Change Annotated .docx1     5InnovationEntrepreneurial Change Annotated .docx
1 5InnovationEntrepreneurial Change Annotated .docx
 
Active Problem Solving
Active Problem SolvingActive Problem Solving
Active Problem Solving
 
Performance appraisal interviews
Performance appraisal interviewsPerformance appraisal interviews
Performance appraisal interviews
 
Program Evaluation Studies TK Logan and David Royse .docx
Program Evaluation Studies TK Logan and David Royse .docxProgram Evaluation Studies TK Logan and David Royse .docx
Program Evaluation Studies TK Logan and David Royse .docx
 
10 things about_evaluation_final_with_hyperlinks
10 things about_evaluation_final_with_hyperlinks10 things about_evaluation_final_with_hyperlinks
10 things about_evaluation_final_with_hyperlinks
 
Consulting framework bidhu
Consulting framework bidhuConsulting framework bidhu
Consulting framework bidhu
 
Demystifying Evaluation
Demystifying EvaluationDemystifying Evaluation
Demystifying Evaluation
 
Good practice in evaluation research
Good practice in evaluation researchGood practice in evaluation research
Good practice in evaluation research
 
Nine Evaluation of TrainingLearning ObjectivesAfter readin.docx
Nine Evaluation of TrainingLearning ObjectivesAfter readin.docxNine Evaluation of TrainingLearning ObjectivesAfter readin.docx
Nine Evaluation of TrainingLearning ObjectivesAfter readin.docx
 
Workbook for Designing a Process Evaluation
 Workbook for Designing a Process Evaluation  Workbook for Designing a Process Evaluation
Workbook for Designing a Process Evaluation
 
Workbook for Designing a Process Evaluation .docx
Workbook for Designing a Process Evaluation .docxWorkbook for Designing a Process Evaluation .docx
Workbook for Designing a Process Evaluation .docx
 
Workbook for Designing a Process Evaluation
 Workbook for Designing a Process Evaluation  Workbook for Designing a Process Evaluation
Workbook for Designing a Process Evaluation
 
APF orlando diy survey workshop 071114 final
APF orlando diy survey workshop 071114 finalAPF orlando diy survey workshop 071114 final
APF orlando diy survey workshop 071114 final
 
Evaluating your public engagement activities (by Suzanne Spicer)
Evaluating your public engagement activities (by Suzanne Spicer)Evaluating your public engagement activities (by Suzanne Spicer)
Evaluating your public engagement activities (by Suzanne Spicer)
 
Performance appraisal survey
Performance appraisal surveyPerformance appraisal survey
Performance appraisal survey
 

Recently uploaded

Thesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.pptThesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.ppt
EverAndrsGuerraGuerr
 
678020731-Sumas-y-Restas-Para-Colorear.pdf
678020731-Sumas-y-Restas-Para-Colorear.pdf678020731-Sumas-y-Restas-Para-Colorear.pdf
678020731-Sumas-y-Restas-Para-Colorear.pdf
CarlosHernanMontoyab2
 
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
MysoreMuleSoftMeetup
 
The approach at University of Liverpool.pptx
The approach at University of Liverpool.pptxThe approach at University of Liverpool.pptx
The approach at University of Liverpool.pptx
Jisc
 
Embracing GenAI - A Strategic Imperative
Embracing GenAI - A Strategic ImperativeEmbracing GenAI - A Strategic Imperative
Embracing GenAI - A Strategic Imperative
Peter Windle
 
The geography of Taylor Swift - some ideas
The geography of Taylor Swift - some ideasThe geography of Taylor Swift - some ideas
The geography of Taylor Swift - some ideas
GeoBlogs
 
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
EugeneSaldivar
 
Palestine last event orientationfvgnh .pptx
Palestine last event orientationfvgnh .pptxPalestine last event orientationfvgnh .pptx
Palestine last event orientationfvgnh .pptx
RaedMohamed3
 
How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17
Celine George
 
special B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdfspecial B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdf
Special education needs
 
Language Across the Curriculm LAC B.Ed.
Language Across the  Curriculm LAC B.Ed.Language Across the  Curriculm LAC B.Ed.
Language Across the Curriculm LAC B.Ed.
Atul Kumar Singh
 
The Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official PublicationThe Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official Publication
Delapenabediema
 
Honest Reviews of Tim Han LMA Course Program.pptx
Honest Reviews of Tim Han LMA Course Program.pptxHonest Reviews of Tim Han LMA Course Program.pptx
Honest Reviews of Tim Han LMA Course Program.pptx
timhan337
 
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
Nguyen Thanh Tu Collection
 
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCECLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
BhavyaRajput3
 
Chapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptxChapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptx
Mohd Adib Abd Muin, Senior Lecturer at Universiti Utara Malaysia
 
Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345
beazzy04
 
1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx
JosvitaDsouza2
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
Peter Windle
 
Acetabularia Information For Class 9 .docx
Acetabularia Information For Class 9  .docxAcetabularia Information For Class 9  .docx
Acetabularia Information For Class 9 .docx
vaibhavrinwa19
 

Recently uploaded (20)

Thesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.pptThesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.ppt
 
678020731-Sumas-y-Restas-Para-Colorear.pdf
678020731-Sumas-y-Restas-Para-Colorear.pdf678020731-Sumas-y-Restas-Para-Colorear.pdf
678020731-Sumas-y-Restas-Para-Colorear.pdf
 
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
 
The approach at University of Liverpool.pptx
The approach at University of Liverpool.pptxThe approach at University of Liverpool.pptx
The approach at University of Liverpool.pptx
 
Embracing GenAI - A Strategic Imperative
Embracing GenAI - A Strategic ImperativeEmbracing GenAI - A Strategic Imperative
Embracing GenAI - A Strategic Imperative
 
The geography of Taylor Swift - some ideas
The geography of Taylor Swift - some ideasThe geography of Taylor Swift - some ideas
The geography of Taylor Swift - some ideas
 
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
 
Palestine last event orientationfvgnh .pptx
Palestine last event orientationfvgnh .pptxPalestine last event orientationfvgnh .pptx
Palestine last event orientationfvgnh .pptx
 
How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17
 
special B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdfspecial B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdf
 
Language Across the Curriculm LAC B.Ed.
Language Across the  Curriculm LAC B.Ed.Language Across the  Curriculm LAC B.Ed.
Language Across the Curriculm LAC B.Ed.
 
The Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official PublicationThe Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official Publication
 
Honest Reviews of Tim Han LMA Course Program.pptx
Honest Reviews of Tim Han LMA Course Program.pptxHonest Reviews of Tim Han LMA Course Program.pptx
Honest Reviews of Tim Han LMA Course Program.pptx
 
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
 
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCECLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
 
Chapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptxChapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptx
 
Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345Sha'Carri Richardson Presentation 202345
Sha'Carri Richardson Presentation 202345
 
1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
 
Acetabularia Information For Class 9 .docx
Acetabularia Information For Class 9  .docxAcetabularia Information For Class 9  .docx
Acetabularia Information For Class 9 .docx
 

Evaluation post mortems

  • 1. A M Y A . G E R M U T H , P H . D . E v a l u a t i o n 2 0 1 0 A m e r i c a n E v a l u a t i o n A s s o c i a t i o n S a n A n t o n i o , T X , N o v e m b e r 1 3 , 2 0 1 0 1 Evaluation Post Mortems Dissecting what went right or wrong and learning from it!
  • 2. What’s a Post Mortem? 2  A formal analysis of the success and failures of the project to-date.  Findings are added to the general knowledge base for future use.
  • 3. Post Mortems in Practice 3  Conducting post-mortems is a common practice in business.  Appears to be less common in evaluation.  Very little written about evaluations that don’t work, including why they don’t work. - Fear of reporting failures?
  • 4. Post Mortems in Evaluation 4  What if we made post-mortems part of the evaluation process? - What would it look like? - Where would it fit? - Would it be beneficial? - If so, why and how?
  • 5. Post Mortems: The Process 5 Two-step process: 1. Provide the persons involved a list of questions about the evaluation that they think about and respond to on their own. 2. Bring all persons together in order to share what they thought and discuss lessons learned.
  • 6. Who to Involve 6  May depend upon where you are in the evaluation cycle.  Ultimately want all stakeholders to weigh in.  Remember – about evaluation process – NOT evaluation findings.
  • 7. General Questions to Consider 7  Are you/we proud of our finished deliverables (project work products)? - If yes, what's good about them? - If no, what's wrong with them?  What was the single most frustrating part of the evaluation?  How would you/we do things differently next time to avoid this frustration?
  • 8. General Questions Cont. 8  What was the most gratifying or professionally satisfying part of the evaluation?  Which methods or processes worked particularly well?  Which methods or processes were difficult or frustrating to use?  If you could wave a magic wand and change anything about the evaluation, what would you change?
  • 9. Evaluation-focused Questions 9  Did our stakeholders, senior managers, customers, and sponsor(s) participate effectively? If not, how could we improve their participation?  How accurate were our original estimates of the time, cost, and other resources required of the evaluation? What did we over- or under- estimate?  Knowing what we know now, would we have chosen the same type of evaluation design as the one we used? If not, what could have pointed us to a design that would have been better suited for such a project?
  • 10. Evaluation-focused Questions Cont. 10  Were our evaluation questions the best ones, or were there other questions we did not fully explore with stakeholders, through our evaluation, etc. that needed addressing?  How would we rate the quality of the data we gathered and what could we have done to have collected more convincing data for formative and summative purposes?  Did our presentation of results highlight the data so that stakeholders could make their own interpretations or understand the ones we made?  What did we do to help stakeholders understand and use the evaluation findings?
  • 11. Further Notes on Post Mortems 11  Separate from just following the program evaluation standards post-mortems have a very formal outcome, the identification of lessons learned.  Separate from a meta-evaluation as meta-evaluations are themselves evaluations and not designed to identify lessons learned, as much as to identify the value in the evaluation that was conducted.  Also, for meta-evaluation to be viewed as unbiased, they do need to be conducted by someone outside of the original evaluation, whereas post-mortems are specifically designed to engage the original evaluators.
  • 12. For More Information 12 http://www.evalthoughts.com/2009/10/evaluation- evaluation-post-mortems.html Amy A. Germuth, Ph.D. EvalWorks, LLC 150 Solterra Way Durham, NC 27705 (919) 401-5403 agermuth@EvalWorks.com Blog: www.EvalThoughts.com