Your SlideShare is downloading. ×
Building the evidential quality of evaluations (Tony Redmond, Uni Manchester)
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Building the evidential quality of evaluations (Tony Redmond, Uni Manchester)


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Building the evidential quality of evaluations ALNAP March 5, 2013 Tony Redmond
  • 2. Humanitarian vs Development
  • 3. Hierarchy of Evidence
  • 4. Most evidence in thehumanitarian sphere is in theweakest categories or outside the pyramid altogether
  • 5. Expert Opinion• Consensus Statements Regarding the Multidisciplinary Care of Limb Amputation Patients in Disasters or Humanitarian Emergencies: Report of the 2011 Humanitarian Action Summit Surgical Working Group on Amputations Following Disasters or Conflict• Delphi Studiens
  • 6. Gathering the Evidence
  • 7. bias• Incentive• Observer• Systematic (non random) error – Confidence intervals
  • 8. RCT Sibbald B, BMJ 1998;316:201• the most rigorous way of determining whether a cause-effect relation exists between intervention and outcome and• for assessing the cost effectiveness of an intervention.
  • 9. Dangerous Data• Wrong inference• Randomisation• Open and transparent
  • 10. HERR• Medical teams are 200 times as cost effective as UKISAR• That is they cost less, stay longer, treat more people and save more lives
  • 11. sampling
  • 12. Convenience sampling• Using those who are willing to volunteer, or cases which are presented to you as a sample.
  • 13. Judgement sampling• Based on deliberate choice and excludes any random process.• Normal application is for small samples from a population that is well understood and there is a clear method for picking the sample.• Is used to provide illustrative examples or case studies.
  • 14. Simple random sampling• Ensures every member of the population has an equal chance of selection.• Produces defensible estimates of the population and sampling error.• Simple sample design and interpretation.
  • 15. Cluster samplingUnits in the population can often be found in geographical groups or clusters. Arandom sample of clusters is taken, then all units within those clusters areexamined.• Quicker, easier and cheaper than other forms of random sampling.• Does not require complete population information.• Useful for face-to-face interviews.• Works best when each cluster can be regarded as a microcosm of the population.• Larger sampling error than other forms of random sampling.• If clusters are not small it can become expensive.• A larger sample size may beneeded to compensate forgreater sampling error.
  • 16. Why is evidence ignored?• Evidence influencing policy• Policy influencing evidence• Weve always done it this way…• ……and it’s done no harm (regardless of the evidence)• Maybe the evidence was always there….
  • 17. Epidemiology/statistics• Observation• Pattern Recognition• Cause and effect or coincidence• Change of practice
  • 18. CONFIDENCEINTERVALS (CI) MEAN VALUE You have a 95% confidence that the TRUE VALUE should fall between the CI if the study is repeated multiple times in a similar population 95% CONFIDENCE INTERVAL
  • 19. assessments• Early warning• Risk analysis• Trending• Needs assessment
  • 20. Evidence based medicine• Clinical trials in the field – in an emergency?• Multiple disciplines
  • 21. Evaluation• What was done?• What was the reason for doing it?• What did they find?• SO WHAT?
  • 22. Benefit cost
  • 23. What you have to evaluate adapted from DFID WASH Evaluation 2011• Internal validity – was what they did good enough?• External validity – relevance to you?• Can you implement it ?• Sustainability?• Wider impact?• COST
  • 24. Is good enough, good enough?
  • 25. Humanitarian• Historically draws on concept of compassion, individuality and reform (transformation)• It is evidence and evaluation that will help us reform and lead us to prevention