Healthcare staff need to be able to judge the validity and relevance of research articles. This presentation will help get you started on doing critical appraisal. Allow at least an hour to work through the exercises provided.
Introduction to CriticalAppraisalJason CurtisSite LibrarianShrewsbury and Telford Health Libraries
Learning OutcomesBy the end of this session you should:• Understand the principles of critical appraisal and why you should undertake it• Be able to appraise published research and judge its reliability• Be able to assess the relevance of published research to your own work
What is Critical Appraisal?‘Critical appraisal is the assessment of evidence by systematically reviewing its relevance, validity and results to specific situations’ Chambers, R (1998)
What is Critical Appraisal?Critical Appraisal is not:• Negative dismissal of a piece of research• Assessment of results alone• Based entirely on detailed statistical analysis• To be undertaken by expert researchers only
What is Critical Appraisal?Critical Appraisal is:• Balanced assessment of strengths of research against its weaknesses• Assessment of research process and results• Consideration of quantitative and qualitative aspects of research• To be undertaken by all health professionals as part of their work
Why Should We Bother?• Published research is not always reliable• Only an estimated 2% is judged clinically relevant• Patients are better informed and come to us having read material on the web or in the media
Exercise 1Work through the Feng Shui exercise provided (you candownload a copy below) Download Feng Shui ExerciseAllow yourself around 5 minutesSome suggested answers are on the next slide
Exercise AnswersThis is not a reliable piece of research, and these are a few possible reasons:• Were the estate agents or buyers told that the houses were built using Feng Shui (were they ‘blinded’?)• The rebuilt houses would be six months newer (a possible ‘confounding’ factor)• The sample size is very small• Were the houses to be rebuilt chosen at random or did their position in the cul-de-sac give them an automatic advantage?
Exercise 2This is a demonstration of how not to conduct research!Read the entry below from the Guardian’s Bad Science column:www.badscience.net/2006/11/the-miracle-cure-for-dyslexia/The article discussed was reporting the results of an RCT about a cure for dyslexia.
Levels of Evidence Systematic Reviews and Meta-analyses Randomised Controlled Trials (RCTs) Cohort Studies Case Control Studies
Systematic Reviews andMeta-AnalysesCharacteristics• Combination of results of several RCTs or other types of evidence• Meta-analyses includes a statistical combination of the results• Considered to be highest-level evidence if conducted properly• Relies on enough RCTs being available• Can include ‘Odds-Ratio’ diagrams
Randomised Controlled TrialsCharacteristics• Treatment group and ‘control’ group• Random assignment to groups• May involve ‘blinding’ of participants and researchers• Used for therapeutic or diagnostic interventions• Some interventions unsuitable for RCTs• Expensive
Cohort StudiesCharacteristics• Two groups of people followed over time• One group has received an intervention or exposure (e.g. smoking)• Groups otherwise closely matched• Groups followed over time• Can be used for causation, diagnostic, harms and therapeutic studies
Case-Control StudiesCharacteristics• Used mainly for causation studies• Patient with outcome matched to control• Investigations made into possible causes in both patients• May be only option in rare conditions
Other Types of Clinical ResearchThese are lesser forms of evidence, but for some interventions, exposures or conditions they may be the only form available• Case studies / Case series• Cross-sectional surveys
QuizWhat Type of clinical research is this?1. Two groups of doctors, one group smokers, the other non-smokers are followed over the course of 20 years to see whether which group are more likely to develop lung cancer
QuizWhat Type of clinical research is this?2. Two groups of patients are studied, one group given physiotherapy for low back pain, the other given advice only. Patients are randomly assigned to either group and followed up after six months
QuizWhat Type of clinical research is this?3. One hundred sets of twins, where one had developed melanoma and the other had not, were studied for possible causation factors
Answer Case-control study(As each twin should be closely matched in most aspects other then one having developed melanoma)
How to do Critical Appraisal• Checklists are available that ask simple questions• There are different checklists for different types of studies (systematic reviews, RCTs etc)• These checklists help you focus on the most important aspects of the article
Sources of ChecklistsCritical appraisal checklists can be downloaded from: Critical Appraisal Skills Programme www.casp-uk.net Centre for Evidence-Based Medicine www.cebm.net/index.aspx?o=1157
Key Steps in Critical AppraisalThe checklists help us answer these questions• What are the results? Are they significant?• Are the results valid? How was the research done?• How will these results help me work with my patients?
What Are The Implications?Some further questions to ask of the research• Does it produce new knowledge?• How applicable is the research to your own patients?Perhaps the ultimate question is…• Should we act on the findings and change current local practice?
But The Good News Is…Some healthcare research has already been appraised Cochrane Library (good quality systematic reviews conducted by the Cochrane Collaboration) Database of Abstracts of Reviews of Effects (critical appraisals of published systematic reviews) Behind the Headlines (appraisals of healthcare research in the media)
Exercise 2• This is an exercise on appraising a published systematic review about interventions for hangovers.• Spend 10 minutes reading the article and then 20 minutes answering the questions on the CASP checklist• Remember, there are no wrong answers! Download the Systematic Review (NHS Athens account required) Download the Appraisal Checklist
AnswersThere are no right or wrong answers to this question,but you may like to read the NHS Centre for Reviewsand Dissemination’s views to see what they suggest, orsee the responses in the BMJ. CRD appraisal of article Rapid Responses to article on BMJ website
Further Reading• Further resources are available on NHS Shropshire’s del.icio.us account at: http://delicious.com/nhsshropshire/appraisal• Further guides and documentation to accompany this presentation are available from Shrewsbury and Telford Health Libraries at: http://library.sath.nhs.uk/appraisal
FinallyWe hope you have found this presentation useful. Yourlocal healthcare library will be able to provide furtherassistance, and will be able to help you search forevidence-based information and supply books, journalarticles and other resources.You can search for your nearest healthcare library onthe Health Library and Information Services Directory atwww.hlisd.org