Indicators of study quality in systematic reviews of qualitative research to inform public health policy making

1,242 views

Published on

Indicators of study quality in systematic reviews of qualitative research to inform public health policy making

Presented at the 2nd European Conference on Qualitative Research for Policy Making, 26-27 May 2011, Belfast, UK

Organised by Merlien Institute

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,242
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Indicators of study quality in systematic reviews of qualitative research to inform public health policy making

  1. 1. Qualitative research and policy making May 2011. BelfastRuth GarsideSenior Research FellowPenTAG, Peninsula Medical School, University of ExeterRuth. Garside@pms.ac.uk
  2. 2. Session plan 75 mins Background to our experience: 10 mins Group work: What does good quality research and its reporting look like? 10mins Checklist examples from Public Health Policy Making in England 10 mins Break out practical session and feedback: 35 mins Round up (and a suggestion!) (10 mins)
  3. 3. Our experience Working with CPHE NICE in the UK Producing systematic reviews of qualitative research Tendency to assume similar concerns as for quantitative research
  4. 4. What makes qualitative research“good quality”?
  5. 5. p.8
  6. 6. Alternative criteria Wallace A, Croucher K, Quilagars D, Baldwin S. Meeting the challenge: developing systematic reviewing in social policy. Policy and Politics 2004; 32(4):455-470.
  7. 7. Question Is the research question clear? E/D1 Theoretical Is the theoretical or ideological perspective of the author (or funder) D perspective explicit?2 Has this influenced the study design, methods, or research findings?3 Study design Is the study design appropriate to answer the question? E4 Context Is the context or setting adequately described? E5 Sampling Is the sample adequate to explore the range of subjects and E6 settings? Has it been drawn from an appropriate population?7 Data collection Was the data collection adequately described? E8 Was it rigorously conducted to ensure confidence in the findings?9 Data analysis Was there evidence that the data analysis was rigorously conducted E to ensure confidence in the findings?10 Reflexivity Are the findings substantiated by the data and has consideration D been given to any limitations of the methods or data that may have affected the results?11 Generalisibility Do any claims to generalisibility follow logically and theoretically from D the data?12 Ethics Have ethical issues been addressed and confidentiality respected? D13 Question Is the research question clear? E14 Theoretical Is the theoretical or ideological perspective of the author (or funder) D perspective explicit?15 Has this influenced the study design, methods, or research findings?16 Study design Is the study design appropriate to answer the question? E
  8. 8. Small group work Are there any challenges with using the checklists provided? Do you agree that the questions they are asking will assess “quality”? Why?/Why not? What would you use?
  9. 9. My preferred approach Technical aspects Trustworthiness Theoretical considerations Practical considerationsAdapted from: Popay J, Using Qualitative Research to Inform Policy and Practice. ONS, Cardiff: April 2008.
  10. 10. 1. Technical aspects Y/P/N Comments 1. Is the research question(s) clear? 2. Is the research question(s) suited to qual. enquiry? Are the following clearly described? 3. Context 4. Sampling 5. Data collection 6. AnalysisAdapted from: Dixon-Woods M, Shaw RL, Agarwal S, Smith JA. The problem of appraising qualitativeresearch. Qual Saf Health Care 2004; 13:233-225
  11. 11. 2. TrustworthinessFor example: Are the design and execution appropriate to the research question? What evidence of reflexivity is there? Do the voices of the participants come through? Are alternative interpretations, theories etc explored? How well supported by the data are any conclusions? Are ethical considerations given appropriate thought? etc.
  12. 12. 3. Theoretical considerationsFor example: Does the report connect to a wider body of knowledge or existing theoretical framework; and, if so Is this appropriate (e.g. not uncritical verification); Does the paper develop explanatory concepts for the findings etc.
  13. 13. 4. Practical considerationsNot “is this research valid?” but rather “what is this research valid for?”For example Does this study usefully contribute to the policy question? Does this study provide evidence relevant to the policy setting? Does this study usefully contribute to the review?Aguinaldo JP. Rethinking Validity in Qualitative Research from a Social Constructionist Perspective:From "Is this valid research?" to "What is this research valid for?". The Qualitative Report 2004; 9(1):127-136.
  14. 14. Presented at the 2nd Europeanconference on Qualitative Research forPolicy Making, 26 -27 May 2011, Belfast For more information Please visit: http://www.merlien.org

×