Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Public involvement in the systematic review process in health and social care: A narrative review of case examples.


Published on

Public involvement in the systematic review process in health and social care: A narrative review of case examples.

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

Public involvement in the systematic review process in health and social care: A narrative review of case examples.

  1. 1. Public involvement in the systematic review process in health and social care: A narrative review of case examples 9th HTAi Annual Meeting, Bilbao, Spain, 25-27th June 2012 Jonathan Boote (1), Wendy Baird (1) and Anthea Sutton (2)(1) NIHR Research Design Service for Yorkshire and the Humber, (2) The University of Sheffield
  2. 2. Background to the review• Perhaps the most important type of health research that the public can become involved in is the systematic review – Systematic review has high status in evidence hierarchies• Impetus for involving the public in the systematic review process has come primarily from the Cochrane Collaboration (Bastian, 1994; Horey, 2010) – The public have been part of the Collaboration since its inception – Public representation on the Cochrane steering group – Cochrane Consumer Network• 2 surveys of the extent to which Cochrane review groups involve the public have been published : Kelson, 1999, Wale et al, 2010• 4 Cochrane review groups have published accounts of public involvement within their respective groups: – Musculoskeletal (Shea et al, 2005), Pregnancy and Childbirth (Sakala et al, 2001), Breast cancer (Ghersi, 2002), Haematological malignancies (Skoetz et al, 2005)
  3. 3. Background to the review• Detailed published accounts of public involvement within individual systematic reviews remain scarce• Purpose of the review was to identify and review case examples of public involvement in the design and conduct of individual systematic reviews, to: – examine the methods, levels and stages of involvement reported – synthesise the identified contributions of involving the public – discuss the tensions, facilitating strategies and recommendations for good practice that have been identified• Excluded from the review were: – Viewpoint or discussion papers on public involvement in systematic reviews – Surveys of public involvement in systematic reviews. – Documents with insufficient detail of the contribution(s) of the public to the systematic review.
  4. 4. Searching for evidence • A literature search was undertaken in January 2011 of the following databases, with the aim of identifying peer-reviewed journal articles and reports published in English on public involvement in the systematic review process: – PsycINFO, MEDLINE, and MEDLINE In-Process, CINAHL, EMBASE, Cochrane Methodology Register, HMIC, invoNET. • Search terms included ‘consumer’, ‘user’, ‘participant’, ‘involvement’, ‘inclusion’, ‘review’, ‘systematic review’ and ‘research’. • A further search of a comprehensive bibliography of references on public involvement in health and social care research (Boote et al, 2012) was undertaken, to identify relevant reports of systematic reviews that involved the public
  5. 5. Identifying evidence• A total of 744 documents were identified in the search of databases, discounting duplicates• abstracts were scrutinised for possible inclusion in the review by the lead researcher, and these inclusions were checked for accuracy by a second researcher• Of these 744 documents, three papers were identified on public involvement in the systematic review process on the basis of paper title and abstract (where available)• The search of the bibliography identified a further four further relevant sources.• In total, seven case examples were found.
  6. 6. Included case examples (1)• Braye S, Preston-Shoot M. Emerging out of the shadows? Service user and carer involvement in systematic reviews. Evidence and Policy: A Journal of Research, Debate and Practice 2005;1(2):173–94.• Smith E, Donovan S, Beresford P, Manthorpe J, Brearley S, Sitzia J, et al.Getting ready for user involvement in a systematic review. Health Expectations 2009;12:197–208.• Serrano-Aguilar P, Trujillo-Martín MM, Ramos-Go˜ni JM, Mahtani-Chugan V, Perestelo-Pérez L, Posada-de la Paz M. Patient involvement in health research: a contribution to a systematic review on the effectiveness of treatments for degenerative ataxias. Social Science and Medicine 2009;69(6):920–5.• Brett J, Staniszewska S, Mockford C, Seers K, Herron-Marx S, Bayliss H. The PIRICOM Study: a systematic review of the conceptualisation, measurement, impact and outcomes of patients and public involvement in health and social care research. UK Clinical Research Collaboration; 2010,
  7. 7. Included case examples (2)• Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database of Systematic Reviews 2006;3(September),ncbi-p:1469493X.• Carr S, Fleischmann P. Systematic review of consumers’ perspective on electro-convulsive therapy. In: Carr S, Coren E, editors. Collection of examples of service user and carer participation in systematic reviews. London: SCIE; 2007.• Rees R, Oliver S. An example from sexual health promotion. In: Carr S, Coren E, editors. Collection of examples of service user and carer participation in systematic reviews. London: SCIE; 2007.
  8. 8. Summary of included papers (1)Paper and Topic area Public Stage, method and type ofcountry involvementBraye and teaching, learning Wide range of different Consultation, advisory group,Preston-Shoot, and assessment of types of public define the scope and interpret2005, UK law in social work findings educationSmith et al, User involvement Members of 24 Collaboration (user-2009; UK in nursing, midwifery national service researcher) and consultation and health visiting user organisations (advisory group) researchSerrano- Degenerative 53 patients with Consultation - scoping theAguilar et al, ataxias degenerative review, Choice of outcomes,2009 ataxias Delphi methodCarr and Patients’ 2 service user User-led, All stages of theFleischman et perspectives on researchers with direct research. Use of user-al, 2007 Electro-convulsive experience of ECT and researchers and advisory therapy representatives of user group and voluntary groups with a stake in ECT research
  9. 9. Summary of included papers (2)Paper and Topic area Public Stage, method and typecountry of involvementRees and HIV health promotion for Representatives of a Consultation – define theOliver, 2007. men who have sex with number of health scope of the review,UK men promotion charities e.g. agree outcomes and Terrence Higgins Trust comment on findings. Advisory groupBrett et al, The conceptualisation, 3 lay people on advisory Collaboration –2007. UK measurement, group. Members of membership of advisory impact and outcomes of UNTRAP and Diabetes group; consultation via a public involvement in User Network Research workshop to discuss health research Group findingsNilsen et al, Methods of consumer A panel consisting Consultation through2006 involvement in developing of members of the email discussion list. healthcare policy and Cochrane Consumer Consulted on the review research, clinical practice Network. protocol, literature search, guidelines and patient and the initial draft of the information material review
  10. 10. Reported contributions of the public to systematic reviews1. Refining the scope of the review – In the review of HIV health promotion for MSM, the public identified HIV+ and young MSM as population subgroups on which the review should focus1. Suggesting and locating relevant literature – In the review of patients’ experiences of ECT, a user group located an unpublished study of ECT for the benefit of the review1. Appraising the literature – User researchers were involved in the review of patients’ experiences of ECT and appraised the literature1. Interpreting the findings – Two reviews reported the use of consultation workshops where preliminary findings were discussed with the public1. Writing up the review – Three of the reviews reported the public being involved in the writing up stage of the review process: either as first author, co-author, or as the author of the review’s foreword
  11. 11. Tensions identified when involving thepublic in the review process• Time pressures – It can take time to build up trust between researchers and the public in the review process• Resourcing problems – Lack of funding at the review development (pre-grant) stages to support involvement• Continuity issues – It can be difficult for the public to attend all review meetings• Concerns about group dynamics – The user-led review of experiences of ECT reported tensions regarding the amount of power the user-researchers had in the review process• Research Ethics Committee involvement – One review reported that they had to obtain approval from a REC to involve the public in the review process• Tensions associated with the representativeness of the members of the public involved – Some reviews mentioned that those who get involved in reviews will not be representative of the wider population
  12. 12. Good practice recommendations for involvingthe public in systematic reviews• Funding and payment – The public should be paid for the time they spend contributing to reviews and that this should be budgeted for• Identifying a lead for public involvement – This person should be the contact point for the public and this role should be costed into the review budget• Training, briefing and information provision – Provision of training, briefing notes and background information (such as a glossary) has been recommended. The extent of this will vary depending on the level of involvement.• Structured methods of involvement – Two reviews used structured processes (Delphi and voting at meetings) to capture the views of the public during the review process. This prevents the review process being dominated by particular, more vocal individuals.
  13. 13. Conclusions and implications• The review found examples of the public being involved in the systematic review process at the 3 main levels of involvement (consultation, collaboration and user control) and using a variety of methods, including the Delphi process, advisory group meeting and consultation workshops• It was found that the public were involved at all the key stages of the review process, but no reports were found of the public being involved in meta- analysis of quantitative dataImplications:• Commissioners of systematic reviews need to recognise that public involvement in the review process means that reviews take longer to deliver whilst being more expensive• Financial support is needed to resource public involvement at the design (pre-grant) stage of systematic reviews• Abstracts of systematic reviews should describe the extent of public involvement (if any) in the review process to facilitate more accurate future reviews of the evidence base
  14. 14. Further reading• Boote J, Baird W, Sutton A (2011), Public involvement in the systematic review process in health and social care: a narrative review of case examples. Health Policy, 102, 105– 116.• Kreis J, Puhan M,Schunemann H, Dickersin K (2012), Consumer involvement in systematic reviews of comparative effectiveness research. Health Expectations, DOI: 10.1111/j.1369-7625.2011.00722.x – Explores current approaches to involving consumers in US-based and key international organizations and groups conducting or commissioning systematic reviews• Vale C, Tierney J, Spera N, Whelan A, Nightingale A, Hanley B (2012), Evaluation of patient involvement in a systematic review and meta- analysis of individual patient data in cervical cancer treatment. Systematic Reviews 2012, 1:23 doi:10.1186/2046-4053-1-23 – Key example of involvement of the public in a meta-analysis along with an evaluation of the process
  15. 15. Contact details: Dr Jonathan Boote Research Fellow and PPI Lead NIHR Research Design Service for Yorkshire and the Humber Email: Tel: 0114 222 0892 RDS website: