• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Systematic Reviews as a Source of Useful Evidence: The Experience of the EPPI-Centre - Professor Sandy Oliver, Institute of Education
 

Systematic Reviews as a Source of Useful Evidence: The Experience of the EPPI-Centre - Professor Sandy Oliver, Institute of Education

on

  • 1,225 views

 

Statistics

Views

Total Views
1,225
Views on SlideShare
1,225
Embed Views
0

Actions

Likes
0
Downloads
11
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Oct 05 added ‘hidden’ before bias Feb 06, added research question and protocol March 06, removed ‘sustainable’ from 4 th bullet. Added need for reflexivity
  • This is what we learnt Earlier reviews were predominantly statistical meta-analyses with no mention of… Health promotion specialists considered them totally inadequate justification for intervening in women’s lives, and possibly interrupting relationships with their family and maternity carers.
  • The current review evidence is based on women’s and health practitioners’ views. It addresses… It is cited in policy support documents internationally
  • Attending a symposium like this, one would be forgiven for thinking that a systematic review is a statistical meta-analysis just like this.

Systematic Reviews as a Source of Useful Evidence: The Experience of the EPPI-Centre - Professor Sandy Oliver, Institute of Education Systematic Reviews as a Source of Useful Evidence: The Experience of the EPPI-Centre - Professor Sandy Oliver, Institute of Education Presentation Transcript

  • Systematic reviews as a source of useful evidence the experience of the EPPI-centre TOWARD AN EVIDENCE-BASED DEVELOPMENT POLICY Launch of the UK office of 3ie at LIDC 11 October 2010, London Sandy Oliver
  • Outline
    • The EPPI-Centre story of systematic reviews
      • Addressing knowledge needs
      • Clinical model to social influence model
      • Diversity in methodology
    • The broader history of systematic reviews for decision-making
      • Innovation and challenge
      • Achievements and culture change
      • Capacity building for development policy
      • Barriers and drivers for innovation
  • Part 1: The EPPI-Centre story
  • Systematic reviews/ evidence syntheses
    • Answer important questions by looking at prior research
    • Bring together and ‘pool’ the findings of primary research
    • Not just a report of the findings of the individual studies in a review
    • Synthesis of the findings of all the included studies in order to answer the review question
  • Reviews should ( just like primary research ):
    • be pieces of research - following principled methods and a research question, with a protocol and reflection on own strengths and limitations
    • take steps to reduce hidden ‘bias’ and ‘error’
    • be accountable, and in some sense replicable and updateable
  •  
  • Support and tools for review groups: (60+ groups, c130 reviews), Education, criminology, employment, speech and language, social care, International development Conducting reviews since 1993 In health promotion, education, transport, social care, work and pensions On-line libraries of research evidence Short courses and Masters level courses in research synthesis Formal links with Cochrane and Campbell Collaborations Methodological work, e.g. Methods for Research Synthesis Project ESRC National Centre for Research Methods
  • Addressing knowledge needs
  • Early reviews disappointing…
    • Few randomised controlled trials evaluating health promotion in the community, mostly American
    • Randomised controlled trials evaluating clinic-based health promotion did not provide sufficient evidence to convince practitioners and service planners
  • Knowledge needs assessment
    • Service planners wanted reviews reporting...
    • clear justifiable methods
    • AND
    • questions framed with professionals and service users
    • detailed descriptions of interventions, resources and training
    • whether interventions worked, AND how and why
  • Peer delivered health promotion
    • What is the effectiveness?
    • > half trials showed at least one positive effect on behaviour
    • ? specific characteristics of an effective model
    • peer leaders delivering didactic information does not change behaviour
    • What is the appropriateness?
    • negative views about peer education
    • not all schools could cope with adult/ young people partnerships
    • young people at greater risk of adverse health behaviours difficult to reach
    • young men more reluctant to take on the role of peer educator
    • Harden A, Oakley A, Oliver S (2001) Peer-delivered health promotion for young people: A systematic review of different study designs. Health Education Journal 60 : 339-353.
  • Smoking cessation in pregnancy Practitioner response to systematic reviews
    • 1995
    • Statistical meta-analyses
    • No mention of
      • Potential harms
      • Theory underpinning interventions
      • Emotional and social outcomes
      • Social context of women smoking
      • Information for implementation
    • Totally inadequate justification for intervening in women’s lives
  • Smoking cessation in pregnancy stakeholder influence of evidence
    • 1995
    • Statistical meta-analyses
    • No mention of
      • Potential harms
      • Theory underpinning interventions
      • Emotional and social outcomes
      • Social context of women smoking
      • Information for implementation
    • Totally inadequate justification for intervening in women’s lives
    • 2005
    • Women’s views sought
    • Health promotion practitioners’ views sought
    • Review addresses
      • Potential harms
      • Theory underpinning interventions
      • Emotional and social outcomes
      • Social context of women smoking
      • Information for implementation
    • Cited in policy support documents internationally
  • Rapid reviews to meet policy timelines
    • To inform the work of the Prime Minister's Commission on the Future of Nursing and Midwifery
    • The socioeconomic value of nursing and midwifery: a rapid systematic review of reviews
    • To inform NICE Guidance on Quitting smoking in pregnancy and following childbirth
    • An equity analysis of a Cochrane systematic review (72 RCTs) of smoking cessation in pregnancy.
  • Clinical model to social influence model
  • Social determinants of health
    • Age, gender and constitutional factors
    • Individual lifestyle factors
    • Social and community networks
    • Living and working conditions
      • Agricultural and food production, education, work environment, unemployment, water and sanitation, health care services, housing
    • General socio-economic, cultural and environmental conditions
  • Social determinants of health
    • Age, gender and constitutional factors
    • Individual lifestyle factors
    • Social and community networks
    • Living and working conditions
      • Agricultural and food production, education, work environment, unemployment, water and sanitation, health care services, housing
    • General socio-economic, cultural and environmental conditions
  • Young people and physical activity
    • Enhancing:
    • Individuals’ knowledge
      • classroom teaching
    • Social and community support:
      • engaging parents in supporting/ encouraging children's activity
      • multi-component, multi-site interventions using a combination of school-based physical education and home-based activities.  
    • Living and working conditions:
      • education and provision of equipment for monitoring TV or video-game use
    • Brunton G, Thomas J, Harden A, Rees R, Kavanagh J, Oliver S, Shepherd J, Oakley A (2005) Promoting physical activity amongst children outside of physical education classes: a systematic review integrating intervention studies and qualitative studies. Health Education Journal 64 : 323-338.
  • Socio-economic position: PROGRESS
    • PROGRESS
    • P lace of residence
    • R ace/ethnicity
    • O ccupation
    • G ender
    • R eligion
    • E ducation
    • S ocio-economic status
    • S ocial capital
  • PROGRESS-Plus
    • PROGRESS +
    • Characteristics subject to prejudice, discrimination and stigma
    • E.g. age, disability, sexual orientation
    • Vulnerable people
    • E.g. older people returning home from hospital or respite care
    • Excluded people
    • E.g. children not attending school, ‘looked after’ children or runaways
  • Reviews with an ‘Equity lens’
    • No studies presented impact data related to participants’ gender, age, religion, education or social capital.
    • Pooling the findings in a statistical meta-analysis suggested that interventions might be less effective for people who are more socio-economically disadvantaged
    • Kavanagh J, Oliver S, Lorenc T, Caird J, Tucker H, Harden A, Greaves A, Thomas J, Oakley A (2009) School-based cognitive-behavioural interventions: A systematic review of effects and inequalities.  Health Sociology Review , 18: 61-78 .
  • Diversity in methodology
  • A systematic review… For testing a hypothesis: Cognitive behavioural interventions in school for young people’s mental health
  • A systematic review… For building theory Food in the school Chosen foods Provided foods Food in the home Influences on foods eaten Food preferences Non-influencing factors Health benefits Knowledge behaviour gap Roles and responsibilities Healthy eating concepts (understanding) ‘ Good’ and ‘bad’ foods Health consequences Limited choices Eating to socialize Contradictions Breaking rules Food rules Understandings of healthy eating
  • Synthesis methods
    • Meta-ethnography
    • Thematic analysis
    • Realist review
    • Textual narrative review
    • Framework analysis
    • Tabular methods
    • Ecological triangulation
    • Meta-analysis
    • Meta-regression
  • Synthesis methods
    • Meta-ethnography
    • Thematic analysis
    • Realist review
    • Textual narrative review
    • Framework analysis
    • Tabular methods
    • Ecological triangulation
    • Meta-analysis
    • Meta-regression
    • Theory building
    • Theory testing
  • Deriving descriptive themes Visualisation of descriptive themes
  • Part 2: The broader history of systematic reviews for decision-making
  • Innovation and challenge
  • Innovation and challenge
    • In Medicine
    • Challenged consultants’ authority within profession
    • In Nursing and Maternity Care
    • Challenged value of ‘tender loving care’
    • Challenged professional knowledge over patients
    • In health promotion
    • Challenged community development and activism
    • In education
    • Challenged ‘craft skills’ and ‘experiential knowledge’ of teachers
    • Challenged qualitative research tradition in higher education
  • Achievements and culture change
  • Changes in policy and practice
    • Evidence-informed practitioners
    • Evidence-informed shared decision-making
    • Evidence-informed activism
    • Evidence-informed guidelines across clinical health and public health
  • Changes in systematic reviews
    • Inspired the Cochrane and Campbell Collaborations
    • Wealth of evidence readily accessible, mainstreamed within guidance
    • Improved the keywording of trials on MEDLINE
    • Greater range of useful reviews
        • Reviews to generate, explore and test theories
        • Reviews of complex interventions
        • Reviews of impact, implementation, sustainability, theory, context and equity
  • Changes in primary research
    • Raised the quality of impact evaluations
    • Raised the accessibility of publications reporting trials
    • Inspired a growth in qualitative nursing research
    • Inspired a striving for relevance: in outcomes and questions
  • Capacity building for development policy
  • Barriers to capacity building
    • Infrastructure: Research and knowledge resources and conventions (e.g. availability of databases, and keywording of available databases)
    • Information and communication technology: access to databases, and internet connectivity
    • Language barriers: languages of publication, language skills of review teams
    • Methodological innovation is required for synthesising evidence in any new area
    • Novice reviewers learning review skills and simultaneously facing all the challenges above
  • What drives innovation?
    • Synthesising different types of data (quantitative, qualitative and mixed data)
    • Researching new settings or populations with different social and political contexts.
    • Requires:
        • interdisciplinary working to prepare systematic reviews
        • engaging new potential users of systematic reviews
        • transparency and clarity to enable debate about methods and how they are applied
        • multidisciplinary teams with expertise both in synthesis methods and in development studies
        • a degree of risk taking (trial and error)
  • Anticipate…
    • Challenges to policy makers, practitioners and researchers
    • Mutual learning
    • Raised standards for doing and using research
    • New methods for research and working together
    • “ If you are poor you actually need more evidence than if you are rich” Dr Hassan Mshinda, Ifakara Centre, Tanzania
  • www.ioe.ac.uk/ssru/ http://eppi.ioe.ac.uk [email_address] Thank you EPPI-Centre Social Science Research Unit Institute of Education University of London 18 Woburn Square London WC1H 0NR Tel +44 (0)20 7612 6397 Fax +44 (0)20 7612 6400 Email eppi@ioe.ac.uk Web eppi.ioe.ac.uk/