Systematic Reviews: the researcher's perspective and the research question. Edoardo Aromataris
Upcoming SlideShare
Loading in...5
×
 

Systematic Reviews: the researcher's perspective and the research question. Edoardo Aromataris

on

  • 1,948 views

Health Libraries Australia Professional Development Day 2012 [Keynote address - part 1]

Health Libraries Australia Professional Development Day 2012 [Keynote address - part 1]

Statistics

Views

Total Views
1,948
Views on SlideShare
1,948
Embed Views
0

Actions

Likes
0
Downloads
49
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • What the next hr is all aboutThe next hour is really about setting the scene for the systematic review and the librarians potential role throughout - where any review project starts - with the question – the most important part in the systematic process. How the researcher will be organised.I’d like to cover a bit about what, as researchers, most of us do know (which may be surprisingly little), what most of us should know, particularly about question development.Throughout i’d like to keep in mind the interaction between researchers and librarians, and some of the things that work and don’t work from the outset and can help them move forward.A lot of this and some of the latest considerations and innovations I know are the topics of importance that will be picked up on and addressed by this afternoon’s speakers.
  • However, considering the broad nature of the topic suggested another option is to conduct scoping review, asking a general question, for example ‘What smoking cessation interventions (population level) does the published evidence suggest are effective and represent ‘value for money’? Effective in this context could represent effectiveness of interventions in terms of health benefit in return for dollars invested (cost effectiveness), and/or outcomes in terms of dollar benefit in return for dollars invested (cost benefit). In the absence of a specific research question to guide the approach of a systematic review, a scoping review of the literature works best and allows multiple interventions and a range of outcome measures to be reviewed. The scoping review uses similar systematic and methodical methods as a systematic review, however studies are selected generally on relevance rather than appraisal and the key data related to the costs and outcomes (health and/or costs) extracted from individual studies to create an overview of the evidence in relation to the question.
  • However, considering the broad nature of the topic suggested another option is to conduct scoping review, asking a general question, for example ‘What smoking cessation interventions (population level) does the published evidence suggest are effective and represent ‘value for money’? Effective in this context could represent effectiveness of interventions in terms of health benefit in return for dollars invested (cost effectiveness), and/or outcomes in terms of dollar benefit in return for dollars invested (cost benefit). In the absence of a specific research question to guide the approach of a systematic review, a scoping review of the literature works best and allows multiple interventions and a range of outcome measures to be reviewed. The scoping review uses similar systematic and methodical methods as a systematic review, however studies are selected generally on relevance rather than appraisal and the key data related to the costs and outcomes (health and/or costs) extracted from individual studies to create an overview of the evidence in relation to the question.
  • Broad groups of who may walk through your door looking to do a systematic review....not mutually exclusive by any means!All sorts – its a methodology that anyone can and does try, so whilst they are knowledgeable in their topic, many aren’t knowledgeable in the methodology, or what research is all about.Subpoint 1 – or soon to be up and comer. This group know their subject matter.Pt2 – well versed in methodsPt 3 – Clinical skills – consume research (or expectation that they do with today’s evidence based/best practice rhetoric) foray into research for the first time Systematic review is an attractive option. Use researcher/nurse switch as example.Pt 4 – #3 may be #4 also. Never done research, struggling to understand – maybe never will
  • Which one have you got in front of you? Irrespective of the answer, the researcher interested in the systematic review will come to you with the most important thing in the entire systematic process (more after morning tea) of doing a systematic review, and that’s their question.Problems?Some actually don’t know what a systematic review is! They think its a literature review. Its research!Who they are may or may not have some impact on their ability to articulate their question and not just articulate it, to actually know what they’re asking...!
  • Least romantic – experimental data collection/collect the numbers necessary.
  • Last point....because the researchers just hasn’t through about it logically
  • Black magic
  • Black magic of search platforms and the algorithms they use.
  • So much we researchers fail to understand and need to understand to have a successful review team, similarly, without doubt, if the librarian understands the subject matter things move more easily, you can over come some of the inadequacies of the researcher
  • Both ‘research’ and ‘search’ are art and science, need to understand and appreciate both.

Systematic Reviews: the researcher's perspective and the research question. Edoardo Aromataris Systematic Reviews: the researcher's perspective and the research question. Edoardo Aromataris Presentation Transcript

  • Systematic reviews: theresearcher‟s perspective and the research question Edoardo Aromataris www.joannabriggs.edu.au
  • Questions for you• Where do you work? What settings? – Health care? Hospital, University, government health service?• Who‟s had some experience with systematic reviews? – Was it...pleasant?• What is a systematic review? www.joannabriggs.edu.au
  • Researcher‟s perspective• The review question• What do we know?• What should we know? – PICO/PICo• Depends on who we are...• Interaction between researchers and librarians• What to expect from us www.joannabriggs.edu.au
  • Steps in a Systematic Review• Formulate review question• Define inclusion and exclusion criteria• Locate studies• Select studies• Assess study quality• Extract data• Analysis/summary and synthesis of relevant studies• Present results• Interpret results/determining the applicability of results (Egger & Davey Smith, 2001:25; Glasziou et al., 2004:2) www.joannabriggs.edu.au
  • Question Development• Aim is to provide a framework for the development and conduct of the review• A good question supports the review, a poor question risks confounding the review• A good question responds to identified priorities and needs www.joannabriggs.edu.au
  • Question DevelopmentDivide question following the PICO/PICo model• Reviews of effects & • Reviews of qualitative & economics: Textual data: – Population – Population – Intervention/Exposure – Phenomena of Interest – Comparator – Context – Outcome – Types of Study Design www.joannabriggs.edu.au
  • Questions of the effects of interventions• Population: – The most important characteristics, including: • demographic factors of the population (e.g. age, gender, ethnicity) • socioeconomic factors • the setting (e.g. hospital, community etc) • illness www.joannabriggs.edu.au
  • Questions of the effects of interventions• Intervention and Comparator – Primary intervention of interest (treatment group) – Comparator (control group) • Passive (placebo, no treatment, standard care, or a waiting list control) • Active (variation of the intervention, a drug, or kind of therapy) www.joannabriggs.edu.au
  • Questions of the effects of interventions• Outcomes – Identify the primary outcome/s in order to reach a clinically relevant conclusion – Secondary outcomes may be required – Avoid use of surrogate outcomes unless clearly reasoned in the background – Consider how the type and timing of outcome measurements impacts on outcome measurement www.joannabriggs.edu.au
  • Example intervention comparison • Are antiseptic washes more effective than non- antiseptic washes at preventing nosocomial infections in patients undergoing surgery? outcome population www.joannabriggs.edu.au
  • Example intervention comparison • Are antiseptic washes more effective than non- antiseptic washes at preventing nosocomial infections in patients undergoing surgery? outcome population www.joannabriggs.edu.au
  • http://guides.library.upenn.edu/content.php?pid=192036&sid=1610308
  • Questions of the experiential evidence• Qualitative and textual reviews: – Re-focus to phenomena of Interest, not intervention, – and Context not comparator• The phenomena of Interest relates to a defined event, activity, experience or process• Context is the setting or distinct characteristics www.joannabriggs.edu.au
  • Example Phenomena of interest population• What are caregivers experiences of providing home-based care to persons with HIV/AIDS in Africa? Context www.joannabriggs.edu.au
  • Example Phenomena of interest population• What are caregivers experiences of providing home-based care to persons with HIV/AIDS in Africa? Context www.joannabriggs.edu.au
  • PICO / PICo• Constructing a well-built clinical question is a fundamental skill• Question should be arranged following the PICO/PICo model• The question operationalises the review by forming the basis for inclusion and exclusion criteria and the search strategy www.joannabriggs.edu.au
  • Inclusion Criteria: Effects• Draws upon: – Population characteristics – Intervention or exposure – Comparator - active or passive – Outcomes of interest• Study type and other elements of the review such as language, year of publication etc www.joannabriggs.edu.au
  • Inclusion Criteria: Experience• Draws upon: – Population characteristics – Phenomena of Interest – Context• Study type and other elements of the review such as language, year of publication etc www.joannabriggs.edu.au
  • Different questions lead to differentevidence• Effectiveness of therapy• Adverse events and harmful outcomes of therapy• Aetiology of disease• Diagnosis and diagnostic test accuracy• Prognosis www.joannabriggs.edu.au
  • Focused versus broad question• Does SSRI antidepressant increase the risk of suicide in teenagers? – Clear guide to the data the reviewer will look for and will arrive at clear conclusions. – Identified outcome - miss full safety profile of drug• What are common adverse events when starting on antidepressant therapy? – Find new events across a range of drugs – Vast amounts of heterogeneous data difficult to synthesise and draw conclusions from. www.joannabriggs.edu.au
  • Broader question?• Is a systematic review appropriate? – „What smoking cessation interventions (population level) does the published evidence suggest are effective and represent „value for money‟? – „What are the main risks to patient safety in primary care?‟• Consider a scoping review www.joannabriggs.edu.au
  • Question Development• Verify that the question has not already been addressed – Search protocols and systematic review reports in the JBI and Cochrane Libraries and others• May be similar topic – shouldn‟t be the same question unless justification is provided – Critically appraise and assess quality of review www.joannabriggs.edu.au
  • Protocol• Guide process – Reasoned approach to the question asked• Detailed review methods a priori• Decrease biased post hoc changes”• Main details are question, eligibility or inclusion criteria, approach to search and remainder of steps www.joannabriggs.edu.au
  • Review Team• Funder• Content experts, researchers• Methodologists with experience in systematic reviews• Statistician• Librarian• End users - policy makers/decision makers www.joannabriggs.edu.au
  • Researchers and Systematic Reviews• Types/breeds of researchers – Academic/expert in their field – Consultant – Clinician – Student/novice• Not always mutually exclusive• Experience? www.joannabriggs.edu.au
  • Breeds...• Academic – Know the topic, do they know the method?• Consultant – May know it all, may know nothing, or somewhere in between... – Still have some grounding in research and what its all about – Methodologist advantage• Clinician – Know the need/questions – Not how to approach the research www.joannabriggs.edu.au
  • PICO / PICo• Constructing a well-built clinical question is a fundamental skill• Divide question following the PICO/PICo model• The question also forms the basis for the search strategy – PICO concepts www.joannabriggs.edu.au
  • From PICO to search• Nursing management of cancer fatigue in patients• Concept map
  • Researchers knowledge• Led from the question through PICO....• ...define eligibility criteria....• ...into an example search strategy...• ....told where they can search...• Onto the next step www.joannabriggs.edu.au
  • Knock, Knock....• ...who‟s there? – Academic/methodologist/clinician – Experience with systematic reviews?• Not everyone understands what a systematic review is!• Who are they and where are they coming from?• The question....what is it? www.joannabriggs.edu.au
  • What‟s the question...?• What concepts? – Same words - different definitions and meanings – Different words – same definitions and meanings• Not everyone comes armed with their own question – Contract/consultancy – Researcher may be the middle man www.joannabriggs.edu.au
  • Researchers may fail to appreciate...• Search is incredibly important part of the review process• No evidence = no review• How and where are some of the defining features of the review process• Will understand „reproducibility‟ www.joannabriggs.edu.au
  • Researcher‟s question• “...major difficulty being able to conceptualise their question...”• “...can‟t describe what they‟re trying to do...”• You may be left to (help) interpret aims of the project! www.joannabriggs.edu.au
  • Search strategy• “...no capacity to be logical...”• “..logic grid doesn‟t reflect what she‟s trying to do...”• “...its not rocket science...”• “...essential elements are language and logic...” www.joannabriggs.edu.au
  • Search strategy• Many researchers understand PICO, they just fail to follow it• Many questions wont have all of the elements of a PICO question• Maybe different elements – Settings, study design, aspects of population/intervention www.joannabriggs.edu.au
  • Search Strategy• Strategy is more important than the search itself and requires a great deal more time – Results driven researcher www.joannabriggs.edu.au
  • Databases• Fail• No idea• “...I‟m going to do a search of OVID....”• Search “PubMed and OVID Medline...”• How much to explain? www.joannabriggs.edu.au
  • ...works both ways...• Can‟t do the experiment unless you understand the tools you‟re working with• Researchers and reviewers are told to refer to the librarian• Need to understand the nature of the work and maybe even the topic www.joannabriggs.edu.au
  • Summary• Question• PICO and conceptualising a question• Researchers come in all shapes and sizes• Search – Critical for review success• Many will not understand what you do, why you do it and the impact it will have on their systematic review! www.joannabriggs.edu.au