Development from the initial call for evidence-based clinical medicine in the seventies and early eighties, to the stated intention of government in the late nineties that all policy must be evidence-based. Brief outline of systematic review methodology What is a systematic review – ie, what methodology underpins it? Transparency and reproducibility How we go about identifying the needs of the researcher? Examples of recent requests Developing a systematic search strategy – the foundation for any good piece of research
Cochrane in Effectiveness and Efficiency: Random Reflections on Health Services (1972) suggested that, as resources would always be limited, they should be used to provide the forms of health care that had been shown to be effective. In particular, he stressed the importance of using evidence from properly conducted randomised controlled trials - which were likely to provide much more reliable information than other sources of evidence. In 1979 he wrote, &quot;It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials.&quot; This challenge led to the establishment in the 1980s of an international collaboration to develop the Oxford Database of Perinatal Trials and subsequently to the UK Cochrane Centre and then the worldwide Cochrane Collaboration.
From Opinion-Based Policy to Evidence-Based Policy Evidence-based policy: &apos;the integration of experience, judgement and expertise with the best available external evidence from systematic research&apos; (Davies, 1999). It’s a balance between professional judgement and expertise and the use of valid, reliable and relevant research evidence.. Sir Muir Gray (1997) has suggested that evidence-based policy and practice involves a shift away from opinion-based decision making to evidence-decision making. Evidence-based decision making draws heavily upon the findings of research that has been gathered and critically appraised. The opinions and judgements of experts that are based upon high quality up-to-date scientific research clearly constitute valid and reliable evidence. Those opinions that are not based upon such scientific evidence, but are unsubstantiated, subjective and opinionated viewpoints do not constitute high quality, valid and reliable evidence. Different types of evidence are generated by different types of research methods and research designs.
An evidence-based approach to research support: applying systematic review methods. Morgan & Mann
Fiona Morgan and Mala Mann
LILAC 1 April 2009
Identifying the Evidence Base
• Background and history
• Outline of systematic review methods
• Identifying the needs of the
• Developing a systematic search
• Archie Cochrane
– British medical researcher
– Contribution to development of
epidemiology as a science
• Cochrane Collaboration (1993)
– Improving healthcare decision-
making globally, through systematic
reviews of the effects of healthcare
Cochrane systematic reviews
• Evidence-based medicine
– Clinical answers for more effective healthcare
– Pinnacle of the evidence pyramid
• Critical summaries of the evidence
– Randomised controlled clinical trials
• Clear methodology
Expansion of evidence-based methods
• 1999 UK Government White Paper
“Government must…produce policies
that really deal with problems; that are…
shaped by the evidence rather than a
response to short-term pressures….
“This Government expects…better use
of evidence and research in policy
What does the researcher want?
• A systematic literature search?
– Finding all the information on a topic area
• Literature review
• Informs research
• A systematic review of the literature?
– Finding and using the best available evidence
to answer a specific question
A systematic review of the literature
• Explicit, reproducible methodology
– Usually conducted by a team
– Systematic literature search
– Study selection using pre-defined inclusion
and exclusion criteria
– Critical appraisal of included papers
– Data extraction
– Statistical or narrative summary
– Highlights strengths and weaknesses
Systematic review or
systematic literature search?
• Same search principles apply
• Methodology adapts effectively
• Aim is to find ALL the relevant
• The finished product only as good
as the information that underpins it.
What is a search strategy?
A search strategy is a plan that helps you
look for the information you need. [PubMed]
Developing a systematic search strategy
1. Define the question and break it down
2. Set limits
3. Search each concept and combine
4. Test and refine the search
5. Select databases and other sources
P = Population/problem
I/E = Intervention/Exposure
C = Comparison/Control
O = Outcome
Living in social
Similar area no
• In an inner-city residential area, does the
introduction of a neighbourhood watch
scheme reduce the level of crime when
compared with no watch scheme?
• In an inner-city residential area, does the
population living in social housing
experience a higher level of crime when
compared with those living in owner
Question must be….
Counsell C (1997). Annals of Internal Medicine, 127: 380-387
“Ask a poor question and you will get a
• Consider inclusion/exclusion criteria
– Age group
– Publication date
– Study design
List all relevant keywords
• Synonyms and acronyms
• American and British spellings
• Hyphenated vs non-hyphenated
Identify relevant subject headings
• Match keywords to headings
• Check how key papers and
references are indexed
• Subject headings –
– To explode or not to explode?
Wildcards & Truncation
* $(n) ? #
Manag* home$1 Colo?r Wom#n
Combining Search Terms
• Using Boolean Logic
–AND narrows the search
• Papers containing both concepts together will be
–OR broadens the search
• Articles containing either concept are retrieved
• NOT removes a concept
• Cuts a concept out of a search
Retrieves records containing terms in any order within a
specified number (n) of words of each other
eg asthma ADJ inhaler
The words must appear next to one another
“an asthma inhaler”
Eg asthma ADJ3 inhaler
The words must be within 3 of each other (in any order):
“an inhaler for asthma”
“using an inhaler for moderate asthma”
1. child abuse.mp. or Child Abuse/
2. child protection.mp.
3. (battered child or shaken baby or
4. 1 or 2 or 3
5. (child* or infant* or baby).mp.
6. non-accidental injur*.mp.
7. non-accidental trauma.mp.
8. (non-accidental* and injur*).mp.
9. soft tissue injur*.mp.
10. physical abuse.mp.
11. (or/6-10) and 5
12. 4 or 11
14. 12 and 13
15. (investigat* adj3 fract*).mp.
16. (radiolog* adj3 fractur*).mp.
17. (roentgen* adj3 fract*).mp.
18. skeletal survey.mp.
19. bone scan*.mp.
20. Isotope Bone Scan*.mp.
23. Tomography, X-Ray Computed/
24. ((paediatric or pediatric) adj3
25. ((paediatric or pediatric) adj3
27. (ageing adj3 fractur*).mp.
28. ((dating or date) adj3 fractur*).mp.
29. (pattern* adj3 fractur*).mp.
32. 26 or 31
Kemp et al. BMJ 2008;337:a1518
Searching for study designs
Collections of search terms designed to
retrieve selections of records
InterTASC Information Specialists' Sub-Group http://
Balance between sensitivity and specificity
• Proportion of relevant articles identified by a
search strategy, expressed as % of all relevant
articles on a given topic
• The ability to identify all relevant articles on a
• Proportion of relevant articles identified by the
search, as a % of all articles (relevant and
• The ability of a search to exclude irrelevant
Centre for Reviews and Dissemination, CRD Report 4 (2009)
Is the strategy adequate?
• Depends on the size of the evidence base
for the topic.
• Rough guide - 1:10 or 1:20 relevant
• Use key papers to test the search strategy
– Does the search pick up all key papers?
– If key papers have been missed why?
– Does search need adjusting?
Range of general, specialist and grey
General ASSIA, CINAHL, Cochrane Library
Embase, Medline, Science Citation
Index, SCOPUS Social Care Online,
Sociological Abstracts, Social
Sciences Citation Index
Specialist Age Line, ChildData, Community Wise,
ERIC, IBSS, IDOX PsycINFO
Grey Literature HMIC, Open SIGLE, ReGARD,
conference abstracts, websites
• Identify grey literature and new or
• Grey literature
– Conference proceedings
– Trial registers
– Internet searching
• Google scholar
• Web sites of relevant organisations
• Contacting experts
– Experts in the field and research networks
• Reference lists of relevant papers
• Systematic review methodology
should be transparent and replicable
• Record sources searched, search
strategies and results
• Use reference management software
to manage the search results
And finally…A few tips
• Check out the literature
• Are there other reviews in the topic
• What sources of information do they
use – databases, grey literature?
• How are key papers indexed in the
• Use and adapt other people’s search
strategies - it’s not plagiarism.