1. How to search the medical literature on the net
Samir Haffar M.D.
Associate Professor of Gastroenterology

2. My students are dismayed when I say to them
“Half of what you are taught as medical students
will in 10 years have been shown to be wrong.
And the trouble is, none of your teachers knows
which half.”
Dr Sydney Burwell
Dean of Harvard Medical School

3. The Term “Evidence-Based Medicine”
The term EBM was first used by
David Sackett & his collegues at
McMaster University in Ontario,
Canada in the early 1990s.
Father of EBM

4. Glasziou P, Del Mar C & Salisbury J. Evidence based medicine Workbook.
BMJ Publishing Group – First edition – London – 2003.
Clinical epidemiology
Evidence-based medicine
Evidence-based practice

5. The 3 components of EBP
“EBM is the integration of best research evidence
with clinical expertise & patient values”
- David Sackett
EBM
Patient
values
Clinical
Expertise
Best research
evidence

6. What is the best evidence?
• The best evidence is the evidence most likely to
provide an unbiased view of the truth
• Bias is difference between study results & truth
• Of course, we can never know the truth, but we can
try to come as close as possible by performing &
using well-designed & well executed studies

7. Observational
Study types
Interventional
or experimental
RCTCohort study
Case-control study
Cross-sectional study
Case series & case report

8. Trial designs
• Systematic review
• Meta-analysis
• Randomized clinical trial
• Cohort study
• Case control study
• Cross-sectional study
• Case series & case report
Primary research
Secondary research

9. Hierarchy of evidence in quantitative studies
McGovern D, Summerskill W, Valori R, Levi M. Key topics in EBM.
BIOS Scientific Publishers, 1st Edition, Oxford, 2001.

10. Level of evidence
Ia
II
III
IV
V
Oxford Centre of EBM
Levels of
evidence
Ib
Bias
Bias
Bias
Bias
Bias
Bias
Validity/StrengthofInference

11. 1- Ask
2- Acquire
4- Apply
5- Assess
Patient
dilemma
Principles
of EBP
Evidence alone does not
decide – combine with other
knowledge & values
3- Appraise
Hierarchy
of evidence
5 A

12. Background & foreground questions
Evolution of questions we ask as we progress from novices posing
background questions to experts posing foreground questions
Guyatt G, et al. User‟s guide to the medical literature.
Essentials of evidence based clinical practice. Mc Graw Hill, 2nd ed, 2008.

13. Background questions
• Sometimes
Involve a single fact such as:
“What is the causative agent of Chagas disease?”
“What is the hemophagocytic syndrome? ”
• Often
Involve much more information such as:
“How do I insert a jugular venous central line?”

14. Hemophagocytic syndrome
• Can masquerade as cirrhosis with ascites
• These patients have Fever
Jaundice
Hepatosplenomegaly
• Usually in setting of lymphoma or leukemia
de Kerguenec C et al. Am J Gastroenterol 2001 ; 96 : 852 – 857.

15. Searching background questions
Easy to search
• Standard textbooks such as Harrison, Nelson….
• Innovative electronic texts such as UpToDate
• Some companies group collections of textbooks
MDConsult http://www.mdconsult.com/
Stat!Ref http:// www.statref.com/
• Entry of single concept such as disease or diagnostic test

16. Textbooks
Textbooks may be a useful source of information
on “background” questions but:
• May provide biased advice
• Rapidly become out of date

17. • Questions that provide the evidentiary basis for specific
clinical decisions
• Best structured using the framework of:
- Patient
- Intervention
- Comparison (if relevant)
- Outcomes
Foreground questions
PICO/PIO

18. P
I
C
O
Patient Patient or Problem in question
Intervention Intervention, test, or interest exposure
Comparaison Comparison interventions (if relevant)
Outcome Outcome(s) of interest
Formulating a foreground question
Good question should have 3 or 4 components
Concept of PICO/PIO

19. Example of PICO/PIO Concept
• 70-year-old man with PUD & stable angina tt by low-
dose aspirin, statin, ACE inhibitor & nitrates as needed
• The patient developed recently an UGI bleeding
UGI endoscopy revealed no ulcer & biopsy was – for HP
• You wonder if clopidogrel is more effective than the
association of PPI & asipirin for recurrent PU bleeding

20. Key components of your clinical question
PICO
P
I
C
O
Patient Elderly patient with aspirin-associated PU
Intervention Clopidogrel
Comparaison PPI + Aspirin
Outcome Recurrent PU bleeding

21. Formulation of the relevant question
In a elderly patient with aspirin-associated PU,
is clopidogrel more effective than PPI & aspirin
to prevent recurrence of PU bleeding?

22. Medical literature
• Published
22 000 biomedical journals
≈ 5 000 journals indexed in Medline
17 000 biomedical books annually
• Gray literature
Conference proceedings
Reports
Theses
Unpublished studies
Glasziou P et al. Systematic reviews in health Care – A practical guide.
Cambridge University Press, Cambridge, 1st edition, 2001

23. The amount of medical literature
0
500000
1000000
1500000
2000000
2500000
Biomedical MEDLINE Trials Diagnostic
MedicalArticlesperYear
5,000
per day
1,500
per day
95 per
day
55 per
day
Glasziou P, Del Mar C. Evidence based practice workbook.
Blackwell Publishing, 2nd edition, 2007.
So much evidence, so little time

24. How can I find what I am searching for?

25. 50,000 articles/yr
from 120 journals
~3,000 articles/yr
meet critical appraisal
& content criteria
(94% noise reduction)
McMaster PLUS project – First level
Critical appraisal filters
 Valid
 Ready for clinical attention
Health Information Research Unit – McMaster University – Canada

26. ~3,000 articles/yr meet
critical appraisal
& content criteria
(95% noise reduction)
McMaster PLUS Project
Clinical Relevancy Filter (MORE)*
~5-50 articles/yr for
authors of evidence-
based clinical topic
reviews
~25 articles/yr for
clinicians (99.95%
noise reduction)
* MORE McMaster Online Rating of Evidence

27. High quality/relevant data – Pearls
Glasziou P, Del Mar C. Evidence based practice workbook.
Blackwell Publishing, 2nd edition, 2007.
Finding high-quality evidence like searching for „rare pearls‟

28. High quality/relevant data
Pearls
If not valid No value
If not relevant No value

29. Availability
Most efficient resources are expensive
• Academic physicians
Free access of their medical school or hospital libraries
• Physicians in private practice in reach countries
Free access to some resources by professional associations
• Health professionals in developing countries
Institutional access through the WHO HINARI 1 project
Free access to developing countries (BMJ PG 2 – NEJM ..)
1 HINARI*: Health InterNetwork Access to Research Initiative
http://www.who.int/hinari/
2 BMJ PG: British Medical Journal Publishing Group

30. Free medical resources on the net
• PubMed Central
• Certain journals
CMAJ – BMJ
Most BioMed Central journals: www.gfmer.ch
• Merck Manual: www.merck.com
• Free Medical Journals: freemedicaljournals.com

31. PubMed Central

32. PubMed Central

33. PubMed Central (A – B)

34. • Very popular with generalists, specialists, & house staff
– Ease of use
– Comprehensiveness
– Inclusion of disease-oriented formation
• Provides recommendations (guidelines) for clinicians

35. Synthesis
< 50 000
• Systematic review of articles
• Clinical practice guidelines
• You assess the information & make decisions

36. Synthesis
Category Comprehensiveness Easy to
use
Cost
nearest $50 late 2007
CDSR1 Therapy & prevention Easy $300 whole library
Free abstracts
Clinical practice
guidelines2
US & other nations
Compare guidelines
Easy Free
Many free full-text
UK national library
for health
Easy
Ontario Medical
Association
Collection of
preappraised guidelines
Easy Free summaries
Many free full-text
1 CDSR: Cochrane Database of Systematic Reviews
2 Clinical practice guidelines: US National Guidelines Clearinghouse database

37. Synthesis
Category Website
Systematic Reviews
CDSR1 http://www.cochrane.org/
Guidelines:
Clinical practice guidelines2
UK National Library for Health
Ontario Medical Association
http://www.guideline.gov/
http://libraries.nelh.nhs.uk/
http://www.gacguidelines.ca/
1 CDSR: Cochrane Database of Systematic Reviews
2 Clinical practice guidelines: US National Guidelines Clearinghouse database

38. Cochrane collaboration
http://www.cochrane.org
Database available free online in many countries

39. Smith G. D. et al. BMJ 1998 ; 316 : 221 - 225
Pioneer in health services research
His visions are at the heart of Cochrane Collaboration
Archie Cochrane (1909 – 1988)
British physician & epidemiologist

40. In a 1979 article*
‘It is surely a great criticism of our profession that we
have not organized a critical summary, by specialty or
subspecialty, adapted periodically, of all relevant RCT’
** Cochrane AL (1979). 1931–1971: A critical review, with particular reference to medical
profession. In: Medicines for Year 2000, Office of health economics, London.
Archie Cochrane

41. Corticosteroids for preterm birth
• 1972 RCT showing improved outcomes for
preterm babies when mothers were given
short course of corticosteroid before birth
• 1972- 89 6 RCTs published confirmed 1972 findings
• 1989 First published meta-analysis
• 1989-91 Seven more studies reported
Corticosteroid reduces odds of babies dying from immaturity by
30 – 50% but thousands of babies died or suffered unnecessarily
since 1972 because doctors did not know about effectiveness of tt

42. Cochrane collaboration
http://www.cochrane.org
Database available free online in many countries

43. The Cochrane collaboration
Established in september 1992 in Oxford
• Cochrane reviews provide substantial amount of
evidence in health care & widely cited in guidelines
• > 6000 people in > 100 countries working for Cochrane
• 25 further centers all over the world
• Much of this work unpaid but vital financial support
has come from governments & other funders

44. Cochrane centers & branches
25 centers & branches

45. The Cochrane library
• CDSR (Cochrane Database of Systematic Reviews)
13 385 reviews as of the 2008, Issue 1
1 786 posted protocols of reviews in progress
• CCTR (Cochrane Controlled Trials Register)
Largest electronic compilation of CT in existence
527 885 citations as of the 2008, Issue 1
• DARE (Database of Reviews of Effects)
Free abstracts – Thousands

46. The Cocrhane collaboration
http://www.cochrane.org

47. Primary Studies
Millions
• Original or primary studies
eg, those stored in MEDLINE & EMBASE
• Needs evaluation before application to clinical
problems

48. Finding good answers
Primary studies

49. Primary studies
Category Comprehensiveness Easy to use Cost
nearest $50 late 2007
CCTR1 All specialties
Best way to find published CT
Easy to use $300 whole library
Free abstracts
1 CCTR Cochrane Controlled Trials Registry
2 CINAHL Cumulative Index to Nursing & Allied Health Literature
3 EMBASE Large European database
PubMed All disciplines
Clinical application articles
Hard to use
(Large size)
Free
CINAHL2 Nursing database Hard to use
(Large size)
High cost
EMBASE3
Similar to PubMed Hard to use
(Large size)
High cost

50. Primary studies
Category Website
CCTR1 http://www.cochrane.org/
PubMed http://www.ncbi.nlm.nih.gov/
CINAHL2 http://www.cinahl.com/
EMBASE3 http://www.elsevier.com/
1 CCTR Cochrane Controlled Trials Registry
2 CINAHL Cumulative Index to Nursing & Allied Health Literature
3 EMBASE Large European database

51. US National Library of Medicine
Moved to its current quarters in Bethesda
on campus of NIH in 1962
Collections include > 7 million books,
journals, technical reports, microfilms
manuscripts, photographs & images
Logo of the NLM
The world's largest medical library

52. • Interface to MEDLINE (NLM*) & easiest way to use it
• From 1950 to date
• 19 million articles as of November 2009
• Growing at rate of 700 000 articles/year
• ≈ 5 000 indexed journals
• > 70 million search done each month
• Search terms by topics, authors or journal
Free on the Internet since the mid-1990s
* US NLM: United States National Library of Medicine

53. Annual addition of articles to PubMed
50 years ago Majority of research not published in English
Currently Almost 90% of articles published in English

54. PubMed tutorial
• Side bar of the PubMed entry page
• Quite detailed tutorial program
• Takes about 2 hours to go right through
• Very helpful

55. PubMed Tutorials
http://www.ncbi.nlm.nih.gov/

56. EMBASE
• Large European database (> 11 million citations)
• Similar to MEDLINE in scope & content
• Strengths in drugs & allied health disciplines
• Clinicians are unlikely to use EMBASE (high cost)
• 70% of citations not included in MEDLINE
http://www.elsevier.com/

57. Medline
Embase
Cochrane Trials Registry
Comparing Medline & Embase
Overlap of 2 databases: 34%
MEDLINE: better coverage of US journals
EMBASE: better coverage of European journals
Smith BJ et al. Med J Aust 1992 ; 157 : 603 – 11.

58. Key statistics for MEDLINE & EMBASE
MEDLINE EMBASE
No. of journal articles 10 million 8 million
No. reference added annually 400 000 references 415 000 references
No. of journals 3900 journals/40 languages 4000 journals/70 countries
Specific thesaurus for indexing MeSH EMTREE
English abstracts 76% of references 80% of references
Date of coverage 1950 to present 1974 to present
Availability Internet (free) – CD Internet (expensive) – CD
Journals published in US 52% 33%
English language articles 88% 90%
Records on database Twice a week 15 days of journal receipt
Egger M et all. Systematic reviews in health care: Meta-analysis in context.
BMJ Publishing Group, London, 2nd edition, 2001.

59. Searching for primary studies
• MEDLINE (US NLM*) one of most available systems
PubMed interface of MEDLINE
• Useful tutorial with trial & error learning
• Type sentence or phrases representing what you search
• Choice of terms based on PICO/PIO format
* US NLM: United States National Library of Medicine

60. Key components of your clinical question
PICO
P
I
C
O
Patient Elderly patient with aspirin-associated PU
Intervention Clopidogrel
Comparaison PPI + Aspirin
Outcome Recurrent PU bleeding

61. Text word & MeSH term*
• Text word
Any word or phrase in the title or abstract of an article
• MeSH terms
Vocabulary used to index all MEDLINE articles
• Example:
“hemorrhage” [Text Word]
“haemorrhage” [Text Word]
“hemorrhage” [MeSH Terms]
* MeSH: Medical Subject Headings in PubMed

62. MeSH Database

63. Tree structure of MeSH in PubMed
The „explode‟ (exp) feature of tree structure allows to
capture entire subtree of MeSH* terms within single word
One of 3 MeSH categories of peptic ulcer
* MeSH: Medical Subject Headings in PubMed

64. Alternative spellings & misspellings
(a) hemorrhage American spelling
[Mesh term]
(b) haemorrhage British spelling
(c) hemorhage Misspelling
(d) haemorhage Misspelling

65. Boolean operator ‘OR’
After its creator George Boole
Akobeng AK. Arch Dis Child 2005 ; 90 : 837 – 840.

66. Akobeng AK. Arch Dis Child 2005 ; 90 : 837 – 840.
Boolean operator ‘AND’
After its creator George Boole

67. Akobeng AK. Arch Dis Child 2005 ; 90 : 837 – 840.
Boolean operator ‘NOT’
After its creator George Boole

68. Boolean search
After its creator George Boole
Boolean operators Meaning
Peptic ulcer OR Hemorrhage
Peptic ulcer AND Hemorrhage
Peptic ulcer NOT Hemorrhage

69. ‘Wildcards’
malig* means malignancy
malignant
malign
but not malunion
malaligned
etc

70. PubMed command – 1
Command What it does?
OR Finds articles containing either of specified words
Ex: child OR adolescent
AND Finds articles containing both specified words
Ex: child AND adolescent
NOT Excludes articles containing the specified word
Ex: child NOT adolescent
Glasziou P, Del Mar C. Evidence based practice workbook.
Blackwell Publishing, 2nd edition, 2007.
Limits Articles restricted in several ways by: date, language, age, …
( ) Use parentheses to group words
Ex: (colorectal cancer) AND (screen AND mortality)

71. PubMed command – 2
Command What it does?
*
Truncation
Acts as a wildcard indicating any further letters
Ex: child * = (child OR childs OR children OR childhood)
Glasziou P, Del Mar C. Evidence based practice workbook.
Blackwell Publishing, 2nd edition, 2007.
[ti] Finds articles with the word in title
Ex: ulcer [ti] = find article with the word ulcer in title
[so] or so Find articles from a specific source
Ex: ulcer AND BMJ [so] = finds articles on ulcer in BMJ
MeSH Controlled vocabulary keywords used in also in Cochrane
Often useful to use both MeSH heading & text words

72. PubMed translation of query into search terms
PICO Element Search terms for PubMed
P Elderly patient
Peptic ulcer
Limit to “aged: 65+ years”
“peptic ulcer” [MeSH term]
I Clopidogrel “clopidogrel” [MeSH term]
C PPI
Aspirin
“proton pump inhibitors” [MeSH term]
“aspirin” [MeSH term]
O Recurrence
Peptic ulcer bleeding
“recurrence” [MeSH term]
“peptic ulcer hemorrhage” [MeSH term]
Other RCT Limit to “randomized controlled trial”

73. PubMed Clinical Queries
• Search faster & more relevant articles
• Category
Etiology, dg, therapy, prognosis, clinical prediction guides
• Search
Narrow specific search
Broad sensitive search

74. PubMed Clinical Queries

75. PubMed Clinical Queries

77. N Engl J Med 2005 ; 352 : 238 – 44.
Clopidogrel vs aspirin & esomeprazole
Difference 7.9 % points
95 % CI: 3.4 – 12.4
P = 0.001 by log-rank test

78. How to improve your PubMed search?
• “Related Articles” link
• Select another set of terms & try again
• “AbstractPlus” format
• “Limits” link: sex, age, language, RCT,…
• “MeSH” terms: Medical Subject Headings

79. Related articles link

80. “Limits” link

81. “Limits” link

82. Question type & study design
Study DesignQuestion
Intervention RCT
Incidence & prognosis Cohort study
Prevalence Cross-sectional study
Etiology & risk factors Cohort or case-control
Diagnosis Cross-sectional study
In each case, SR of all available studies better than individual study

83. How many questions can doctors answer each day?
64 residents in 2 hospitals
401 consultations
280 questions (2 for 3 patients)
80 answers (29%)
Sources of answers:
– Textbooks (31%)
– Articles (21%)
– Consultants (17%)
Green ML et al.
Am J Med 2000;109:218–233.
Residents
103 GPs
Questions collected over 2.5 d
1101 questions
702 answers (64%)
Sources of answers:
– Available print
– Human resources
– Literature search: 2 answers
Ely JW et al.
BMJ 1999;319: 358–361.
GPs

84. Steps
– Answerable question
– Search
– Appraise
– Apply
EBM & Systematic Review
Find a systematic review
Steps
– Answerable question
– Search ++++
– Appraise x 2
– Synthesize
– Apply
Time: 6 months – Team
2 000 articles
This patient dies
Systematic reviewEBM
Time: 90 seconds
< 20 articles
This patient survives

85. Type of Question
RCTs
Appraisal
Stop
Others
Diagnosis, prognosis, etiology
No RCTs
PubMed Clinical Queries
Low quality
PubMed general
Other databases
No Study
Stop
Appraisal
Cohort
Cross-sectional
Stop
SR
Appraisal
PubMed
Clinical Queries
Intervention
Cochrane
SR
Stop
Cochrane
Library

86. Type of Question
Intervention Others
Diagnosis, prognosis, etiology
Cochrane
SR
Stop
SR
Appraisal
Stop
RCTs
Appraisal
Stop
No RCTs
PubMed Clinical Queries
Stop
Appraisal
Cohort
Cross-sectional
Low quality
PubMed general
Other databases
No Study
Cochrane
Library
PubMed
Clinical Queries

87. Limitations of evidence
• Evidence simply doesn‟t exist
• Some of what is available
Not easily accessible
Not clinically relevant
Not of good quality
Not applicable to your patient
It takes time to develop high quality evidence
& more time to get it to publication

88. Learning through play
• Try all “buttons”
• Make lots of “mistakes”
• Have fun

89. Mc Graw Hill
2008
References
BMJ Publishing Group
2003
Elsevier
2009

90. Thank You