Shortening the distance to research evidence

Shortening the distance to
research evidence
The librarian as knowledge
translation activist
Raechel Damarell

Outline
• Search filters
– Research
– Implementation
– Dissemination
– Evaluation
• Flinders Filters: a new research initiative and
opportunity for librarians

What is a search filter?
• Tool to retrieve specific subset of literature
• Much more than an expert search
– Developed experimentally following a strict (and
usually transparent) research methodology
– Has a known level of performance in the database for
which it was developed
• The InterTASC Information Specialists' Sub-
Group Search Filter Resource

Why use a search filter?
Searching without a filter

Why use a search filter?
Searching with a filter

Types of search filters
• Methodological vs subject-based
– Cochrane HSSS; PubMed Clinical Queries
• 1st, 2nd, and 3rd generation
– subjectively derived/effectiveness unknown
– subjectively derived/effectiveness known
– objectively derived/effectiveness known

Subject-based filters
• Diagnostic imaging techniques
• Heart failure
• Nephrology
• Return to work
• Alcohol-impaired driving
• Sleep
• Ultrasound for DVT
• Knowledge translation
• Cardiovascular biology
• Occupational origin of disease
• Nurse staffing research
• Palliative care
• Patient safety
• Mental health
• Spanish studies
• Measurement properties of
measurement instruments
• Administrative data studies
• Primary health care
• Occupation health interventions
• Evidence based nursing

Methodological filters
• Adverse effects
• Diagnostic studies
• Economic evaluations
• Epidemiological studies
• Etiology
• Guidelines
• Health services research
• Observational studies
• Prognosis
• Qualitative research
• Quality of life
• RCTs and other trials
• Systematic review
• Therapy studies

Basic filter methodology
• Establish an Expert Advisory Group
– Scoping conceptual boundaries
– Preliminary consensus on potential search terms
– Advice on Gold Standard Set

The Gold Standard Set
• Set of representative citations of known
relevance indexed in the database of interest
• Target set: all citations in it should be retrieved
by final filter
• Provides means of iteratively improving a
search strategy

Gold Standard Set options
• Journal ‘hand search’ (e.g. Palliative Care Filter)
• Included references of:
– Clinical Practice Guidelines (Heart Failure)
– Systematic Reviews (Lung Cancer)
– All citations from range of specialty journals on topic (PHC)
• Others:
– Delphi study method (Best Supportive Care)
– Set retrieved by a search on the topic

• Division of Gold Standard Set
– Term Identification Set
– Filter Development Set
– Filter validation Set(s)

Performance measures
Sensitivity (or Recall)
The number of relevant citations retrieved by the search filter
from the gold standard as a proportion of the total number of
relevant citations (100% = all relevant citations retrieved)
Precision
The number of relevant records retrieved as a proportion of the
total number of records retrieved (100% = all citations retrieved
relevant)
Specificity
Number of irrelevant citations not retrieved (correct exclusions)

– Scoping conceptual boundaries
– Preliminary consensus on potential search terms
– Advice on Gold Standard Set
• Division of Gold Standard Set
– Term Identification Set; Filter Development Set; Filter
Validation Set(s)
• Implementation: making it available

Why PubMed?
• Accessibility: Largest freely available
biomedical database
• Currency: PubMed’s ‘in process’ and ‘as
supplied by publisher’ citations
• Immediacy: Search filter can be saved as
hyperlink for one-click searching
• Implementable: Can be embedded in any
webpage

The Palliative Care Filter
• Developed in 2004 based on 1999-2001
dataset
• Specificity = 99% but sensitivity <50%
• Combined with 60+ topics of clinical
importance to palliative care clinicians
• Subject of three published studies (4th
currently being written up)

What the users don’t see
((advance care planning[mh] OR attitude to death[mh] OR
bereavement[mh] OR terminal care[mh] OR hospices[mh] OR life support
care[mh] OR palliative care[mh] OR terminally ill[mh] OR death[mh:noexp]
OR palliat*[tw] OR hospice*[tw] OR terminal care[tw] OR 1049-9091[is] OR
1472-684X[is] OR 1357-6321[is] OR 1536-0539[is] OR 0825-8597[is] OR
1557-7740[is] OR 1552-4264[is] OR 1478-9523[is] OR 1477-030X[is] OR
0749-1565[is] OR 0742-969X[is] OR 1544-6794[is] OR 0941-4355[is] OR
1873-6513[is] OR 0145-7624[is] OR 1091-7683[is] OR 0030-2228[is]) AND
Medline[sb]) OR ((advance care plan*[tw] OR attitude to death[tw] OR
bereavement[tw] OR terminal care[tw] OR life supportive care[tw] OR
terminally ill[tw] OR palliat*[tw] OR hospice*[tw] OR 1049-9091[is] OR
1472-684X[is] OR 1357-6321[is] OR 1536-0539[is] OR 0825-8597[is] OR
1557-7740[is] OR 1552-4264[is] OR 1478-9523[is] OR 1477-030X[is] OR
0749-1565[is] OR 0742-969X[is] OR 1544-6794[is] OR 0941-4355[is] OR
1873-6513[is] OR 0145-7624[is] OR 1091-7683[is] OR 0030-2228[is]) NOT
Medline[sb]) AND English[la]

The Heart Failure Filter
• Developed in collaboration with Centre for
Cardiovascular & Chronic Care (UTS)
• Gold standard: Evidence based guidelines
• Sensitivity: 97%; Precision: 75%
• Methodology article published in BMC Medical
Research Methodology
• Innovative methodology for PubMed translation
(article in progress)
• Combined with 40+ topics of clinical importance to
heart failure/palliative care clinicians

What the users don’t see
((heart failure[tw] OR ventricular dysfunction,
left[mh:noexp] OR cardiomyopathy[tw] OR left
ventricular ejection fraction[tw]) AND Medline[sb]) OR
((heart failure[tw] OR left ventricular dysfunction[tw] OR
cardiomyopathy[tw] OR left ventricular ejection
fraction[tw] OR cardiac resynchronization[tw] OR LV
dysfunction[tw] OR left ventricular systolic
dysfunction[tw] OR left ventricular diastolic
dysfunction[tw] OR cardiac failure[tw]) NOT medline[sb])
AND english[la]

Palliative Care Filter effectiveness studies
Study 1
• Compiled more recent gold standard based on
included studies in 29 palliative care systematic
reviews
• Found that Palliative Care Filter performed better in
more recent dataset than in original one (˃ 5%)

Study 2
• Real life PubMed searches of pc clinicians for pc
literature (n=37) analysed
• Each run in gold standard set and retrieval compared
with that of PCF
• Results highlighted difficulties in searching:
– Very limited number of search terms used for complex topic
– Incorrect search construction
– Negative correlation between stated confidence and actual
performance (i.e. overconfidence is issue)
– Search filter retrieved 68% of gold standard; average
clinician search retrieved 25%

Study 3
• Were palliative care clinicians aware of topic
searches built on the PCF?
• Of those aware:
– Were they using them? If yes, what did they think? If no, why
not?
• If not aware:
– What did they think after using search? (intervention study)
• Follow up study
– Were they still using searches after 6 weeks?

Flinders Filters
• Established in 2012 as Flinders co-
funded research venture
• Committed to producing over 2 years:
– Two health filters
– Two non-health filters
– Four research papers that make a
contribution to search filter methodology

Flinders Filters
Completed projects:
• Primary Health Care Search Filter (PHCRIS)

Flinders Filters
PHC Search Filter
launch
All welcome

Flinders Filters
New projects:
• ‘Indigenous health’ and ‘Australian Aboriginal health’ filters
(Lowitja Institute)
• Best Supportive Care (Duke University)
• Dementia
• Sexual health

Flinders Filters
Urgent issues for FF:
• Non-health filters: What? Who with? Where
appropriate?
• Finding time to write for publication
• More librarians needed to join team

Shortening the distance to research evidence

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (10)

Similar to Shortening the distance to research evidence

Similar to Shortening the distance to research evidence (20)

Recently uploaded

Recently uploaded (20)

Shortening the distance to research evidence