Assessing an adaptive,
profile-based PubMed
search tool
Marisa Conte
meese@umich.edu
NLM Associate Fellow
University of Michigan Health Sciences Libraries

MiSearch
 Adaptive biomedical literature search tool
 Utilizes implicit relevance feedback
 Builds statistical profile to predict which results will be most
relevant to user

Objective
 To compare the performance of a locally-
developed third-party PubMed search tool
to PubMed in terms of user satisfaction,
search success and efficiency

Collaboration

Methodology
 N = 11 librarians (12 recruited)
 Tasks derived from mediated search files
 Each subject completed 2 tasks with each
search tool
 Each task completed by 3 subjects with
each search tool
 Pre-training conducted by tool developer

Data collection
 Web-based surveys
 Demographic survey
 Task-specific survey
 PubMed search histories
 MiSearch data logfiles

Sample questions
 In patients with diabetes mellitus, does
increasing hemoglobin A1c reduce the risk
of amputation and kidney failure?
 What are the attitudes, beliefs and socio-
cultural factors that influence cervical
cancer screening in the US? What about
specifically among immigrant groups? And
I’m only interested in research from 1990 –
present.

Limitations
 Convenience sample
 Unclear survey questions
 Study tasks too complicated
 Problems with data collection

Sample results – Query 2
Average
satisfaction MiSearch PubMed
Search activity 1 1.7
Tool 1.3 1.7
Speed 1.3 1.7
Accuracy 1.3 2
Completeness 1.7 2
1 = low

Overall satisfaction with searches
60
50
% responses
40
30
PubMed
20 Misearch
10
0
High Fair Low
User satisfaction

Overall satisfaction with tools
70
60
% responses
50
40
30 PubMed
20 MiSearch
10
0
High Fair Low
User satisfaction

Suggestions
 Improve database response time
 Add more filters
 Add a “reset” function
 Make search details more transparent

Next steps
 Librarians suggested changes which:
 were easy to implement
 dramatically improved the tool’s functionality
 First usability test – helped establish formal
usability protocols and heuristics
 New roles for librarians

New roles for librarians
 Usability experts
 Heuristic evaluation of tools
 Portal development
 Formal usability studies
 Weblog analyses
 Education, promotion, dissemination
 Research collaborations

Resources
 MiSearch: http://misearch.ncibi.org/
 NCIBI: https://portal.ncibi.org/gateway/
 States, D.J., Ade, A.S., Wright, Z.C., Bookvich,
A.V. & Athey, B.D. (2008) Misearch Adaptive
PubMed Search Tool. Bioinformatics.
PMID 18326507

Acknowledgments
 Jean Song, Nirit Glazer, Barbara Mirel, David States
 Librarians from UM’s Health Sciences and Shapiro
Science Libraries
 National Center for Integrative Biomedical Informatics
(NIH grant #U54DA021519)
 National Library of Medicine
This research was supported in part by an appointment to the NLM
Associate Fellowship Program sponsored by the National Library of
Medicine and administered by the Oak Ridge Institute for Science and
Education.