Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Use of the Crowdsourcing Methodology to                                                   Generate a Problem-Laboratory Te...
You’ve finished this document.
Upcoming SlideShare
SBMI Second Life Orientation for Online Classes Spring 2015
Next
Upcoming SlideShare
SBMI Second Life Orientation for Online Classes Spring 2015
Next

Share

Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base

We evaluated the use of a previously described crowdsourcing methodology to generate a problem-laboratory test knowledge base, identifying appropriately linked problem-laboratory test pairs by clinicians during e-ordering. Existing evaluation metrics, including patient frequency and link ratio, were not correlated with appropriateness for 600 links manually validated. Further research is necessary to better evaluate these associations.

  • Be the first to like this

Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base

  1. 1. Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base Allison B. McCoy, PhD1, Adam Wright, PhD2, Jacob A. McCoy, MS3, Dean F. Sittig, PhD1 1The University of Texas Health Science Center at Houston – School of Biomedical Informatics 2Brigham and Women’s Hospital, Harvard Medical School Objective Methods Results DiscussionTo determine whether crowdsourcing is We applied crowdsourcing to create a Clinicians asserted 17,555 links (4,067 distinct Crowdsourcing has previously beeneffective at generating a knowledge base of knowledge base of problem-laboratory test problem-laboratory test pairs) during the study demonstrated to be effective at identifyingrelated clinical problems and laboratory test pairs. We retrieved all problem-laboratory period. Of the 600 links evaluated, 429 appropriate problem-medication pairs. Despiteresults using links asserted by clinicians tests pairs linked by clinicians at a large, multi- (71.5%) were determined to be appropriate. similar processes used by clinicians induring e-ordering. specialty, ambulatory academic practice Of these, 2,715 (66.8%) were asserted for ordering medications and laboratory tests, it between June 1, 2010 and May 31, 2011. only one patient and 224 (5.5%) were did not generalize well when applied to We calculated the frequency of patients with asserted for 10 or more patients. generating a problem-laboratory test Background knowledge base. each link and the proportion of patients with a The table depicts the appropriateness for eachKnowledge bases that link clinical problems to co-occurring laboratory test and problem for threshold group. No threshold groups One potential cause of the poor performancemedications, laboratory tests, and other whom a link had been manually asserted (i.e., achieved appropriateness greater than or is the use of laboratory test results forclinical data can be applied to improve patient link ratio). We stratified links into threshold equal to 95%, and we did not observe an diagnostic purposes; because the cliniciansafety and meet Meaningful Use criteria in a groups according to each measure. Blinded to increasing trend in appropriateness as the has not determined whether a patient has avariety of ways, including generating patient the threshold values, we reviewed 25 patient frequency or link ratio increased. given disease, the problem has not beensummaries, facilitating medication randomly selected pairs from each group to added to the patient’s list and is therefore notreconciliation, and delivering clinical decision determine whether the pair was appropriate. linked during ordering. In this case, clinicianssupport. While many problem-medication may arbitrarily select a problem to link or linkknowledge bases are available, few problem- Appropriateness Thresholds for Problem-Laboratory Test Link Inference to a health maintenance code, which welaboratory test knowledge bases exist. Current excluded from our analysis.methods for generating such knowledge 0 Link Ratiobases, such as association rule mining, are 0.1 0.1-0.19 0.2-0.29 0.3-0.49 >=0.5 Total This study is limited in that it was performed inlimited or computationally complex. 1 72% 68% 68% 80% 72% 72% a single setting with a small subset of linked problem-laboratory test pairs. Additional Patient Link 2 76% 64% 72% 76% 64% 76% Frequency 3-4 84% 76% 84% 80% 88% 84% research is necessary to determine whether Crowdsourcing more data may improve the findings, and 5-9 76% 64% 72% 80% 68.8% 76%Crowdsourcing, defined as outsourcing a task whether the methods are generalizable. >=10 56% 68% 56% 48% 77.8% 56%to a group of people, represents a novel Total 72.8% 68% 70.4% 72.8% 74% 72.8%method for generating such a knowledgebase. The approach takes advantage of Summary of Conclusionsmanually linked laboratory tests to clinical Example Links Use of the crowdsourcing methodology withproblems by clinicians during standard e- the initially proposed evaluation metrics didordering, a task required by many institutions Appropriate Problem-Laboratory Test Pairs Inappropriate Problem-Laboratory Test Pairs not adequately identify appropriate problem-for billing purposes. Laboratory Laboratory laboratory test links. Further research may Problem Frequency Ratio Problem Frequency Ratio Test Test better evaluate these associations. Abdominal Pain Fecal 4 0.11 Chronic Renal TSH, 3rd 1 0.03Sample Screen for Linking to a Problem Leukocyte Stain Insufficiency Generation Abdominal Pain CBC 7 0.01 Aortic TSH, 3rd 2 0.12 Acknowledgements Acute Streptococcus 6 0.60 Aneurysm Generation This project was supported by Contract No. Pharyngitis Test Rapid Aortic Hemoglobin A1c 3 0.23 10510592 for Patient-Centered Cognitive Support Aneurysm under the SHARP Program from the Office of the Atrial Fibrillation Prothrombin 54 0.26 National Coordinator for Health Information Time with INR Aortic Stenosis Hepatic 4 0.57 Function Panel Technology and NCATS grant UL1 TR000371. Abdominal Pain Calprotectin, 13 0.30 Stool Hyperlipidemia PSA Total+ 13 0.54 Please contact the first author via email: % Free allison.b.mccoy@uth.tmc.edu

We evaluated the use of a previously described crowdsourcing methodology to generate a problem-laboratory test knowledge base, identifying appropriately linked problem-laboratory test pairs by clinicians during e-ordering. Existing evaluation metrics, including patient frequency and link ratio, were not correlated with appropriateness for 600 links manually validated. Further research is necessary to better evaluate these associations.

Views

Total views

619

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×