Successfully reported this slideshow.
Your SlideShare is downloading. ×

Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 1 Ad

Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base

We evaluated the use of a previously described crowdsourcing methodology to generate a problem-laboratory test knowledge base, identifying appropriately linked problem-laboratory test pairs by clinicians during e-ordering. Existing evaluation metrics, including patient frequency and link ratio, were not correlated with appropriateness for 600 links manually validated. Further research is necessary to better evaluate these associations.

We evaluated the use of a previously described crowdsourcing methodology to generate a problem-laboratory test knowledge base, identifying appropriately linked problem-laboratory test pairs by clinicians during e-ordering. Existing evaluation metrics, including patient frequency and link ratio, were not correlated with appropriateness for 600 links manually validated. Further research is necessary to better evaluate these associations.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base (20)

Advertisement

More from Allison McCoy (11)

Advertisement

Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base

  1. 1. Use of the Crowdsourcing Methodology to Generate a Problem-Laboratory Test Knowledge Base Allison B. McCoy, PhD1, Adam Wright, PhD2, Jacob A. McCoy, MS3, Dean F. Sittig, PhD1 1The University of Texas Health Science Center at Houston – School of Biomedical Informatics 2Brigham and Women’s Hospital, Harvard Medical School Objective Methods Results Discussion To determine whether crowdsourcing is We applied crowdsourcing to create a Clinicians asserted 17,555 links (4,067 distinct Crowdsourcing has previously been effective at generating a knowledge base of knowledge base of problem-laboratory test problem-laboratory test pairs) during the study demonstrated to be effective at identifying related clinical problems and laboratory test pairs. We retrieved all problem-laboratory period. Of the 600 links evaluated, 429 appropriate problem-medication pairs. Despite results using links asserted by clinicians tests pairs linked by clinicians at a large, multi- (71.5%) were determined to be appropriate. similar processes used by clinicians in during e-ordering. specialty, ambulatory academic practice Of these, 2,715 (66.8%) were asserted for ordering medications and laboratory tests, it between June 1, 2010 and May 31, 2011. only one patient and 224 (5.5%) were did not generalize well when applied to We calculated the frequency of patients with asserted for 10 or more patients. generating a problem-laboratory test Background knowledge base. each link and the proportion of patients with a The table depicts the appropriateness for each Knowledge bases that link clinical problems to co-occurring laboratory test and problem for threshold group. No threshold groups One potential cause of the poor performance medications, laboratory tests, and other whom a link had been manually asserted (i.e., achieved appropriateness greater than or is the use of laboratory test results for clinical data can be applied to improve patient link ratio). We stratified links into threshold equal to 95%, and we did not observe an diagnostic purposes; because the clinician safety and meet Meaningful Use criteria in a groups according to each measure. Blinded to increasing trend in appropriateness as the has not determined whether a patient has a variety of ways, including generating patient the threshold values, we reviewed 25 patient frequency or link ratio increased. given disease, the problem has not been summaries, facilitating medication randomly selected pairs from each group to added to the patient’s list and is therefore not reconciliation, and delivering clinical decision determine whether the pair was appropriate. linked during ordering. In this case, clinicians support. While many problem-medication may arbitrarily select a problem to link or link knowledge bases are available, few problem- Appropriateness Thresholds for Problem-Laboratory Test Link Inference to a health maintenance code, which we laboratory test knowledge bases exist. Current excluded from our analysis. methods for generating such knowledge 0 Link Ratio bases, such as association rule mining, are 0.1 0.1-0.19 0.2-0.29 0.3-0.49 >=0.5 Total This study is limited in that it was performed in limited or computationally complex. 1 72% 68% 68% 80% 72% 72% a single setting with a small subset of linked problem-laboratory test pairs. Additional Patient Link 2 76% 64% 72% 76% 64% 76% Frequency 3-4 84% 76% 84% 80% 88% 84% research is necessary to determine whether Crowdsourcing more data may improve the findings, and 5-9 76% 64% 72% 80% 68.8% 76% Crowdsourcing, defined as outsourcing a task whether the methods are generalizable. >=10 56% 68% 56% 48% 77.8% 56% to a group of people, represents a novel Total 72.8% 68% 70.4% 72.8% 74% 72.8% method for generating such a knowledge base. The approach takes advantage of Summary of Conclusions manually linked laboratory tests to clinical Example Links Use of the crowdsourcing methodology with problems by clinicians during standard e- the initially proposed evaluation metrics did ordering, a task required by many institutions Appropriate Problem-Laboratory Test Pairs Inappropriate Problem-Laboratory Test Pairs not adequately identify appropriate problem- for billing purposes. Laboratory Laboratory laboratory test links. Further research may Problem Frequency Ratio Problem Frequency Ratio Test Test better evaluate these associations. Abdominal Pain Fecal 4 0.11 Chronic Renal TSH, 3rd 1 0.03 Sample Screen for Linking to a Problem Leukocyte Stain Insufficiency Generation Abdominal Pain CBC 7 0.01 Aortic TSH, 3rd 2 0.12 Acknowledgements Acute Streptococcus 6 0.60 Aneurysm Generation This project was supported by Contract No. Pharyngitis Test Rapid Aortic Hemoglobin A1c 3 0.23 10510592 for Patient-Centered Cognitive Support Aneurysm under the SHARP Program from the Office of the Atrial Fibrillation Prothrombin 54 0.26 National Coordinator for Health Information Time with INR Aortic Stenosis Hepatic 4 0.57 Function Panel Technology and NCATS grant UL1 TR000371. Abdominal Pain Calprotectin, 13 0.30 Stool Hyperlipidemia PSA Total+ 13 0.54 Please contact the first author via email: % Free allison.b.mccoy@uth.tmc.edu

×