What can we derive from this? This is not an indicator of the quality of research by Adam Wright and other co-authors However, it tells the following Exposure into research Co-authors Research exposure of co-authors Innovative method of using social networking concepts in the area of research Possible candidate for Research 2.0 ?
KLAS helps healthcare providers make informed technology decisions by offering accurate, honest, and impartial vendor performance information. HIMSS Analytics is a wholly-owned, not-for-profit subsidiary of the Healthcare Information and Management Systems Society (HIMSS) and deliver high quality data and analytical expertise. The company collects and analyzes healthcare organization data relating to IT
Based on comprehensive analysis of the clinical decision support knowledge base in use at Partners HealthCare system
(Certification Commission for Health Information Technology) Different certification criteria for different type of system
There were some ambiguities in the methods that the authors undertook. The authors did not mention what systems were picked initially when first contacted the companies and their customers. Also how were these 9 systems shortlisted, especially because of the generally positive response received from the vendor organizations. Authors did not mention who did they speak with. Did they speak with technical people of did they speak with sales representatives. These questions are important because users don’t have a complete knowledge of the capabilities of the system and depending on whom they asked there can be biases. Claudia and James
As mentioned before the features were evaluated along 4 axes… The results were presented pseudonomously in order to protect the confidentiality of vendors and sensitivity of product capabilities.
We’ll discuss this table again
Limitations as identified by the authors
There are a few methodological issues that are identified by me and the class comments in the blog. First, I was curious about the acceptance of the functional taxonomy they have used. On further research I found that there are 5 articles referencing to this taxonomy, out of the 5; 4 are referenced by the original authors of this study. Only 1 study cited this research in 2 instances but did not refer to the taxonomy information itself…it basically referenced other information of the paper.
CATCH-IT JOURNAL CLUB WRIGHT, A., SITTIG, D. F., ASH, J.S., SHARMA, S., PANG, J. E., AND MIDDLETON, B. (2009). CLINICAL DECISION SUPPORT CAPABILITIES OF COMMERCIALLY-AVAILABLE CLINICAL INFORMATION SYSTEMS. JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION, 16(5), 637 – 644. Presented by: Talat Ashraf HAD 5726 Master of Health Informatics University of Toronto Date: November 2, 2009 11/2/2009 CATCH-IT Journal Club
ABOUT THE PAPER 11/2/2009 CATCH-IT Journal Club Partner HealthCare System is a Boston based integrated healthcare system that includes primary care, specialty care, community hospitals, academic medical centers, and other health-related entities (2). Partners is an important entity for this research since it is related to several important aspects of this research, such as the taxonomy that is used.
11/2/2009 CATCH-IT Journal Club Source: Biomed Experts (4)
ADAM WRIGHT’S NETWORK OF RESEARCH 11/2/2009 CATCH-IT Journal Club Source: Biomed Experts (4)
SITTIG’S NETWORK OF RESEARCH 11/2/2009 CATCH-IT Journal Club Source: Biomed Experts (4)
ASH’S NETWORK OF RESEARCH 11/2/2009 CATCH-IT Journal Club Source: Biomed Experts (4)
PRIOR RESEARCH BY AUTHORS 11/2/2009 CATCH-IT Journal Club Authors Article Year Main Findings Wright A , Bates DW, Middleton B , Hongsermeier T, Kashyap V, Thomas SM, Sittig DF . Creating and sharing clinical decision support content with Web 2.0: Issues and examples. 2009 evaluate the potential of Web 2.0 technologies to enable collaborative development and sharing of CDSS through the lens of three case studies; analyzing technical, legal and organizational issues for developers, consumers and organizers of clinical decision support content in Web 2.0 Wright A , & Sittig D F A Four-Phase Model of the Evolution of Clinical Decision Support Architectures 2008 The authors review the history of evolution of CDSS from 1959 to present and propose a 4 phase architecture model for how the mechanism of integration of CDSS into Clinical Information systems have evolved, beginning with stand alone systems through sophisticated levels of integration Wright A, Sittig D F. A framework and model for evaluating clinical decision support architectures. 2008 Develop a four-phase model for evaluating architectures for clinical decision support that focuses on: defining a set of desirable features for a decision support architecture; building a proof-of-concept prototype; demonstrating that the architecture is useful by showing that it can be integrated with existing decision support systems and comparing its coverage to that of other architectures.
PRIOR RESEARCH BY AUTHORS 11/2/2009 CATCH-IT Journal Club Authors Article Year Main Findings Wright A ; Goldberg H; Hongsermeier T; Middleton B A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network 2007 Developed a functional taxonomy of rule-based clinical decision support along four axes: Trigger, input data elements, interventions and offered choices Sittig DF , Wright A , Osheroff JA, Middleton B , Teich JM, Ash JS , Campbell E, Bates DW. Grand challenges in clinical decision support. 2007 Discusses the ten grand challenges in the development of CDS applications based on an iterative consensus building. The ten grand challenges include i mprovement of HCI, dissemination of best practices in CDS design, development and implementation, creation of internet accessible CDS repositories, and so on.
For example, Medication-related CDS (for CPOE, ePrescription)
Evaluation of system features enabling CDS implementations
Particular types of information (such as allergies, drugs)
Particular system features (such as admission, prescribing)
Evaluation of other researchers’ evaluation of CDS capabilities
Unable to locate any research with similar research interest
A few examples of such research is provided in the following slide
11/2/2009 CATCH-IT Journal Club Source: Google Scholar, SCOPUS
OTHER CDS EVALUATIONS 11/2/2009 CATCH-IT Journal Club By Year Evaluation of Description Kuperman et al. 2007 Medication-related CDS
Discusses about two types of applications
Basic: drug-allergy & drug-drug interaction, dosage guidance, duplicate therapy, etc.
Advanced: drug-pregnancy & drug-disease checking, dosage support for geriatric patients, etc.
Garg et al. 2005 Practitioner’s performance in using CDSS Reviews the controlled trials assessing the effects of computerized CDSS in improvement of practitioner’s performance Kaplan 2001 Other authors’ evaluation Evaluates and discusses the strengths and weaknesses of the methods of evaluation, outcome measures, barriers to systems use, etc. Sniffman et al. 1999 Computer-based Guideline Implementation Systems Reviews prior research for computer-based guideline implementation systems; identifies features, recommendations, etc.
Step 1 : Identified several Clinical Information Systems (CIS) using figures from KLAS (Orem, UT) and HIMSS Analytics (Chicago, IL) that are CCHIT certified
Step 2 : Contacted companies (that developed the systems) and customers of the systems with initial inquiry
Step 3 : Shortlisted a purposive sample of 9 systems
Step 4 : Analyzed system features
3 authors interviewed individuals and evaluated the systems
A Taxonomy with 42 elements used for analysis
In case of uncertainty
Contacted users or contacts within vendor company
Referred to product manuals
Carried out hands-on evaluations
11/2/2009 CATCH-IT Journal Club Source: Wright et al. (1)
TAXONOMY USED 11/2/2009 CATCH-IT Journal Club Source: Wright et al. (7) Axes Explanation Example of Rule Triggers Event that cause decision support rule to be invoked When penicillin allergy is entered; check medications list Input Data Data elements used by a rule to make inferences If entered ‘female’ as gender; only show options related to female such as mammogram, pregnancy test etc Interventions Possible actions a decision support module can take If physician wants to override an alert; it asks the physician to provide a reason Offered Choices Provide the users with flexibility and offer them choices Allow users to change the dosage of drug
OTHER AVAILABLE TAXONOMIES FOR CDS 11/2/2009 CATCH-IT Journal Club - Following are other available taxonomies for CDS Source: Wright et al. (7) By Type Components Osheroff et al. Available Methods Documentation forms or templates, relevant data display, order creation facilitators, time-based checking and protocol or pathway support, reference information and guidance, and finally, reactive alerts and reminders. Berlin et al. Supported Scenarios Five categories: context, knowledge and data source, decision support, information delivery, and workflow Wang et al. Hierarchical (Tree Structure) View
A tree structure comprising of three hierarchies:
Miller et al. Dimensional View Three dimensions used: Type of Intervention, Workflow, Level of Disruption
CCHIT CERTIFICATION CRITERIA (CDS CRITERIA ONLY) 11/2/2009 CATCH-IT Journal Club Source: CCHIT (8) For Ambulatory EHRs For Inpatient EHRs
Highlight abnormal test results
Alert prescriber if:
Patient is allergic to a drug being ordered
Drug interactions may occur
A follow up test is recommended
Newly entered allergy for Patient’s drug
Drug side effects may occur based on diagnosis
Availability more cost-effective therapy
Reasoning behind an alert
Alert severity adjustment based on clinical rule
Use guidelines for disease and wellness management
Reminders for due/overdue care: alert, generate report, generate letter
Patient education materials: generate, tailor to patient, include procedure and test education
Issue alerts for:
Patient allergic to drug being prescribed
New allergy added for drug already given
Patient on similar drug
More cost-effective drug available
Allow overriding if appropriate
Adjustable severity based on clinician role
Dosing guidance and warning using demographics, lab results, scientific reference
Display for nurse at the time of administration any previous alerts, patient’s results & allergies
Allow use of barcodes to determine patient, drug, dose, time, and route
Require nurse to complete critical verifications prior to giving medications
Use of taxonomy that has not been validated by the research community
High potential for biased information used in conducting research
Use of data that are not validated
Results are questionable due to methodological issues
11/2/2009 CATCH-IT Journal Club
LEARNING FROM THE STUDY 11/2/2009 CATCH-IT Journal Club Source: Wright et al. (1) Category Learning Buyers of CIS
Must carefully inspect systems before making purchasing decision
Know the capabilities that are important to them
Know the functionalities that the system offers
Know what to trade-off
Developers of decision support systems (DSS) Need to be aware of capabilities for IS in which CDS components will run, and develop the rules according to the capability supported by the IS. For example, develop to a common denominator portable across all targeted systems, or develop contingencies supporting systems to its capability. CIS vendors Be aware of their own products as well as their competitors’, as this may be important for increasing sophistication among other vendors and an increasingly demanding customer base. Certification bodies Should consider certification of CDS capabilities, and as features become more common, they should turn those features into certification criteria. Also, they should put the less commonly available capabilities into the roadmaps for certification criteria in future. CDS Researchers Develop a convincing methodology to evaluate CDS capabilities in commercially available CIS, and include factors such as quality (non-functional aspects), importance, and usage of capabilities.
1. Wright A, Sittig D F, Ash J S, Sharma S, Pang J E, and Middleton B. Clinical Decision Support capabilities of Commercially-available Clinical Information Systems. Journal of the American Medical Informatics Association 2009; 16(5): 637-644.
2. Parners Healthcare. What is Partners?. Accessed via http://www.partners.org/about/about_whatis.html . Accessed on October 20, 2009
3. Scopus. Scopus Journal Search. Accessed via http://simplelink.library.utoronto.ca/url.cfm/54186 . Accessed on October 22, 2009
4. BioMed Experts. Accessed via http://www.biomedexperts.com . Accessed on October 15, 2009.
5. DMICE: People – Students. Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University. Accessed via http://www.ohsu.edu/ohsuedu/academic/som/dmice/people/students/index.cfm . Accessed on October 20, 2009
6. Clinical and Quality Analysis, Information Systems. Clinical and Quality Analysis Staff. Accessed via http://www.partners.org/cqa/Staff.htm . Accessed on October 18, 2009.
7. Wrigh A, Goldberg H, Hongsermeier T, and Middleton B. A Description and Functional Taxonomy of Rule-Based Decision Support Content at a Large Integrated Delivery Network. Journal of the American Medical Informatics Association 2007; 14(4): 489-496.
8. CCHIT. Concise Guide to CCHIT Certification Criteria. Accessed via http://www.cchit.org/sites/all/files/ConciseGuideToCCHIT_CertificationCriteria_May_29_2009.pdf . Accessed on October 10, 2009.
9. Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, Campbell E, Bates DW. Grand challenges in clinical decision support. Journal of Biomedical Informatics 2008; 41(2):387-392.