Can Clinicians Create High-Quality Databases?


Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Good Afternoon everyone, My name is Ritu Khare and I am a PhD student at the ischool at Drexel University. In this talk, I am going to present our work titled Can clinicians create high-quality databases. Wherein we conduct a study on a flexible EHR system.
  • Let us begin with the motivation of this work.
  • one of the primary activities that healthcare professionals are engaged in is clinical data collection. And for this they rely on healthcare IT systems, particularly databases. However these systems are usually designed by an outside vendor or an IT expert. A major flaw with this setting is that these systems are usually inconsistent with respect to the needs of the clinicians + the modification of these systems is either very time-consuming or simply impossible. This definitely causes certain unintended consequences at times.
  • To deal with this inflexible design of the HITs, we focus on a specific Health IT system , EHRs, and propose a flexible EHRs. This system would allow clinicians to modify and extend the existing system databases based on their customized data collection needs. without affecting the quality of the underlying database.
  • I will now describe the design of the flexible ehr system.
  • To design the system, we follow a form-based approach – we leverage on the fact that clinicians are very familiar with forms and that forms contain rich information on databases. The system has 3 components: first component allows clinicians to design forms based on their data collection needs, the second component generates a tree structure corresponding to the clinician designed form, and the third component explores the tree structure of extract database information.
  • Here is an example.
    An example is a clinician who wants to collect information about patients such as personal information and vital signs. Given the flexible EHR system, the clinician interacts with the form design interface to build a form corresponding to her needs. System then generates a tree structure corresponding to the form. This tree structure is translated into a database by the database design algorithm.
  • I will start with the first module which is the form design interface, we developed this interface so that clinicians can easily design all kinds of forms using the interface. They can design simple as well as advanced forms as shown here. Our primary concern while designing the interface was simplicity. We kept only those features that are prevalent in the healthcare domain and with which the clinicians would already be familiar.
    Also, we kept the terminology very intuitive. I can give some examples here. We use terms like title, category, field and format, and subcategories and subfields within a category. For advanced forms the interface allows users to add supporting text, unit to any field. And the interface also supports some advanced formats like – extended checkboxes. Plus two fields could be conditionally linked to each other.
  • To give a better picture – if the user needs to design a form like this – she can follow these steps one by one. Here Here is a screenshot of the interface we developed – the left hand side is a placeholder for the user actions – it allows users to specify form components and right-hand side shows the form being designed.
    This provides mechanism for clinicians to specify their needs.
  • After a form is designed by the clinician, the tree generation module generates a form tree. We use three kinds of nodes in the form tree – oval shapes are element nodes that represent text labels, rectangular shapes are format nodes that represents formats like textboxes, radio-button groups and checkbox groups , and triangular nodes are value nodes that represent pre-specified values in the form such as blue-cross, fever. This tree accurately captures the relationship among various form components. For instance, 2 categories like patient and status become siblings, a category and its field share parent-child relationship – like patient and name, and Format and associated values share parent child relationship.
  • The main goal of the algorithm is to generate a database which is high-quality , one of the important properties we consider is normalization. The database design algorithm traverses the form tree in depth first order and generates database elements based on the form patterns. The first pattern is the textbox pattern – for this pattern the algorithm induces a functional dependency in the database that reflects the relationship among the three nodes in the pattern.
    Another pattern is the radiobutton which suggests that the field with radiobutton is independent of the entity representing the parent node nj and that there is a M-1 cardinality between nj and n, and so we remove any possibility of transitive dependency and create two separate tables in the database. The checkbox pattern is similar except for the fact that we translate the checkbox values as yes/no columns as its possible to select more than one checkboxes at the same time. Then we have another pattern that we term as the category subcategory pattern – because they represent 2 separate independent entities , we remove possibilities of transitive dependencies and split them into 2 tables connected via a join table. The form pattern 5 which is sibling category pattern is treated in a similar manner.
  • Very briefly, here is an example of a simple form and its corresponding database.
  • After designing the system, we implemented it and evaluated it by conducting a user study with health care professionals.
  • We recruited 5 nurse professionals with absolutely no knowledge of databases and who had experience of using paper forms. We gave them 2 tasks. 1st – build task – they had to build a system version of a given paper form using the form design interface. 2nd task was more advanced – model and build form – wherein participants were just given a problem statement that they had to model into a form and build it using the interface.

    We conducted 2 rounds – first with small scale forms that generated small scale databases and second with large scale forms that generated large-scale databases. So for each participant we have 2 rounds and 2 tasks per round.
  • First we measured the duration ratio – that gives an idea of the efficiency of participants in performing the tasks. And found the following results. Each bar represents the performance of one participant in a given round and a given task. The only outlier is the participant P3 who considered several design alternatives in the model and build task and hence took longer. We also measured that assistance ratio that gives an idea on the number of assistances needed by various participants and hence their confidence. The only outlier in this case is participant P5 who had difficult understanding the terminology used by the interface.
  • Overall we find that the participants could finish all tasks with 100% effectiveness in 95% of the cases. Also the duration ratio was within reasonable limits – 1-9 minutes to design simple forms and 7-19 minutes to design longer and advanced forms.
  • Also, we find that the efficiency consistently improved from round 1 to round 2. and so did their confidence in performing the build task. Most of the participants showed improved understanding of form concepts and started applying their knowledge of form concepts to consider design alternatives.
  • Through this work, we have made two main contributions.
    1. With the user study we found that participants gave high-performance in terms of accuracy and time taken to perform the tasks. This suggests that the system has the potential to reduce the current problems of healthcare IT system. Also we observed improvement in participants performance and understanding – this suggests that the system is adoptive and is easy to learn.
    2. Database design algorithm – is guided by high-quality principles and generates a database which is normalized with respect to the needs of the clinicians. And so this generated is comparable to a professionally developed system.
  • Some of our immediate goals are – to improve the user study by involving more participants and different sections of the healthcare professionals. Also form design interface can be improved by adding a design recommendation component and including more features. Database design algorithm – can be enhanced by considering even more form patterns and by including a merging component that effectively merges the generated database with an existing one.
  • Thank you for your attention.
    Certified systems – is it possible to give the power the to clinicians themselves
  • Can Clinicians Create High-Quality Databases?

    1. 1. RITU KHARE, YUAN AN, IL -YEOL SONG, XIAOHUA HU THE ISCHOOL AT DREXEL DREXEL UNIVERSITY P H I L A D E L P H I A , P A , U S A Can Clinicians Create High-Quality Databases? A Study on A Flexible Electronic Health Record (fEHR) System
    2. 2. Presentation Part 1 2 1. Motivation:  Inflexible Design of Existing HITs 2. Solution 3. Evaluation 4. Conclusion
    3. 3. Motivation 3  Clinicians rely on health information technologies (HIT) in clinical data collection  related to patients, diseases, treatments, etc.  Current HITs are vendor or IT professional designed systems  inconsistent  with the data collection needs of the clinicians (Gurses et al. ,2009).  inflexible  in that it is either impossible or time-consuming to evolve them according to clinicians' current and changing needs (Gurses et al. 2009, An et al. 2009).  unintended consequences (Ash et al. 2004)  creation of more work for the clinicians (Lee 2007) – inefficient workflow  HIT failures (Harrison et al. 2007).
    4. 4. We propose… 4  A flexible design of HITs that would allow the clinicians to easily and quickly modify the system based on their needs (Gurses et al. ,2009).  Focusing on a specific implementation of HIT, the electronic health record(EHR) (Linder et al. 2007, DesRoches et al. 2008), we propose a flexible EHR (fEHR) system  to enable the clinicians to modify and extend the underlying database based on their data collection needs  While ensuring that the extended database retains high- quality.
    5. 5. Presentation Part 2 5 1. Motivation:  Inflexible Design of Existing HITs 2. Solution:  A Flexible Electronic Health Record (fEHR) System 3. Evaluation 4. Conclusion
    6. 6. Form-based approach 3 components  clinicians' high familiarity quotient on forms  rich information embedded in forms to guide DB design. 6 The fEHR system The fEHR System Form Design Interface Tree Generation Database Design Algorithm 1 2 3 Enables clinicians to build forms Generates a tree structure corresponding to the form Designs database based on the tree semantics 6
    7. 7. The flexible EHR System- Example Scenario 7 I want to collect patient’s information, personal and vital signs, etc Database The fEHR System Form Design Interface Tree Generation Database Design Algorithm Clinician 1 2 3
    8. 8. Simple Form Advanced Form 8 1. Form Design Interface Title Category Field Format Subcategory Supporting Text Unit Extended Checkbox Condition Subfield SIMPLE! Features(form patterns) Terminology (intuitive)
    9. 9. 1. Form Design Interface 9 Input: User actions (based on data collection needs) Output: Form 1. Enter the Title “Patient Encounter Form” 2. Enter the category “Patient” 3. Enter the field “Name” 4. Pick a format “textbox” 5. Enter the field “Age” 6. …
    10. 10. 2. Tree Generation 10 Input: Form Output: Form Tree
    11. 11. 3. Database Design Algorithm 11  Goal: High-quality database (normalization)  Traverses the form tree in depth first order Input: Form Tree Output: Database Textbox Pattern Radiobutton Pattern Checkbox Pattern Category/subcategory Pattern Sibling categories Pattern We consider 5 more patterns (not presented here) mj.ID -> mj.m M:1
    12. 12. 3. Database Design Algorithm - Examples 12 Example 1 Example 2 Textbox pattern Sibling categories pattern Radiobutton pattern Checkbox pattern Category-subcat. pattern Textbox pattern
    13. 13. Presentation Part 3 13 1. Motivation:  Inflexible Design of Existing HITs 2. Solution:  A Flexible Electronic Health Record (fEHR) System 3. Evaluation:  User Study with Health Professionals 4. Conclusion:
    14. 14. Participants and Tasks User Study Settings  5 nurse professionals.  No knowledge of database  Moderate computer users  Familiar with Paper-based Forms  2 Tasks  Build task  Replicate a paper-based form on the system  Model and build task  Model and build a given need (in natural language) into a form using the system interface.  2 rounds (form scale = no. of steps to design a form)  Round 1: Small scale needs  Avg. form scale = 17  Avg. Database Scale  4.2 tables  5.8 non-key attributes  1.8 values  3.2 foreign key references  Round 2: Large scale needs  Avg. form scale 47.4  Avg. Database Scale  6.2 tables  13.8 attributes  10.4 values  4.6 foreign key references 14 Usability Evaluation Implementation using: MySQL, JAVA, JSP, JavaScript, HTML, CSS, Lucene Indexing Package, yFiles Package
    15. 15. 15 Measurements Duration Ratio = Time(in min)/ Form Scale(#of steps to build form) Assistance Ratio = # of assistances sought/ Form Scale(#of steps to build form) Outlier: P5: had difficulty in form terminology (needed more assistance) Outlier: P3: considered design alternatives (high duration ratio)
    16. 16. Effectiveness Efficiency  In 19/20 cases, participants finished the tasks with 100% effectiveness.  Exception: a building error committed by a participant who skipped a component while building forms.  Duration ranged from 1 to 9 minutes for simple small-scale needs, and 7 to 19 minutes for advanced longer needs.  Exception: A participant who considered several design alternatives . 16 Findings 1
    17. 17. Findings 2 17  System Adoption  Efficiency : consistently improved from round 1 to round 2.  Confidence:  Very confident for specifying small-scale needs for both the tasks.  Improved from round 1 to round 2 for the build task.  Did not improve for model-and-build task from round 1 to 2.  Understanding: improved greatly in round 2.  They started synthesizing their knowledge of form concepts and domain knowledge to consider different design alternatives.
    18. 18. Presentation Part 4 18 1. Motivation:  Inflexible Design of Existing HITs 2. Solution:  A Flexible Electronic Health Record (fEHR) System 3. Evaluation:  User Study with Health Professionals 4. Conclusion:  Contribution and Future Work
    19. 19. Contributions  User study with the form design interface  Potential to reduce the current problems of HITs, particularly, the inefficiency faced by clinicians, and the inconsistency between clinician's needs and databases.  Participants showed high-performance in terms of effectiveness and efficiency.  Adoptive: helps the clinicians to learn and improve their need modeling and form building skills.  Improvement in participants’ efficiency, confidence, and understanding in using the system.  Database Design Algorithm  Comparable to any vendor or expert designed Healthcare databases.  High-Quality Principles  Normalized Database with respect to the clinician's needs  Correctness, completeness, & compactness. 19
    20. 20. Immediate Long Term  User Study  More Participants  Broad section of healthcare professionals  Form Design Interface  Design Recommendation  More Form Features  Database Design Algorithm  More Patterns  Merging Component  Scalability Experiments  Form Filling Component  Handle Record Conflict  Data Recommendation (domain/range checks)  User Study  Tree Generation  Handle elsewhere designed forms 20 Future Work
    21. 21. A C K N O W L E D G E M E N T S : T O T H E R E V I E W E R S O F I H I 2 0 1 0 R E F E R E N C E S : [ 1 ] T O [ 2 3 ] ( I N F U L L T E X T ) . 21 Thank you