Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Healthcare use of workflow engine technology  with emphasis on data  analysis and decision support Vojtech Huser MD PhD La...
Objectives <ul><li>1. Describe the abstract notion of a  workflow engine  and workflow technology in general  </li></ul><u...
Introduction <ul><li>Workflow technology: focus on my research for the last 8.5 years </li></ul><ul><li>This talks is a mi...
Workflow Technology (WT) Vojtech Huser, MD, PhD
Workflow Technology (WT) <ul><li>Business process management system (BPMS) </li></ul><ul><li>Workflow  = The automation of...
WT history <ul><li>History </li></ul><ul><ul><li>1960 Carl Adam Petri </li></ul></ul><ul><ul><li>1999 SDO, standard termin...
Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Vojtech Huser, MD, PhD
Humans & machines working together Vojtech Huser, MD, PhD
Executable flowchart Vojtech Huser, MD, PhD
Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Minimum components Vojtech Huser, MD, PhD language editor execution engine
Vojtech Huser, MD, PhD
Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Vojtech Huser, MD, PhD
Vojtech Huser, MD, PhD
Workflow engine Vojtech Huser, MD, PhD
Engine Vojtech Huser, MD, PhD
Use of WT <ul><li>Pioneer domains </li></ul><ul><ul><li>Banking </li></ul></ul><ul><ul><li>Manufacturing </li></ul></ul><u...
Flowcharts Vojtech Huser, MD, PhD
Flowcharts <ul><li>Present in clinical guidelines </li></ul><ul><li>Can summarize several pages of text </li></ul><ul><li>...
Clinical Guidelines Quality Improvement measures Process definition (workflow editor)
Examples of WT use in healthcare Vojtech Huser, MD, PhD Bed management Infections control (MRSA) J. Emanuele and L. Koette...
Stroke guideline (WfMS) Vojtech Huser, MD, PhD
Soarian Vojtech Huser, MD, PhD
Vojtech Huser, MD, PhD
Current Soarian Example Vojtech Huser, MD, PhD Main Line Health  (Harm Sherpbier) Soarian: Workflow engine (Tibco), MLM (A...
Rheumatology decision support  Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
My research in workflow technology
Vojtech Huser, MD, PhD 1. 2. HealthcareWorkflow.wordpress.com
Vojtech Huser, MD, PhD
Vojtech Huser, MD, PhD
Vojtech Huser, MD, PhD Medinfo 2007
Implemented processes <ul><li>RetroGuide (v2)  (2004-2008) (published) </li></ul><ul><ul><li>Quality improvement </li></ul...
Problems which HealthFlow tries to address? <ul><li>EHR systems need user-friendly customization </li></ul><ul><li>Represe...
HealthFlow System Vojtech Huser, MD, PhD code.google.com/p/HealthFlow
HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD code.google.com/p/HealthFlow
HealthFlow: RetroGuide + FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
HealthFlow :  key concepts <ul><li>Flowchart layer + code layer (applications) </li></ul><ul><li>Set of HF external applic...
Vojtech Huser, MD, PhD self service tools (i2b2)
HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Code Layer  (node; right click properties) Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Together Workflow Engine (Community edition) Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
Example of retrospective execution Vojtech Huser, MD, PhD 0.9% of patients with high risk patients with subsequent MI
Example of execution Vojtech Huser, MD, PhD
Process mining <ul><li>Arrive at process definition by examining retrospective data </li></ul><ul><li>Compare processes fr...
Chronic kidney disease (mining)
 
 
Benefits and challenges  of using workflow engines in healthcare Vojtech Huser, MD, PhD
Decision support (wish list) <ul><li>Understandable format </li></ul><ul><ul><ul><li>Graphical - task network paradigm – S...
Benefits of WT <ul><li>Ability to customize processes within an EHR </li></ul><ul><li>Graphical representation </li></ul><...
Challenges of WT <ul><li>Process administration </li></ul><ul><ul><ul><li>Owner of a process </li></ul></ul></ul><ul><ul><...
Conclusion <ul><li>4 objectives </li></ul><ul><ul><li>Workflow engine, flowcharts (guidelines), current use in healthcare,...
 
 
 
 
Extra slides
Vojtech Huser, MD, PhD
 
Evaluation <ul><li>18 users, 2 parts, qualitative </li></ul><ul><li>9 task questions  (T1-T9) </li></ul><ul><ul><ul><ul><l...
Evaluation  (SQL and RG) <ul><li>Which technology do you prefer (SQL or RG)? </li></ul><ul><ul><ul><ul><li>94% of particip...
Evaluation <ul><li>General important features:  (general question) </li></ul><ul><ul><li>Highest ranking features </li></u...
Upcoming SlideShare
Loading in …5
×

Healthcare use of workflow engine technology with emphasis on data analysis and decision support

4,332 views

Published on

Healthcare use of workflow engine technology with emphasis on data analysis and decision support



1. Describe the abstract notion of a workflow engine and workflow technology in general
2. Understand the relationship of flowcharts (common in medical guidelines) to executable models of processes used by workflow engines
3. Understand current use of workflow engines in healthcare in production environment and in research context (phenotype modeling, data analysis, clinical decision support, process mining and discovery)
Includes description of some of my research projects
4. List the evidence for benefits and challenges of using workflow engines in healthcare

Published in: Health & Medicine, Technology
  • Be the first to comment

Healthcare use of workflow engine technology with emphasis on data analysis and decision support

  1. 1. Healthcare use of workflow engine technology with emphasis on data analysis and decision support Vojtech Huser MD PhD Laboratory for Informatics Development National Institutes of Health, Clinical Center Bethesda, MD, USA
  2. 2. Objectives <ul><li>1. Describe the abstract notion of a workflow engine and workflow technology in general </li></ul><ul><li>2. Understand the relationship of flowcharts (common in medical guidelines) to executable models of processes used by workflow engines </li></ul><ul><li>3. Understand current use of workflow engines in healthcare in production environment and in research context (phenotype modeling, data analysis, clinical decision support, process mining and discovery) </li></ul><ul><ul><ul><li>Includes description of some of my research projects </li></ul></ul></ul><ul><li>4. List the evidence for benefits and challenges of using workflow engines in healthcare </li></ul>Vojtech Huser, MD, PhD
  3. 3. Introduction <ul><li>Workflow technology: focus on my research for the last 8.5 years </li></ul><ul><li>This talks is a mixture of </li></ul><ul><ul><li>Workflow technology tutorial </li></ul></ul><ul><ul><li>Larger context of WT </li></ul></ul><ul><ul><li>My prior and current work </li></ul></ul>Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  4. 4. Workflow Technology (WT) Vojtech Huser, MD, PhD
  5. 5. Workflow Technology (WT) <ul><li>Business process management system (BPMS) </li></ul><ul><li>Workflow = The automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules. </li></ul><ul><li>Workflow Process = A set of one or more linked procedures or activities which collectively realize a business objective or policy goal, normally within the context of an organizational structure defining functional roles and relationships (definition vs. instance) </li></ul><ul><li>Worklist Handler = A software component that manages the interaction between the user and the worklist maintained by a workflow engine (task display, completion, acceptance, referral) </li></ul>Vojtech Huser, MD, PhD WfMC: Terminology & Glossary, Document Number WFMC-TC-1011, Feb 99 http://www.wfmc.org/index.php?option=com_docman&task=doc_download&gid=93&Itemid=74 HealthcareWorkflow.wordpress.com
  6. 6. WT history <ul><li>History </li></ul><ul><ul><li>1960 Carl Adam Petri </li></ul></ul><ul><ul><li>1999 SDO, standard terminology </li></ul></ul><ul><ul><li>2000s </li></ul></ul><ul><ul><ul><li>2002: XPDL 1.0, 2008 XPDL 2.1 </li></ul></ul></ul><ul><li>Since 2000s </li></ul><ul><ul><li>offered by large IT companies (MS, Oracle, SAP, IBM, Fujitsu) </li></ul></ul><ul><li>Healthcare </li></ul><ul><ul><li>2008 Gartner report </li></ul></ul>Vojtech Huser, MD, PhD
  7. 7. Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  8. 8. Vojtech Huser, MD, PhD
  9. 9. Humans & machines working together Vojtech Huser, MD, PhD
  10. 10. Executable flowchart Vojtech Huser, MD, PhD
  11. 11. Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  12. 12. Minimum components Vojtech Huser, MD, PhD language editor execution engine
  13. 13. Vojtech Huser, MD, PhD
  14. 14. Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  15. 15. Vojtech Huser, MD, PhD
  16. 16. Vojtech Huser, MD, PhD
  17. 17. Workflow engine Vojtech Huser, MD, PhD
  18. 18. Engine Vojtech Huser, MD, PhD
  19. 19. Use of WT <ul><li>Pioneer domains </li></ul><ul><ul><li>Banking </li></ul></ul><ul><ul><li>Manufacturing </li></ul></ul><ul><ul><li>Shipping </li></ul></ul><ul><ul><li>Government authorities </li></ul></ul><ul><ul><li>Pharmaceutical </li></ul></ul><ul><li>Healthcare </li></ul><ul><ul><li>Gartner 2008 report: <5% use </li></ul></ul><ul><ul><ul><ul><li>More in the third section </li></ul></ul></ul></ul>Vojtech Huser, MD, PhD
  20. 20. Flowcharts Vojtech Huser, MD, PhD
  21. 21. Flowcharts <ul><li>Present in clinical guidelines </li></ul><ul><li>Can summarize several pages of text </li></ul><ul><li>Take-home message </li></ul><ul><li>Algorithmic nature (step by step) </li></ul><ul><li>Usually very well defined terms, thresholds and treatment options </li></ul><ul><li>Used even more often in local guidelines </li></ul>Vojtech Huser, MD, PhD
  22. 22. Clinical Guidelines Quality Improvement measures Process definition (workflow editor)
  23. 23. Examples of WT use in healthcare Vojtech Huser, MD, PhD Bed management Infections control (MRSA) J. Emanuele and L. Koetter, &quot;Workflow Opportunities and Challenges in Healthcare,&quot; in 2007 BPM & Workflow Handbook, 2007. L. Koetter, &quot;MRSA infection control with workflow technology,&quot; Spring AMIA Conference, Orlando, FL, 2007. R. Hess, &quot;The Chester County Hospital: Case Study,&quot; in 2007 Excellence in Practice: Moving the Goalposts., 2007.
  24. 24. Stroke guideline (WfMS) Vojtech Huser, MD, PhD
  25. 25. Soarian Vojtech Huser, MD, PhD
  26. 26. Vojtech Huser, MD, PhD
  27. 27. Current Soarian Example Vojtech Huser, MD, PhD Main Line Health (Harm Sherpbier) Soarian: Workflow engine (Tibco), MLM (Arden Syntax)
  28. 28. Rheumatology decision support Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  29. 29. My research in workflow technology
  30. 30. Vojtech Huser, MD, PhD 1. 2. HealthcareWorkflow.wordpress.com
  31. 31. Vojtech Huser, MD, PhD
  32. 32. Vojtech Huser, MD, PhD
  33. 33. Vojtech Huser, MD, PhD Medinfo 2007
  34. 34. Implemented processes <ul><li>RetroGuide (v2) (2004-2008) (published) </li></ul><ul><ul><li>Quality improvement </li></ul></ul><ul><ul><ul><li>Osteoporosis </li></ul></ul></ul><ul><ul><ul><li>Cholesterol management </li></ul></ul></ul><ul><ul><ul><li>Blood pressure control in diabetics </li></ul></ul></ul><ul><ul><li>Decision support </li></ul></ul><ul><ul><ul><li>Computerized glucose management protocol </li></ul></ul></ul><ul><ul><ul><li>Adverse Drug Events (naloxon, sleep apnea) </li></ul></ul></ul><ul><ul><li>Research </li></ul></ul><ul><ul><ul><li>Hodgkin's lymphoma and pregnancy </li></ul></ul></ul><ul><ul><ul><li>Hepatitis C </li></ul></ul></ul><ul><li>+ several other experimental </li></ul><ul><li>RetroGuide (v3) </li></ul><ul><ul><li>20+ scenarios </li></ul></ul><ul><ul><ul><li>US Prev. Services TF </li></ul></ul></ul><ul><ul><ul><li>Health plan rules </li></ul></ul></ul><ul><ul><ul><li>hsCRP </li></ul></ul></ul><ul><ul><ul><li>Genetic counseling </li></ul></ul></ul><ul><ul><ul><li>Thyroid disease, ACEi monitoring </li></ul></ul></ul><ul><ul><ul><li>Osteoporosis </li></ul></ul></ul><ul><li>FlowGuide </li></ul><ul><ul><ul><li>Surgical patient discharge (feasibility) </li></ul></ul></ul><ul><ul><ul><li>Cholesterol management </li></ul></ul></ul><ul><ul><ul><li>Rheumatoid arthritis </li></ul></ul></ul>Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com Huser (2010) J Biomed Inform
  35. 35. Problems which HealthFlow tries to address? <ul><li>EHR systems need user-friendly customization </li></ul><ul><li>Representation must be able to represent complex logic and understandable </li></ul><ul><li>Execution must be integrated into an routine-care EHR system (standalone engine vs. integrated) </li></ul><ul><li>Same representation/execution must be deployed at multiple institutions </li></ul><ul><li>Switchable representation (retrospective experiment, prospective observation) </li></ul><ul><li>Re-uses existing cross-industry technology </li></ul>Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  36. 36. HealthFlow System Vojtech Huser, MD, PhD code.google.com/p/HealthFlow
  37. 37. HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD code.google.com/p/HealthFlow
  38. 38. HealthFlow: RetroGuide + FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  39. 39. HealthFlow : key concepts <ul><li>Flowchart layer + code layer (applications) </li></ul><ul><li>Set of HF external applications (EAs) </li></ul><ul><li>Single patient execution model (DSS) </li></ul><ul><li>Works with time ordered chart </li></ul><ul><li>Resembles manual chart review </li></ul><ul><li>Concept of current position in EHR </li></ul><ul><li>Use of variables </li></ul><ul><li>No AI component </li></ul><ul><ul><li>No autonomous DM algorithm, advanced ML methods </li></ul></ul><ul><ul><li>Simple execution of human-specified steps (prior knowledge), no machine input </li></ul></ul><ul><ul><ul><li>Data querying, answering questions, data exploration </li></ul></ul></ul>Vojtech Huser, MD, PhD
  40. 40. Vojtech Huser, MD, PhD self service tools (i2b2)
  41. 41. HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  42. 42. Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  43. 43. Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  44. 44. Code Layer (node; right click properties) Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  45. 45. HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  46. 46. Together Workflow Engine (Community edition) Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  47. 47. HealthFlow: RetroGuide, FlowGuide Vojtech Huser, MD, PhD HealthcareWorkflow.wordpress.com
  48. 48. Example of retrospective execution Vojtech Huser, MD, PhD 0.9% of patients with high risk patients with subsequent MI
  49. 49. Example of execution Vojtech Huser, MD, PhD
  50. 50. Process mining <ul><li>Arrive at process definition by examining retrospective data </li></ul><ul><li>Compare processes from different settings </li></ul><ul><li>Even younger field then workflow technology </li></ul><ul><li>Examples </li></ul><ul><ul><li>Prior work on Chronic Kidney Disease </li></ul></ul><ul><ul><li>Current work on Clinical Research Protocols </li></ul></ul><ul><li>www.processmining.org </li></ul>
  51. 51. Chronic kidney disease (mining)
  52. 54. Benefits and challenges of using workflow engines in healthcare Vojtech Huser, MD, PhD
  53. 55. Decision support (wish list) <ul><li>Understandable format </li></ul><ul><ul><ul><li>Graphical - task network paradigm – SAGE,GLIF, ProForma </li></ul></ul></ul><ul><ul><ul><ul><ul><li>Textual (Arden Syntax), GELLO </li></ul></ul></ul></ul></ul><ul><li>Several implementations (compare to databases, BI, prog. IDE) </li></ul><ul><ul><ul><li>Choice of editor </li></ul></ul></ul><ul><ul><ul><li>Choice of execution engine </li></ul></ul></ul><ul><li>Handles well logic and context </li></ul><ul><ul><li>Logic </li></ul></ul><ul><ul><ul><ul><li>>66yo, history of fracture -> recommend bone mineral density scan </li></ul></ul></ul></ul><ul><ul><li>Context </li></ul></ul><ul><ul><ul><ul><li>Display when and to whom, </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Resource modeling (nurse role, current nurse, hospitalist; in-house machine vs. referral) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Modeling the opportunities for decision support (SAGE) </li></ul></ul></ul></ul><ul><li>Explanation function </li></ul><ul><ul><ul><li>Authoring, Reviewing </li></ul></ul></ul><ul><ul><ul><ul><li>Not-developed-here syndrome, owner of logic vs. happy user vs. victim </li></ul></ul></ul></ul><ul><li>Knowledge exchange (business model) </li></ul>
  54. 56. Benefits of WT <ul><li>Ability to customize processes within an EHR </li></ul><ul><li>Graphical representation </li></ul><ul><ul><ul><li>author from scratch, modify slightly the logic, modify to fit local context </li></ul></ul></ul><ul><ul><li>Programmer vs. analyst (hard coded processes) </li></ul></ul><ul><ul><li>Understandability (clinician, explanation function) (flowcharts) </li></ul></ul><ul><li>Auditing / change management </li></ul><ul><ul><li>EHR with APIs/links (add task, track tasks, see roles, provide event triggers) </li></ul></ul><ul><ul><li>Observation mode/ Intervention mode </li></ul></ul><ul><li>Ability to share workflows </li></ul><ul><ul><li>Sharing on a graphical level only (no “code”) </li></ul></ul><ul><li>Cross industry technology </li></ul>
  55. 57. Challenges of WT <ul><li>Process administration </li></ul><ul><ul><ul><li>Owner of a process </li></ul></ul></ul><ul><ul><ul><ul><ul><li>Medical home, infections control nurse, consumer (PHR) </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Physician autonomy </li></ul></ul></ul><ul><ul><ul><ul><ul><li>Cook book medicine </li></ul></ul></ul></ul></ul><ul><li>Workflow technology enabled EHR systems </li></ul><ul><ul><ul><ul><li>Add-on integration (TheraDoc, iList) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Direct integration (Siemens Soarian, % of hardcoded steps) </li></ul></ul></ul></ul><ul><li>Graphical modeling </li></ul><ul><ul><ul><ul><li>Several standards (BPMN, YAWL) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Workflow patterns </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Handling exceptions (model in a flowchart), swim-lanes (roles) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Very complex processes (code + HTML docs, just flowchart, simplified flowchart) </li></ul></ul></ul></ul>
  56. 58. Conclusion <ul><li>4 objectives </li></ul><ul><ul><li>Workflow engine, flowcharts (guidelines), current use in healthcare, benefits and challenges </li></ul></ul><ul><li>WT is an emerging technology </li></ul><ul><ul><ul><li>Compared with databases, started in 1999 </li></ul></ul></ul><ul><ul><ul><li>Not everyone means executable process definitions (within a workflow engine) when they say “workflow” </li></ul></ul></ul><ul><li>Cross industry standard to watch/use </li></ul><ul><ul><ul><li>vs. healthcare specific guidelines engines </li></ul></ul></ul><ul><li>Thank you for attention. </li></ul><ul><ul><ul><li>[email_address] </li></ul></ul></ul><ul><ul><ul><li>http://healthcareworkflow.wordpress.com </li></ul></ul></ul>
  57. 63. Extra slides
  58. 64. Vojtech Huser, MD, PhD
  59. 66. Evaluation <ul><li>18 users, 2 parts, qualitative </li></ul><ul><li>9 task questions (T1-T9) </li></ul><ul><ul><ul><ul><li>T5: Find all patients who had at least 2 creatinine lab results flagged too high (but they must be at least 180 days apart). </li></ul></ul></ul></ul><ul><li>5 choice questions (C1-C5) </li></ul><ul><ul><ul><ul><li>Solved problem 1 -> extended problem 2 + choices A), B), C) </li></ul></ul></ul></ul><ul><li>Results </li></ul><ul><ul><li>Statistically significant difference in scores (paired t-test, 2-sided) </li></ul></ul><ul><ul><ul><li>RG: 11.1 ± 1.8 vs. SQL: 6.3 ± 2.1 (p<<0.0001) </li></ul></ul></ul>Vojtech Huser, MD, PhD
  60. 67. Evaluation (SQL and RG) <ul><li>Which technology do you prefer (SQL or RG)? </li></ul><ul><ul><ul><ul><li>94% of participants preferred RG </li></ul></ul></ul></ul><ul><ul><li>and why do you prefer it? </li></ul></ul><ul><ul><ul><ul><li>1. easy to learn/use/understand </li></ul></ul></ul></ul><ul><ul><ul><ul><li>2. temporal modeling capabilities </li></ul></ul></ul></ul><ul><ul><ul><ul><li>3. more intuitive/natural/logical </li></ul></ul></ul></ul><ul><li>Disadvantages of SQL? </li></ul><ul><ul><ul><ul><li>1. must know exact syntax/be expert </li></ul></ul></ul></ul><ul><ul><ul><ul><li>2. difficult to use </li></ul></ul></ul></ul><ul><ul><ul><ul><li>3. insufficient support for temporal criteria </li></ul></ul></ul></ul><ul><li>Disadvantages of RG? </li></ul><ul><ul><ul><ul><li>1. need to know function of various apps and new terminology </li></ul></ul></ul></ul><ul><ul><ul><ul><li>2. none </li></ul></ul></ul></ul><ul><ul><ul><ul><li>3. hard to understand what data user gets back </li></ul></ul></ul></ul><ul><ul><ul><ul><li>4. can be slow for queries involving a larger population </li></ul></ul></ul></ul>Vojtech Huser, MD, PhD Hancock (2004), Lacey (2004)
  61. 68. Evaluation <ul><li>General important features: (general question) </li></ul><ul><ul><li>Highest ranking features </li></ul></ul><ul><ul><ul><li>Intuitive modeling paradigm </li></ul></ul></ul><ul><ul><ul><li>Short training time </li></ul></ul></ul><ul><li>Technology acceptance: (RG only) </li></ul><ul><ul><li>Unified Theory of Acceptance and Use of Technology (UTAUT) </li></ul></ul><ul><ul><li>Performance expectancy, Effort Expectancy, Behavioral intention </li></ul></ul><ul><ul><li>Favorable scores predicting high actual use </li></ul></ul><ul><li>CDS architecture evaluation (Wright, 2009) </li></ul>Vojtech Huser, MD, PhD Venkatesh (2003), Hulse (2006), Hu (1999), Chismar (2002), Wilson (2004), Hoof (2003)

×