Medical Informatics Grand Rounds                                         Aug 17, 2012Cleveland Clinics Usability StudyEsta...
Medical Informatics Grand Rounds                                       Aug 17, 2012Connect with Me          Email         ...
Medical Informatics Grand Rounds                                         Aug 17, 2012Cross reference + Category labelsNew ...
Medical Informatics Grand Rounds                                        Aug 17, 2012New Methods to Analyze QuantitativeOpe...
Background        Planning         Data Collection   Analysis   ImpactAbout• Cleveland Clinic One of Top 4 Hospital in US ...
Background    Planning      Data Collection   Analysis   ImpactAbout•Sole IA for ClevelandClinic.org•Thought leadership: U...
Background     Planning   Data Collection   Analysis   ImpactInfluence Decision Makers• Who they are• How they think
Background     Planning   Data Collection   Analysis   ImpactInfluence Decision Makers• Who they are• How they think
Background     Planning   Data Collection   Analysis    ImpactInfluence Decision Makers• Who they are• How they think     ...
Background    Planning     Data Collection   Analysis   Impact State of Site Structure• Inconsistent navigation           ...
Background   Planning   Data Collection   Analysis   Impact A website is…
Background   Planning   Data Collection   Analysis   Impact A website is…             Site Map                   Navigatio...
Background    Planning     Data Collection   Analysis   Impact State of Site Structure• Inconsistent navigation
Background   Planning   Data Collection   Analysis   ImpactGoal:Consistent User Experience across ccf.orgSolution:Standard...
Background   Planning   Data Collection   Analysis   ImpactState of Sitea standard   Needs Structure        site map     a...
Background   Planning   Data Collection   Analysis   Impact Settle Design Debate• Started out as a design dilemma• Real pr...
Background   Planning   Data Collection     Analysis       ImpactPlan the study• Quantitative             Corporate cultur...
Background    Planning     Data Collection   Analysis   ImpactResearch Questions• Elements of a sitemap   •Number of categ...
Background   Planning   Data Collection   Analysis   ImpactGenerate Cards• Full Content Inventory• 71 cards
Background   Planning   Data Collection   Analysis   ImpactOnline opencard sort• Sort all 71 cards• See card label &descri...
Background   Planning   Data Collection   Analysis   Impact Cross reference + Category labels  New Methods to Analyze Quan...
Background       Planning     Data Collection       Analysis          ImpactStatistical Analysis Methods• 179 completes, 7...
Background   Planning   Data Collection   Analysis   ImpactIn-house Cluster Analysis
Background      Planning    Data Collection       Analysis          ImpactStatistical Analysis MethodsCluster (Dendrogram)...
Background   Planning   Data Collection   Analysis   ImpactIn-house Multidimensional Scaling
Background                    Planning    Data Collection       Analysis          ImpactStatistical Analysis MethodsCluste...
Background   Planning   Data Collection   Analysis          ImpactCell Plot Analysis                                      ...
Background   Planning   Data Collection   Analysis                    ImpactCategory Labels                               ...
Background       Planning     Data Collection         Analysis           ImpactKeyword Frequency  ?          Participant #...
Background    Planning   Data Collection   Analysis   ImpactImpact of the study• Established a standard sitemap Accepted b...
Background   Planning   Data Collection   Analysis   ImpactApplication of Standard Sitemap
Background      Planning    Data Collection   Analysis   ImpactConclusions & Discussions• Tailor UX study to corporate cul...
Background      Planning      Data Collection     Analysis   ImpactTreatment Outcomes • Qualitative • Before product launc...
Background   Planning   Data Collection   Analysis   ImpactTreatment Outcomes                         Scenario            ...
Background   Planning   Data Collection   Analysis   ImpactTreatment Outcomes                          Website            ...
Background            Planning         Data Collection          Analysis              Impact  Task: Evaluate quality of ca...
Background         Planning       Data Collection    Analysis        Impact  Evaluate quality of care                     ...
Background        Planning      Data Collection     Analysis           Impact  Outcome BookOn task “evaluate qualify of me...
Medical Informatics Grand Rounds                                                                                          ...
Medical Informatics Grand Rounds                           Aug 17, 2012Special Thanks to•Amy Sternad•Michelle Tackla Walla...
Background   Planning   Data Collection   Analysis   Impact
Upcoming SlideShare
Loading in …5
×

User Experience Design on Cleveland Clinic Corporate Website | Medical Informatics Grand Rounds

1,068 views

Published on

Presented to Cleveland Clinic Medical Informatics Grand Rounds meeting, Aug 17, 2012

Published in: Technology, Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,068
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • So we set up the study in optimalsort, Market research department were the ones who did the actual data collection.
  • So we set up the study in optimalsort, Market research department were the ones who did the actual data collection.
  • Spencer’s spreadsheet accommodates 40 cards Built-in tools: cluster analysis
  • Spencer’s spreadsheet accommodates 40 cards Built-in tools: cluster analysis
  • Spencer’s spreadsheet accommodates 40 cards Built-in tools: cluster analysis
  • Spencer’s spreadsheet accommodates 40 cards Built-in tools: cluster analysis
  • Spencer’s spreadsheet accommodates 40 cards Built-in tools: cluster analysis
  • Qualitative data – emotional persuasion boss cube, first line of defense. Chair/manager making requests. In the past we let them drive. Now we have sitemap to guide them Department requests UX study. No proposal necessary. No struggle with participant incentives.
  • Qualitative data – emotional persuasion
  • Qualitative data – emotional persuasion
  • Qualitative data – emotional persuasion
  • Qualitative data – emotional persuasion
  • Qualitative data – emotional persuasion
  • Qualitative data – emotional persuasion
  • Qualitative data – emotional persuasion
  • User Experience Design on Cleveland Clinic Corporate Website | Medical Informatics Grand Rounds

    1. 1. Medical Informatics Grand Rounds Aug 17, 2012Cleveland Clinics Usability StudyEstablish Standard Sitemap for consistent user experience Kaitlan Chu | Information Architect | Cleveland Clinic
    2. 2. Medical Informatics Grand Rounds Aug 17, 2012Connect with Me Email ChuK@ccf.org Twitter @KaitlanChu LinkedIn linkedin.com/in/KaitlanChu SlideShare slideshare.net/KaitlanChu
    3. 3. Medical Informatics Grand Rounds Aug 17, 2012Cross reference + Category labelsNew Methods to Analyze QuantitativeOpen Card Sort:Cleveland Clinics First UX Study Formal, full-cycle, by in-house staff Kaitlan Chu | Information Architect | Cleveland Clinic
    4. 4. Medical Informatics Grand Rounds Aug 17, 2012New Methods to Analyze QuantitativeOpen Card Sort:Cleveland Clinics First UX Study Designed to maximize stakeholder buy-in Kaitlan Chu | Information Architect | Cleveland Clinic
    5. 5. Background Planning Data Collection Analysis ImpactAbout• Cleveland Clinic One of Top 4 Hospital in US 41,000+ Employees•Digital Marketing ClevelandClinic.org 13,000 Pages 120,000 visits/day Most visited US hospital site $5/visit in contribution margin
    6. 6. Background Planning Data Collection Analysis ImpactAbout•Sole IA for ClevelandClinic.org•Thought leadership: Usability / Navigation•Sign off on site changes•First to conduct UX studies in house•Academic research background•Analytics & front-end development, Fortune 100
    7. 7. Background Planning Data Collection Analysis ImpactInfluence Decision Makers• Who they are• How they think
    8. 8. Background Planning Data Collection Analysis ImpactInfluence Decision Makers• Who they are• How they think
    9. 9. Background Planning Data Collection Analysis ImpactInfluence Decision Makers• Who they are• How they think Quantitative Studies Authoritative Sources
    10. 10. Background Planning Data Collection Analysis Impact State of Site Structure• Inconsistent navigation What is navigation?
    11. 11. Background Planning Data Collection Analysis Impact A website is…
    12. 12. Background Planning Data Collection Analysis Impact A website is… Site Map Navigation Menu
    13. 13. Background Planning Data Collection Analysis Impact State of Site Structure• Inconsistent navigation
    14. 14. Background Planning Data Collection Analysis ImpactGoal:Consistent User Experience across ccf.orgSolution:Standard Institute Sitemap
    15. 15. Background Planning Data Collection Analysis ImpactState of Sitea standard Needs Structure site map applicable to all departments• Inconsistent navigation
    16. 16. Background Planning Data Collection Analysis Impact Settle Design Debate• Started out as a design dilemma• Real problem – inconsistent site structures• Solution – standard sitemap
    17. 17. Background Planning Data Collection Analysis ImpactPlan the study• Quantitative Corporate culture• Open card sort No presumption• Online, remote Content inventory• Put together a team Participant Recruiting: Panel• Budget $300 Statistical Analysis• Analysis guidelines Hard to find; Invent our own
    18. 18. Background Planning Data Collection Analysis ImpactResearch Questions• Elements of a sitemap •Number of categories •Content within categories •Cross-reference ? •Category label ?
    19. 19. Background Planning Data Collection Analysis ImpactGenerate Cards• Full Content Inventory• 71 cards
    20. 20. Background Planning Data Collection Analysis ImpactOnline opencard sort• Sort all 71 cards• See card label &description• Name eachcategory
    21. 21. Background Planning Data Collection Analysis Impact Cross reference + Category labels New Methods to Analyze Quantitative Open Card Sort: Cleveland Clinics First UX Study
    22. 22. Background Planning Data Collection Analysis ImpactStatistical Analysis Methods• 179 completes, 71 cards => 12710 lines in excel New Method Cluster (Dendrogram) Multidimensional Scaling Cell Plot
    23. 23. Background Planning Data Collection Analysis ImpactIn-house Cluster Analysis
    24. 24. Background Planning Data Collection Analysis ImpactStatistical Analysis MethodsCluster (Dendrogram) Multidimensional Scaling Cell Plot
    25. 25. Background Planning Data Collection Analysis ImpactIn-house Multidimensional Scaling
    26. 26. Background Planning Data Collection Analysis ImpactStatistical Analysis MethodsCluster (Dendrogram) Multidimensional Scaling Cell Plot• Elements of a sitemap •Number of categories •Content within categories •Cross-reference ? •Category label ?
    27. 27. Background Planning Data Collection Analysis ImpactCell Plot Analysis Cross reference
    28. 28. Background Planning Data Collection Analysis ImpactCategory Labels • Elements of a sitemap• What keywords to use •Number of categories •Content within categories• What NOT to use •Cross-reference •Category label ?
    29. 29. Background Planning Data Collection Analysis ImpactKeyword Frequency ? Participant #1 #2, #3, #4………… Participant #179 words words Patient 1378 Info 632 General 152 Care 81 Appointment 80 Resource 60 Become 17
    30. 30. Background Planning Data Collection Analysis ImpactImpact of the study• Established a standard sitemap Accepted by all departments• Stakeholders see the value Greatly increased budget line• Increase perceived value of myteam & UX studies
    31. 31. Background Planning Data Collection Analysis ImpactApplication of Standard Sitemap
    32. 32. Background Planning Data Collection Analysis ImpactConclusions & Discussions• Tailor UX study to corporate culture• Analyze data in as many methods as possible • Cluster • Multidimensional Scaling 3D • Cell Plot • Keyword Frequency New study findings:• Pre-test online study in-person Treatment Outcomes • - Mix qualitative & quantitative • - Test the usability of online study• Pilot test with participants represent real users• Provide decent participant incentives• Number of cards?
    33. 33. Background Planning Data Collection Analysis ImpactTreatment Outcomes • Qualitative • Before product launch - Opportunity to improve • One-on-one, in-person, lab-based study Camera Participant Moderator
    34. 34. Background Planning Data Collection Analysis ImpactTreatment Outcomes Scenario Task
    35. 35. Background Planning Data Collection Analysis ImpactTreatment Outcomes Website Task
    36. 36. Background Planning Data Collection Analysis Impact Task: Evaluate quality of careTask Description:You’re comparing a few hospitals to take your uncle to treat his epilepsy. Avoid keywordWhere do you go to evaluate the quality of medical services and expertise the “Outcomes”center provide?For example,• How many patients did the epilepsy center successfully treat last year?• Is the epilepsy center using up-to-date technology for diagnosis and treatments?• Does the center have leading experts in the field?Your Task: Where do you go to evaluate the quality of medical services andexpertise the center provide? E.g. Number of successfully treated patients; up-to-date technology; and leading experts in the field.Research questions:•Does an average patient understand “outcomes”?•Do they read Outcome book?•We should place it in “For Medical Professionals”
    37. 37. Background Planning Data Collection Analysis Impact Evaluate quality of care Diagnosis Research 4 Outcomes Outcome Book Outcome Book 1 About Us Outcomes•“Outcomes” is associated with quality of medical care.•Does an average patient understand “outcomes”? Yes•Do they read Outcome book? Yes, at least 2 read cover to cover•We should place it in “For Medical Professionals”. Patients nevernavigate to that section.
    38. 38. Background Planning Data Collection Analysis Impact Outcome BookOn task “evaluate qualify of medical care”:Participant:“The site [outcome book pdf] itself is sort of like you’re reading atextbook. So I blew right over that the first time.The comparison chart with national data is a great chart. Once youcaught my attention with that then I would spend more time readingthe other stuff [charts and data].”
    39. 39. Medical Informatics Grand Rounds Aug 17, 2012References•Capra, M. (2005) Factor Analysis of Card Sort Data: An Alternative to Hierarchical Cluster Analysis. Human Factors and Ergonomics Society AnnualMeeting Proceedings, Computer Systems , pp. 691-695(5). Human Factors and Ergonomics Society.•Hinkle, V. (2008, October) Card-Sorting: What You Need to Know about analyzing and Interpreting Card Sorting Results. Retrieved on February 152011 from Usability News October 2008, Vol. 10 Issue 2 (http://www.surl.org/usabilitynews/102/cardsort.asp).•Krystal M. Lewis, Peter Hepburn, (2010) "Open card sorting and factor analysis: a usability case study", Electronic Library, The, Vol. 28 Iss: 3, pp.401 –416•Paul, Celeste Lyn (2008) “A modified Delphi Approach to a New Card Sorting Methodology” Journal of Usability Studies 4 (1), 7-30.•Puskala, A. (Nov 17 2009) Card Sort cluster analysis tool 1.1. Retrieved on February 15 2010 from User Point (http://www.userpoint.fi/tools/card_sort_cluster_analysis_tool/).•Spencer, D. (2009) Card Sorting: Designing Usable Categories. New York: Rosenfeld Media.•Spencer, D. (2007, June 7) Card sort analysis spreadsheet. Retrieved on February 15 2011 from RosenfeldMedia.com (http://rosenfeldmedia.com/books/cardsorting/blog/card_sort_analysis_spreadsheet).•Tullis, T. & Albert, B. (2008) Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Burlington, MA: MorganKaufmann Publishers.•Tullis, T. & Wood, L.E. (2005). How Can You Do a Card-sorting Study with LOTS of Cards? Poster presented at the Annual Meeting of the UsabilityProfessionals Association, June 24 - July 1, Montreal, QB, Canada.•Free tool to generate cluster analysis for large open card sort http://www.userpoint.fi/tools/card_sort_cluster_analysis_tool/
    40. 40. Medical Informatics Grand Rounds Aug 17, 2012Special Thanks to•Amy Sternad•Michelle Tackla Wallace•Amanda Beacher•Jieying Jane Chen•Zhijie Li•Scott Linabarger•Lynn Pecl
    41. 41. Background Planning Data Collection Analysis Impact

    ×