The Chain of Trust —  Quality and Credibility in Health Information Sources PF Anderson,  [email_address] April 23, 2008 ©...
Background <ul><li>In 2006 I was asked by two nursing faculty to develop:  </li></ul><ul><ul><li>a quality assessment tool...
My earlier tools <ul><li>Evaluation Tool for Clinician-Oriented Web Sites  </li></ul><ul><ul><li>http://www-personal.umich...
Why? <ul><li>Everyone wants the good information </li></ul><ul><li>Nobody agrees what that is </li></ul><ul><li>Like “matc...
Why? (cont.) <ul><li>What’s good depends on: </li></ul><ul><ul><li>The question </li></ul></ul><ul><ul><li>Who’s asking </...
What’s missing <ul><li>Professional codes define professional responsibilities with ever-greater accuracy. Huge efforts al...
What’s needed <ul><li>Trust  </li></ul><ul><li>Respectful dialogue </li></ul><ul><li>Tools to support dialogue </li></ul>
Solution? <ul><li>Yes, “trust your gut”,  but … </li></ul><ul><li>Train the “gut feeling” </li></ul><ul><ul><li>Discuss & ...
Potential audience <ul><li>Students, undergrad </li></ul><ul><li>Students, grad & professional </li></ul><ul><li>Patients ...
Challenges  <ul><li>Easy to use </li></ul><ul><li>Easy to remember </li></ul><ul><li>Broad audience / multipurpose </li></...
Process  <ul><li>Review as many existing similar tools as discoverable, for all audiences, from elementary school children...
Other information  quality assessment tools <ul><li>ABC (The Good, The Bad, & the Ugly):  </li></ul><ul><ul><li>Accuracy, ...
Aha moment - “Great home page” Ahmad Risk <ul><li>Full text:  </li></ul><ul><ul><li><http://www-personal.umich.edu/~pfa/pr...
Strategy <ul><li>Keyword based </li></ul><ul><li>Concept clustering </li></ul><ul><li>Likert scale </li></ul><ul><li>Level...
Assumptions <ul><li>All information sources have some good information </li></ul><ul><li>All information sources have some...
Chain <ul><li>Candor </li></ul><ul><li>Honesty </li></ul><ul><li>Accountability </li></ul><ul><li>Information quality </li...
Trust <ul><li>Timeliness </li></ul><ul><li>Relevant </li></ul><ul><li>Unbiased </li></ul><ul><li>Scope </li></ul><ul><li>T...
Scoring <ul><li>0 = No way </li></ul><ul><li>5 = I really really like it! </li></ul><ul><li>Lowest possible score = 0 </li...
Connect your gut feeling to the evidence <ul><li>Places information in context </li></ul><ul><li>Makes more overt aspects ...
Levels of evidence  in EBHC
A broader  view
How do they relate?
An alternate  view
What is “best  available evidence”? <ul><li>Example 1: Common condition, common topic, large research base? </li></ul><ul>...
Get your own copy <ul><li>From this page:  </li></ul><ul><ul><li>http://www-personal.umich.edu/~pfa/pro/courses/ </li></ul...
More information <ul><li>Links: </li></ul><ul><ul><li>http://del.icio.us/rosefirerising/assessment+authority.credibility <...
Upcoming SlideShare
Loading in...5
×

Chain of Trust, a web quality assessment tool

2,169

Published on

Presentation for the School of Dentistry Bootcamp series on April 23, 2008. Uploaded originally at that time, but Slideshare for some inexplicable reason deleted the file. Hope it sticks this time. The Chain of Trust / Levels of Evidence tool was originally developed for use by college undergraduate students, shown adapted here for use in healthcare. It is appropriate for a wide variety of audiences.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,169
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Chain of Trust, a web quality assessment tool

  1. 1. The Chain of Trust — Quality and Credibility in Health Information Sources PF Anderson, [email_address] April 23, 2008 © 2008 Regents of the University of Michigan. All rights reserved.
  2. 2. Background <ul><li>In 2006 I was asked by two nursing faculty to develop: </li></ul><ul><ul><li>a quality assessment tool </li></ul></ul><ul><ul><li>for both health information and term paper sources </li></ul></ul><ul><ul><li>that could be used with incoming freshman </li></ul></ul><ul><ul><li>as an assignment. </li></ul></ul><ul><li>Existing tools were either too specific, too difficult to use, or not appropriate for the target audience. </li></ul>
  3. 3. My earlier tools <ul><li>Evaluation Tool for Clinician-Oriented Web Sites </li></ul><ul><ul><li>http://www-personal.umich.edu/~pfa/pro/courses/EvalClin.pdf </li></ul></ul><ul><li>Evaluation Tool for Patient-Oriented Web Sites </li></ul><ul><ul><li>http://www-personal.umich.edu/~pfa/pro/courses/EvalPtEd.pdf </li></ul></ul><ul><li>NOTE: In prior testing, these tools had proven effective, but difficult to use in teaching, and time consuming for scoring. </li></ul>
  4. 4. Why? <ul><li>Everyone wants the good information </li></ul><ul><li>Nobody agrees what that is </li></ul><ul><li>Like “match the tool to the task”, information one person recommends won’t be right for another person </li></ul><ul><li>Censoring what information is available or “acceptable” creates a lack of trust </li></ul><ul><li>Criticizing information choices impairs communication (and trust) </li></ul>
  5. 5. Why? (cont.) <ul><li>What’s good depends on: </li></ul><ul><ul><li>The question </li></ul></ul><ul><ul><li>Who’s asking </li></ul></ul><ul><ul><li>Literacy </li></ul></ul><ul><ul><li>Prior educational context </li></ul></ul><ul><ul><li>Prior knowledge in this topic </li></ul></ul><ul><ul><li>Prior experiences </li></ul></ul><ul><ul><li>Attitudes & assumptions </li></ul></ul><ul><ul><li>Intangible personal preferences </li></ul></ul><ul><ul><li>Other context </li></ul></ul>
  6. 6. What’s missing <ul><li>Professional codes define professional responsibilities with ever-greater accuracy. Huge efforts also go into ensuring trustworthy performance. . . .The efforts to prevent abuse of trust are gigantic, relentless and expensive; and inevitably their results are always less than perfect. Have these countermeasures begun to restore trust, or just to reduce suspicion? . . . Patients, it is said, no longer trust doctors . . . and in particular no longer trust hospitals or hospital consultants. &quot;Loss of trust&quot; is in short a clich é of our times. </li></ul><ul><ul><li>O'Neill, Onora. A Question of Trust. (BBC Reith Lectures 2002: Lecture 1. &quot;Spreading Suspicion,&quot; April 6, 2002, pp. 2 - 3.) Retrieved February 3, 2004, from http://www.bbc.co.uk/radio4/reith2002/pdf/lecture1_text.pdf; also available at http://www.bbc.co.uk/radio4/reith2002/1.shtml </li></ul></ul><ul><ul><li>Cited in: MLA Guide, Introduction: http://www-personal.umich.edu/~pfa/mlaguide/free/power.html </li></ul></ul>
  7. 7. What’s needed <ul><li>Trust </li></ul><ul><li>Respectful dialogue </li></ul><ul><li>Tools to support dialogue </li></ul>
  8. 8. Solution? <ul><li>Yes, “trust your gut”, but … </li></ul><ul><li>Train the “gut feeling” </li></ul><ul><ul><li>Discuss & define common elements </li></ul></ul><ul><ul><li>Articulate contextual elements </li></ul></ul><ul><ul><li>Describe environmental constraints </li></ul></ul><ul><ul><li>Articulate criteria </li></ul></ul><ul><li>In other words </li></ul><ul><ul><li>Spell it out </li></ul></ul>
  9. 9. Potential audience <ul><li>Students, undergrad </li></ul><ul><li>Students, grad & professional </li></ul><ul><li>Patients </li></ul><ul><li>Healthcare professionals </li></ul><ul><li>Friends and family </li></ul><ul><li>Personal use </li></ul><ul><li>Anyone </li></ul>
  10. 10. Challenges <ul><li>Easy to use </li></ul><ul><li>Easy to remember </li></ul><ul><li>Broad audience / multipurpose </li></ul><ul><ul><li>Incoming freshmen </li></ul></ul><ul><ul><li>Training of healthcare professionals </li></ul></ul><ul><li>Broad range of information available </li></ul><ul><li>Blurred media boundaries </li></ul><ul><ul><li>traditional print peer-reviewed publications </li></ul></ul><ul><ul><li>popular press & news media </li></ul></ul><ul><ul><li>general web </li></ul></ul><ul><li>Potential range of literacy skills </li></ul><ul><li>Potential range of awareness of health information quality issues & importance </li></ul>
  11. 11. Process <ul><li>Review as many existing similar tools as discoverable, for all audiences, from elementary school children to professionals </li></ul><ul><li>Collect and organize concepts represented in the various tools </li></ul><ul><ul><li>Note frequency of distribution for common concepts </li></ul></ul><ul><ul><li>Note unique and valuable concepts </li></ul></ul><ul><li>Prioritize </li></ul><ul><li>Seek unifying structure </li></ul>
  12. 12. Other information quality assessment tools <ul><li>ABC (The Good, The Bad, & the Ugly): </li></ul><ul><ul><li>Accuracy, authority, bias/objectivity, currency, coverage </li></ul></ul><ul><ul><li>http://lib.nmsu.edu/instruction/evalcrit.html </li></ul></ul><ul><li>Internet Detective </li></ul><ul><ul><li>http://www.vts.intute.ac.uk/detective/where.html </li></ul></ul><ul><li>Five criteria (Cornell) </li></ul><ul><ul><li>http://www.vts.intute.ac.uk/detective/where.html </li></ul></ul><ul><li>MedlinePlus: Guide to Healthy Web Surfing: </li></ul><ul><ul><li>http://www.nlm.nih.gov/medlineplus/healthywebsurfing.html </li></ul></ul>
  13. 13. Aha moment - “Great home page” Ahmad Risk <ul><li>Full text: </li></ul><ul><ul><li><http://www-personal.umich.edu/~pfa/pro/greatrisk.html> </li></ul></ul><ul><li>Candour </li></ul><ul><li>Honesty </li></ul><ul><li>Quality </li></ul><ul><li>Informed consent </li></ul><ul><li>Privacy </li></ul><ul><li>Professionalism </li></ul><ul><li>Responsible partnering </li></ul><ul><ul><li>That the website chooses its partner with care and that those partners strengthen the chain of trust among its community. </li></ul></ul><ul><li>Accountability </li></ul>
  14. 14. Strategy <ul><li>Keyword based </li></ul><ul><li>Concept clustering </li></ul><ul><li>Likert scale </li></ul><ul><li>Levels of evidence - extended </li></ul>
  15. 15. Assumptions <ul><li>All information sources have some good information </li></ul><ul><li>All information sources have some bad information </li></ul><ul><li>Peer review tends to be better information, but cannot be guaranteed </li></ul><ul><li>Branding or certificate programs likewise tend to be better but cannot be guaranteed </li></ul><ul><li>Who watches the watchers? </li></ul>
  16. 16. Chain <ul><li>Candor </li></ul><ul><li>Honesty </li></ul><ul><li>Accountability </li></ul><ul><li>Information quality </li></ul><ul><li>Neighborly </li></ul>
  17. 17. Trust <ul><li>Timeliness </li></ul><ul><li>Relevant </li></ul><ul><li>Unbiased </li></ul><ul><li>Scope </li></ul><ul><li>Trustworthy </li></ul>
  18. 18. Scoring <ul><li>0 = No way </li></ul><ul><li>5 = I really really like it! </li></ul><ul><li>Lowest possible score = 0 </li></ul><ul><li>Highest possible score = 50 </li></ul><ul><li>Perfectly alright to use the tool without the numbers - it is the ideas that are important. </li></ul>
  19. 19. Connect your gut feeling to the evidence <ul><li>Places information in context </li></ul><ul><li>Makes more overt aspects such as </li></ul><ul><ul><li>Type of question being asked </li></ul></ul><ul><ul><li>Range of information available </li></ul></ul><ul><ul><li>What is the best evidence available </li></ul></ul><ul><ul><li>Environmental factors or constraints </li></ul></ul>
  20. 20. Levels of evidence in EBHC
  21. 21. A broader view
  22. 22. How do they relate?
  23. 23. An alternate view
  24. 24. What is “best available evidence”? <ul><li>Example 1: Common condition, common topic, large research base? </li></ul><ul><ul><li>Gold standard: systematic reviews, meta-analyses, randomized controlled clinical trials (RCTs, CCTs) </li></ul></ul><ul><li>Example 2: Rare condition, little to no research? </li></ul><ul><ul><li>Expert opinion, If you can find it </li></ul></ul><ul><ul><li>Patient perspective, support groups, personal web pages </li></ul></ul><ul><li>… and all the range between the two … </li></ul>
  25. 25. Get your own copy <ul><li>From this page: </li></ul><ul><ul><li>http://www-personal.umich.edu/~pfa/pro/courses/ </li></ul></ul><ul><li>Horizontal version: </li></ul><ul><ul><li>http://www-personal.umich.edu/~pfa/pro/courses/ChainOfTrustLoEHoriz2.pdf </li></ul></ul><ul><li>Vertical version: </li></ul><ul><ul><li>http://www-personal.umich.edu/~pfa/pro/courses/ChainOfTrustLoEVert2.pdf </li></ul></ul>
  26. 26. More information <ul><li>Links: </li></ul><ul><ul><li>http://del.icio.us/rosefirerising/assessment+authority.credibility </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×