Preliminary Findings from a Randomized Mixed-Methods Cross-Over Study to Compare the Performance of DVD and Web-based Patient Decision Support
Preliminary findings from a randomized mixed-methods cross-over study to compare the performance of DVD and web-based patient decision support Dominick L. Frosch, PhD Associate Investigator, PAMFRI Associate Professor of Medicine, UCLA July 31, 2012
Knowledge gaps in decision supportn The internet is increasingly becoming the primary medium for providing patients decision support.n Few data are available to guide optimal design of internet decision support and ensure comparable effectiveness compared to video.n A handful of studies have examined the efficacy of internet decision support, but only one study (2003) directly compared internet and video decision support.
Review of Video and Internet Presentationn 98% of participants assigned to the Video group viewed the tape.n 54% of those assigned to Internet viewed the entire presentation of 47 slides. 40% never reviewed any part of the presentation. Frosch, Kaplan & Felitti, JGIM, 2003.
PSA Knowledge 54.5 43.5 3 Video2.5 Internet 21.5 10.5 0 Pre Post p<.001
PSA Knowledge:Video & Internet Completers 54.5 43.5 3 Video2.5 I-Net Comp. 21.5 10.5 0 Pre Post
Research questionsn How do internet and video decision support compare in terms of: – Increases in patient knowledge – User acceptability – Decision qualityn What proportion of web-based decision support do individuals review and how much time do they spend with it?
Study design Figure 1. Research design Primary care patients (N=90) Complete baseline quantitative survey - 1 Review Review RA probes DVD content Health selection Crossroads Complete quantitative survey - 2 RA probes Review Review content Health DVD selection Crossroads Complete quantitative survey - 3
Study measuresn Decision/disease specific knowledgen Quantitative user ratings of decision supportn Decision qualityn Role preferencesn Click-stream (para) data on HC usen Think-out loud while using HCn In-depth interviews following each review of decision support
Study setting and samplingn Palo Alto Medical Foundation in Silicon Valley, CA.n Participants are active primary care patients at PAMF who have/are: – Diabetes – Benign Prostatic Hyperplasia – Overdue to consider colon cancer screeningn Participants are randomly selected from primary care panels and invited by mail/phone.
Participant demographics N=30Age 64.6 (SD=12.9)Gender 63.3% MaleMarital status 76.7% married or living with partnerEthnicity 83.3% Caucasian 10.0% Asian 6.7% LatinoEducation 80.0% College graduate or moreHousehold income 85.7% >$50,000Health literacy score (range 0-8) 7.9 (SD=.35)
How much do patients view on the Health Crossroads? DVD first WWW first70.00% 66.00%60.00% 54.00% 50.00% 50.00% 51.00%50.00% 46.00% 45.00%40.00% 36.00%30.00%20.00%10.00%0.00% All BPH CRC Diabetes
How much time do patients spend on the Health Crossroads? DVD first WWW first Video length50.0 4545.040.035.0 33 32.2 3130.0 26.7 27.4 25.2 2425.0 21.020.0 17.6 17.9 16.615.010.0 5.0 0.0 All BPH CRC Diabetes
How many videos in the Health Crossroads do patients click on? DVD first WWW first6.0 5.05.04.03.0 2.52.0 1.8 1.8 1.5 1.4 1.41.0 0.80.0 All BPH (25) CRC (27) Diabetes (19)
Time spent watching videos in the Health Crossroads DVD first WWW first3.5 3.13.0 2.62.5 2.4 2.02.0 1.6 1.61.5 1.4 1.31.00.50.0 All BPH CRC Diabetes
Reflections on using Health Crossroads“I didn’t even look at the videos but just the presence of the videos seemed like it wasgetting in my way, it was just filling up space, and you know I didn’t come looking for avideo…these days not knowing what’s going to happen when you click on a video, if you’regoing to get an advertisement, and uh, and then you know, something you know a video,will take 10 minutes to say something that you can read in a minute.” MALE, 54, CRC"My reaction is to take the DVD only because everybody likes to watch a DVD, but thewebsite is better. But see I was forced to read the website. If I had the DVD, I will actuallyput the DVD in and play it. I would go to [the website]; I dont know if I would do as intensivereading as I did here, um, um. There has to be a catch but I dont know what the catchis...There has to be follow-up. If theres no follow up, then they might just flip through [thewebsite] and jot down and see a picture and read a line, but not really read the whole thing."MALE, 61, DIABETES
Changes in knowledge - Combined DVD first WWW first90.0% 82.0% 79.0% 77.0%80.0%70.0% 68.0% 65.0%60.0% 56.0%50.0%40.0%30.0%20.0%10.0%0.0% Baseline First review Second review All
Changes in knowledge - BPH DVD first WWW first80.0% 73.0%70.0% 63.0% 60.0%60.0% 48.0%50.0% 43.0% 40.0%40.0%30.0%20.0%10.0%0.0% Baseline First review Second review BPH
Changes in knowledge – CRC Screening DVD first WWW first120.0% 95.0% 98.0%100.0% 92.0% 78.0% 80.0%80.0%60.0% 55.0%40.0%20.0% 0.0% Baseline First review Second review CRC
Changes in knowledge – Diabetes DVD first WWW first82.0% 80.0% 80.0% 80.0%80.0% 78.0%78.0%76.0%74.0% 73.0%72.0% 71.0%70.0%68.0%66.0% Baseline First review Second review Diabetes
Understanding how patients use HC“It wasn’t too busy, you know um, I didn’t get bored, I could easily find something you know,some websites are too much you know, this was simple here’s all the different tests you cantake and these are all the procedures, I liked the chart you know the summary thing causeyou forget and with the summary then you don’t have to click from page to page cause youhave it all right there so that’s probably what I liked best was the summary chart.” FEMALE,52, CRC"I liked [the website] better because you could read something if werent sure about it youcan go back and look at it." "In any given amount of time, you can absorb just so muchinformation. So the DVD is a little too fast, but that cant be helped. So the website is a verygood supplement." MALE, 85, BPH
Predictors of Health Crossroadsknowledge gain Variable R2 Change BetaAge .10 -.17Education .21 .28Proportion of pages .05 .18viewedNumber of videos clicked .10 .40 Cumulative R2 = .46
Which medium do patients prefer? DVD first WWW first70.0% 64.3%60.0%50.0% 43.8%40.0% 37.5%30.0% 21.4% 18.7%20.0% 14.3%10.0%0.0% Prefer video Equal ratings Prefer Health Crossroads
Preferences for Health Crossroads"Maybe I shouldnt say I would never use the DVD. If [my doctor] gave me the DVD as apackage and said view the DVD, you know, and then go to the website, you know, I woulddo that. As I was watching [the DVD], I was thinking that I would never go back to this as atool. I would watch it once...and be done with it." MALE, 66, DIABETES"I am not an internet person. I dont sit there and look at this stuff all day long, but I can readit. I can go back to it. I can access it immediately with a mouse, and get what I want. [I] donthave to watch the whole thing again to figure out where in the picture it was and I have to gofind this out. So I like that first website very much." MALE, 85, DIABETES“The DVD you didn’t have any choice you just had to watch whatever came up but this oneyou could choose what you wanted to see.” FEMALE, 64, DIABETES"Everybody sits at a computer and its just so easy you know, but you get a DVD in mail andgo "Oh I want to watch that" and you throw it on your desk, but you never do it. Buteveryone sits at a computer and its just "click" and were there." FEMALE, 52, CRC
Conclusionsn Data appear to confirm that DVD may be the superior option, for ensuring that patients are exposed to all necessary information.n Knowledge gains appear to be greater for DVD.n More viewing of videos embedded in HC appears to drive greater knowledge gains.n Some suggestions that having both mediums is valuable for patients – start with DVD, but use HC to reinforce knowledge gains.
Limitations and next steps n Findings are preliminary, based on 1/3 of ultimate sample of 90 participants. n Study is exploratory, not designed or powered for null-hypothesis significance testing. n Non-naturalistic laboratory study, but qualitative data give us insight into how people might use decision support at home. n Participant recruitment is currently 85% complete. Analysis will be complete by year-end.