WST PhD presentation for PenTAG 17may11
Upcoming SlideShare
Loading in...5
×
 

WST PhD presentation for PenTAG 17may11

on

  • 433 views

 

Statistics

Views

Total Views
433
Views on SlideShare
433
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

WST PhD presentation for PenTAG 17may11 WST PhD presentation for PenTAG 17may11 Presentation Transcript

  • Information Graphics inHealth Technology Assessment Will Stahl-Timmins 17th May 2011
  • Information Graphics
  • maximumvalueupperquartilemedianlowerquartileminimumvalue View slide
  • maximumvalueupperquartilemedianlowerquartileminimumvalue View slide
  • PhD research intro A journey My PhD (so far) Opportunities
  • PhD research intro A journey My PhD (so far) Opportunities
  • Health Technology Assessment
  • Health Policy Medical Practice EBMHTA Scientific Evidence
  • Information Graphics in Health Technology Assessment Virtual presentations of information, which usegraphical elements (eg position, colour, size, etc) to present scientific evidence, informing health policy-making in terms of recommendations for the adoption of specific health interventions.
  • Research QuestionHow should information graphics be designed, produced and used in health technology assessment?
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • TAR review• 50 of 98 NICE TAR reports reviewed• dated Oct 2003 - Nov 2007• content analysis• graphics categorised
  • • 965 graphics used
  • • 965 graphics used• graphics in every report but one
  • • 965 graphics used• graphics in every report but one• 0.20 graphics per page
  • • 965 graphics used• graphics in every report but one• 0.20 graphics per page• 0.58 tables per page
  • 1.0• 965 graphics used 0.8• graphics in every report 0.6 but one 0.4• 0.20 graphics per page 0.2• 0.58 tables per page 0 GRAPHICS TABLES PER PAGE PER PAGE
  • TIME SERIES CEAC LINETHRESHOLD ANALYSIS 301 102 124 37 38 OTHER STATE TRANSITION 78 41 DECISON TREE FLOW 5 OTHER 124 BAR CHART SCATTER PLOT AREA /POSITION 88 55 44 OTHER 187 FOREST PLOT OTHER 331 22 OTHER 353
  • USED IN NICE-COMMISSIONED TECHNOLOGY ASSESSMENT REPORTS, 2003-2007 CIRCLES REPRESENT THE NUMBER OF REPORTS THAT USED A TYPE OF GRAPHIC AT LEAST ONCE, BY REPORT SECTION INTRO/ SYSTEM. SYSTEM. MODEL MODEL MODEL CONC. APPEN- BACKG. REVIEW REVIEW REVIEWS METHODS RESULTS DICES METHODS RESULTSLINE TIME SERIES 10 6 10 8 8 9 CEAC 31 2 2 THRESHOLD 1 1 4 1 1 1 8 1 OTHER 3 1FLOW STATE TRANSITION 6 2 8 2 30 2 DECISION TREE 1 4 1 20 5 14 2 OTHER 3 5AREA 10 10 1 BAR CHART 2 2 4 2 SCATTER PLOT 1 16 2 1/POSITION OTHER 3 2 1 2 1OTHER FOREST PLOT 25 7 1 2 OTHER 5 4 4 1 ALL CALCULATIONS FOR CIRCLE SIZES ARE AREA-BASED. SO, A CIRCLE REPRESENTING 50 REPORTS HAS A DIAMETER OF 10mm, AND AN AREA OF 7.9mm2. A CIRCLE REPRESENTING 25 REPORTS 10 20 30 40 50 HAS AN AREA OF 3.9mm2 AND A DIAMETER OF 7.1mm.
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • NICE technical advisors telephone interviews - needs assessment • 5 interviews • ~30 minutes • gist transcribed • framework analysis
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, “It’s difficult 1989) - because it depends on the - individual appraisal” selective focussing needed (Thomas, 2005) Some things were thought to lead to complex data included: - Multiple outcome measures - Many subgroups - Sequential treatments - Mixed treatment comparisons - Many variables in SA - Many disease states in model
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005) “In terms of [presenting a large] quantity of information, then the problem is usually with the clinical effectiveness, and summarising that.” Data must be split over several pages, or slides. Also sensitivity analysis of models mentioned.
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005) The most commonly mentioned type of data that needed to be compared to another was the ICER, the overall measure of the cost-effectiveness of an intervention. One interviewee noticed that it was frequently necessary in committee meetings to “flip backwards and forwards” between slides when questions were asked about the certainty of evidence.
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - When asked comparison needed (Spence 2007) if time was limited (in any part - time limited (Resnikoff, 1989) appraisal process), interviewees of the - responded: selective focussing needed (Thomas, 2005) “Time is always limited” or “Yes, is the short answer!” All five interviewees stated that time was always limited for decision-makers to familiarise themselves with the necessary information before an appraisal committee.
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005)
  • • Looking for instances of: - complexity (Remus, 1984; 1987) - summary/overview needed (Tufte 2001) - comparison needed (Spence 2007) - time limited (Resnikoff, 1989) - selective focussing needed (Thomas, 2005) Most interviewees seemed very uncomfortable about the idea. “it’s really not fair, on- it’s not right to give some of them extra detail.”
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • 1. Small multiple techniques including10 information graphics Sankey diagrams for overview of studies in a systematic review The Friday Information Graphic 2. Two-way sensitivity analysis matrix / bubble chart 16th Oct 2009 Link diagrams for showing 3. Parallel coordinates for probabilistic connections between search sensitivity analysis strategies in multiple systematic reviews 4. Technology assessment report graphical overview 5. Sankey Markov overview 6. ‘Whirlpool’ display for enhancing tornado diagram in deterministic sensitivity analysis 7. Survival synthesis bubble chart Peninsula Technology Information Graphics in Assessment Group Health Technology Assessment www.pms.ac.uk/pentag www.pms.ac.uk/infographics Noy Scott House Will Stahl-Timmins Barrack Road Exeter EX2 5DW wstahl-timmins@pms.ac.uk +44 (0) 1392 406 967 1 8. Distribution-based forest plot 9. Search strategy link diagram 10. Individual patient display for discrete event simulation
  • Evaluation?
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preferencenumerical presentation graphical presentation
  • measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preferencehttp://www.pms.ac.uk/ infographics/
  • measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preferencehttp://www.pms.ac.uk/ infographics/
  • measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preferenceseed 1seed 2 25+ participants...seed 3
  • measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference increasing complexitysingle group males and females low, medium, high risk males and females
  • measurements test (experiment) online decision gradually increasing task study decision complexity (subgroups) sample: general internet-using measurements: public - respondent decision accuracy, -driven sample time & preference
  • measurements test (findings)
  • measurements test (findings)Study Duration:36 days (15th June - 21st July 2009)
  • measurements test (findings)Study Duration:36 days (15th June - 21st July 2009)244 entries were recorded duringthis time.
  • measurements test (findings)Study Duration:36 days (15th June - 21st July 2009)244 entries were recorded duringthis time.48 excluded as possible duplicates,leaving 196 for the analysis
  • measurements test (findings)Study Duration:36 days (15th June - 21st July 2009)244 entries were recorded duringthis time.48 excluded as possible duplicates,leaving 196 for the analysis99 participants received thegraphical presentation first.97 received the numerical one first.
  • Randomised to receive: Numerical first (N=97) Graphical first (N=99) did not did not complete completetask 1 (N=19) task 1 (N=22) Task 1did not complete did not complete task 2 (N=7) Task 2 task 2 (N=7)did not complete did not complete task 3 (N=3) Task 3 task 3 (N=2)did not complete did not completedetails collection (N=6) Details details collection (N=2) collectiondid not complete did not complete task 4 (N=7) Task 4 task 4 (N=2) did not complete Task 5 task 5 (N=2)did not complete did not complete task 6 (N=2) Task 6 task 6 (N=2) did not give preference (N=2) Preference did not give preference (N=5) collection 38 people gave a 25 people liked 43 people gave a preference for the the displays preference for the numerical display equally graphical display
  • Average Deaths (mean) Numerical - Graphical group 7787 7787 7787 Graphical - Numerical groupMax. Possible Deaths 7224 7224 7224 95% confidence 5920 mean (g-n group) mean (n-g group) 5466 95% confidenceMin. Possible Deaths 5074 5114 5015 4748 Y1 Y2 Y3 Y4 Y5 Y6 N=78 N=71 N=69 N=65 N=63 N=61 N=77 N=70 N=67 N=54 N=54 N=52
  • Average Times (mean) Numerical - Graphical Graphical - Numerical200 seconds 95% confidence mean (g-n group) mean (n-g group) 95% confidence 0 seconds Y1 N=78 Y2 N=71 Y3 N=69 Y4 N=65 Y5 N=63 Y6 N=61 N=77 N=70 N=67 N=54 N=54 N=52
  • 20mins 0secs num. task graph. task num. task graph. task num. task graph. task Numerical Undecided Graphical Preference Preference
  • rs = .484p < 0.05
  • Average Times (mean) Numerical - Graphical Graphical - Numerical200 seconds 95% confidence mean (g-n group) mean (n-g group) 95% confidence 0 seconds Y1 N=78 Y2 N=71 Y3 N=69 Y4 N=65 Y5 N=63 Y6 N=61 N=77 N=70 N=67 N=54 N=54 N=52 numerical presentation graphical presentation
  • Average Times (mean) Numerical - Graphical “in this case, i think the Graphical - Numerical200 seconds graphical plus the numerical makes for more confustion. one or the other is sufficient” 95% confidence mean (g-n group) “[the graphical presentation] mean (n-g group) seemed more confusing - 95% confidence too manty differnt elements to look at - visual noise” 0 seconds “I found the screen Y1 Y2 Y3 Y4 Y5 Y6 cluttered.” N=78 N=71 N=69 N=65 N=63 N=61 N=77 N=70 N=67 N=54 N=54 N=52 numerical presentation graphical presentation
  • Average Times (mean) Numerical - Graphical “in this case, i think the Graphical - Numerical200 seconds graphical plus the numerical makes for more confustion. one or the other is sufficient” 95% confidence mean (g-n group) “[the graphical presentation] mean (n-g group) seemed more confusing - 95% confidence too manty differnt elements to look at - visual noise” 0 seconds “I found the screen Y1 Y2 Y3 Y4 Y5 Y6 cluttered.” N=78 N=71 N=69 N=65 N=63 N=61 N=77 N=70 N=67 N=54 N=54 N=52 numerical presentation graphical presentation
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • COGS (Clinical effectivenessOverview Graphical Summary
  • COGS testTask-based cognitive interviewingSpeak-aloud protocol9 expert users (HTA systematicreviewers)Randomised, sequencial comparisonto reportQuantitative results (time and accuracy)Qualitative results (actions and wordsof participants - framework analysis)
  • Randomised, crossover design report TASK TASK TASK TASK TASK TASK TASK TASK given first 1 2 3 4 5 6 7 8 TASK TASK TASK TASKgraphic TASK TASK TASK TASK TASK TASK TASK TASK 9 10 11 12 given first 1 2 3 4 5 6 7 8 12 tasks
  • Task 4: Can you tell me about selection bias in the Peters et al. (2007) trial please? COGS display report section 1 4 5 8 9 2 3 6 7 6.9% 9.3% 12.0% 12.3% 13.5% 16.4% 17.0% 17.9%24.1%
  • Task 8: Of the unilateral cochlear implants vs non-technological support trials, which reported at least one significant outcome measure, and which measures were these? report section COGS display 1 4 5 8 9 2 3 6 7 5.4% 5.1% 6.3% 7.9% 13.3%15.0% 15.1% 18.2% 32.4%
  • +&,-.*,/0.*12&!"#$%#&$&(#&)*#&+,-. /00$0#10)2#34"#56 345 675 two-sample t(69) = 4.4 p < 0.001 645 75 45 %&()* !"#$
  • task accuracy COGS: 74.3% report: 46.4%c2 (1, N = 63) = 5.12, p = 0.024
  • Those given COGS firsttook a mean of 99.5%of their COGS task timewith the report.Those given the reportfirst took a mean of268.5% of their COGStask time with thereport.two-sample t(7) = 4.0, p = 0.005
  • Interview 8:Those given COGS first “I would say I got much more of an overview, just from looking at thattook a mean of 99.5% graphical summary”of their COGS task timewith the report. Interview 3: (pointing to graphic) “This really helps to have all of thoseThose given the report elements brought together, so you can get a more holistic view offirst took a mean of where is it from and how big is it,268.5% of their COGS what’s the study design.”task time with thereport. Interview 1: “I speculate that I would have had a much, much less detailed idea of thetwo-sample t(7) = 4.0, p = 0.005 quality of the evidence if I’d been confronted with that [the report] first.”
  • stated preference for COGSkey stated preference for report did not state preference during taskusingreport display 1 using COGS familiar- isation 1 task 1 task 2 task 3 task 4 general reliability 1 display 2 familiar- isation 2 task 5 task 6 task 7 task 8 general reliability 2 display 3 familiar- isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
  • stated preference for COGSkey stated preference for report did not state preference during taskusingreport display 1 using COGS Interview 7: familiar- isation 1 task 1 “I do think [the overall quality of the task 2 evidence is] easier to see with this, task 3 actually. It’s a good way of task 4 general presenting it.” reliability 1 display 2 Interview 2: familiar- isation 2 task 5 task 6 “again, I’m going to use the task 7 graphical summary because it’s far task 8 general reliability 2 more useful [for this task], I think.” display 3 familiar- isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
  • stated preference for COGSkey stated preference for report did not state preference during taskusingreport display 1 using COGS familiar- isation 1 task 1 task 2 task 3 task 4 general reliability 1 display 2 familiar- isation 2 task 5 task 6 task 7 task 8 general reliability 2 display 3 familiar- isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
  • stated preference for COGSkey stated preference for report did not state preference during taskusingreport display 1 using COGS Preferred elements (N): familiar- isation 1 task 1 The outcomes display (2) task 2 task 3 The quality grid (2) task 4 general Follow-up display (1) reliability 1 display 2 Being able to compare familiar- characteristics, quality and isation 2 task 5 outcomes together (1) task 6 task 7 Being able to compare task 8 characteristics between studies (1) general reliability 2 The study design symbols (3) display 3 familiar- Age display (1) isation 3 task 9 task 10 task 11 task 12 probe general questions useful for this review? useful for other reviews? validate tasks interactive version
  • COGS test - conclusions Search time reduced - however, there was less information available overall in COGS. Gives overview Failed to present study designs successfully - revisions to key needed Different intervention areas will need different data
  • Design/size arrowsHeight of arrow is proportional toN (number of people tested) larger study smaller study pre/post design (same cohort is measured before and a er intervention). pre-intervention N = 29 post-intervention N = 20 pre-intervention N=7 post-intervention N=2 cross-sectional / Intervention Intervention non-randomised N = 29 N=7 cohort design Control N = 20 Control N = 2 Intervention N = 29 Intervention N = 7 randomised design N=9 N = 49 Control N = 20 Control N = 2 Intervention N = 29 retrospective non-randomised Intervention N = 7 cohort study design Control N = 20 Control N = 2 survey design N = 49 N=9
  • length of follow-up 0 yrs 5 10 10 cross-sectional Intervention N = 29design (no follow-up) Control N = 20 Intervention N = 29 5 year follow-up Control N = 20 Intervention 12 year follow-up N = 29 Control N = 20
  • follow-up outcome measures 0yr 5 10 15 GASP CUNY CAP CDT CID CNC CPT FMWT GSL ESP Intervention N = 21N = 43 Control N = 22
  • outcome measures used no. of design, size baseline study cog func be glo author location centres & follow-up MMSE sex ages quality ADCS-ADL ADCS-CGIC ADAS-cog other other other CIBIC MMSE DAD PDS CDR QoL SIB NPI GDS 0yr 1 2 0 10 20 30 55 75 95 Donepezil 1mg N = 42 M F Rogers & Rand Donepezil 3mg N = 40 M F ? N = 161 Donepezil 5mg N = 39 M F Char Blind Analy 1mg 3mg 1996 F Placebo N = 40 M 5mg Donepezil 5mg N = 154 M F 5mgRogers et al. Rand M F Char 10 Donepezil 10mg N = 157 Blind mg 1998 (A) N = 473 Analy M F Placebo N = 162 Donepezil 5mg N = 157 M F 5mgRogers et al. Rand M F Char 10 Donepezil 10mg N = 158 mg Blind 1998 (B) N = 468 Analy M F Placebo N = 153 Donepezil 5mg N = 271 M F 5mg Burns et al. Rand M F Char 10mg Donepezil 10mg N = 273 Blind 1999 Analy N = 818 M F Placebo N = 274 Greenberg Donepezil 5mg (D) Rand et al. group 1 (p-D-p-p) N=30 M F Char group 2 (p-p-D-p) N=30 M F Blind N = 60 Analy 2000 Placebo (p) Donepezil 5mg N = 134Homma et al. M F Rand Char Blind 2000 N = 268 M F Analy Placebo N = 129 Donepezil 10mg N = 214 Mohs et al. M F Rand Char ADCS-CGIC ADCS-ADL Blind ADAS-cog 2001 N = 431 M F Analy MMSE CIBIC Placebo N = 217 DAD other other other CDR GDS PDS QoL NPI SIB 0yr 1 2 0 10 20 30 55 75 95 cog func be glo
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • State Occupancy Charts (SOCs) temozolomide vs placebo for the treatment of newly diagnosed high-grade glioma1 — State Occupancy Chart2 — State Occupancy & Absolute Quality of Life3 — State Occupancy & Absolute Costs Per Person4 — Incremental State Occupancy5 — Incremental QALYs6 — Incremental Costs
  • State Occupancy Chart placebo arm treatment arm 1 state occupancy state occupancy week 1 surgery 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows the numberof simulated people in the six week weekdi erent states of the model, 39 39over the 260 one-week cycles week weekof the model. 52 52 week weekDuring the rst week of treat- 65 65ment, all of the people week weekin the model were assumed 78 78to undergo surgery, whichis represented with a seperate week weekstate in the model. 91 91 week weekFrom weeks 2-6, patients can 104 104either be in a post-operationrecovery (treatment-free) state, week week 117 117or move to death in anyof these ve weeks. week week 130 130In weeks 7-12, patients will week weekundergo radiotherapy, have 143 143progressive disease or be dead. week week 156 156From week 13 onwards, themodel becomes a fairly typical week weekthree-state model, with patients 169 169either in a stable state, having week weekprogressive disease, 182 182or dead. week week 195 195 – surgery (week 1) week week 208 208 – post-op recovery (weeks 2-6) week week – radiotherapy 221 221 (weeks 7-12) – stable disease week week (week 13+) 234 234 – progressive week week disease 247 247 – death week week 260 260
  • State Occupancy & Absolute Quality of Life 2 placebo arm treatment arm week 1 surgery 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is information graphic showsthe absolute quality of life week weekexperienced by the simulated 39 39patients in the model. e week weekshades of grey provide a scale 52 52from black (a utility of 1 per week weekperson) to white (a utility of 0 65 65per person). week week 78 78 ese shades of grey arepresented in bars whose length week weekcorrespond to the number 91 91of people in that state in the week weekmodel during that week, as in 104 104graphic 1 — State Occupancy. week week 117 117 e slowly lightening e ect inthe progressive state is caused week weekby the gradual decomposition 130 130of utility values in this state in week weekthe model. e simulated 143 143patients experience less quality week weekof life the longer they spend in 156 156this state. e values presentedhere are the average (mean) of week weekthe utility scores experienced 169 169by the cohort in that week of week weekthe model. 182 182 week week 195 195 utility of 1 week week (per person) 208 208 week week 221 221 utility of 0.5 week week 234 234 week week 247 247 utility of 0 week week 260 260
  • State Occupancy & Absolute Costs Per Person 3 placebo arm treatment arm 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is information graphic showsthe absolute costs incurred per week weekperson in the model, on a scale 39 39from black (£3000 per person) week weekto white (£0 per person). 52 52 week week ese shades of grey are 65 65presented in bars whose lengthcorrespond to the number week weekof people incurring that cost 78 78in that state in the model, as in week weekgraphic 1 — State Occupancy. 91 91 e small dark bars appearing week week 104 104between the progressive anddeath states represent the week weekone-o costs assigned to death 117 117in the model. e length of week weekthese bars is again proportional 130 130to the number of people dyingduring that week of the model. week week 143 143Similar dark boxes appear week weekbetween the stable and 156 156progressive states to indicatethe higher costs assigned week week 169 169to a patient’s rst week in theprogressive state. week week 182 182Costs for surgery in week week week1 are o the scale at £5953 per 195 195person, but this cost is identical week weekin both arms of the model. 208 208 week week 221 221 £3000 per person week week 234 234 £1500 per person week week 247 247 £0 per person week week 260 260
  • Incremental State Occupancy 4 stable progressive death week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 e di erence in stateoccupancy between the two week weekarms of the model are shown 39 39here. week week 52 52A bar extending to the week weekle shows that there were more 65 65people in that state during that week weekweek in the placebo arm than 78 78in the temozolomide arm.Extending to the right indicates week weekmore people in the temozolo- 91 91mide arm. week week 104 104 e length of the bars, indicat-ing the incremental di erence week week 117 117between the state occupancy ofthe two arms, are proprtional week weekto graphic 1: State Occupancy. 130 130A 10% shi is indicated by a week weekthin vertical white line. 143 143 week week 156 156 week week 169 169 week week 182 182 week week 195 195 week week 208 208 week week 221 221 5% shi week week 234 234 10% shi 15% shi week week 247 247 20% shi week week 260 260
  • Incremental QALYs 5 stable progressive total week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows thedi erences between the two week weekarms of the model in terms of 39 39the quality adjusted life years week week(QALYs) that would be 52 52experienced by a simulated week weekcohort of 1000 people. 65 65 week weekA bar extending to the le 78 78shows that, during that week,more QALYs were experienced week weekby the people in the placebo 91 91arm than the temololomide week weekarm. A bar extending to the 104 104right represents more QALYsexperienced in the temozolo- week week 117 117mide arm. week week in vertical white lines 130 130show the number of QALYs week weekthat would be experienced 143 143in a cohort of 1000 simulated week weekpatients. A bar that reaches one 156 156line represents one QALY. week week e death state is not shown, 169 169as no QALYs are experienced week weekin that state, as it was assigned 182 182a utility of 0. week week 195 195 e “total” column on the farright shows a sum of the values week week 208 208from the other two states. week week 221 221 incremental QALYs week week 234 234 1 2 week week 247 247 3 week week 260 260
  • Incremental Costs 6 stable progressive death total week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows thedi erences in costs between week weekthe two arms of the model, 39 39again with a cohort of 1000 week weeksimulated patients. 52 52 week weekAs before, a bar extending 65 65to the le shows higher costs week weekin the placebo arm, and 78 78a bar extending to the rightshows higher costs in week weekthe temozolomide arm. 91 91 week week e thin vertical white lines 104 104show cost thresholds inincrements of £100,000 week week 117 117 e “total” column on the week weekfar right shows a sum of the 130 130values from all three states week weekof the model. 143 143 week week 156 156 week week 169 169 week week 182 182 week week 195 195 week week 208 208 incremental costs week week £100,000 221 221 £200,000 week week £300,000 234 234 £400,000 week week £500,000 247 247 £600,000 week week 260 260
  • SOC testTask-based cognitive interviewingProbing protocol (Interviewer-led)6 expert users (HTA modellers)Stand-alone evaluation (no comparator)Tasks used to assess understandingQualitative results (participants askedfor opinions - framework analysis)
  • Task 1Q: How many peoplehave progressivedisease in week 52 ofthe temozolomide armof the model?A: 32.7%
  • Task 1
  • Task 2Q: Where do the costs tend to come from ineach arm of the model?
  • Task 2
  • Task 6Q: Where does the greatest difference betweenthe costs of the two arms lie?
  • Task 6
  • SOC test - conclusions Participants largely understood meaning of displays Main function is to give overview, adding a temporal display to existing methods Considered useful by participants Could be used to display SA? Applicability to other models - with many more states?
  • placebo arm 0% 25% 50% 75% 100% £5.9m week 1 surgeryweek 26week 39 £1.6mweek 52 weeks 7-12 radiotherapyweek 65week 78week £6.7m 91week104 weeks 2+ progressive diseaseweek117week £3.1m130week143 weeks 2+ deathweek156week169week182week195week chemotherapy drugs208week221 radiotherapy £3000 per personweek234 hospital inpatient £1500 per personweek247 hospital outpatientweek £0 per person260
  • Total Incremental Costs and QALYs total incremental total incremental 7 QALYs costs week 1 surgery week 1 week 2 weeks 2-6 post-op recovery week 7 weeks 7-12 radiotherapy week week 13+ stable/progressive/death 13 week week 26 26 is graphic shows how week weekadopting temozolomide 39 39treatment would a ect week weekquality of life and costs 52 52incurred over time. week week 65 65 e grey bars show the change week weekin costs and QALYs each week, 78 78and the solid black lines showthe cumulative e ects of week week 91 91adopting the treatment. week week 104 104It should be noted that theweekly values are presented week weekon a di erent scale to the 117 117cumulative values. is week weekis necessary for the two 130 130to be compared overlaid week weekin this manner. 143 143 week week 156 156 week week 169 169 week week 182 182 week week 195 195 week week 208 208 week week incremental 221 221 costs / QALYs week week 234 234 cumulative incremental week week costs / QALYs 247 247 week week 260 260 total QALY gain: total cost: 223.1 £8.1m ICER = £36,171 per QALY
  • Overall Conclusions
  • TARreview NICE interviews Methods study Design & critique COGS SOC test test
  • Research QuestionHow should information graphics be designed, produced and used in health technology assessment?
  • Design
  • DesignRESEARCH OUTPUTS Research Area ECEHH
  • Production1. Using standard visualisation tools in spreadsheet software (current situation – suitable for HTA professionals)2. Developing new specialist software for use by HTA professionals (such as currently used for Forest plots)3. Designing graphics on an individual basis (ie. in collaboration with trained information design professionals)
  • Use Likely that a combination of all three production methods will continue Depends on:- complexity of information to be presented- available resources- available skills- which (or whether) specialist software tools are developed
  • Production1. Using standard visualisation tools in spreadsheet software (current situation – suitable for HTA professionals)Suitable for simpler reports: - small number of trials in review - few subgroups, sequencial treatments or other complicating factors - simple treatment pathway for model
  • Production2. Developing new specialist software for use by HTA professionals (such as currently used for Forest plots)COGS software would be suitable forgiving overview of more complexsystematic reviewsSOC suitable for models in which time is akey considerationLikely to be other graphics - these wouldfurther testing and evaluation
  • Production3. Designing graphics on an individual basis (ie. in collaboration with trained information design professionals)Suitable for the most complex reviews andmodels, where: - different media become useable / dominant - particular information needs highlighting (area of world, timing of trials, etc)
  • Kessler Illg et al. Nikolopoulos MED-EL Staller Manrique Nikolopoulos Harrison et al. et al. et al. et al. et al. et al. 1997 1999 1999 2001 2002 2004 2004 2005 auth date ages 0 5 10 15 200 5 10 15 20 N = 49 N = 82 N = 78 N = 82 N = 82 N = 167 N = 126 N = 182 0yr 5 10 150yr 5 10 15 Pr Pr Pr Pr Pr Pr Pr Pr Se Se Se Se Se Se Se Se As As As As As As As As At At At At At At At At Po Po Po Po Po Po Po Po design/size follow-up quality Ot Ot Ot Ot Ot Ot Ot Ot 1ST 1ST 2ST 2 ST AVGN AVGN AB AB BKB BKB
  • 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Harrison et al.Nikolopolous et al. Manrique et al. Staller et al. MED-ELNikolopolous et al. Illg et al. Kessler et al.
  • Future researchDeveloped graphics- Use and monitoringEvaluation of new graphics- Other graphics designed for PhD- Different designers’ work- Different media?Different audiences- Public- Medical professionals
  • Current work
  • European Centre for Environment and Human Health
  • UK CARBON EMISSIONS IN 2009, THE UK’S DEPARTMENT FOR ENERGY AND CLIMATE CHANGE2009 CALCULATED THAT WE EMMITTED 564 TONNES OF CO2 - CARBON DIOXIDE. HERE’S HOW THAT 8t BREAKS DOWN INTO DIFFERENT 195t SECTORS. 10t ENERGY 18t TRANSPORT BUSINESS RESIDENTIAL 123t 50t AGRICULTURE WASTE 86t INDUSTRIAL 79t PUBLIC SECTOR
  • ATMOSPHERIC COLUMN ATMOSPHERIC SERVICES SATELLITES SERVICES AT ALL THREE ALTITUDES LOW EARTH ORBIT 160—2000 km PLASMA AND METEORS SOUNDING ROCKETS 50—1500 km DISPERSION OF AIR POLLUTION STRATOSPHERE 10—50 km PROPERTIES WEATHER BALLOONS 0—40 km UPPER AND LOWER TROPOSPHERE UPPER TROPOSPHERE AIRCRAFT 1—10 km CRUISING 6—12 kmLOWER TROPOSPHERE POWER STATIONS 0—1 km 80—350 m COLUMN BASE: WIND TURBINES 80—130 m 1 km2