The document discusses evaluating the cost-effectiveness of diagnostic tests through modeling. It provides an overview of how cost-effectiveness analyses are applied to determine if a diagnostic test represents value for money. The modeling requires estimating test accuracy, modeling patient outcomes for different test results, and calculating an incremental cost-effectiveness ratio to compare the new test to current practice. It provides an example of modeling different diagnostic strategies for deep vein thrombosis.

1. School of Health and Related Research (ScHARR) The University of Sheffield Regent Court 30 Regent Street Sheffield, S1 4DA Website: www.sheffield.ac.uk/scharr

2. Health Economics and Decision Science (HEDS) The purpose of HEDS section within ScHARR is to promote excellence in national and international healthcare resource allocation decisions, through applied and theoretical research funded by the public and private sector; and supporting the effective implementation of the results of such research through education, training and management interventions 27/10/11 © The University of Sheffield

3. Portfolio of specialities of HEDS 27/10/11 © The University of Sheffield

4. RESEARCH

5. Information resources Bayesian Statistics in Health Economics Evidence review and synthesis Health Economics and Outcomes Research Cost-effectiveness modelling to support Healthcare Decision Making Main Focus of Research Areas

6. 27/10/11 © The University of Sheffield Consultancy within HEDS Modelling alongside clinical trials Health related quality of life Clinical Trial Simulation Health Economics Modelling Evidence Synthesis Costing studies Systematic reviewing

7. Disease Areas Acute coronary syndrome Age-related macular degeneration Alcohol Alzheimer’s Ankylosing spondylitis Asthma Back pain Breast cancer Cardiovascular conditions Colon cancer Colorectal cancer Dementia Depression Dyskaryosis Epilepsy Flushing Foot ulcers Immune thrombocytopenic purpura Incontinence Influenza Irritable bowel syndrome Leg ulcers Lymph node metastases Menopause Mental health problems Multiple sclerosis Overactive bladder Prostate cancer Psoriatic arthritis Pulmonary hypertension Renal disease Rheumatoid arthritis Schizophrenia Sexual health Surgical procedures in sex reassignment Type 1 and Type 2 diabetes Vaccination scheduling Venous ulcers

8. Innovation & Knowledge Transfer Contact details Further information about consultancy projects and research areas is available at: http://www.sheffield.ac.uk/scharr/consultancy For an informal chat, contact the HEDS programme director, Roberta Ara: Email: [email_address]

9. Cost-effectiveness modelling to support Healthcare Decision Making 27/10/11 © The University of Sheffield

10. The Decision Analytic Modelling team is a leading healthcare modelling group internationally, with a wide ranging, high impact programme of methodological and applied research. Research focuses on Operational Research modelling including both health technology assessment modelling and supporting service organisation and delivery.

11. In particular: Cost-effectiveness modelling to support HTA submissions Budget impact modelling Systematic reviews & critique of cost-effectiveness literature Critique / validate existing model Country adaptations Model adaptations

12. Examples of cost-effectiveness studies conducted in HEDS 27/10/11 © The University of Sheffield

13. Evaluating the cost-effectiveness of diagnostic tests Dr Matt Stevenson Reader in Health Technology Assessment; NICE Appraisal Committee Member; Contributor to the NICE Diagnostic Assessment Programme manual

14. Overview A brief description of how (and why) cost-effectiveness analyses are applied A brief overview of the analyses required to model diagnostic tests in sepsis A very brief description of the reasons why diagnostic evaluations are more difficult than pharmaceutical evaluations

15. Cost-effectiveness analyses In the last 10 years there has a been a considerable increase in the importance of cost-effectiveness analyses. This was due to the relatively fixed budget and a combination of ageing populations and emerging expensive interventions. This has led to the formation of funding agencies in England and Wales, Scotland, Australia and Canada.

16. Does a diagnostic test represent value for money? Diagnostic tests with high prices may be cost-effective (i.e. a worthwhile use of a limited budget). Conversely, diagnostic tests with low prices may not be cost-effective. The ‘gold standard’ approach for determining whether the price of a diagnostic test is justified is through an economic evaluation, or cost-effectiveness analysis.

17. Cost-effectiveness analyses The goal of funding agencies is to provide the greatest amount of health for society within the budget, and thus opportunity cost is a key principle. That is, what health would be lost if money was diverted from one intervention in order to fund another. The process is typically to estimate the cost-effectiveness of an intervention through modelling, and comparing this result with a value assumed to represent opportunity cost.

18. Previous Diagnostic evaluations Whilst the majority of evaluations undertaken relate to pharmaceuticals, evaluations of diagnostic tests have been conducted: Carotid Artery Imaging - Wardlow et al. HTA Thrombophilia Testing - Simpson et al. HTA BMD Scanning (Strontium Ranelate) – Stevenson et al HTA Diagnostic strategies for DVT – Goodacre et al HTA Diagnostic pathways for minor head injuries – Pandor et al. HTA Non-Invasive Liver Testing – Stevenson et al HTA (in press)

19. Methods for evaluating diagnostic tests There have been, for some time, clear methods guide for undertaking evaluation of pharmaceutical interventions. Recently NICE has set up a Diagnostic Assessment Programme which has issues an interim statement of the methods it expects to be followed in evaluating diagnostics. http://www.nice.org.uk/media/164/3C/DAPInterimMethodsStatementProgramme.pdf

20. Simplified Overview of the modelling required The following slides discuss the steps that would be required to generate an estimate of the cost-effectiveness of a diagnostic test (or series of diagnostic tests). The overview is a simplification. More detailed discussion is provided in the previously listed HTA reports (all free to download) and the Diagnostic Methods statement

21. Estimating Test Accuracy The sensitivity and specificity of a diagnostic test must be estimated. These values would be combined with the estimated prevalence of the condition being tested for, to form an expectation of the number of true positives, true negatives, false positives and false negatives generated by the diagnostic test.

22. Modelling the patient experience For each of the four groups defined, an estimation of the events that would occur to the patient must be modelled. These may differ due to underlying risks and the chosen medical management. The modelling would include factors such as the risks of mortality, risk of morbidity, length of stay within hospital, costs for initial and subsequent care, treatment-related adverse-events and the quality of life for patients in each potential health state.

23. Modelling the patient experience Ultimately, an estimation of the life years, quality adjusted life years (QALYs*) and costs can be attributed to each of the four groups. These can be weighted by the proportions in each group to form a total cost and total QALY for patients post diagnosis. The costs of the diagnostic tests performed are then added. * The QALY is a combination of life years and patient utility. A person living for 10 years at a utility of 0.5 would gain 5 QALYs; a person living for 4 years at a utility of 0.75 would gain 3 QALYs

24. Calculating an ICER* Assume that post diagnosis, an average patient was expected to gain 10 QALYs at a cost of £20,000 under current best practice. These values became 11 QALYs at a cost of £18,000 following a new diagnostic test, which costs £4,000 per patient. In this instance the increase in cost is £2,000 (£18,000 - £20,000 + £4,000) The increase in QALYs is 1. (11 – 10) * An Incremental Cost Effectiveness Ratio.

25. Calculating an ICER In this example, the ICER would be £2,000 per QALY gained (£2,000 / 1) This would be compared with an estimation of the cost of gaining a QALY in interventions that are likely to be replaced.* Thus if this were the result from a real technology appraisal the diagnostic test would be likely to be recommended for use. * NICE has estimated this to be in the region of £20,000-£30,000

26. Implications for diagnostic pricing Where a new diagnostic test has a large impact on mortality or on the utility of a patient, then the QALY gained over the current diagnostic will be greater. ICER = Δ Cost / Δ QALY Thus, for a constant ICER, such a test would be able to command a higher price than a test with a smaller QALY gain.

27. Sequences and subgroups Note that sequences of tests and only incorporating tests on a subgroup of the population are possible. The following slide shows the predicted optimal strategy for diagnosing whether a patient has deep vein thrombosis. The costs of diagnostic tests, the risks of death, morbidity, recurrence, treatment-related adverse-events and the costs of treating future events were all considered in the model.

28. Example of diagnostic algorithm Taken from Goodacre et al. QJM 2006; 99:377–388

29. Returning to the example of sepsis Current gold standard is blood culture, but this has poor sensitivity. A new test is available (SeptiFast © ) which has higher accuracy than blood culture tests, but is relatively expensive. The price may preclude use in all patients with suspected sepsis. A decision support system is also available, (Treat © ) which can categorise patients into high, medium and low risk. This may allow SeptiFast © to be used more efficiently.

30. Estimating the cost-effectiveness of a Treat © and SeptiFast © diagnostic strategy For each Treat © category, the patient experience must be modelled taking into account true positives, false positives, true negatives and false negatives. This will incorporate (amongst others) Risks of mortality Risks of post-infection sequalae Length of stay in hospital These variables are expected to be lower where there is appropriate management of a patient with sepsis.

31. Estimating the cost-effectiveness of a Treat © and SeptiFast © diagnostic strategy Thus there will be a QALY gain (and cost reduction) associated with the new diagnostic tests. It is expected that these will be greatest in those patients denoted high risk. Factoring in the costs of the diagnostic (Treat © for all patients, SeptiFast © for the groups being evalauted) will allow the cost-effectiveness of diagnostic strategies to be evaluated.

32. Additional complications with evaluating diagnostic tests There are reasons why evaluating diagnostic tests are more difficult than evaluating pharmaceuticals. Due to time restrictions these will be mentioned very briefly under broad headings.

33. Complicating Issues The need to understand the patient pathway Data reporting Missing and unobtainable data Meta-analyses Correlation between tests Imperfect gold standards Required operator skill Spectrum bias Incidental findings Estimating the costs of diagnostic tests

34. NICE Decision Support Unit Dr Paul Tappenden Senior Research Fellow Acting DSU Director http://www.nicedsu.org.uk / 27/10/11 © The University of Sheffield

35. Background to DSU Established in 2002. Formal collaboration between the Universities of Sheffield (lead), York and Leicester. Peripheral members at LSHTM, Brunel and Bristol. May also involve highly specialist expertise from elsewhere. Expertise covers all key areas of HTA. Commissioned by NICE to provide a research and training resource to support the Institute's Technology Appraisal Programme.

36. Main strands of work Appraisal-specific work Rapid evaluation of technical issues/analysis around health economic evaluation/modelling problems within specific technology appraisals An independent yet highly qualified voice Methods development Ongoing role in developing Methods guidance for NICE DSU Technical Support Documents Training for industry and NICE Appraisal Committee members Other methods work

37. Appraisal-specific work ~7 appraisals per year Analysis of additional evidence submitted as part of the appraisal process Cetuximab for 1 st line metastatic colorectal cancer Lapatinib for metastatic breast cancer Provision of modelling expertise unavailable elsewhere (software) Donepezil for Alzheimer’s Disease Evaluating Patient access schemes (PAS) Tocilizumab for Rheumatoid Arthritis

38. The NICE Methods Guide for Technology Appraisal Setting the agenda for, organising and facilitating the NICE methods guide for technology appraisal Recent work includes managing the update to the 2008 NICE methods guide – 6 workshops covering issues around evidence synthesis, utilities, uncertainty analysis, subgroups etc. Update due in 2011/2012 Conduit for leading academic centres in HTA

39. DSU Technical Support Documents (TSDs) Series of independent, non-prescriptive reports commissioned to support NICE Methods Guide 14 TSDs completed or near completion Main areas: Health utilities (x5) Evidence synthesis (x7) Model development and reviewing evidence for informing model parameters Survival modelling using individual patient data Use of regression methods in HTA models

40. Other DSU methods research Report and related publications demonstrating the feasibility of applying Value of Information methods. Feasibility projects relating to computerised decision support systems and orphan drugs Equity arguments applied to orphan drugs Discounting of costs and benefits The role of patient valuations of health states The incorporation of uncertainty into cost-effectiveness models The incorporation of equity weights into cost effectiveness models and decision making

41. Methods training Bridge between NICE and the pharmaceutical industry in providing methods training “ Masterclasses” held: Measurement and valuation of health-related quality of life Network meta-analysis & indirect comparisons NICE Appraisal Process, modelling, what makes a good submission, uncertainty analysis

42. Future updates Join our mailing list at : http://www.nicedsu.org.uk

43. Health economic modelling in bowel cancer Dr Paul Tappenden Senior Research Fellow ScHARR, University of Sheffield [email_address] 27/10/11 © The University of Sheffield

44. Health economics and cancer Key area of methodological expertise in ScHARR for >15 years Key areas include breast, bowel, cervical, CML, kidney, prostate, lung Range of decision-makers/clients including: NICE NETSCC / NHS R&D Programme NHS Cancer Screening Programmes Department of Health Local decision-makers

45. Key areas of application Modelling interventions for the early detection / prevention of cancer Modelling interventions for the treatment of diagnosed cancer Modelling whole disease and treatment pathways (Whole Disease Modelling)

46. 1. Modelling interventions for the early detection / prevention of cancer Methodological development in modelling natural history disease progression Handling competing risks Length / lead-time biases Consideration of full trade-off between costs and benefits Calibration of unobservable parameters (disease progression, presentation behaviours etc) Evaluation of screening programmes in breast, bowel, cervical and prostate cancer Evaluation of early awareness campaigns

47. An example – screening for CRC 27/10/11 © The University of Sheffield

48. 2. Modelling interventions for the treatment of diagnosed cancer Curative / palliative treatments for diagnosed cancer Numerous appraisals for NICE - main areas covered include bowel and breast Key issues around methods for handling Extrapolation beyond trial duration Handling treatment crossover Treatment sequences Evidence networks across multiple trials

49. An example – bevacizumab for MCRC 27/10/11 © The University of Sheffield

50. 3. Whole Disease Modelling Usefulness of models is in part determined by the scope of the decision it is intended to inform. Single isolated point versus whole pathway model. “ Modelling the bigger picture” – development of models which can represent whole disease and treatment pathways

51. A lot of effort so why bother? Consistent basis for economic evaluation across the pathway Structurally capable of evaluating any intervention at any point in the pathway Capturing upstream and downstream knock-on impacts Shift to potentially more useful economic decision rules Methodological challenges Obtaining agreement regarding pathways Handling geographical variability Programming / model run time Calibration of unobservable parameters Non-trivial investment of time at outset but payoff may be considerable

52. An example – Colorectal Cancer Whole Disease Model

53. From piecewise CPQ to constrained maximisation 27/10/11 © The University of Sheffield

54. Colorectal cancer screening : using mathematical modelling to inform policy decisions Dr Sophie Whyte Research Fellow ScHARR, University of Sheffield Email: s.whyte@sheffield.ac.uk 27/10/11 © The University of Sheffield

55. Contents Background The current NHS Bowel Cancer Screening Programme The flexible sigmoidoscopy screening trial Should the screening programme be changed? Methods Possible screening strategies to consider The value of modelling in this context Modelling challenges Modelling solutions Results and conclusions Results Policy implications 27/10/11 © The University of Sheffield

56. The current NHS Bowel Cancer Screening Programme Roll out commenced in 2006 and now screening covers the whole of England. Persons aged 60-69 (being increased to age 74) are offered biennial screening. The guaiac faecal occult blood (gFOB) test is used for screening and positives(≈2%) receive follow up with colonoscopy. Uptake is 52% 27/10/11 © The University of Sheffield

57. Flexible sigmoidoscopy(FS) screening trial* Trial reported in 2010 170,432 persons aged 55-64 were randomised to FS screening or to a control group (no screening). 40,674 persons underwent FS screening After 10 years of follow-up the incidence of CRC was reduced by 33% and mortality was reduced by 43% in the FS group. *Atkin et al 2010, Once-only flexible sigmoidoscopy screening in prevention of colorectal cancer: a multicentre randomised controlled trial, The Lancet 27/10/11 © The University of Sheffield

58. Should the screening programme be changed in light of new evidence? Project for NHS cancer screening “Reappraisal of options for CRC screening” Update to “Appraisal of options for CRC screening” completed by Tappenden et al in 2004. The reappraisal uses: Data from existing screening programme, Data from FS trial, Data on newer immunochemical FOBTs 27/10/11 © The University of Sheffield

59. Methods: Possible screening scenarios Biennial screening with guaiac or immunochemical FOBT for ages 60-74 One-off FS screening (for different ages) Combination screening strategies e.g. FS at age 55 then biennial FOBT at ages 60-74 27/10/11 © The University of Sheffield

60. Methods: Advantages of modelling The number of possible screening strategies is very large To demonstrate a significant impact on incidence and mortality rates screening trials require: very large numbers of participants a long follow-up period Hence modelling is very useful for the evaluation of screening strategies 27/10/11 © The University of Sheffield

61. Methods: Modelling challenges Considerable uncertainty surrounding values of some parameters essential for colorectal cancer screening modelling. Uncertainty in underlying prevalence of precancerous conditions: 27/10/11 © The University of Sheffield

62. Methods: Modelling challenges Uncertainty in screening test characteristics Sensitivity to CRC of the Haemoccult gFOBT : range 0.25-0.96* Uncertainty(very little data) on growth rates of colorectal cancer and precancerous conditions as: (1) difficult to observe and (2) cancers are removed if found. 27/10/11 © The University of Sheffield * Burch JA, Soares-Weiser K, St John DJ, et al. Diagnostic accuracy of faecal occult blood tests used in screening for colorectal cancer: a systematic review. J Med Screen 2007;14(3):132-7.

63. CRC natural history model structure 27/10/11 © The University of Sheffield

64. Methods: Modelling solutions 27/10/11 © The University of Sheffield Polyp detection rate at FS screening Polyp prevalence Sensitivity of screening test to polyps

65. Methods: Modelling solutions 27/10/11 © The University of Sheffield Probability of diagnosis of CRC (due to symptoms or chance detection) State allocation diagram CRC incidence in absence of screening

66. Methods: Modelling solutions 27/10/11 © The University of Sheffield Parameter values are estimated by using a Bayesian approach in which sets of parameter values for which the model predictions have a good fit to observed data are generated.*

67. Results and conclusions CRC screening is generally cost-effective: Tappenden et al found FS screening to be cost saving and FOBT screening the have an ICER<£3K. “ Reappraisal Project” results will be available by the end of October. DOH Press Release 2 nd October 2010 “ New Action on Cancer to Save Thousands More Lives Every Year Increase detection through a revolutionary new bowel cancer screening technology flexible sigmoidoscopy – a £60 million investment over the next four years to incorporate the latest breakthrough in bowel cancer screening into our existing national programme – saving 3,000 lives a year.” 27/10/11 © The University of Sheffield

68. Potential impact on policy decisions Results of work will allow us to advise NHS cancer screening on various aspects of FS screening implementation. At what age will a one-off FS screen provide the greatest health benefits? How cost effective are combined screening strategies? Would it be cost effective to create a national polyp centre to process all pathology generated through screening? What level of FS uptake is required to achieve the same health benefits as the current screening programme? 27/10/11 © The University of Sheffield

69. Calibration method The Metropolis-Hastings algorithm explores the multi-dimensional parameter space to generate multiple sets of parameters for which the model predictions have a good fit to observed data. 27/10/11 © The University of Sheffield

70. Cost utility analysis of interventions to return employees to work following long term sickness absence Hazel Squires Research Fellow ScHARR, University of Sheffield Email: h.pilgrim@sheffield.ac.uk 27/10/11 © The University of Sheffield

71. Introduction Background Project process Modelling methodology Results Conclusions Limitations/ further research

72. Background People can apply for Incapacity Benefit after they have been on sick leave for >6 months. Sickness absence in the UK costs the country around 1% of the annual GDP. There is a need to prevent people from going onto Incapacity Benefit.

73. Process Effectiveness literature review Very limited evidence identified – only 3 UK papers Cost-effectiveness literature review No studies identified which consider the implications of RTW within the UK setting over a sufficient time frame to capture all impacts on costs and benefits of the interventions Health economic model

74. Model scope Population: A cohort of employed men & women that have been on between 1 week and 6 months of sick leave due to musculoskeletal disorders. Interventions: Workplace intervention Physical activity and education intervention Physical activity, education and workplace visit C omparator: Usual care for musculoskeletal disorders Outcomes: Cost per quality-adjusted life year (QALY) gained Cost per day on sick leave avoided Perspective: NHS and PSS Societal Employer

75. Modelling methodology (1) Markov model which simulates the experience of a hypothetical cohort of employees who are currently on long term sick leave over a working lifetime. At work On sick leave for 1 week -6 months On sick leave for 6-12 months On sick leave for 12-18 months On sick leave for 18+ months Retirement or death

76. Modelling methodology (2) Costs and QALYs have been calculated for both cohorts starting at age 41 (average age of sickness absence from evidence) through to retirement at age 66. Discount rate of 3.5% for both costs and utilities. One-way sensitivity analysis and threshold analysis carried out.

77. Quality of life Utility scores estimated based on data from the British Household Survey Panel (BHSP) Based on whether the individual is at work or on sick leave Sf-36 SF-6D Transformed using SPSS Impact of potential confounding variables assessed using regression Utility at work Utility on sick leave Age<35 0.83 0.66 Age 35-45 0.8 0.59 Age 45-55 0.76 0.61 Age>55 0.76 0.61

78. Costs State in the model Perspective NHS & PSS Societal Employer At work £0 £0 £0 1 wk to 6 months sick leave Cost of usual care and intervention incurred by NHS NHS & PSS costs + Employer costs - Transfer costs Cost of intervention incurred by employer + Cost of replacing employee + Production loss over friction period + Salary of replacement employee after friction period + Occupational sick pay + Employer’s NI contribution 6-12 months sick leave Cost of usual care Cost of usual care Occupational sick pay + Employer’s NI contribution 12 months+ sick leave Cost of usual care Cost of usual care

79. Key model assumptions (1) If an employee has not returned to work within 6 months given the intervention they are subsequently no more likely to RTW than an employee who is given usual care The intervention is only given the first time that the person goes onto LTS The probability of going onto further episodes of LTS is the same for the intervention cohort and the usual care cohort

80. Key model assumptions (2) The employee is assumed to receive 15 weeks on full pay and 16.4 weeks on half pay Wages and productivity of a replacement worker are assumed to be the same as the employee on LTS The probability of dying is assumed to be no different for people on LTS to people who are at work

81. Results: NHS/ Societal perspective (1) Cost per QALY gained = £2,758

82. Results: NHS/ Societal perspective (2)

83. Results: Employer perspective Cost per day on sick leave avoided = £0.34

84. One way sensitivity analysis If the difference in health utility between being at work and being on LTS is ≤0.02 then cost per QALY >£20,000. If the person is aged 55 when going onto LTS then cost per QALY >£8,000. All other assumptions tested within the model had a limited impact on model results

85. Conclusions The model suggests that interventions to return employees to work do not have to be very effective to be considered cost-effective in comparison to other technologies routinely funded by the NHS. This can be explained by the high costs of productivity loss and Incapacity Benefit in comparison to the relatively low cost of the intervention.

86. Model limitations Effectiveness data based on generally poor evidence carried out in non-UK countries Costs of interventions are uncertain Lack of long term follow up data Only able to consider interventions for musculoskeletal disorders due to limited data on interventions for other sickness absence Does not take into account different employee/ employer characteristics (although some tested in sensitivity analyses)

87. Further research Further research is required that: Is within the UK setting; Provides follow up data beyond 12 months; Reports comparable return to work outcomes between studies; Reports quality of life data of the employees who are both at work and on sick leave.

88. Public health modelling: lessons learned from a contraception case study Hazel Squires, Jim Chilcott, Nick Payne, Lindsay Blank, Monica Hernandez, Louise Guillaume ScHARR, University of Sheffield

89. Introduction Outline methods & results of contraception case study Discuss key issues in public health modelling based upon this work

90. Aim of NICE contraceptive services project To assess the effectiveness and cost-effectiveness of interventions to encourage young people to use contraceptives and contraceptive services 27/10/11 © The University of Sheffield

91. Model scope Population: Young people aged between 14 & 16 years within secondary school who have not previously been a parent. Interventions: School-based dispensing of hormonal contraceptives to 14-16 year olds School-based ‘dispensing’ of condoms to 14-16 year olds Comparator: School nurse only Outcomes: Cost per age 14 – 16 Pregnancy Averted Cost per Abortion Averted Perspective: Public sector perspective (incl. & excl. Benefit payments) 27/10/11 © The University of Sheffield

92. Conceptual model 27/10/11 © The University of Sheffield Education/ employment impacts Impact on ‘unintended’ pregnancies Intervention to encourage contraceptive use Sexually transmitted infection (STI) rate Treatment of STIs Impact upon contraceptive services Abortion Miscarriage/ ectopic Pregnancy/ stilbirth Birth Mistimed Unwanted Low birth weight baby Maternity care Social care Mental health problems (eg. depression) Child health problems Crime Government-funded Benefits

93. Methods Markov model developed within Excel. Cohort of 100,000 14-year olds who have not had a baby followed over lifetime. Each year there is a probability of becoming pregnant. Following conception, dependent upon age, there is a probability of birth, abortion, miscarriage, ectopic pregnancy or stillbirth. Costs are associated with each of these outcomes. 27/10/11 © The University of Sheffield

94. 27/10/11 © The University of Sheffield Outcomes of teenage birth Family, societal and individual characteristics Long term outcomes of child Immediate birth outcomes Long term outcomes of parent(s) Low social class Teenage birth Poor education Poor employment More claims for Means-tested Benefits Teenage pregnancy Crime Low birth weight Foetal death Poorly educated mother Poor behaviour/ education as child Other observable or unobservable characteristics A B C

95. Literature review: long term outcomes Econometric literature review assessing long term consequences of a teenage birth in the UK Controlling for observable & unobservable characteristics (eg. personality) which might predispose a young woman to teenage motherhood Eg. Comparing those who’ve had a teenage miscarriage with those that have had a baby as a teenager 27/10/11 © The University of Sheffield

96. Model schematic 27/10/11 © The University of Sheffield t t+1 t+1 t Conception (may be mistimed or unwanted) No conception (may be delayed) Abortion Birth Miscarriage/ ectopic pregnancy/ stillbirth Additional proportion of low birth weight babies Additional Benefit claims of teenage parents Cohort of young people STI No STI t+1 t t t+1

97. Key model assumptions The negative impacts of a teenage birth include: 4% more likely to receive Income Support & associated Benefits until age 35 years; 90% will receive Income Support immediately, hence the earlier the birth, the more years it is received for; <1% more babies will be low birth weight. 50% of averted teenage births are mistimed & 50% are unwanted. STIs are assumed to be transmitted to 1 person only. 27/10/11 © The University of Sheffield

98. Health outcomes Usually increasing life years is considered to be a good thing. In this case, contraceptives aim to prevent life. There is no long term quality of life evidence associated with motherhood by age, adjusted for socioeconomic status Similarly, there is no quality of life evidence associated with the child by age of motherhood 27/10/11 © The University of Sheffield

99. Cost-effectiveness decision rules 27/10/11 © The University of Sheffield More expensive Less effective More expensive More effective Less expensive Less effective Less expensive More effective Difference in effectiveness Difference in costs NO YES

100. Results (excl. Benefits) Cohort of 100,000 14-year olds followed over a lifetime 27/10/11 © The University of Sheffield Expected cost per age 14 – 16 pregnancy averted = £37 (condoms) & £110 (hormonal) compared with no intervention Approx. 50% probability that cost-saving

101. Results (incl. Benefits) Cohort of 100,000 14-year olds followed over a lifetime 27/10/11 © The University of Sheffield Dominates no intervention

102. Key issues in public health modelling Model scoping & granularity Extrapolating outcomes over the long term Valuing outcomes Handling uncertainty 27/10/11 © The University of Sheffield

103. Model scoping & granularity (1) Public health interventions & their settings are complex. Often need to capture potentially adaptive behaviour (eg. sexual behaviour). Often several health areas to be modelled (in this case, pregnancies & STIs). Intersectoral impacts (not just health) 27/10/11 © The University of Sheffield

104. Model scoping & granularity (2) Formal OR problem structuring methods such as cognitive mapping could be used: To determine the model scope To make sure that all important costs & outcomes associated with the intervention are captured Further research around the use of these methods applied to public health modelling is required 27/10/11 © The University of Sheffield

105. Extrapolating outcomes over the long term Trials generally collect intermediate outcomes. The relationship between these outcomes & final outcomes needs to be understood. Long term outcomes (eg. employment) are dependent upon many factors; hence econometric techniques will generally be required to disentangle the impacts of the intervention. 27/10/11 © The University of Sheffield

106. Valuing outcomes Within health economics, benefits are generally measured in terms of QALYs gained Within public health economic modelling, the QALY measure may not be sufficiently broad eg. intersectoral costs/ consequences There is also generally limited evidence around utilities Further research is required around alternative outcome measures 27/10/11 © The University of Sheffield

107. Handling uncertainty Within health economics, uncertainty within model parameters is formally assessed using probabilistic sensitivity analysis (PSA) The structure of public health models is also often highly uncertain eg. modelling mistimed pregnancies It is important to parameterise structural uncertainties & include these within the PSA as outlined by Bojke et al. (2006). 27/10/11 © The University of Sheffield

108. Conclusions Dispensing contraceptives within schools to 14 – 16 yr olds is likely to be cost-effective Key areas for methodological development within public health modelling: Model scoping & level of granularity Extrapolating outcomes over the long term Valuing outcomes Handling uncertainty

109. High dose lipid-lowering therapy Is this strategy cost-effective? Roberta Ara Senior Research Fellow ScHARR, University of Sheffield Email: r.m.ara@sheffield.ac.uk

110. Background Statin therapy for secondary CVD should “ usually be initiated with a drug with a low acquisition cost (taking into account required daily dose and product price per dose)” ( www.nice.org.uk/TA094, 2006 ) Savings: £1 billion over 5 years {Moon BMJ, 2006} 69% PCT switch to generic statins -> NHS savings ≥ £85m/year (www.institute.nhs.uk, 2007)

111. The decision problem To evaluate the cost-effectiveness of high dose statins (atorvastatin 80mg/d, rosuvastatin 40mg/d & simvastatin 80mg/d) versus simvastatin 40mg/d in individuals with acute coronary syndrome.

112. Markov health states

113. Clinical Data Literature review: 28 RCTs > 12 week duration Benefit: LDL-c Synthesis: MTC Relationship: LDL-c & CV events

114. Mean Relative Risks (95% CI) Atorvastatin 80mg/d Rosuvastatin 40mg/d Simvastatin 40mg/d Simvastatin 80mg/d Non-fatal MI 0.425 (0.302 - 0.544) 0.378 (0.241 - 0.51) 0.588 (0.500 - 0.675) 0.510 (0.404 - 0.613) Non fatal stroke 0.623 (0.508 - 0.737) 0.593 (0.465 - 0.717) 0.730 (0.647 - 0.813) 0.679 (0.580 - 0.777) Stroke death 0.809 (0.429 - 1.242) 0.794 (0.384 - 1.261) 0.863 (0.593 - 1.173) 0.837 (0.516 - 1.206) CHD death 0.580 (0.445 - 0.715) 0.545 (0.397 - 0.692) 0.699 (0.601 - 0.796) 0.642 (0.527 - 0.758)

115. Adherence to therapies Scenario A Scenario B Yr 1 Yr 5 Yr 1 Yr 5 S40 ITT ITT ITT ITT A80 ITT ITT 95% 85% R40 ITT ITT 95% 85% S80 ITT ITT 95% 85%

116. Treatment costs Annual cost Sensitivity analyses Atorvastatin 80mg/d £368 £92 Rosuvastatin 40mg/d £387 Simvastatin 40mg/d £17 Simvastatin 80mg/d £34 Monitoring cost £76

117. Results C omparing with simvastatin 40mg/d: Atorvastatin 80mg/d Rosuvastatin 40mg/d Simvastatin 80mg/d Scenario A £17,469 £12,484 £5,319 Scenario B £17,217 £12,277 £5,226

118. Results – reduced cost for Atorvastatin simvastatin 40mg/d simvastatin 80mg/d atorvastatin 80mg/d rosuvastatin 40mg/d 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 £0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 Threshold ratio Probability cost effective

119. Summary & Conclusion Rosuvastatin 40mg/d is currently the most cost-effective alternative Atorvastatin 80mg/d is the most cost-effective (@ 25% of current price) Simvastatin 80mg/d is not recommended Current PCT policies intended to minimise primary care drug acquisition costs result in sub-optimal treatment for patients with ACS

120. A cost-effectiveness model of prostate cancer screening Matthew Mildred, Jim Chilcott, Silvia Hummel ScHARR, University of Sheffield

121. Contents Introduction to the project and topic Disease natural history model Data and model calibration Validation Results Conclusions 05/04/2011 © The University of Sheffield

122. The project Client: UK National Screening Committee Purpose: Help determine IF a national prostate cancer screening programme should occur AND which screening strategy is best. Objectives: Estimate costs, benefits and resource implications of alternative screening options. 05/04/2011 © The University of Sheffield

123. Introduction to prostate cancer The prostate is a small gland in men behind the bladder. The most common cancer in men in UK (excluding non-melanoma skin cancer) In 2008: Over 37,000 men diagnosed Over 10,000 men died from prostate cancer 05/04/2011 © The University of Sheffield

124. Aim of screening: Reduce cancer mortality, morbidity and treatment costs through early diagnosis and intervention. 05/04/2011 © The University of Sheffield Current evidence: In 2009 two large RCTs reported apparently inconsistent results in terms of the death rate ratio: ERSPC – significant reduction in PCa death rate PLCO – no statistically significant reduction

125. Challenges: Effectiveness of different screening programmes unknown. Scarce data around disease process due to its unobservable nature. Multiple unknown parameters in cancer screening model. 05/04/2011 © The University of Sheffield

126. Solution: Develop loosely parameterised cancer screening simulation model. Calibrate unobservable model parameters to observed data. Estimate impact of prostate cancer screening using calibrated model. 05/04/2011 © The University of Sheffield

127. About the model: Disease natural history model (Simul8) Calibration module (Excel, Visual Basic) Simulation model of prostate cancer screening (Simul8) Resource impact model (Excel) 05/04/2011 © The University of Sheffield

128. Screening strategies investigated 05/04/2011 © The University of Sheffield No. Screens Screening Age (years) Screening Interval (years) Single 50 N/A 55 60 65 70 Repeat 50-70 2, 4 50-74 1, 2, 4 55-70 2, 4 55-74 2, 4

129. Outputs: Age-specific incidence Age-specific mortality Prostate cancer stage distributions Over-detection rate Lead time Life years gained, QALYs gained Probability of developing prostate cancer Etc... 05/04/2011 © The University of Sheffield

130. Definitions & terms used 05/04/2011 © The University of Sheffield PCa Onset Screen Detection PCa Mortality Clinical Diagnosis Other Cause Mortality PCa Onset Screen Detection PCa Mortality Clinical Diagnosis Other Cause Mortality Lead-time Lead-time Over-detection: Relevant:

131. Disease natural history model 05/04/2011 © The University of Sheffield

132. Data 05/04/2011 © The University of Sheffield Data Source Age specific cancer incidence Office of National Statistics Cancer stage distributions ProtecT RCT UK Cancer Registry (ERIC) Gleason score distributions ProtecT RCT UK Cancer Registry (ERIC) PSA/biopsy test characteristics ERSPC RCT (Rotterdam section) Progression Free Survival ERSPC RCT (Rotterdam section) Overall Survival ERSPC RCT (Rotterdam section)

133. Calibration process 05/04/2011 © The University of Sheffield

134. Total SSE during calibration 05/04/2011 © The University of Sheffield

135. Validation: Incidence 05/04/2011 © The University of Sheffield

136. Validation: PCa mortality 05/04/2011 © The University of Sheffield

137. Validation: BAUS 05/04/2011 © The University of Sheffield

138. Results: Incidence 05/04/2011 © The University of Sheffield

139. Results: Mortality 05/04/2011 © The University of Sheffield

140. Over-detection & Lead time: 05/04/2011 © The University of Sheffield Once at 50 50-74 every 4 years 50-74 every 2 years 50-74 every year Over-detection rate 18% 44% 45% 46% Lead time (for over-detected cases) 15.2 yrs 11.6 yrs 12.5 yrs 13.0 yrs

141. Conclusions: A minimal life gain is offset by the high levels of disease management and over-diagnosis: One off screening: life gain of 0.004 years (1.2 days) with 36 years of additional disease management Repeat screening: life gain of 0.03 years (10-11 days) with 67-84 years of additional disease management 05/04/2011 © The University of Sheffield

142. Have you heard our findings? 05/04/2011 © The University of Sheffield BBC News 06/12/2010 http://www.bbc.co.uk/news/health-11930979

143. Acknowledgements: Dr Anne Mackie and Prof Julietta Patnick at the UK National Screening Committee The South West Public Health Observatory The British Association of Urological Surgeons The ProtecT team 05/04/2011 © The University of Sheffield

144. Healthcare costs alone do not describe the total costs directly attributable to Ankylosing Spondylitis R. Ara 1 , R. Rafia 1 , J. C. Packham 2,3 , K L Haywood 4 , E.L. Healey 2,4 1 University of Sheffield, 2 Staffordshire Rheum Centre, 3 University of Warwick, 4 Arthritis Research Campaign, Keele. 27/10/11 © The University of Sheffield

145. Introduction Ankylosing Spondylitis (AS) is a chronic, progressive disease and prognosis is often poor. In addition to direct health care costs, AS is associated with substantial indirect costs due to privately funded health care, absenteeism from work and early retirement. 27/10/11 © The University of Sheffield

146. Objectives To explore the total costs directly attributable to AS patients attending rheumatological centres in the UK. To explore the relationship between total costs and disease severity (classified using the Bath AS Disease Activity Index (BASDAI) and the Bath AS Functional Index (BASFI) variables) 27/10/11 © The University of Sheffield

147. Methods Health care costs included: medications, inpatient, GP/ outpatient, physiotherapy, hydrotherapy Non medical indirect costs included: productivity losses (unemployment, early retirement, absenteeism, presenteeism) Productivity losses were assessed using human capital approach Absenteeism = number of days off sick (average 5 working days per week) Presenteeism = relative to age/gender adjusted gross pay & WLQ-16 Two-step model used to explore relationship between disease severity and costs using data collected at baseline 1: logistic regression for probability incurring cost 2: generalised linear model conditional on incurring any cost Performance of the estimated model tested on data collected at 6 months

148. Results (1) Healthcare Utilisation: 41% reported no health care resources 45% reported no NHS funded healthcare resources 26% reported ≥ 1 medication 71/162 received anti-TNF treatment 10 hospitalisations due to AS (mean duration 11 days) 215 reported ≥ 1 GP consultation Number consultations > for greater disease activity (1.7 vs 2.6) Number consultations > for non workers (2.8 vs 1.9) 19% ≥ 1 physiotherapy session (33% of these self funded) 11% ≥ 1 hydrotherapy session (46% of these self-funded) Number sessions > non workers (7.1 vs 13.1) Number sessions > for greater disease activity (7.0 vs 12.3) 27/10/11 © The University of Sheffield

149. Results (2) Total Costs: The distribution of costs is heavily skewed with small number incurring high costs and substantial proportion incurring none. Average 3 month direct health care costs = £420 ± £988 (median = £27) Average 3 month total costs = £2,837 ± £3,358 (median = £1,228) Mean 3 month total costs sub-grouped by disease severity bands

150. Results (3) Absenteeism/ presenteeism 25% of working age reported not working or early retirement due to AS 25% (85/339) of workers reported time off sick 93% (315/339) of workers reported a reduction in productivity while at work Mean number of days off sick due to AS = 5.7 (range 1 – 62) Mean reduction in productivity at work = 20% (range = 1 – 90%) 27/10/11 © The University of Sheffield

151. Results (4) Using BASDAI & BASFI to predict AS disease costs Model 1 : Probability of incurring costs = 3.26106 +0.13308*BASFI + 0.27695*BASDAI – 0.01323*BASFI*BASDAI + 0.30472*male – 0.04869*age – 0.01171*disease duration Model 2 : 3 month total costs, conditional on costs incurred = 6.89107 + 0.26335*BASFI + 0.12048*BASDAI - 0.01411*BASFI*BASDAI + 0.46456*male – 0.01679*age + 0.00364*disease duration Actual and predicted total 3 month costs

152. Conclusion This study shows that direct healthcare costs alone do not describe the total costs associated with AS and that productivity losses associated with AS are considerable. Additional research examining interventions and measures that improve both presenteeism and absenteeism will have the largest affect on the cost to society. 27/10/11 © The University of Sheffield

153. The direct healthcare resource costs associated with Ankylosing Spondylitis patients attending a UK secondary care rheumatology unit RM Ara 1 , JC Packham 2 , KL Haywood 3 1 University of Sheffield; 2 Staffordshire Rheumatology Centre, Stoke on Trent; 3 Royal College of Nursing Institute, Oxford, UK 27/10/11 © The University of Sheffield

154. Introduction Anti-TNF inhibitors such as etanercept (ETN), are now licensed for treating patients with severe Ankylosing Spondylitis (AS). These treatments have larger cost implications than previously available therapies and comprehensive economic evaluations of these novel treatments in individual disease areas are increasingly requested by policy decision makers to inform reimbursement decisions. 27/10/11 © The University of Sheffield

155. Objectives To explore the direct healthcare resources utilised by AS patients attending a UK secondary care rheumatology unit to inform an economic evaluation of ETN in AS patients. To establish if resources, and thus health care costs, vary by disease severity (as classified using the Bath AS Disease Activity Index (BASDAI) and the Bath AS Functional Index (BASFI) variables).

156. Methods Costs: Costs were assessed using a micro-costing approach starting with a detailed inventory and measurement of resources consumed by the patient. Unit cost multipliers were applied to the quantity of each type of resource consumed. The mean cost per patient is estimated using the total cost divided by the total number of patients included. Resources included : NHS physiotherapy, outpatient appointments; inpatient days, laboratory based tests (FBR, ESR, LFT, U&E), X-rays, scans, prescribed medications. 27/10/11 © The University of Sheffield

157. Results (1) Mean Annual Disease Costs The mean total annual costs is estimated to be £1,837 (s.d. = £2,764; median = £772). The total annual costs range from £101 to £18,012 with an asymmetric distribution (just 11/147 patients have annual costs concentrated beyond £7,000). The mean annual cost per patient is correlated with both measures of disease activity (BASDAI) and functional disability (BASFI) The mean annual costs by levels of disease activity and functional disability Mean (proportion) annual costs by resource type and disease severity BASDAI < 60 BASDAI > 60 BASFI < 60 BASFI > 60 Physiotherapy £281; 22% £1,575;45% £123; 13% £1,547; 43% Hospitalisation £303; 23% £785; 23% £243; 26% £779; 22% Medication £315; 24% £479; 14% £225; 24% £615; 17% Outpatient/tests £397; 31% £668; 19% £351; 37% £688; 19%

158. Results (2) Using BASDAI & BASFI to predict AS disease costs Both the BASDAI and BASFI measurements were moderately correlated with the log transformed annual costs (Pearson correlation = 0.40 and 0.48 respectively; p < 0.001). The model # with the best predictive ability was Annual direct costs = exp(0.006*BASDAI + 0.016*BASFI + 5.862) # Adj R 2 =0.24 hence BASDAI and BASFI only partially account for the variability in costs. A 10 unit increase in both BASDAI and BASFI measurements incurs an increment of approximately £130 for scores of 40 and below; and an increment of approximately £480 for scores of 70 and above.

159. Discussion/Future Research The results of this costing study give an indication of the range of direct healthcare costs associated with patients who have AS and are eligible for anti-TNF treatments in the UK. Due to time constraints only readily accessible data such as clinical visits; inpatient care, technical procedures including radiographic examinations, prescribed medications, and physiotherapy appointments were included. The study would benefit from inclusion of additional information such as the number of GP visits. Further research into the most (cost)-effective provision of physiotherapy to patients with AS would be constructive. As AS is a progressive debilitating disease, it may be appropriate to include non-medical resources such as aids and appliances or formal household care and paid productivity losses such as absence from paid work.

160. A treatment option for patients with severe active Ankylosing Spondylitis: the costs and benefits associated with etanercept RM Ara 1 , A Reynolds 2 , P Conway 2 1 University of Sheffield; 2 Wyeth Pharmaceuticals, UK 27/10/11 © The University of Sheffield

161. Introduction Ankylosing Spondylitis (AS) is a progressive inflammatory disease that can cause irreversible skeletal damage. Typically presenting in young males, the long term prognosis is poor. High levels of pain and severe loss of physical function can have a large effect on health related quality of life (HRQoL). The Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) and the Bath Ankylosing Spondylitis Functional Index (BASFI) are validated and established measures in AS patients.

162. Objective To explore the costs and benefits associated with etanercept (ETN) in patients with severe AS in the UK under the BSR criteria. 27/10/11 © The University of Sheffield

163. Methods The model (written in Excel 2003) estimates the costs and benefits associated with ETN (2x 25mg/week) plus NSAIDS compared with NSAIDs alone. Effectiveness of ETN is based on patient level data obtained from two RCTs Placebo data are used to inform the comparator (88% of patients received NSAIDS) Effectiveness of treatment are measured by changes in BASDAI and BASFI at weeks 12 and 24 Initial response is supported by three year open label data [6,7] Direct healthcare costs associated with AS are obtained from a UK costing study and are estimated using the relationship [8] Annual direct costs = exp(0.006*BASDAI + 0.016*BASFI + 5.862) QoL data (EQ-5D) collected during the RCT are used to estimate quality adjusted life years (QALYS) using the relationship Utility = 0.923 – 0.004*BASFI – 0.004*BASDAI A 25 year time horizon is used to reflect the chronic progressive nature of AS and results are also presented for shorter horizons Univariate sensitivity analyses are used to explore the impact of varying individual key parameters Costs and benefits are discounted at 3.5% per annum

164. Assumptions Natural disease progression: BASFI increases at 0.7 units per annum For responders to ETN: BASDAI & BASFI measurements remain constant at values observed in RCTS BASDAI & BASFI measurements revert to baseline values and follow natural disease progression after withdrawal from ETN 10% annual withdrawal from ETN due to lack of efficacy/ adverse effects Age and sex related life expectancy is adjusted using ratio of 1.5 and assumed equal in both arms 27/10/11 © The University of Sheffield

165. Base case results 27/10/11 © The University of Sheffield

166. Univariate sensitivity analyses The tornado diagram shows that the three variables that have the largest impact of the 25 year results are: the values used to represent HRQoL the annual withdrawal rates the health care costs directly attributable to AS 27/10/11 © The University of Sheffield Tornado diagram showing the parameters which have the largest impact on the results

167. Monte Carlo results Using a 25 year horizon and a £30k per QALY threshold, ETN would be considered a cost-effective treatment when compared to NSAIDs in individuals with severe AS. The corresponding cost effectiveness acceptability curve shows that ETN is 93% likely to be cost effective when using a £25k per QALY threshold. 27/10/11 © The University of Sheffield Cost effectiveness plane ETN plus NSAIDs compared with NSAIDs

168. Areas for future research Potential impact of radiological progression on long term disability Efficacy of ETN in individuals with BASDAI < 40 Strengthen relationship between BASDAI/BASFI and both costs and utilities BASDAI & BASFI further explore the relationship to disease costs Effect of ETN on early retirement Develop prognostic algorithms to identify patients who would benefit the most from ETN Quantify natural disease progression and disease progression for individuals responding to ETN

169. An economic evaluation of etanercept in the treatment of patients with psoriatic arthritis RM Ara and R Rafia ScHARR, University of Sheffield 27/10/11 © The University of Sheffield

170. Introduction Psoriatic Arthritis (PsA) is a chronic, progressive disease and prognosis is often poor. Anti_TNF agents such as etanercept are effective alternatives for patients whose current treatment is symptomatic.

171. Objective To explore the costs and benefits associated with etanercept in the treatment for patients with psoriatic arthritis to inform a NICE submission.

172. Methods The model compares ciclosporin to anti-TNF agents (etanercept, adalimumab, infliximab) and it is assumed that patients have failed methotrexate and sulphasalazine prior entering the model. Relationship between HAQ and costs associated with PsA and HAQ and health related quality of life are then used to estimate the long term costs and benefits accrued. Treatment effectiveness was derived from a Mixed Treatment Comparison published in a previous STA. Treatment discontinuation was modelled using PsARC response rate at 12 and/or 24 weeks using results from the MTC. Long term withdrawal rate was derived from data from a recent observational study conducted by the BSRBR. A set of 27 univariate sensitivity analysis was performed to test the robustness of the model to main assumptions. The overall uncertainty was examined using a Monte Carlo approach. Costs and quality of life benefits were discounted at 3.5% per annum as per NICE recommandations for economic evaluation The model was extended beyond the trial duration to a 50 years time horizon

173. Relationship between HAQ and HrQoL Relationship estimated from PRESTA trial for the base case: Utility (EQ-5D) = 0.8996 – 0.4559 * HAQ – 0.0010 * Age + 0.0201 * male + 0.0031 *Age * HAQ – 0.0388 * male * HAQ

174. Relationship between HAQ and Direct health care Costs Health care costs include both primary and secondary care resources. The relationship between HAQ and costs was examined using a Generalised Linear Model assuming a poison distribution and a log link: Cost = 3.5367 + 2.0484 * HAQ + 0.0260 * Age – 0.0119 * Age * HAQ Actual and predicted annual cost by HAQ

175. Base case results

176. Uncertainty in results The model results were sensitive to both the magnitude of the rebound of HAQ after withdrawal from treatment, and annual HAQ progression rates, relationship between HAQ and QoL and discount rates. Results from the PSA, demonstrate one is 65% confident that etanercept is a cost-effective strategy when using a threshold of £20,000 per QALY. 27/10/11 © The University of Sheffield Cost-Effectiveness- Acceptability Curve

177. Conclusion While this evaluation provides evidence on the potential cost-effectiveness of etanercept compared to other anti-TNF agents or ciclosporin for adults with PsA, there are limitations with the evidence based used to populate the model. 27/10/11 © The University of Sheffield

178. Evidence Review and Synthesis 27/10/11 © The University of Sheffield

179. Extensive experience of systematic review and evidence synthesis work for National Institute for Health and Clinical Excellence (NICE), NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) NICE Public Health Collaborating Centre (PHCC) Multiple clients in industry. 27/10/11 © The University of Sheffield

180. Systematic review of clinical evidence High quality systematic reviews for submissions to key reimbursement bodies or for publication Clinical reviews Critical appraisal of reviews Review updates Other review methods Mapping reviews Reviews of model parameters Rapid reviews Research reports 27/10/11 © The University of Sheffield

181. Generalised evidence synthesis Classical and Bayesian meta analysis, including network meta analysis RCTs Observational studies Diagnostic studies Qualitative studies (meta synthesis) Mixed methods

182. Examples of Systematic reviews conducted in HEDS 27/10/11 © The University of Sheffield

183. PET and MRI for the assessment of axillary lymph node metastases in early stage breast cancer A Systematic review Susan Harnan Research Associate ScHARR, University of Sheffield Email: s.harnan@sheffield.ac.uk 27/10/11 © The University of Sheffield

184. Background Axillary staging is important for breast cancer staging and treatment planning Current techniques include Axillary lymph node dissection (ALND) Sentinel lymph node biopsy (SLNB) Sampling techniques such as four node sampling (4-NS) All these procedures (ALND, SLNB and 4-NS) have short and long term adverse effects such as lymphoedema 27/10/11 © The University of Sheffield

185. Diagnostic pathway NICE diagnostic pathway *Either fine needle aspiration cytology (FNAC) or core biopsy. Positron emission tomography (PET) and magnetic resonance imaging (MRI) may offer alternatives with fewer adverse events Could replace SLNB/4-NS Or be added to the pathway 27/10/11 © The University of Sheffield

186. HTA report National Institute for Health Research Health Technology Assessment (HTA) programme commissioned research to evaluate PET and MRI: Diagnostic accuracy Cost-effectiveness Patient outcomes Work carried out by ScHARR: Katy Cooper, Yang Meng, Sue Harnan, Sue Ward, Patrick Fitzgerald, Diana Papaioannou With clinical advisers: Lynda Wyld, Christine Ingram, Iain Wilkinson, Eleanor Lorenz 27/10/11 © The University of Sheffield

187. Methods Searched eleven databases Inclusion criteria: Assessed diagnostic accuracy of PET or MRI for assessment of axillary metastases in patients newly diagnosed with early-stage invasive primary breast cancer PET studies only included if ≥ 20 patients Studies with > 20% non-early stage, non-newly diagnosed or DCIS were excluded. Quality Assessment of Diagnostic Accuracy Studies (QUADAS) checklist used. 27/10/11 © The University of Sheffield

188. Diagnostic accuracy studies Patients are classified against a reference standard (eg ALND or SLNB) as true positive, true negative, false positive or false negative Sensitivity is % of positive patients correctly identified Specificity is % of negative patients correctly identified Studies only included if TP, TN, FP and FN were reported or could be calculated. Bivariate random effects model used to meta-analyse data as sensitivity and specificity are linked.

189. Results 27/10/11 © The University of Sheffield

190. PET results Quality generally acceptable, though some problems with patient spectrum, blinding, availability of relevant clinical information and reporting of uninterpretable results. 26 studies of PET or PET/CT 27/10/11 © The University of Sheffield

191. PET results – forest plots

192. PET sensitivity analyses

193. MRI results Quality generally good. Some problems with patient spectrum, availability of relevant clinical information and uninterpretable results. Small numbers, different methods Nine studies of MRI, meta-analysed using highest reported sensitivity and specificity from each study

194. MRI results – forest plot 27/10/11 © The University of Sheffield

195. MRI results Criteria for positivity varied across studies, including variables of size, morphology and uptake. Uptake pattern appeared to give better combined sensitivity and specificity. Sensitivity analyses Sensitivity analyses for size and nodal status not possible as these were not reported Lower specificity and a trend towards higher sensitivity was seen for studies with only early-stage, newly-diagnosed non-DCIS patients, but wide range. Study quality did not affect estimates (note small sample, little variation in scores) 27/10/11 © The University of Sheffield

196. Adverse effects No adverse effects were reported for PET Mild to moderate adverse effects reported for MRI include mild rash following USPIO administration, claustrophobia and back pain. Cautions and contraindications exist for both including pregnancy (PET), allergy to contrast agents, renal and liver dysfunction, pacemakers, metallic implants (MRI). 27/10/11 © The University of Sheffield

197. Discussion PET, PET-CT and MRI have lower sensitivity and specificity than ALND, SLNB and 4-NS PET and PET-CT are similar to ultrasound in terms of sensitivity and specificity, MRI is slightly higher. MRI has higher sensitivity than PET or PET-CT, but similar specificity. USPIO-enhanced MRI gave highest estimates of sensitivity and specificity, but these are based on a small number of studies All results vary widely between studies and caution should be taken when interpreting results 27/10/11 © The University of Sheffield

198. Discussion (2) Based on these estimates, if PET or MRI are used to replace SLNB/4-NS in the diagnostic pathway, more women will be at greater risk of recurrence or metastatic spread (false negatives) and more women will undergo ALND unnecessarily (false positives) However PET and MRI have few adverse effects and much fewer women would undergo SLNB/4-NS and be at risk of the long term problems associated with them. Decision modeling is needed for overall evaluation of benefits and harms. Decision modeling will help evaluate the effects if PET or MRI are used in addition to SLNB/4-NS in the diagnostic pathway. 27/10/11 © The University of Sheffield

199. PET results – ROC plot 27/10/11 © The University of Sheffield All PET studies, showing ROC curve (solid line), mean sensitivity/specificity (black spot) and 95% confidence region (dashed ellipse)

200. MRI results – ROC plot 27/10/11 © The University of Sheffield All MRI studies, showing ROC curve (solid line), mean sensitivity/specificity (black spot) and 95% confidence region (dashed ellipse)

201. Systematic reviews of relevant data Myfanwy Lloyd Jones Senior Research Fellow ScHARR, University of Sheffield Email: m.lloydjones@sheffield.ac.uk

202. More specifically Drawing on the experience of the pilot diagnostics project, how do systematic reviews of diagnostic interventions differ from systematic reviews of therapeutic interventions, when both are undertaken to inform NICE decision-making? Review question PICO(S) components Meta-analysis 27/10/11 © The University of Sheffield

203. The review question Is there a bigger difference between the overall project question and the question that forms the focus of the systematic review of clinical effectiveness in an assessment of a diagnostic intervention than in an assessment of a therapeutic intervention? Therapeutic intervention: strontium ranelate TAR (2005) 27/10/11 © The University of Sheffield

204. Therapeutic intervention Research question defined in the protocol: To establish the clinical and cost effectiveness of strontium ranelate for the prevention of osteoporotic fractures in postmenopausal women with osteoporosis Systematic review of clinical effectiveness: What is the clinical effectiveness of strontium ranelate for the prevention of osteoporotic fractures in postmenopausal women with osteoporosis, with or without prior fracture? 27/10/11 © The University of Sheffield

205. In PICOS terms Population: postmenopausal women with osteoporosis, with or without prior fracture Intervention: strontium ranelate Comparator: placebo/no treatment; specified active treatments if evidence available Outcomes: Principal: incident fractures (vertebral & nonvert) Secondary: adverse effects, health-related quality of life etc Study design: RCTs 27/10/11 © The University of Sheffield

206. Diagnostic intervention Decision problem defined in the protocol: Will using non-invasive liver assessment tools in patients with suspected alcohol-related liver fibrosis who might otherwise be candidates for biopsy or referral to specialist care reduce the number of referrals or biopsies and improve the health outcomes and quality of life of those patients? Systematic review of clinical effectiveness: The subject of some debate 27/10/11 © The University of Sheffield

207. In PICOS terms: population & intervention Population: patients suspected of having liver fibrosis related to alcohol consumption Intervention: the scope and protocol specified 4 non-invasive liver assessment tools (NILTs) ? A fifth – how binding is the protocol? 27/10/11 © The University of Sheffield

208. In PICOS terms: the comparator The current UK decision-making process? What the NICE team wanted: A relevant comparator for the model Problematic given the lack of consensus among clinicians about the potential role of NILTs A problematic comparator for the systematic review because no study is likely to have been set up with that comparator A specific diagnostic test (reference standard)? Feasible in terms of the evidence base What many clinicians are likely to want to know is how a NILT compares with the reference standard (at least if the reference standard is what they generally use) 27/10/11 © The University of Sheffield

209. In PICOS terms: outcomes Test accuracy compared with the reference standard Number of test failures or other withdrawals Adverse events Number of referrals to specialist care Number of biopsies needed (presumably because of test failures or borderline results) Longer-term health outcomes Quality of life Which of these should be the primary outcome? 27/10/11 © The University of Sheffield

210. In PICOS terms: study design Protocol: “The best available level of evidence will be included, with priority given to controlled studies if available” 27/10/11 © The University of Sheffield

211. Population (1) Therapeutic intervention: reasonable match between review question and study populations (main exclusions related to conditions which could interfere with bone metabolism, and recent treatment with antiosteoporotic agents; 1 study excluded women aged >78) Diagnostic intervention: mismatch introducing spectrum bias 27/10/11 © The University of Sheffield

212. Population (2) In diagnostic studies, if the reference standard is invasive (eg liver biopsy) or particularly expensive, then for ethical or cost reasons the study populations are likely not to be representative of the full range of people eligible for the index test Either the whole study population is unrepresentative because of disease severity (eg already scheduled for liver biopsy because of suspected severe fibrosis) Or the whole population is more representative, but liver biopsy is only performed in the subset whose index test results suggest more severe disease

213. Intervention Therapeutic intervention: Description and other useful data from EMEA and BNF Diagnostic intervention: No structured descriptions of the technology – not even sponsor submissions: Data pulled together from assorted articles and manufacturers’ websites 27/10/11 © The University of Sheffield

214. Comparator: what is it? Therapeutic intervention: No active intervention (placebo or no treatment) Active intervention likely to have a direct effect on relevant clinical outcomes Diagnostic intervention: Subject of debate: test or care pathway? Primary comparator: liver biopsy Secondary comparators: tests used to identify conditions associated with liver fibrosis

215. Comparator: issues Studies of diagnostic test accuracy assume that the reference standard has 100% sensitivity and specificity, which is clearly unlikely Some reference standards (eg liver biopsy) are known to be imperfect Discordance between the results of the index test and reference standard may result from error in either test Some studies may try to determine, for each individual discordant result, whether the reference standard or the new test is more likely to be correct, but will never identify cases where the results are concordant but both wrong 27/10/11 © The University of Sheffield

216. Outcomes: therapeutic intervention Primary outcome: fracture Clinical relevance clear 27/10/11 © The University of Sheffield

217. Outcomes: diagnostic intervention Primary outcome: subject of debate Test accuracy? Fundamental: if the new test is inaccurate, no point in going further Problematic, and often poorly reported Patient outcomes? Clinical outcomes: obviously relevant, but may depend heavily on clinician or patient choices subsequent to the test result (eg, for NILTs, would a positive test result make the patient more or less likely to stop drinking?) Safety/ patient acceptability relative to current practice 27/10/11 © The University of Sheffield

218. Reporting of outcomes: therapeutic intervention (1) The Consort statement says that “for each outcome, study results should be reported as a summary of the outcome in each group (for example, the number of participants with or without the event and the denominators, or the mean and standard deviation of measurements), together with the contrast between the groups, known as the effect size” For binary outcomes, the effect size could be the relative risk, odds ratio, or risk difference 27/10/11 © The University of Sheffield

219. Reporting of outcomes: therapeutic intervention (2) 1 27/10/11 © The University of Sheffield Number (%) Relative risk (95% CI) Endpoint Strontium ranelate (n=719) Placebo (n=723) Nonvertebral fracture 112 (15.6) 122 (16.9) 0.90 (0.69 to 1.17)

220. Reporting of outcomes: therapeutic intervention (3) Vertebral fracture: SOTI trial reported %ages plus RR; numbers of participants in each treatment group with fracture obtained from investigator The STRATOS trial and, at the time of our review, the TROPOS trial only reported %ages plus RR Nonvertebral fracture: SOTI and TROPOS reported numbers of participants in each treatment group with fracture, with RR (and %ages for TROPOS) STRATOS just reported %ages, as adverse events 27/10/11 © The University of Sheffield

221. Reporting of outcomes: diagnostic intervention (1) The STARD (Standards for Reporting of Diagnostic Accuracy) guidelines require “a cross tabulation of the results of the index tests (including indeterminate and missing results) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard ” 27/10/11 © The University of Sheffield

222. Reporting of outcomes: diagnostic intervention (2) 27/10/11 © The University of Sheffield Reference standard positive Reference standard negative Index test positive True positive (TP) False positive (FP) Index test uninterpretable Uninterpretable (U 1 ) Uninterpretable (U 2 ) Index test negative False negative (FN) True negative (TN) Sensitivity = (TP/(TP+U 1 +FN))x100 Specificity = (TN/(TN+U 2 +FP))x100

223. Reporting of outcomes: diagnostic intervention (3) 27/10/11 © The University of Sheffield

224. Reporting of outcomes: diagnostic intervention (4) NILTs produce results on a continuous scale, but liver biopsy results are expressed in terms of ordinal scoring systems, so when NILTs are compared with liver biopsy the results are also generally presented as ordinal data (fibrosis stages) Most of the studies in the pilot project identified the threshold values for those stages from the ROC curve, rather than validating prospectively identified values Only 1/11 studies in the pilot project which compared a NILT with liver biopsy published a reasonably clear tabulation of test accuracy 27/10/11 © The University of Sheffield

225. Study design Therapeutic intervention: Only RCTs included Diagnostic intervention: No relevant RCTs identified: Where RCTs of diagnostic test accuracy exist, they are generally underpowered to detect a difference between 2 tests 15 cross-sectional studies, 2 of which were extended to form cohort studies looking at longer-term outcomes Non-RCTs carry the risk of double-counting: often not clear whether different publications by the same research group include some of the same patients 27/10/11 © The University of Sheffield

226. Meta-analysis (1) Therapeutic intervention: Generally straightforward using Review Manager In the strontium ranelate example, not possible because the relevant data for all 3 included studies were not published and could not be obtained from the original investigators Diagnostic intervention: Complex, only partially possible in Rev Man Data often not published in suitable format 27/10/11 © The University of Sheffield

227. Meta-analysis (2) Not appropriate if studies have heterogeneous populations, and in diagnostic studies that heterogeneity includes different prevalences of the condition of interest How similar do populations need to be before the data can be combined? Requires special techniques if studies use different diagnostic thresholds to define a positive result Is it generally better to use the results of the best quality study which has the greatest clinical relevance to the study question? 27/10/11 © The University of Sheffield

228. Systematic reviews of relevant data Myfanwy Lloyd Jones Senior Research Fellow ScHARR, University of Sheffield Email: m.lloydjones@sheffield.ac.uk

229. The aim of the assessment To answer the research question: Will using the specified non-invasive liver assessment tools in patients with suspected alcohol-related liver fibrosis who might otherwise be candidates for biopsy or referral to specialist care reduce the number of referrals or biopsies and improve the health outcomes and quality of life of those patients?

230. The specified tests Three patented blood tests: The Enhanced Liver Fibrosis (ELF) test FibroTest FibroMAX Transient elastography FibroScan Test results are affected by alcohol consumption

231. Outcomes of interest: Diagnostic test accuracy (including numbers of test failures) Number of patients tested in primary care requiring referral to secondary care for further investigation or treatment Number of patients requiring liver biopsy Number of patients giving up alcohol, or significantly reducing alcohol consumption, because of test result Long-term patient outcomes (disease progression, complications related to liver disease, need for liver transplantation, mortality) Adverse effects of testing Health-related quality of life Cost-effectiveness

232. Reference standard tests Liver biopsy Also used in some studies Hepatic venous pressure gradient (HVPG) measurement to identify portal hypertension Upper intestinal endoscopy to identify oesophageal varices

233. Systematic review of clinical effectiveness Population: patients with suspected alcohol-related liver fibrosis Intervention: any of the 4 specified tests Comparators: Primary: liver biopsy Secondary: HVPG measurement; upper endoscopy Outcomes: previously listed Study design: cohort or cross-sectional studies (prospective or retrospective)

234. Included studies: ELF test Rosenberg 2004: patients with chronic liver disease (subgroup with ALD); test accuracy vs liver biopsy Parkes in press: follows cohort of English patients from Rosenberg study; survival (median follow-up 6.86 years)

235. Included studies: FibroTest (1) 2 studies specifically in patients with known or suspected alcohol-related liver disease, both of test accuracy vs liver biopsy: Naveau 2005 Nguyen-Khac 2008 Naveau 2009: 5 and 10 year survival of patients included in Naveau 2005

236. Included studies: FibroTest (2) 3 studies in mixed aetiology liver disease: 1 study in patients with chronic liver disease: - Thabut 2003: test accuracy vs endoscopy 1 study in patients undergoing transjugular liver biopsy for clinical reasons: - Thabut 2007a: test accuracy vs HPVG 1 study in patients with severe cirrhosis: - Thabut 2007b: survival at 2 and 6 months

237. Included studies: FibroMAX No relevant studies identified

238. Included studies: FibroScan (1) 6 studies specifically in patients with known or suspected alcohol–related liver disease, all looking at test accuracy vs liver biopsy: 4 biopsied all patients Kim 2009, Mueller 2010, Nahon 2008, Nguyen-Khac 2008 2 biopsied subset only: Janssens 2010: biopsy only in those with FS score > 9.6, indicating severe fibrosis ( > F3) (this study also looked at test accuracy vs HPVG) Melin 2005: biopsy only in patients with FS score >13 kPa (chosen as threshold for cirrhosis in patients with hepatitis C)

239. Included studies: FibroScan (2) 3 studies in mixed aetiology liver disease: 1 study in patients undergoing transjugular liver biopsy for clinical reasons: Bureau 2008: test accuracy vs HVPG 2 studies in patients with cirrhosis: Lemoine 2008: test accuracy vs HVPG Nguyen-Khac 2009: test accuracy vs endoscopy

240. AUROCs from key studies: test vs liver biopsy Test Degree of fibrosis Study AUROC (95% CI) ELF ‘ Moderate/severe’ Rosenberg 0.94 (0.84-1.00) FibroTest F2-F4 Naveau 0.83 (0.81-0.87) Nguyen-Khac 0.79 (0.69-0.90) Cirrhosis (F4) Naveau 0.95 (0.94-0.96) Nguyen-Khac 0.84 (0.72-0.97) FibroScan Severe fibrosis (F3-F4) Kim 0.98 (0.94-1.02) Mueller 0.91 + 0.03 Nahon 0.94 (90-0.97) Nguyen-Khac 0.90 (0.82-0.97) Cirrhosis Kim 0.97 (0.93-1.01) Mueller 0.92 (0.87-0.97) Nahon 0.87 (0.81-0.93) Nguyen-Khac 0.94 (0.87-0.98)

241. Sensitivity and specificity: ELF Test (subgroup with ALD) Degree of fibrosis Study Threshold score Sensitivity Specificity ‘ Moderate/severe’ Rosenberg 0.087 100% 16.7% 0.431 93.3% 100%

242. Sensitivity and specificity: FibroTest (all patients ALD) Degree of fibrosis Study Threshold score Sensitivity Specificity Moderate-severe (F2-F4) Naveau 0.30 84% 66% 0.70 55% 93% Cirrhosis (F4) Naveau 0.30 100% 50% 0.70 91% 87%

243. Sensitivity and specificity: FibroScan (all patients ALD, all biopsied) Degree of fibrosis Study Threshold score Sensitivity Specificity Moderate-severe (F2-F4) Nguyen-Khac 7.8 kPa 80% 90.5% Severe (F3-F4) Mueller 8.0 kPa 91% 75% Nguyen-Khac 11 kPa 86.7% 80.5% Nahon 11.6 kPa 87% 89% Cirrhosis (F4) Mueller 11.5 kPa 100% 77% 12.5 kPa 96% 80% Nguyen-Khac 12.8 kPa 100% 75.4% 19.5 kPa 85.7% 84.2% Nahon 22.7 kPa 84% 83% Kim 28.5 kPa 90% 87%

244. Sensitivity and specificity: FibroScan (all patients ALD, only subset biopsied) Degree of fibrosis Study Threshold score Sensitivity Specificity Severe (F3-F4) Janssens 13.9 kPa 81% 59% 15.8 kPa 75% 70% 16.5 kPa 72% 70% 17.0 kPa 72% 76.5% 17.3 kPa 69% 76.5% Cirrhosis (F4) Janssens 19.6 kPa 80% 76% 21.1 kPa 75% 80% 23.5 kPa 65% 83%

245. FibroScan: exclusion of patients with laboratory signs of ASH Degree of fibrosis No of patients AUROC Threshold score Sensitivity Specificity F4 101 0.921 11.5 100% 77% 12.5 96% 80% F4 without GOT >100 U/L 86 0.944 11.5 100% 84% 12.5 95% 90% F3-F4 101 0.914 8.0 91% 75% F3-F4 without GOT >100 U/L 80 0.922 8.0 87% 87%

246. Evidence re outcomes of interest (1) Diagnostic test accuracy: NILTs appear to have reasonable accuracy, but the evidence is not drawn from full spectrum of patients Number of patients tested in primary care requiring referral to secondary care: no data Number of patients requiring liver biopsy following NILT: no evidence of what this would be in clinical practice rather than study setting

247. Evidence re outcomes of interest (2) Numbers of patients giving up or significantly reducing alcohol consumption following test result: only reported in one study, and not subdivided by test results. Long-term patient outcomes: ELF test and FibroTest may have some prognostic value Adverse effects of NILTs: no specific AEs, but tests which use a blood sample have AEs associated with venepuncture

248. Liver biopsy: an imperfect reference standard Diagnostic studies assume that the reference standard has 100% sensitivity and specificity Not true of liver biopsy: Sampling error (may affect >30% of samples) ‘ Significant’ inter- and intra-observer variation in interpretation of samples Some studies try to determine whether, in discordant cases, the index test or the biopsy is more likely to be correct What if the index test and the biopsy are concordant but both wrong?

249. Liver biopsy: ethical aspects affect study design Adverse events: Minor adverse events: 9.39% Severe adverse events (including death): 0.86% Death: 0.08% Consequently, study populations not representative of full range of people with suspected ALD: Either whole study population is unrepresentative because of disease severity (eg already scheduled for biopsy) Or whole population is more representative, but biopsy is only performed in subset with NILT result suggesting more severe disease

250. How much homogeneity is required for meta-analysis? The studies of FibroTest and FibroScan included in this review are heterogeneous in terms of: Prevalence of the condition of interest Diagnostic threshold for identifying that condition Because of this heterogeneity, it seems inappropriate to combine their results using meta-analysis

251. Health Economics and Outcomes Research 27/10/11 © The University of Sheffield

252. Areas of Particular Strength Quality of life and outcomes measurement, wellbeing and equity Primary economic evaluation in clinical studies, costing studies and service evaluation Statistical methods in health economics

253. Research centre Centre for Well-being in Public Policy (CWiPP) The goals of the centre are to consider as to how people’s health and wellbeing can be defined, measured and improved in ways that help policy-makers determine the best use of scarce resources, and to investigate the determinants of well-being insofar as these are relevant to policy formulation 27/10/11 © The University of Sheffield

254. For example: HEDS in collaboration of the Department of Economics, University of Sheffield run a number of joint research projects in the area of health and well-being, including the Independent Review of the Effects of Alcohol Pricing and Promotion project funded by the Department of Health 27/10/11 © The University of Sheffield

255. Examples of health outcome studies conducted in ScHARR 27/10/11 © The University of Sheffield

256. Developing and testing condition-specific preference-based measures: Lessons learnt and policy implications John Brazier, Donna Rowen, Ifigeneia Mavranezouli, Aki Tsuchiya , Tracey Young, Yaling Yang, Michael Barkham

257. Acknowledgements COSMeQ team: John Brazier 1 , Aki Tsuchiya 1 , Ifigeneia Mavrazenouli 1,2 , Tracey Young 1 , Yaling Yang 3 , Rachel Ibbotson 4 , Michael Barkham 1 1 University of Sheffield 2 University College London 3 Brunel University 4 Sheffield Hallam University COSMeQ project, MRC-NIHR Methodology Research Programme project number 06/97/04

258. Economic evaluation Economic evaluation using cost utility analysis (CUA) increasingly used to inform resource allocation decisions CUA measures health outcomes using the Quality Adjusted Life Year (QALY) QALY combines quantity and quality of life into single measure of health outcome by adjusting life years using a quality weight QALY can be used across all health care interventions for all patient groups

259. Quality-adjusted life years Source: Drummond et al, 1997

260. Health utility measurement Purpose of health utility measurement is to provide the quality adjustment weight, ‘Q’, for the QALY Typically obtained using generic preference-based measures Condition-specific preference-based measures (CSPBM) are an alternative, and more commonly used when generic measure is inappropriate generic measure is unavailable

261. Health utility measurement Preference-based measure has two components: Health state description Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead)

262. Health utility measurement Preference-based measure has two components: Health state description Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead) Condition label? Narrow focus? Multi-dimensional? Side effects, comorbidities?

263. Health utility measurement Preference-based measure has two components: Health state description Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead) Condition label? Narrow focus? Multi-dimensional? Side effects, comorbidities?

264. Health utility measurement Preference-based measure has two components: Health state description Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead) Condition label? Narrow focus? Multi-dimensional? Side effects? Comorbidities? Performance? Comparability?

265. Outline of presentation Developing CSPBM from existing measure impact of condition labels on health state utility values impact of adding a generic dimension to a condition-specific preference-based measure to capture co-morbidities or side-effects Testing CSPBM performance in comparison to measure derived from and generic EQ-5D

266. Impact of condition labels on health state utility values

267. Impact of labelling: rationale Generic measures typically valued with no condition label Often CSPBMs include condition label as is embedded in original measure it is derived from e.g. ‘experience asthma symptoms as a result of air pollution’ No consensus in previous studies examining impact of labelling Small sample size Within-subject design Small severity range of states

268. Study design Between-subject study Respondents valued: same 8 states produced using EORTC-8D, non-labelled CSPBM cancer derived from EORTC QLQ-C30 range of health states of differing severity one of three versions: no label, irritable bowel syndrome label, cancer label using MVH TTO protocol Sampling strategy to ensure representativeness across label groups and to UK general population

269. Example health state (51224434) (no label) / Due to having irritable bowel syndrome / Due to having cancer You have very much trouble taking a short walk outside of the house You are not limited in pursuing your hobbies or other leisure time activities Your physical condition or medical treatment interferes a little with your social activities Pain interferes a little with your daily activities You feel depressed very much You are tired very much You are constipated and/or have diarrhoea quite a bit You feel nauseated very much

270. Analysis ANOVA using generalized linear model to determine significant differences in respondent characteristics across groups t tests to determine significant differences in health state values across groups Regression analysis (RE GLS model) to determine impact on elicited utility values due to condition label, state severity, interaction of label and severity, respondent characteristics, experience of condition

271. Respondent characteristics No label (n=81) IBS label (n=80) Cancer label (n=80) ANOVA P-value Mean age (s.d.) 48.08 (21.57) 51.78 (18.93) 47.1 (18.85) 0.30 Female 62% 63% 56% 0.68 Married/partner 63% 53% 56% 0.40 Employed or self-employed 36% 39% 41% 0.78 Unemployed 3% 3% 6% 0.35 Long-term sick 7% 4% 6% 0.60 Full-time student 12% 3% 6% 0.05 Housework 9% 14% 15% 0.44 Retired 30% 33% 23% 0.35 Experience of labelled condition 40% 78% in themselves 19% 13% in caring for others 14% 36%

272. Descriptive statistics Original study (n=344) No label (n=81) IBS label (n=79-80) Cancer label (n=79-80) Health state Modelled utility value Mean (s.d.) Mean (s.d.) Mean (s.d.) 11111111 1 0.96 (0.13) 0.99 (0.06) 0.96 (0.12) 31212241 0.75 0.74 (0.32) 0.81 (0.23) 0.80 (0.22) 13423411 0.72 0.67 (0.30) 0.71 (0.37) 0.64 (0.36) 44321321 0.65 0.66 (0.35) 0.68 (0.37) 0.56 (0.50) 23141224 0.64 0.63 (0.36) 0.69 (0.36) 0.57 (0.45) 24432411 0.64 0.66 (0.33) 0.65 (0.40) 0.54 (0.44) 51224434 0.51 0.49 (0.41) 0.53 (0.42) 0.41 (0.49) 54444444 0.29 0.20 (0.49) 0.17 (0.49) -0.03 (0.50)

273. Regression results

274. Regression results

275. Regression results Sociodemographic variables: female, married, retired, unemployed, student, housework, long-term sick, secondary school is highest level of education States Cancer interaction terms Labelling 11111111 x Cancer -0.041 IBS 0.008 31212241 -0.197*** 31212241 x Cancer -0.007 13423411 -0.284*** 13423411 x Cancer -0.079 Experience of condition 44321321 -0.304*** 44321321 x Cancer -0.147** Cancer in themselves -0.157* 23141224 -0.313*** 23141224 x Cancer -0.128** Cancer in caring for others 0.134** 24432411 -0.317*** 24432411 x Cancer -0.148** IBS in themselves -0.036 51224434 -0.456*** 51224434 x Cancer -0.136** IBS in caring for others 0.064 54444444 -0.785*** 54444444 x Cancer -0.254*** Constant 0.967***

276. Regression results Sociodemographic variables: female, married, retired, unemployed, student, housework, long-term sick, secondary school is highest level of education States Cancer interaction terms Labelling 11111111 x Cancer -0.041 IBS 0.008 31212241 -0.197*** 31212241 x Cancer -0.007 13423411 -0.284*** 13423411 x Cancer -0.079 Experience of condition 44321321 -0.304*** 44321321 x Cancer -0.147** Cancer in themselves -0.157* 23141224 -0.313*** 23141224 x Cancer -0.128** Cancer in caring for others 0.134** 24432411 -0.317*** 24432411 x Cancer -0.148** IBS in themselves -0.036 51224434 -0.456*** 51224434 x Cancer -0.136** IBS in caring for others 0.064 54444444 -0.785*** 54444444 x Cancer -0.254*** Constant 0.967***

277. Regression results Sociodemographic variables: female, married, retired, unemployed, student, housework, long-term sick, secondary school is highest level of education States Cancer interaction terms Labelling 11111111 x Cancer -0.041 IBS 0.008 31212241 -0.197*** 31212241 x Cancer -0.007 13423411 -0.284*** 13423411 x Cancer -0.079 Experience of condition 44321321 -0.304*** 44321321 x Cancer -0.147** Cancer in themselves -0.157* 23141224 -0.313*** 23141224 x Cancer -0.128** Cancer in caring for others 0.134** 24432411 -0.317*** 24432411 x Cancer -0.148** IBS in themselves -0.036 51224434 -0.456*** 51224434 x Cancer -0.136** IBS in caring for others 0.064 54444444 -0.785*** 54444444 x Cancer -0.254*** Constant 0.967***

278. Discussion Inclusion of condition label can affect health state values, dependent upon specific condition severity of state Experience of condition affects values Why does this occur? Greater precision? Should health state values reflect this difference? We argue not on grounds of comparability, to avoid distortion of values based on prior knowledge or preconceptions

279. Impact of adding a generic dimension

280. Impact of adding generic dimension: rationale Condition-specific measures criticised for inability to capture side-effects or co-morbidities One option is to include additional dimensions in classification system to capture this. i.e. ‘add-on’ What is the impact on utility values? What is the impact on preference weights for existing dimensions? (i.e. in value set or ‘tariff’) What is the impact when using the add-on in the patient population?

281. Study design Add-on pain/discomfort dimension (taken from EQ-5D) to AQL-5D, asthma-specific condition specific measure to produce AQL-6D Valuation studies for 3 level versions of AQL-5D and AQL-6D (states selected using orthogonal array) Respondents valued: 8 health states of differing severity with descriptions produced using AQL-5D or AQL-6D using MVH TTO protocol Sampling strategy to ensure representativeness across measure and to UK general population

282. Example health states Feel concerned about having asthma none of the time Feel short of breath as a result of asthma some of the time Experience asthma symptoms as a result of air pollution none of the time Asthma interferes with getting a good night's sleep all of the time Overall, moderate or some limitation in every activity done due to asthma Feel concerned about having asthma none of the time Feel short of breath as a result of asthma some of the time Experience asthma symptoms as a result of air pollution none of the time Asthma interferes with getting a good night's sleep all of the time Overall, moderate or some limitation in every activity done due to asthma Have extreme pain or discomfort

283. Analysis t tests to determine significant differences in health state values across groups Regression analysis to determine preference weights OLS models on mean level data RE GLS model and RE Tobit Compared coefficients across measures for non-pain variables using the z score test Applied AQL-5D and AQL-6D weights to clinical trial (COGENT examining computerized decision support systems) to examine impact

284. Respondent characteristics AQL-5D (n=91) AQL-6D (n=91) ANOVA p-value Mean age (s.d.) 52.13 (17.54) 50.01 (17.27) 0.41 Female 62% 59% 0.76 Married/partner 70% 76% 0.40 Employed or self-employed 31% 35% 0.53 Unemployed 11% 7% 0.30 Long-term sick 3% 9% 0.12 Full-time student 3% 1% 0.31 Housework 10% 15% 0.27 Retired 36% 29% 0.27 Have asthma 25% 23% Have moderate pain or discomfort 40% 31% Have extreme pain or discomfort 4% 9%

285. Descriptive statistics AQL-5D health state Mean (s.d.) n AQL-6D health state Mean (s.d.) n T test 11111 0.97 (0.14) 60 111111 0.98 (0.07) 61 0.492 12132 0.70 (0.41) 62 121323 0.56 (0.40) 60 0.061 23131 0.64 (0.44) 60 231311 0.78 (0.24) 61 0.034 33333 0.26 (0.53) 91 333333 0.30 (0.48) 91 0.576

286. Regression results AQL-5D AQL-6D Z score Concern2 0.032 0.034 -0.052 Concern3 0.041 0.047** -0.165 Breath2 0.019 -0.001 0.502 Breath3 0.167*** 0.047* 3.177*** Weather2 0.024 -0.016 1.035 Weather3 0.057** 0.033 0.734 Sleep2 0.016 -0.001 0.471 Sleep3 0.106*** 0.091*** 0.431 Activities2 0.074*** 0.040* 0.978 Activities3 0.307*** 0.150*** 4.213*** Pain2 0.071*** Pain3 0.301*** Constant 0.061 0.023

287. Application to trial data AQL-5D AQL-6D Mean (s.d.) Mean (s.d.) n All patients 0.733 (0.188) 0.833 (0.144) 2791 Asthma symptoms score 0<NASS≤20 (least severe) 0.884 (0.087) 0.933 (0.069) 625 20<NASS≤40 0.808 (0.083) 0.893 (0.072) 528 40<NASS≤60 0.759 (0.114) 0.859 (0.090) 583 60<NASS≤80 0.697 (0.149) 0.805 (0.122) 478 80<NASS≤100 (most severe) 0.489 (0.189) 0.655 (0.157) 539 No pain/discomfort 0.817 (0.119) 0.930 (0.060) 1245 Moderate pain/discomfort 0.677 (0.198) 0.787 (0.108) 1373 Extreme pain/discomfort 0.563 (0.226) 0.495 (0.126) 173

288. Discussion Add-on pain/discomfort dimension had significant level coefficients and affected level coefficients of other dimensions Inclusion of add-on affected utilities in trial – utility values higher unless pain/discomfort was at level 3 Why did this occur? Comparability between measures requires that impact of different dimensions on preferences is additive, whether or not they are included in the classification These results cast doubt on this

289. Performance of CSPBMs

290. Examining performance: rationale Increased development and usage of condition-specific preference-based measures (CSPBM) How does CSPBM compare to original measure (CSM) in terms of performance? How does CSPBM compare to generic measure(s) in terms of performance? Original rationale for using CSM is due to its greater relevance and sensitivity, is this true for CSPBM?

291. Analysis Validity: ability to discriminate between patients with different levels of severity defined in terms of their specific condition t tests, effect sizes Responsiveness: sensitivity to change in trial data before and after treatment t tests, effect sizes, standardised response mean, floor and ceiling effects Agreement between measures Pearson correlation coefficient

292. Measures Condition Original measure Preference-based measure Generic EQ-5D Asthma AQLQ AQL-5D Common mental health problems CORE-OM CORE-6D Cancer EORTC QLQ-C30 EORTC-8D

293. Brief summary of results Little evidence of information loss moving from original measure to CSPBM, performed similarly Validity: asthma and cancer measures similar to EQ-5D, mental health measure better Responsiveness: mental health measure better than EQ-5D, reverse for cancer measure All show consistently high agreement Less ceiling effects for CSPBM than EQ-5D, suggesting greater refinement at upper end of health Mean change in utility scores before and after treatment lower for condition-specific measures than EQ-5D

294. Discussion Large amount of resources to develop CSPBM, but may not offer much improvement Mean change actually smaller using CSPBM than EQ-5D Research suggests this is associated with MVH value set Side effects and comorbidities? Comparability with EQ-5D may actually be an advantage if only condition-specific data available

295. Overall conclusions Utility values affected by inclusion of condition labels, we recommend classification systems should avoid labels to avoid distortions by prior knowledge or preconceptions to ensure comparability in economic evaluations across patient groups The role of the classification system is important, as the relationship between dimensions may not be additive Implications for condition-specific measures with narrow focus Implications for generic measures with ‘absent’ dimensions Condition-specific preference-based measures seem to perform comparably to the original measure and to generic EQ-5D Implications for development of future measures Implications for usage if only condition-specific data is available

296. Policy implications The original condition-specific measure used to derive the PBM: must have desirable psychometric properties must include all important dimensions must offer advantage over generic measures Care should be taken as the classification system can affect health state utility values Contrary to what may be expected, CSPBM perform similarly to EQ-5D, with lower mean change but greater refinement at upper end of health

297. Publications Brazier J, Rowen D, Tsuchiya A, Yang Y, Young T (2011) The impact of adding an extra dimension to a preference-based measure. Social Science and Medicine , 73(2), 245-253. Brazier J, Rowen D, Mavranezouli I, Tsuchiya A, Young T, Yang Y, Barkham M, Ibbotson R . Developing and testing methods for deriving preference-based measures of health from condition specific measures (and other patient based measures of outcome). Health Technology Assessment . Forthcoming 2011. Mavranezouli I, Brazier J, Rowen D, Barkham M. The development of a condition-specific preference-based measure for common mental disorders from the Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM) using Rasch analysis. HESG Paper presented at Cork, 23-25 July 2010 Mavranezouli I, Brazier J, Young A, Barkham M. Using Rasch analysis to form plausible health states amenable to valuation: the development of the CORE-6D from a measure of common mental health problems (CORE-OM). Quality of Life Research 2011;20(3):321-33.

298. Publications Rowen D, Brazier J, Tsuchiya A, Young T, Ibbotson R. It's all in the name, or is it? The impact of labelling on health state values. Medical Decision Making 2011;Forthcoming. Rowen D, Brazier JE, Young TA, Gaugrist S, Craig BM, King MT, et al. Deriving a preference-based measure for cancer using the EORTC QLQ-C30. Value in Health, forthcoming. Yang Y, Brazier J, Tsuchiya A, Young T. Estimating a Preference-Based Index for a 5-Dimensional Health State Classification for Asthma Derived From the Asthma Quality of Life Questionnaire. Medical Decision Making 2010;Forthcoming. Young T, Yang Y, Brazier J, Tsuchiya A. The Use of Rasch Analysis in Reducing a Large Condition-Specific Instrument for Preference Valuation: The Case of Moving from AQLQ to AQL-5D. Medical Decision Making 2010;Forthcoming.

299. Deriving a preference-based measure for cancer using the EORTC QLQ-C30 Donna Rowen, John Brazier, Tracey Young, Sabine Gaugris, Benjamin Craig , Madeline King, Galina Velikova

300. Background Cost effectiveness arguments are increasingly important in influencing reimbursement decisions Generic measures are not valid or responsive for many cancer conditions (Garau et al, 2009) Frequently used condition-specific measures, such as EORTC QLQ-C30, have great clinical utility but cannot be used to estimate QALYs To maximise the value of QLQ-C30 data sets (past and future) it is useful to have the capacity to calculate a QALY

301. EORTC QLQ-C30 QLQ-C30 is one of the most commonly used measures in cancer Validity well established for many conditions in cancer 30 questions/items covering: the most common cancer symptoms (such as pain, fatigue, nausea and vomiting) various aspects of function (including physical, role, social, emotional and cognitive functioning) QLQ-C30 is summarised using fourteen scales, plus one global quality of life scale

302. Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors

303. Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor

304. Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension

305. Stage IV Validation –stages I to IV repeated on another time period Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension

306. Stage IV Validation –stages I to IV repeated on another time period Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage V Valuation study to elicit general population TTO values by interview for a sample of health states

307. Stage IV Validation –stages I to IV repeated on another time period Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage V Valuation study to elicit general population TTO values by interview for a sample of health states Stage VI Model TTO values to produce utility values for all health states defined by classification system

308. 27/10/11 © The University of Sheffield Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors

309. Stage I – Establishing factors Dataset used is screening phase of Multiple Myeloma trial, n =655 Used factor analysis to identify structurally independent factors across 27 items (excluding financial impact and global QoL) We used expert judgement alongside these factor and other psychometric analyses Findings were analysed to establish whether they were in accordance with the QLQ-C30 scaling

310. Stage I – Establishing factors The 27 items were grouped into the 4 identified factors: Physical functioning, role functioning, social functioning, pain and items ‘need a rest’, ‘felt weak’ and ‘difficulty concentrating’ Emotional functioning plus ‘trouble sleeping’ All items covering eating and digestion Items ‘short of breath’, ‘were you tired’ and ‘difficulty remembering’ Undertook second factor analysis on factors 1 and 4 (subset of 18 items)

311. Factors used for Stage II Factors included: Physical functioning, role functioning and pain Social functioning Emotional functioning Nausea, vomiting, constipation, diarrhea Fatigue and trouble sleeping Factors excluded: Cognitive functioning Shortness of breath Excluded due to necessary prioritisation of factors showing greater change, greater problems and greater responsiveness with symptoms that were unlikely to be captured through the other dimensions

312. 27/10/11 © The University of Sheffield Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor

313. Stage II – Selecting items and dimensions Used psychometric analysis, Rasch analysis and expert judgement to select items and dimensions Psychometric analysis: missing data, ceiling effects, floor effects, correlation with domain score, distribution of responses and responsiveness over time (using SRM) Rasch analysis: item level ordering, differential item functioning, model fit, item range and spread across latent space

314. Stage II - Considerations All within the overall constraint of producing a classification that generates states amenable to valuation To make the valuation easier factors have been separated into their separate attributes these form the dimensions for the classification system

315. Chosen dimensions and items Dimension Items Question Physical functioning 2+3 Trouble taking a long walk with extra level added from item 3, trouble taking a short walk Role functioning 7 Limited in pursuing hobbies or other leisure time activities Pain 19 Pain interfere with daily activities Social functioning 27 Physical condition or medical treatment interfered with social life Emotional functioning 24 Felt depressed Nausea 14 Felt nauseated Constipation and diarrhea 16+17 Constipated and diarrhoea merged Fatigue and trouble sleeping 18 Tired

316. Health state classification system Physical functioning: You had no / a little / quite a bit / very much trouble taking a long walk / very much trouble taking a short walk outside of the house Role functioning: You were not limited / limited a little / quite a bit / very much in pursuing your hobbies or other leisure time activities Pain: Pain did not interfere / interfered a little / quite a bit / very much with your daily activities Social functioning: Your physical condition or medical treatment did not interfere / interfered a little / quite a bit / very much with your social activities Emotional functioning: You did not feel depressed / felt depressed a little / quite a bit / very much Nausea: You did not feel nauseated / felt nauseated a little / quite a bit / very much Constipation and diarrhea: You were not constipated and did not have diarrhea / were constipated and/or had diarrhea a little / quite a bit / very much Fatigue and sleep disturbance: You were not tired / tired a little / quite a bit / very much

317. 27/10/11 © The University of Sheffield Stage IV Validation –stages I to II repeated on another time period

318. Stage IV - Validation Selections were validated using data at cycle 5 of treatment (n=471) in the trial Cycle was chosen as: responses have changed (standardised response mean across 27 EORTC items = 0.104) some respondents have side effects problem of attrition is reduced in comparison to other treatment cycles

319. 27/10/11 © The University of Sheffield Stage V Valuation study to elicit general population TTO values by interview for a sample of health states

320. Valuation survey 350 interviews were conducted using members of the UK general population A health state is made up of 8 sentences across 5 dimensions, generating a total of 81,920 possible health states Each respondent valued 8 health states using time-trade off (TTO) and ranking (in accordance with the NICE reference case) 85 states were valued (81 states generated using orthogonal array and 4 extra states added)

321. 27/10/11 © The University of Sheffield Stage VI Model TTO values to produce utility values for all health states defined by classification system

322. Modelling the TTO data Elicited values are only for a subset of states – we need values for every state Data was analysed using a variety of models: OLS, RE MLE, Mean model, Episodic RUM Standard model defined as: Dependent variable, y, is TTO disvalue (1–TTO value) for health state i =1,2 … n valued by respondent j =1,2… m X is a vector of dummy explanatory variables for each level λ of dimension δ of the health state classification

323. Preference weights – ERUM OLS pf2 0.052 pf3 0.077 pf4 0.103 pf5 0.104 rf2 0.044 rf3 0.050 rf4 0.076 pain2 0.054 pain3 0.064 pain4 0.070 ef2 0.032 ef3 0.053 ef4 0.132 sf2 0.029 sf3 0.046 sf4 0.132 fat2 0.038 fat3 0.052 fat4 0.084 nau3 0.025 nau3 0.027 nau4 0.052 cd2 0.011 cd3 0.035 cd4 0.059 R-squared = 0.56 Inconsistencies = 0 Significant coefficients = 22/25 MAE = 0.046 MAE>0.05 = 33 MAE>-0.10 = 6

324. Examples You had a little trouble taking a long walk You were limited a little in pursuing your hobbies or other leisure time activities Your physical condition or medical treatment interfered a little with your social activities Pain interfered a little with your daily activities You felt a little depressed You were a little tired You were constipated and/or had diarrhoea a little You felt a little nauseated You had very much trouble taking a short walk outside of the house You were limited very much in pursuing your hobbies or other leisure time activities Your physical condition or medical treatment interfered very much with your social activities Pain interfered very much with your daily activities You felt depressed very much You were tired very much You were constipated and/or had diarrhoea very much You felt nauseated very much Utility value = 0.715 Utility value = 0.291

325. Stage IV Validation –stages I to IV repeated on another time period No item level reduction required Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage V Valuation study to elicit general population TTO values by interview for a sample of health states Stage VI Model TTO values to produce utility values for all health states defined by classification system 24 items across 5 factors Classification system of 8 dimensions using 10 QLQ-C30 items 27 EORTC QLQ-C30 items

326. Discussion This study has shown the feasibility of applying this approach to the QLQ-C30, but: Will the health state classification look the same across all cancer conditions ? Does this approach need to be extended to disease specific modules of the EORTC? Will the valuations be the same across countries? Can the preference-based QLQ-C30 be used to inform cross programme resource allocation decisions? Study forms part of a wider cross-country study that will examine the use of PBMs from the QLQ-C30 in a variety of countries and different patient groups

327. References Garau M, Shah K, Towse A, et al. Assessment and appraisal of oncology medicines: does NICE's approach include all relevant elements? What can be learnt from international HTA experiences? Report for the Pharmaceutical Oncology Initiative (POI) 2009. Rowen D, Brazier JE, Young TA, Gaugris S, Craig BM, King MT, Velikova, G. Deriving a preference-based measure for cancer using the EORTC QLQ-C30. Value in Health , 14(5), 721-731. For further details see:

328. Deriving preference based indices for dementia for use in economic evaluation Brendan Mulhern 1 , Donna Rowen 1 , John Brazier 1 , Sarah Smith 2 , Martin Knapp 34 , Donna Lamping 2 , Vanessa Loftus 4 , Tracey Young 1 , Robert Howard 4 , Sube Banerjee 4 1 Health Economics and Decision Science, University of Sheffield 2 London School of Hygiene and Tropical Medicine 3 London School of Economics and Political Science 4 Institute of Psychiatry, London Contact: b.mulhern@sheffield.ac.u k

329. Contents Introduction Development of dementia specific health state classification systems for patient and carer report Deriving utility values for each classification system Discussion and limitations 27/10/11 © The University of Sheffield

330. Measurement in dementia Total care and health expenditure in UK £50 billion by 2050 Economic evaluation of emerging treatments in dementia requires valid measurement tools Debate about validity of generic preference based measures (e.g. EQ-5D) in dementia Differences in patient and proxy report Generics missing relevant HRQL dimensions (e.g. Cognition) 27/10/11 © The University of Sheffield

331. Dementia specific utility measures Dementia-specific utility measures developed for both self and proxy report from DEMQOL system Provide dementia specific utility values Address concerns about using generic measures DEMQOL system (Smith et al., 2007) DEMQOL (28 items + global health item) DEMQOL-Proxy (31 items + global health item) Investigates HRQL related to dementia Interviewer administered Reliable and valid for use across all dementia severity levels 27/10/11 © The University of Sheffield

332. 27/10/11 © The University of Sheffield Step V Valuation exercise to elicit TTO health state values Step VI Model TTO valuation results to produce utility values Methodology and development process Step I Establish dimensions using Exploratory Factor Analysis Step II Eliminate items per dimension using Rasch analysis Step III Select items per dimension using Rasch analysis and input from clinicians and DEMQOL system developers Step IV Exploration of item level reducing per dimension

333. Step I: DEMQOL dimensionality Exploratory factor analysis applying a range of models with 2-10 factors Mild/moderate patient data collected in clinical service (n=644) Five factor model used for classification system Explains 45.5% of the variance 4 items do not load on any factor so removed from classification system at this stage 27/10/11 © The University of Sheffield

334. Step I: DEMQOL-Proxy dimensionality Exploratory factor analysis applying a range of models with 2-10 factors Mild/moderate carer data collected in clinical service (n=683) Five factor model used for classification system Explains 49.3% of the variance 8 items do not load on any factor so removed from classification system at this stage 27/10/11 © The University of Sheffield

335. Step II: DEMQOL item elimination methods Item elimination criteria: Response ordering Differential item functioning Uniform Non uniform Fit to Rasch model Fit residuals Chi squared statistic 27/10/11 © The University of Sheffield Item with disordered response levels Item displaying DIF

336. Step II/III: DEMQOL item selection methods Item selection criteria: Spread at logit 0 Item range (severity) Fit to the Rasch model Overall dimension Item fit Person fit 27/10/11 © The University of Sheffield Item with ordered response levels Graph displaying spread at logit 0

337. 27/10/11 © The University of Sheffield Step II/III: DEMQOL item elimination and selection Dimension Eliminated Item selected Fit residual Chi Severity range Spread at logit 0 Cognition Fit: 1 item Worry about forgetting things that happened recently -0.158 0.740 -0.769 to 2.037 0.12 to 0.68 Negative emotion DIF: 1 item Felt frustrated -0.626 0.155 -0.753 to 1.348 0.21 to 0.68 Positive emotion No items Felt cheerful 1.208 0.165 -3.104 to 2.427 0.08 to 0.96 Social relationships Order: 3 items Fit: 1 item Worry about making yourself understood -0.087 0.472 -0.205 to 1.280 0.22 to 0.55 Loneliness No items Felt lonely 0.293 0.256 -1.044 to 2.309 0.09 to 0.74

338. 27/10/11 © The University of Sheffield Step II/III: DEMQOL-Proxy item elimination and selection Dimension Eliminated Item selected Fit residual Chi Severity range Spread at logit 0 Cognition DIF: 1 item Fit: 4 items Worry about forgetting people’s names -0.628 0.211 -0.947 to 0.324 0.42 to 0.72 Negative emotion DIF: 1 item Felt frustrated 1.126 0.695 -1.007 to 1.923 0.15 to 0.76 Daily activities Order: 2 items Fit: 1 item No item selected n/a n/a n/a n/a Positive emotion Fit: 1 item Felt lively 0.629 0.735 -3.243 to 3.276 0.04 to 0.96 Appearance No items Worry about keeping yourself looking nice 0.903 0.938 -1.630 to 1.452 0.19 to 0.84

339. Overall goodness of fit of dimensions 27/10/11 © The University of Sheffield

340. Step IV: Item level reduction Item level reduction investigated for selected items Examine: Threshold probability curves Distribution of responses across item levels Original four level response categories used 27/10/11 © The University of Sheffield

341. DEMQOL-U classification system COGNITION 1. I do not worry at all about forgetting things that happened recently 2. I worry a little about forgetting things that happened recently 3. I worry quite a bit about forgetting things that happened recently 4. I worry a lot about forgetting things that happened recently NEGATIVE EMOTION 1. I do not feel frustrated at all 2. I feel frustrated a little 3. I feel frustrated quite a bit 4. I feel frustrated a lot POSITIVE EMOTION 1. I feel cheerful a lot 2. I feel cheerful quite a bit 3. I feel cheerful a little 4. I do not feel cheerful at all RELATIONSHIPS 1. I do not worry at all about making myself understood 2. I worry a little about making myself understood 3. I worry quite a bit about making myself understood 4. I worry a lot about making myself understood LONELINESS 1. I do not feel lonely at all 2. I feel lonely a little 3. I feel lonely quite a bit 4. I feel lonely a lot 27/10/11 © The University of Sheffield

342. DEMQOL-Proxy-U classification system COGNITION 1. I do not worry at all about forgetting what day it is 2. I worry a little about forgetting what day it is 3. I worry quite a bit about forgetting what day it is 4. I worry a lot about forgetting what day it is NEGATIVE EMOTION 1. I do not feel frustrated at all 2. I feel frustrated a little 3. I feel frustrated quite a bit 4. I feel frustrated a lot POSITIVE EMOTION 1. I feel lively a lot 2. I feel lively quite a bit 3. I feel lively a little 4. I do not feel lively at all APPEARANCE 1. I do not worry at all about keeping myself looking nice 2. I worry a little about keeping myself looking nice 3. I worry quite a bit about keeping myself looking nice 4. I worry a lot about keeping myself looking nice 27/10/11 © The University of Sheffield

343. 27/10/11 © The University of Sheffield Step V Valuation exercise to elicit TTO health state values Step VI Model TTO valuation results to produce utility values Methodology and development process Step I Establish dimensions using Exploratory Factor Analysis Step II Eliminate items per dimension using Rasch analysis Step III Select items per dimension using Rasch analysis and input from clinicians and DEMQOL system developers Step IV Exploration of item level reducing per dimension

344. Step V: Valuation study (defining health states) Health states generated using DEMQOL-U and DEMQOL-Proxy-U classification Generated using one level of each dimension . 27/10/11 © The University of Sheffield

345. Step V: Valuation study (state selection) DEMQOL-U: 1024 states DEMQOL-Proxy-U: 256 states Each respondent valued 8 states using Time Trade Off (TTO) 7 mixed states and worst state States selected using simulation DEMQOL: 87 unique states across 13 blocks DEMQOL-Proxy: 70 unique states across 12 blocks 27/10/11 © The University of Sheffield

346. Step V: Valuation study (sample) Representative sample of UK general population. NICE and Washington Panel recommend general population values for QALY estimates Sample 310 for DEMQOL-U 290 for DEMQOL-Proxy-U Obtained by sampling 600 households in urban and rural areas in North England 27/10/11 © The University of Sheffield

347. Step V: Valuation study (procedure) Interview process: EQ-5D and DEMQOL/DEMQOL-Proxy classification system completed for own health 8 health states with ‘full health’ and ‘dead’ ranked in order from best to worst. Practice TTO task Value 8 health states Matched methods with UK valuation of EQ-5D Respondents not informed that health states dementia specific 27/10/11 © The University of Sheffield

348. Step V: Valuation study (DEMQOL-U results) Mean values for each state Histogram of observed values 27/10/11 © The University of Sheffield

349. Step V: Valuation study (DEMQOL-Proxy-U results) Mean values for each state Histogram of observed values 27/10/11 © The University of Sheffield

350. Step VI: Modelling to produce utility values (methods) Mean and individual level multivariate regression used to estimate utility decrements Ordinary least squares (individual/mean level) Random and fixed effects to account of repeated observations Random effects Tobit model to take account of TTO observations at 1 Performance indicators (model): No. of inconsistent coefficients No. of significant coefficients Performance indicators (predictions): Root mean squared error (RMSE) and mean absolute error (MAE) at state level No of states with absolute error (AE) greater than 5% and 10% Plots of actual and predicted health state values 27/10/11 © The University of Sheffield

351. Step VI: Modelling to produce utility values (DEMQOL-U results) Model 2 selected 12 significant variables No inconsistent coefficients Good predictive ability low RMSE Lowest percentage of absolute errors at the state level greater than 0.05 and 0.10. No systematic bias in the predictions by severity, but some health states had large prediction errors. Range 0.986 (best state) to 0.243 (worst state), 27/10/11 © The University of Sheffield

352. Step VI: Modelling to produce utility values (DEMQOL-Proxy-U results) ‘ Appearance’ inconsistent in all models Health worsens but predicted utility increases Resolved by merging adjacent inconsistent levels and re-estimating the model to Model 6 selected due to higher number of significant coefficients. Range 0.937 (best state) to 0.363 (worst state) 27/10/11 © The University of Sheffield

353. Discussion (1) Possible to generate dementia specific utility scores when DEMQOL/DEMQOL-Proxy completed Utility scores used as the ‘Q’ quality adjustment weight of the QALY DEMQOL-U and DEMQOL-Proxy-U enable utilities to be generated across full dementia severity range Dementia-specific measures and utility scores designed for a cognitively impaired population Offer potential advantage over generic measures Address some of the concerns around using generic measures in dementia First attempt to derive a PBM for proxy-report Complementary but key differences Use both measures in tandem 27/10/11 © The University of Sheffield

354. Discussion (2) DEMQOL has interpretable domains amenable to the generation of health state classification systems representing HRQL in dementia Psychometric and Rasch techniques valid method of generating health states for a specific mental disorder Do different methods select different items and if so which are most representative of the condition-specific HRQL issues? Areas of HRQL are not represented in the classification system (e.g. daily activities) The results have not been validated on another dataset and item responsiveness has not been tested TTO used but other valuation techniques (standard gamble, discrete choice) may produce different utility values Comparability with EQ-5D? 27/10/11 © The University of Sheffield

355. Mapping generic quality of life measurements to disease specific variables in patients with Ankylosing Spondylitis R.M. Ara, J.Brazier University of Sheffield 27/10/11 © The University of Sheffield

356. Introduction The disease specific instruments the Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) and Bath Ankylosing Spondylitis Functional Index (BASFI) are frequently used to measure disease progression and response to treatment in patients with ankylosing spondylitis (AS). However, economic evaluations demand health related benefits to be measured in terms of quality adjusted lifeyears. These require utility measurements which can be generated using generic instruments such as the EQ-5D (EuroQol) or the SF-6D 27/10/11 © The University of Sheffield

357. Objective To identify which QoL instrument (EuroQol or SF-36) is the most appropriate to measure health state utility values in patients with AS. To derive a relationship between BASDAI and BASFI variables and the preference based index.

358. Methods Data on the disease specific (BASDAI and BASFI) and HRQoL (EQ-5D and SF-36) variables were collected during a multi-centre European etanercept (ETN) RCT (n=356) Differences in the utility scores resulting from the two utility measures were explored through graphical measures (box and scatter plots examining the data at the extremes of the distributions). Data was analysed using STATA (version 8( and SPSS (version 12) with significance set at P < 0.001. Linear regressions were used to explore a possible relationship between BASDAI, BASFI and health state utility values. 27/10/11 © The University of Sheffield

359. Exploration of data Table 1: Correlations between the SF-6D, EQ-5D, BASDAI and BASFI 27/10/11 © The University of Sheffield

360. Direct comparison of the SF-6D and EQ-5D scores The pattern of correlation between SF-6D and EQ-5D preference based indices are not linear A ceiling effect in the EQ-5D data where a wide range in SF-6D indices correspond to EQ-5D = 1 A floor effect in the SF-6D where a wide range in EQ-5D indices correspond to low SF-6D indices ) The EQ-5D indices cluster with gaps between EQ-5D = 1 and those < 1, and around EQ-5D = 0.45 (caused by the N3 term) 27/10/11 © The University of Sheffield Scatter plot of paired SF-6D and EQ-5D utility scores

361. Comparing dimensional scores There is evidence of correlation between similar dimensions: mobility with physical function; and pain/discomfort with body pain. The lowest correlations are observed between self care with both mental health and vitality. All patients reported 1 for the anxiety/depression question on the EQ-5D questionnaire . 27/10/11 © The University of Sheffield Correlations between the 8 overall dimensional scores in the SF-36 (scale 0-100 where 0 is worse) and the 5 scores from the EQ-5D (scale 1-3 where 3 is worse)

362. Predicting health state utility values The correlations between the BASDAI and BASFI and EQ-5D utility measurements are reasonably strong at -0.69 and -0.70 respectively. Using the BASDAI and BASFI as the independent variables and the EQ-5D as the dependent variable in a linear regression; the disease specific measurements explained 52% of the variance in utility. The model predicts health state utility values over the range 0.09 to 0.92, corresponding to BASDAI and BASFI measurements of 100 and 0 respectively. Utility = 0.920 - 0.004*BASFI - 0.004*BASDAI (p<0.001; MAE=0.13) 27/10/11 © The University of Sheffield Using BASDAI & BASFI to predicted health state utility values

363. Discussion/Conclusion There are large differences in the EQ-5D and SF-6D health state utility values The differences in the health state utility values are comparable with other results [7] The results supports evidence which suggests utility values derived using different instruments in spine patients are not directly comparable [6]. The EQ-5D health state utility values differ from those observed by Maetzel et al . [8] but are similar to those reported by Kobelt et al . [9] and Boonen et al .[10] The SF-6D is likely to underestimate the magnitude of improvement in quality of life for patients starting at the lower end of the scale The EQ-5D may be more sensitive to changes in patients with severe AS and thus may be the most appropriate instrument for measuring HRQoL in patients with severe AS

364. Deriving an algorithm to convert the 8 mean SF-36 dimension scores into a mean EQ-5D preference-based score from published studies when patient level data are not available Roberta Ara and John Brazier ScHARR, University of Sheffield 27/10/11 © The University of Sheffield

365. Introduction Policy decision makers like NICE recommend generic preference-based measures such as EQ-5D and HUI are used to value health related quality of life (HRQoL) in economic evaluations. Many important research studies do not use a generic preference-based measures. One approach to tackle this is to map non-preference based measure onto a generic preference-based measure, such as the EQ-5D. Existing mapping functions have been estimated and tested in individual level data, but researchers may not have access to individual level data. This presentation presents a mapping function designed and tested for use in sub-groups of patients.

366. Objective To derive a method to predict a mean sub-group EQ-5D preference-based index score using published mean statistics of the eight dimension scores describing the SF-36 health profile.

367. Methodology DATASETS: Individual patient level data (n=6,350) pooled from 12 studies covering a wide range of health conditions are used to explore relationships between the 8 dimension scores of the SF-36 and the preference-based EQ-5D index. STATISTICAL METHODS: The EQ-5D index is regressed onto the eight health dimension scores using ordinary least square (OLS) regressions. The general model is defined as: EQ =  +  1 x i +  2 d i +  3 r i +  whereby EQ represents the EQ-5D preference-based index, x the vector of main effects (PF, RP BP, GH, VT, SF, RE, MH), d the vector of demographs (age and gender), r the vector of main effect squares, i individual respondents and   the stochastic error term of the regression (the residual).

368. Results

369. Out-of Sample Cohort Results

370. Goodness of fit using out-of sample summary statistics

371. Discussion/Conclusion The models provide analysts with a mechanism to estimate EQ-5D utility data from published mean SF-36 dimension scores for use in economic analyses. The errors observed at the individual level still exist when applying the algorithms to summary data, but the magnitude of the errors is much smaller. We advocate Model EQ(1) as the first choice for predicting mean EQ-5D scores from mean dimension SF-36 scores when patient level data is not available. If comparing differences between study arms or changes over time, Model EQ(2) is the preferred choice. Caution should be taken when applying the algorithm to conditions: - likely to have very low mean utility scores - where energy is an important symptom Further research is required to validate the results in other health conditions.

372. Predicting the SF-6D preference-based index using mean SF-6D health dimension scores: approximating health related utilities when patient level data are not available John Brazier and Roberta Ara ScHARR, University of Sheffield 27/10/11 © The University of Sheffield

373. Introduction The SF-6D preference-based index can be derived from individual level SF-36 data to calculate QALYs for use in economic evaluation. However, SF-36 results are usually published in terms of profile of 8 mean dimension scores. As patient level data is required to derive the SF-6D (i.e. 11 items), a substantial proportion of this evidence cannot be used in cost-effectiveness models that rely on a synthesis of published evidence. 27/10/11 © The University of Sheffield

374. Objective To derive an algorithm to predict the preference-based SF-6D index using the eight mean health dimension scores from the SF-36 (v2) when patient level data are not available. 27/10/11 © The University of Sheffield

375. Methodology DATASETS: Individual patient level data (n=6,890) is pooled from 15 studies covering a wide range of health conditions STATISTICAL METHODS: The SF-6D index is regressed onto the eight health dimension scores using ordinary least square (OLS) regressions. The general model is defined as: SF =  +  1 x i +  2 d i +  3 r i +  whereby SF represents the SF-6D preference-based index; x the vector of main effects (the 8 dimension scores PF, RP, BP, GH, VT, SF, RE, MH); d the vector of demographic characteristics (age and gender); r the vector of first order interactions between the main effects (i.e. the eight squares and 28 products of the main effects);   vectors of unknown parameters; i individual respondents, and the stochastic error term of the regression (the residual). 27/10/11 © The University of Sheffield

376. Regression Results 27/10/11 © The University of Sheffield

377. Validation: Using mean scores to predict subgroup values 27/10/11 © The University of Sheffield

378. Conclusion This study presents a mechanism to estimate preference-based SF-6D scores from the eight mean dimension scores derived from the SF-36 questionnaire. This study is unique in that it looks at predicting a preference-based SF-6D score using published dimension scores. This will be very useful to those seeking to populate cost effectiveness models from a synthesis of published evidence (e.g. as required for submissions to NICE). Further research is required to validate the results in different health conditions. 27/10/11 © The University of Sheffield

379. Health and well-being research in ScHARR John Brazier Professor of Health Economics ScHARR, University of Sheffield Email: j.e.brazier@sheffield.ac.uk

380. Health technology assessment HTA is concerned with the cost-effectiveness of different healthcare treatments Does the technology work? If so, who for? At what cost? How does it compare with alternative treatments? © The University of Sheffield

381. Health technology assessment HTA is concerned with the cost-effectiveness of different healthcare treatments Does the technology work? If so, who for? At what cost? How does it compare with alternative treatments? © The University of Sheffield Measure of outcome that is comparable across treatments and patient groups QALY

382. QALY QALY combines both quantity and quality of life into a single measure QALY can be used across all health care interventions for all patient groups

383. Health utility measurement Health utility measurement provides the quality adjustment weight, ‘Q’, for the QALY with two components: Health state description Corresponding utility value Example: SF-6D (others include EQ-5D and HUI3)

384. SF-6D Physical Functioning Your health does not limit you in vigorous activities …….  Your health limits you a little in vigorous activities ……....  Your health limits you a little in moderate activities …......  Your health limits you a lot in moderate activities …….....  Your health limits you a little in bathing and dressing ......  Your health limits you a lot in bathing and dressing ……...  Role Limitations You have no problems with your work or other regular daily activities as a result of your physical health or any emotional problems.......…………..  You are limited in the kind of work or other activities as a result of your physical health……  You accomplish less than you would like as a result of emotional problems……...…...........  You are limited in the kind of work or other activities as a result of your physical health and accomplish less than you would like as a result of emotional problems……  Social Functioning Your health limits your social activities none of the time …...  Your health limits your social activities a little of the time .....  Your health limits your social activities some of the time …..  Your health limits your social activities most of the time …...  Your health limits your social activities all of the time ……....  Pain You have no pain……...…………………………  You have pain but it does not interfere with your normal work (both outside the home and housework) ……..  You have pain that interferes with your normal work (both outside the home and housework) a little bit …………  You have pain that interferes with your normal work (both outside the home and housework) moderately …........  You have pain that interferes with your normal work (both outside the home and housework) quite a bit ………...  You have pain that interferes with your normal work (both outside the home and housework) extremely ………...  Mental Health You feel tense or downhearted and low none of the time ......  You feel tense or downhearted and low a little of the time .....  You feel tense or downhearted and low some of the time …..  You feel tense or downhearted and low most of the time …...  You feel tense or downhearted and low all of the time ………  Vitality You have a lot of energy all of the time …….....  You have a lot of energy most of the time ........  You have a lot of energy some of the time …...  You have a lot of energy a little of the time …..  You have a lot of energy none of the time ……  The following questions ask about different aspects of your health. There are six groups of statements, each covering a different aspect of health. Please tick one statement in each group to show the statement which best describes your health over the past 4 weeks.

385. SF-6D scoring algorithm Physical functioning PF23 -0.035 PF4 -0.044 PF5 -0.056 PF6 -0.117 Social functioning SF2 -0.057 SF3 -0.059 SF4 -0.072 SF5 -0.087 Pain PN23 -0.042 PN4 -0.065 PN5 -0.102 PN6 -0.171 Role limitation RL 234 -0.053 Mental health MH 23 -0.042 MH4 -0.100 MH5 -0.118 Vitality V234 -0.071 V5 -0.092 Interaction term MOST -0.061 (if at 3 most serious levels for PF or at 2 most serious levels for any other dimension)

386. Development of preference-based measures SF-6D from SF-36 Valuation studies conducted in Australia, Brazil, Hong Kong, Japan, Portugal, Singapore, UK Generic child-specific measure, CHU-9D Condition-specific QALY measures for asthma, overactive bladder, cancer, sexual functioning, paediatric atopic dermatitis, flushing Enables data collected in study to be used to populate cost-effectiveness models ‘ Bolt-on’ measures for EQ-5D (and SF-6D) for when a gap is identified the generic Finally broader ‘wellbeing’ measures

387. Valuation methods Currently time trade-off method is complex and biased by time preference Developing simpler ordinal methods (e.g. Pair wise tasks – do you prefer state A or B?) Use of willingness to pay to capture non-health benefits NICE currently uses general population values, but open to criticisms Current research examining use of patient and carer values vs. general population © The University of Sheffield

388. Use of preference-based measures What to do when EQ-5D is not used? Estimate regression models between measure used in trial and generic measure like EQ-5D (i.e. ‘Mapping’ or ‘cross walking’) When EQ-5D is inappropriate (i.e. Not sensitive or relevant to population)? Examples include in vision, hearing, cancer, mental health, skin Literature reviews and synthesis of utility values Undertaken in vision, breast cancer, osteoporosis Use in cost effectiveness models

389. Beyond health effects Estimation of utility impact on carers and family of children e.g. Study in ADHD for Shire Can involve primary data collection Productivity effects – the impact of health on the productivity Carer effects – impact of carer costs, leisure time and productivity Is a QALY and QALY regardless of who gets it – work for DH on weighting QALYs for the Burden of Illness (for UK DH Policy makers) Well-being QALY

390. Key recent studies Use of generic and condition specific measures in NICE decision making Medical research Council A ssessing the Impact of Children with ADHD on the health and wellbeing of their families Shire Pharmaceuticals Inc. V alidating generic preference-based measures of health in mental health populations and estimating mapping functions for widely used specific measures Medical Research Council Estimation of a preference-based single index measure of health form the EORTC QLQ-C30 for use in multiple myeloma patients Janesson-Cilag Ltd Development and testing of methods for deriving preference-based measures from condition specific measures National Institute of Health Research (NIHR) Health Technology Assessment (HTA) Methods Programme A review and meta analysis of utility values in breast cancer Pfizer Ltd. An update of a systematic review of health state utility values for Osteoporosis-related conditions Lilly UK Developing a broader measure for economic evaluation MRC For further information see http://www.sheffield.ac.uk/scharr/sections/heds/mvh

391. The Development of the Child Health Utility 9D (CHU9D) and other Paediatric Preference Based Measures Dr Katherine Stevens School of Health and Related Research The University of Sheffield, UK

392. Overview Background – economic evaluation and QALYs Issues in child QALY measures Current options Development of the CHU9D Discussion

393. Economic evaluation Quality adjusted life years (QALYS)

394. Economic evaluation Scarce resources and competing needs Health care resource allocation decisions Economic evaluation can help Compare the difference in costs and benefits of alternatives

395. Economic evaluation Different types of economic evaluation Main difference is benefit measurement In cost utility analysis benefits are typically measured in quality adjusted life years - QALYs

396. Quality adjusted life years (QALYs) Combine length of life and quality of life into a single measure Increased survival rates e.g. cancer, premature birth, heart conditions Impact on HRQoL not just length of life Increase either or both Most useful type for resource allocation decisions

397. How do we calculate QALYs? Collect information on length of life Quality adjustment where 0 =dead and 1=perfect health E.g. 3 years at 0.8 = 3*0.8=2.4 QALYs How do we get this quality adjustment?

398. Off the shelf measures Off the shelf measures are commonly used in clinical trials and health care research studies They consist of a descriptive system and a set of preference weights which assign a preference weight to each health state These preferences are often based on the adult general population Adult and child QALY measures

399. Issues in child QALY measures

400. What should be included in the descriptive system? Dimensions that are relevant to the purpose of the measure Health care resource allocation decisions Quality of life vs health related quality of life e.g. Items on toys, housing, environment Dimensions that are relevant to children

401. Why are children different?

402. Why are children different? Differences not just in language but at the conceptual level Dimensions that are relevant for children but not for adults e.g. school Dimensions that are not relevant for children but are relevant for adults e.g. work, sexual relationships

403. Age range Teenagers, younger children, babies Confounding with development in younger children, particularly under 5 years of age HRQoL measures may demonstrate an improvement but this could be due to natural development, not to an improvement in QoL

404. Taken from the HUI2 Unable to walk at all Required the help of another person to eat, bathe, dress or use the toilet

405. Who provides the information? Growing recognition children have their own unique views and a right to express them in matters affecting them Increasingly recognized in clinical trials and HSR that descriptions of the experience of a health state should be elicited from the patients – to reflect the actual experience of the disease and its treatment

406. Whose preferences? Publicly funded health care system Tax payers Adults as rational and informed individuals Adult general population vs patients Children?

407. Current options available

408. What exists? Currently 3 existing preference based measures available and 1 in development The Health Utilities Index 2 (HUI2) The AQoL 6D The EQ 5D Y The Child Health Utility 9D (CHU9D)

409. The HUI2 The Health Utilities Index 2 6 dimensions,( sensation, mobility, emotion, cognition, self care and pain ) each with 4-5 levels Application: generic Age range: 5 + years Mode of completion: self, proxy, administered Recall period: 1,2, 4 weeks or usual self

410. HUI2 example health state Sees, hears, or speaks with limitations even with equipment Requires mechanical equipment (such as canes, crutches, braces, or wheelchair) to walk or get around independently. Occasionally fretful, angry, irritable, anxious, depressed, or suffering “night terrors”. Learns and remembers very slowly and usually requires special educational assistance. Requires mechanical equipment to eat, bathe, dress, or use the toilet independently. Occasional pain. Discomfort relieved by non prescription drugs or self-control activity without disruption of normal activities.

411. The AQoL 6D 6 dimensions, (Independent Living, Mental Health, Coping, Relationships, Pain, Senses) Application: generic Age range: adolescents Mode of completion: self completion Recall period: current

412. AQoL 6D Example Health State Physical ability Excellent Social and family My close friendships make me generally relationships unhappy. There are some group activities I am not involved in because of my health. Mental health I usually feel sad, I often feel worried, I am sometimes calm and sometimes agitated Coping Excellent Pain I suffer from severe pain. Pain often interferes with my usual activities. Vision, hearing & Excellent Communication

413. The EQ-5D Y Same dimensions as the original adult EQ 5D Empirical work done with children to alter language but assumption that concepts remain the same Further research needed to determine if descriptive system complete No preference weights (yet)

414. Development of the CHU9D

415. The Child Health Utility 9D Other instruments - Content generated from a mixture of literature reviews, expert opinion and interviews with relevant populations -parents and paediatricians Adaptation from adult instruments Evidence that parent’s views are affected by their own health status, knowledge, experience and expectations Development of a child centred measure

416. Development of the descriptive system Aim – to find out how health affects children’s lives 1 to 1 interviews with 70 children age 7-11 years with a wide range of acute and chronic health problems to find out how they say health affects their lives. interviews with children at 2 primary schools in Sheffield Sampled purposively on basis of health and balanced for gender, age and ethnicity

418. Interviews Topic guide Tell me about your health/health problems. How does this affect you (probing for school, home…)

419. Forming dimensions WORRYING What was going to happen to them Health would get worse Would always have the health problem Physical symptoms I felt like really, really worried, worry me, like, um it felt really scratchy and itchy just really worried like it’s never gonna stop and it’s never gonna go away. (Female, 8, fair health)

420. SLEEP Symptoms – coughing, sick, itching Worrying Medication – broken sleep Yeah I couldn’t sleep cos it really hurt my throat when I slept, so I couldn’t sleep at all. (B110, female, 11, good health) In the night I wake up because I’m scratching it. And how does that feel? Horrible because I can’t get back to sleep. (B28, male, 9, excellent health)

421. Dimension 7-9 Years 9-11 Years 1 Worried Scared Worried 2 Sad Upset Sad Upset Unhappy Miserable 3 Annoyed Frustrated Annoyed Frustrated Angry 4 Hurt Pain Hurt Pain 5 School work Learning 6 Daily Routine Daily Routine 7 Tired Weak Drowsy Tired Weak Energy Weary 8 Joining in activities that want to Joining in activities that want to 9 Sleep Sleep 10 Embarrassed 11 Jealous

422. Developing levels/scales Development of levels/scales using the data It’s just really frustrating I just feel really frustrated It was really annoying I was really annoyed it’s quite annoying really annoys me they just kind of annoy me a bit . a bit annoyed. it still gets a bit annoying it was really annoying

424. Testing the measure Study 1: 150 children in 2 schools 7-11 years Balance of age, gender, ethnicity 1 to 1 administration Study 2: 97 children in hospital 7 – 11 years Balance of age, gender, ethnicity 1 to 1 administration Mix of chronic and acute conditions. Surgical and medical patients, inpatient and day care.

425. Psychometric Performance Practicality – excellent – response and completion rates, low time to complete Majority able to self complete Good face and construct validity Reduction of dimensions and levels Final descriptive system

426. The Child Health Utility 9D (CHU9D) 9 dimensions, (worried, sad, pain, tired, annoyed, schoolwork, sleep, daily routine, activities) each with 5 levels Application: generic Age range: 7-11 years Mode of completion: self completion Recall period: today/last night

427. CHU9D HUI2 AQoL 6D EQ 5D Y Worried Emotion Mental Health Anxiety and depression Sad Annoyed Tired Pain Pain Pain Pain/discomfort Sleep Daily Routine Self Care Independent Living Self care School work Cognition Usual activities Joining in activities Mobility Mobility Usual activities Sensation Senses Relationships Coping

429. Example health state I feel very worried I feel quite sad I feel a little bit annoyed I have a lot of pain I feel a little bit tired I have many problems sleeping I have many problems with my daily routine I have a few problems with my schoolwork I can join in with some activities

430. Obtaining preference weights Valuation interviews were undertaken with 300 members of the UK adult general population Modelling Every health state defined by the descriptive system has a preference weight This can be used to calculate QALYs

431. Obtaining preference weights 9 dimensions, 5 levels = 1,953,125 states Valuation interviews were undertaken with 300 members of the UK adult general population Standard gamble and ranking Sample of health states

432. Obtaining preference weights Orthogonal design for main effects only 64 health states in the design Each respondent : Completed descriptive system Ranking of 9 states (including the worst state) SG of 9 states (ping pong version using chance board)

433. Obtaining preference weights Regression modelling to estimate models that could predict a value for every health state defined by the system. A range of models were tested and were evaluated based on their predictive performance

434. Types of Model Standard OLS Fixed effects Random effects Mean, median Rank Data set not large enough to test all possible interactions

435. Assessing Models Significance of coefficients and expected sign Coefficients increasing in absolute size Adjusted R 2 MAE and RMSE % predicted within 0.1 and 0.05 of mean value Graph of observed and predicted Test for bias in predictions

436. Obtaining preference weights 2478 observations from 282 respondents (6% of respondents were excluded. Mean health state values range 0.388 - 0.932 23 negative valuations (0.93%) Interactions made things worse Best 3 models were OLS, RE and mean

437. Further Modelling All models had a few inconsistencies or insignificant levels Further modelling to estimate a parsimonious consistent regression model using the general to specific approach, by combining inconsistent levels and removing non significant levels This was done on the two best performing models (the mean and OLS restricted)

438. Further Modelling Test OLS Mean Inconsistencies 0 0 % within +/-0.1 98.41 98.41 % within +/-0.05 73.02 76.19 MAE 0.0343 0.0349 RMSE 0.0426 0.0431 T test -0.770 -0.336

439. Comparison of measures How was the content generated? Who completes it? Whose values? CHU9D Interviews with children Child or proxy UK adult general population HUI2 Literature review followed by views of parent/child Child or proxy Parents (Canada) Adult general population (UK) AQoL 6D Adaptation of adult instrument Adolescent Adolescent or adult EQ 5D Y Adaptation of adult instrument Child/Adolescent None yet

440. Application Range of paediatric health care research studies in the UK, US and Australia, including economic evaluations alongside clinical trials, observational studies, methodological studies and routine data collection in clinics Clinical areas include: obesity; diabetes; ADHD; intensive care; dentistry; sleep; heart defects; mental health and clinical genetics

441. Application in adolescents Validation in other age groups Recent research to test the validity of using the CHU9D in an adolescent population (11-17 years) Community based sample, web based Demonstrated good construct validity and practicality Research into obtaining adolescent preferences

442. Discussion Content of descriptive system – children only Use in other age groups and testing in clinical populations Adult preference weights Further research is needed to investigate the impact of children’s preferences Methods that could be used to obtain child preferences.

443. Summary Issues with child QALY measurement There are options available Many issues outstanding for further research www.chu9d.org

444. Bayesian Statistics in Health Economics 27/10/11 © The University of Sheffield

445. The Centre for Bayesian Statistics in Health Economics (CHEBS), in collaboration with the School of Mathematics and Statistics is internationally respected for its research in Bayesian statistics. CHEBS disseminates its research by publishing papers in scientific journals and presentations at scientific meetings, and provides training in the application of Bayesian methods in health economics . 27/10/11 © The University of Sheffield

446. Research Themes Evidence synthesis Modelling health-state preference data Elicitation of expert’s probabilities Meta-modelling Bayesian clinical trial simulation 27/10/11 © The University of Sheffield

447. Information Resources 27/10/11 © The University of Sheffield

448. The Information Resource Group has established itself as a national key player in providing information support to health technology assessment and health services research. 27/10/11 © The University of Sheffield

449. Research areas S upport to health and social services research, including systematic reviews E ducation in systematic reviews methods and health informatics L ibrary and Enquiry services E -learning and Continuing Professional Development C reation and evaluation of electronic information resources T raining in information skills for health and information professionals E vidence Based Information Practice and Health Information Project Evaluations D evelopment of information syntheses and summaries in health and social care

450. Courses provided Evidence Synthesis of Qualitative Research in Europe (ESQUIRE) Issues and Challenges for Qualitative Research in Evidence Synthesis (InCQuiRES) Folio Australia Folio Shall Systematic Reviews

451. Examples of information studies conducted in HEDS 27/10/11 © The University of Sheffield

452. The anatomy of e-learning Lynda Ayiku and Anthea Sutton Information Specialists School of Health and Related Research (ScHARR), University of Sheffield 27/10/11 © The University of Sheffield

453. Introduction CPD needs of health information professionals because… Demands on NHS knowledge services have become increasingly sophisticated, requiring that health information professionals’ roles evolve to include: knowledge management training in information and evidence-seeking skills involvement in clinical decision making implementation of policies 27/10/11 © The University of Sheffield

454. Barriers to traditional face-to-face CPD training… Despite their requirements for training in these areas, health information professionals often find it difficult to get away from the workplace to attend training due to: budgetary constraints shortage of time small numbers of staff and lack of appropriate staff cover 27/10/11 © The University of Sheffield

455. FOLIO was created to overcome these difficulties… The online courses enable participants to study without the need to leave the workplace The courses are free of charge and are available for all information professionals who support staff working within the UK National Health Service (NHS) 27/10/11 © The University of Sheffield

456. What is FOLIO? (1) FOLIO is the…. Facilitated Online Learning as an Interactive Opportunity programme 27/10/11 © The University of Sheffield

457. What is FOLIO? (2) FOLIO is a programme of continuing professional development (CPD) e-learning courses for health information professionals Part of the National Library for Health Librarian Development Programme (NLH LDP) NLH LDP provides support and training for librarians moving into new roles Course topics are based on a Training Needs Analysis (TNA) that was carried out in 2003. TNA revealed that technical skills, management skills, professional skills and contextual knowledge were required 27/10/11 © The University of Sheffield

458. Completed FOLIO Courses Up to yet, 13 FOLIO courses have been delivered… Management skills Making your case successfully (MACHIAVEL) Managing change for health information professionals (MCHIP) Managing for service quality (MSQ) Contextual knowledge Information for social care (I4SC) Understanding the business of clinical care (CLINICOS) 27/10/11 © The University of Sheffield

459. Completed FOLIO Courses Professional skills The FOLIO customer care course (FRONTIER) Planning and conducting an information needs analysis (PACINA) Maximising the impact of your service (MAXIM) Getting to grips with knowledge management (G2G) Designing, conducting and analysing surveys and questionnaires (ASQ) Learning and teaching skills Designing and delivering information skills training courses (INFOSKILLS) Evaluating information skills training courses (INFOSKILLS2) Introduction to e-learning (E-FOLIO)

460. FOLIO Team The FOLIO courses are delivered by a team at ScHARR. The FOLIO Team comprises: Andrew Booth (Programme Director) Anthea Sutton and Lynda Ayiku (Learning Resource Co-ordinators) Plus external input from experts in the field In addition, clerical support is provided by the Course Administrator, Tricia Campsell 27/10/11 © The University of Sheffield

461. Aims of the FOLIO programme 27/10/11 © The University of Sheffield

462. (1) Informing Providing participants with an overview of topics relevant to their work Teaching is provided via course materials such as: Briefings Guest lectures Interactive PowerPoint presentations - etc 27/10/11 © The University of Sheffield

463. (2) Engaging We provide a mixture of formal and informal teaching methods to: engage and motivate participants increase the potential to meet a range of preferential learning styles among participants 27/10/11 © The University of Sheffield

464. (3) Socialising Online learners often feel isolated when undertaking e-learning courses This can lead to de-motivation and participant withdrawal from e-learning courses The FOLIO courses include a ‘buddy’ system to help lessen feelings of isolation Buddies are required to collaborate with each other on specific FOLIO exercises and activities 27/10/11 © The University of Sheffield

465. Planning FOLIO courses: Lessons learnt 27/10/11 © The University of Sheffield

466. Lessons learnt… Workload Fewer, evenly spread exercises and tasks Timescales giving participants time to interact with buddies and complete tasks etc. Participants now receive a clear indication of time needed on the FOLIO courses (2-4 hrs per week) 27/10/11 © The University of Sheffield

467. Lessons learnt… Buddy system Participants given the opportunity to follow the buddy route or the self-directed route Larger buddy groups- overcomes problem of ‘orphan’ buddies 27/10/11 © The University of Sheffield

468. Current and upcoming FOLIO courses Current course (Jun-Jul 2006): Promoting and marketing library and information services (ProMISe) [Sept-Oct 2006] Future courses: Moving into supervision : Supervisory skills for paraprofessionals and/or newly qualified professionals (MOVES) [Sep-Oct 2006] Breaking out of the Box: Extending the health LIS professional role- skills and strategies (BREAKOUT) [Nov-Dec 2006] 27/10/11 © The University of Sheffield

469. Any Questions? Contact the FOLIO course team at: [email_address] Keep an eye on the FOLIO website for future courses and developments: www.nelh.nhs.uk/folio 27/10/11 © The University of Sheffield

470. Portals and Pitfalls: Developing web-based research portals for the NHS and University of Sheffield Andy Tattersall, Claire Beecroft, Anna Cantrell, Information Specialists, ScHARR, University of Sheffield 27/10/11 © The University of Sheffield

471. Rationale for Developing the Portals (1) Establishment of the NIHR RDS Lack of knowledge about information management among NHS researchers Need to simplify gathering of information from a wide range of multi-media sources Changing nature of research- information resources are becoming more diverse 27/10/11 © The University of Sheffield

472. Rationale for Developing the Portals (2) ScHARR, University of Sheffield- key UK Public Health research school Research is conducted among teams with members from throughout the school ScHARR is keen to innovate- hence recruitment of a specialist in web technologies 27/10/11 © The University of Sheffield

473. How portals can help you Keep you up to date with what interests you Provide a point of reference Help you find, share and collate information Help you interact Combine text, links, images audio and video Be accessible anywhere Provide a snapshot on a topic, organisation, country, the world Provide entertainment Portals are adaptable and moderately easy to master Be automated (for the most part) Make your life simpler? 27/10/11 © The University of Sheffield

474. How portals can hinder you They are not as automated as we would like RSS feeds can break Web pages can go out of date or just disappear There is always new information, links and people to add Multiple moderation is needed for specialist topics Not all content is applicable to everyone – UK/US angles Information overload Pages can be slow loading Need for a decent Internet connection Sponsored links They could make your working life more complicated 27/10/11 © The University of Sheffield

475. Choosing a portal provider 27/10/11 © The University of Sheffield

476. Choosing a portal provider (2) Initial evaluation focused on: Pageflakes, Netvibes and iGoogle Eventually chose Netvibes for a number of reasons: Comprehensive and adaptable Content can be spread over a number of tabs Many widgets available: rss/Atom feeds, calendars, search engine boxes, notes, bookmarks, Flickr photos, Facebook, YouTube, Twitter, email and user-created modules Netvibes provides reliable support to users. 27/10/11 © The University of Sheffield

477. Widgets “ A web widget is a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation” 27/10/11 © The University of Sheffield Wikipedia (2009) http://en.wikipedia.org/wiki/Web_widget

478. The ScHARR Portal 27/10/11 © The University of Sheffield

479. Journal feeds and podcasts 27/10/11 © The University of Sheffield

480. Tailoring Your Information Health News Special Research Topics and Interest Groups Library Journals Videos Funding Feeds Maps Customised Search Engines 27/10/11 © The University of Sheffield

481. Supporting Specialists (1) 27/10/11 © The University of Sheffield

482. Supporting Specialists (2) 27/10/11 © The University of Sheffield

483. What next ? (1) Evaluation of the portals Horizon scanning for new portal providers Horizon scanning for new web 2.0 (and web 3.0) tools 27/10/11 © The University of Sheffield

484. What next ? (2) Working with NHS to overcome access issues Increasing the number of portals Sharing with other RDS Investigating commercial potential 27/10/11 © The University of Sheffield

485. Bite Size Technology Sessions to Support Research, Teaching and Collaboration Andy Tattersall Information Specialist ScHARR, University of Sheffield Email: a.tattersall@sheffield.ac.uk 27/10/11 © The University of Sheffield

486. 27/10/11 © The University of Sheffield

487. 27/10/11 © The University of Sheffield

488. ScHARR The School of Health and Related Research (ScHARR) specialises in health services and public health research ScHARR concentrates on postgraduate teaching and delivers a teaching and learning portfolio based on research-based, international, multi-disciplinary and world-class curricula 27/10/11 © The University of Sheffield

489. A growing problem Too much choice – which one is best? Too little time – too much time wasted Lack of awareness – lack of application 27/10/11 © The University of Sheffield

490. 27/10/11 © The University of Sheffield

491. 27/10/11 © The University of Sheffield

492. The Ingredients Informal promotion via email, blogs and in house posters Informal interactive presentations from a mixture of academics, technical, clerical and professional staff Staff, PGR and PGT students welcome 27/10/11 © The University of Sheffield

493. 27/10/11 © The University of Sheffield

494. 27/10/11 © The University of Sheffield

495. Structure 20 minute session 10 minutes for questions Guest speakers from across the University Slides made available and recordings uploaded to Intranet, YouTube and Vimeo 27/10/11 © The University of Sheffield

496. The story so far Started in Autumn 2010 29 previous Bite Size sessions - including Google Docs, Prezi, PowerPoint tools, professional social networks, uSpace, Echo360, Pubget, Research Net Contribution, Assessment Methods, Screencasting, Wikis, Electronic Voting Systems, Senate teaching awards, Google Scholar, Mendeley, rss, social media, Google Apps, voice works, data copyright and the Cloud, MOLE 2, mobile phone apps, video capture. 27/10/11 © The University of Sheffield

497. To come The Research Excellence Framework, supervising PhD students, what the Teaching Support Unit can do for you, Scirus scientific search engine, Google Maps for Research, Plagiarism, Medline, Cinahl, what ScHARR Library can do for you, Interactive whiteboards, creating effective posters, how to give a memorable presentation, research costing 27/10/11 © The University of Sheffield

498. 27/10/11 © The University of Sheffield

499. Opening the Information Literacy Toolbox Helen Buckley Woods Information Specialist ScHARR, University of Sheffield Email: h.b.woods@sheffield.ac.uk

500. Housekeeping permission slip and recorder

501. Outline of session Introduction Jigsaw activity Break Reflection and feed back on groupwork Summary of session and evaluation Closing comments 27/10/11 © The University of Sheffield

502. Aim An opportunity to explore the relationship between theory and practice in information literacy teaching.

503. Learning Outcomes By this end of this session participants should be able to: Analyse four areas of interest in learning and teaching Evaluate the effectiveness of active learning techniques Describe how they will apply what they have learnt in their practice.

504. Active learning What is it? Methods Evidence - effectiveness Seen as most appropriate method Explore through Jigsaw Atherton J S (2011) Teaching and Learning; Lectures [On-line: UK] retrieved 3 July 2011 from h ttp://www.learningandteaching.info/teaching/lecture.htm

505. Active Learning “ Learning is not a spectator sport. Students do not learn much just by sitting in classes listening to teachers, memorizing pre-packaged assignments, and spitting out answers. They must talk about what they are learning, write about it, relate it to past experiences and apply it to their daily lives. They must make what they learn part of themselves.” Chickering, Arthur W. and Gamson, Zelda F. (1987) Seven Principles for good practice in undergraduate education. American Association for Higher Education & Accreditation Bulletin [online], 39 (7) 3-76. Available from: http://www.aahea.org/buletins/bulletins.htm [3 July 2011]

506. Question Write down any challenges you have experienced when supporting learners. Discuss with a partner Share with the group

507. Jigsaw Activity Aim: acquire lots of information in short space of time Analyse four areas of interest in learning and teaching 1. Planning learning 2. Working with small groups 3. Working with large groups 4. Motivating learners

508. Jigsaw Group Activity Aim: acquire lots of information in short space of time 2 groups 4 different areas – you will become the expert in 1 of these areas Review and digest the information Meet with fellow experts Return to your original group and present your findings

509. Reflection How did it feel to be a learner? Was it effective? How useful would it be for you as a facilitator? How would you adapt the session? Consider the four themes – what was notable for you about the material we looked at?

510. Conclusion and Evaluation In this session Listening, note taking, discussion, reflection, working alone, group work, peer teaching, feedback, answering/asking questions. Engaging with different materials Explicit or mediated theory

511. Supporting the Health Researcher of the Future Andrew Booth & Andrew Tattersall , Information Resources Group ScHARR, University of Sheffield

512. Presentation Plan The Context of Research Support (AB) The Potential of Web 2.0 (AB)  The Way Forward? (AB) Some Examples of Good Practice (AT) What we are doing in ScHARR/Yorkshire & Humber (AT) Questions (AB/AT)

513. The Context of Research Support Andrew Booth

514. From Gamekeeper….

515. Researchers’ Use of Academic Libraries and their Services Significant differences of perceptions and views between researchers and librarians Communication channels need to be improved. How? Research community uses social networking to exchange and share research-based information. Role of libraries presently ill-defined. Researchers don’t readily recognize content on their desktop is provided through library . Researchers’ Use of Academic Libraries and their Services A report commissioned by the Research Information Network and the Consortium of Research Libraries

516. Research habits Users &quot;power-browse&quot; or skim material, using &quot;horizontal&quot; (shallow) research. Most spend only a few minutes looking at academic journal articles and few return to them. &quot;It almost seems that they go online to avoid reading in the traditional sense,” Not just &quot;screenagers&quot;. “Undergraduates to professors….exhibit a strong tendency towards shallow, horizontal, flicking behaviour in digital libraries. Factors specific to the individual, personality and background are much more significant than generation .&quot; INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE

517. Researchers information literacy training should focus on information management, not information retrieval – 1 (Booth, 2007) Inappropriate to meet IL needs of researchers using instruction methods based on undergraduates; Researchers do not follow neat stepwise progression from state of unknowing (“information need”) to knowing that underpins most IL instruction. Information management, rather than information retrieval, should be focus of IL instruction for researchers.

518. Researchers information literacy training should focus on information management, not information retrieval – 2 (Booth, 2007) Information retrieval should focus on “area scanning”, footnote chasing and known author searching rather than keyword searching IL training should be “socialised” through formal collaboration …..and integration with existing research programmes or research groups. Training should focus on practically based outcomes e.g. production of log book or portfolio. Training should optimally be tailored to individual and delivered at time of need.

519. RIP - Library as Place “ the library has changed from being the place for researchers to visit for help with information searching and for picking up the actual information, to being the “living room” for undergraduate students, making the researchers who visit the library feel outnumbered, and sometimes unwelcome.” Haglund and Olsson (2008)

520. Rethinking the Library Web Site “ Libraries spend huge amounts of time and money to work on the structure and content of the library Web page, while few researchers use it as a starting point for information searching. Many researchers….used the Web of their own department as a starting point, and this is where the library should establish a presence with direct links targeted to that particular group”. Haglund and Olsson (2008).

521. The Potential of Web 2.0 Andrew Booth

522. The Health Researcher of Today: Expects service that optimizes benefits of technology: anywhere, anytime, personalized, at the point of need, instantly. Requires services that are seamless, integrated and open access

523. Web 2.0 Includes specific applications that actively engage users (advanced searches for information and production of information). Young predominant early users of social networking and user-generated content. Social networking sites mainly used for simple social interactions but sometimes linked to information searches. Not only Google Generation display advanced online behaviour. Other “Generations” also able and willing to engage in complex online activities. INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE - A British Library / JISC Study

524. Impact of Web 2.0 Blogs, wikis, podcasts, social networks/other online features offer new educational opportunities. Application of e-learning technologies is evolving. More than publishing content online - dynamic learning environment in which to actively share knowledge and…experiences. Blogs, wikis and social networking stimulate dynamic and proactive engagement in learning ( or indeed investigation! ) process. INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE

525. Some Candidate Web 2.0 Applications One stop shops – Wikis & Portals Project-based Blogs/Wikis/Discussion Fora Collaborative tools – Shared Slides/ Documents/Bookmarking/Tagging “ How To”s – Instructional Resources (Wikis)/Videos The Business of Research – e.g. Publication/Research Assessment/Funding Opportunities (RSS feeds)

526. Some Examples of Good Practice Andrew Tattersall

527. Dublin Public Library Portal

528. Tropika.Net

529. http://greylit.pbworks.com/

530. Central Medical Library, University Medical Center Groningen (UMCG)

531. Central Medical Library , University Medical Center Groningen (UMCG)

532. What we are doing in ScHARR/Yorkshire & Humber Andrew Tattersall

533. Context Independent University-based library serving health services researchers and students and providing tertiary information services for NHS researchers (Trent RDSU up to March 2010) Move away from physical use to use of virtual services New Yorkshire & Humber Research Design Support Service (from October 2008)

534. Personal Homepages and Web Portals

535. How could it help us? Keep you up to date with what interests you Entertain you Be a point of reference Be formal and informal Be automated (for the most part) Help you find/share/collate information Help you interact Be accessible anywhere Combine text, links, images audio and video Give you snapshot on topic, organisation, country, the world Adaptable and moderately easy to master Make your life simpler?

536. H ow could it hinder us? Not as automated as we would like RSS feeds can break Web pages go out of date or just disappear Always new information, links and people to add Multiple moderation needed for specialist topics Not all content is applicable to everyone – UK/US angles Information overload Pages can be slow loading Need for decent Internet connection Could make your working life more complicated Sponsored links

537. Widgets “ A web widget is a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation” Wikipedia (2009) http://en.wikipedia.org/wiki/Web_widget

538. The ScHARR Portal

539. Journal feeds and podcasts

540. Tailoring Your Information Health News Special Research Topics and Interest Groups Library Journals Videos Funding Feeds Maps Customised Search Engines

541. Evaluation of The ScHARR Portal “ Easy access to resources - saving time” “ Could give quick access to interesting info and share what's going on around ScHARR” “ I value the links to anything to do with my current project and report writing” “ Will be a handy one-stop place for lots of information” “ I can get the right feeds for me on there”

542. The Way Forward? Andrew Booth

543. At the moment…… We are at the “If we Build them a Portal will they come?” phase of development

544. However….. Evidence both locally and internationally that health researchers have an appetite….. Not for the technology itself, But for the “What’s In It for Me” of new technologies They will need Guides……and Architects

545. Where will become less important than How!

546. Above All…… Health researchers will want us to open our toolbox of “make your job easier/more effective tools” Will include both free Web 2.0 tools and conventional products (e.g. reference management; citation tools etc)

547. Welcome to the era of Web Tool Point Zero!

548. References - 1 Booth A (2008) Google: It's all at the Co-op now! Health Info Internet ; 62: 3-4. Booth A (2007) Researchers require tailored information literacy training focusing on information management, not simply information retrieval. Report for Research Information Network Consultative Group on Librarianship and Information Science. http://www.rin.ac.uk/training-research-info-spec CIBER. Information behaviour of the researcher of the future – ( A British Library/JISC Study) http://www. bl.uk/news/pdf/googlegen.pdf

549. References - 2 Haglund L and Olsson P (2008). The Impact on University Libraries of Changes in Information Behavior Among Academic Researchers: A Multiple Case Study. Journal of Academic Librarianship 34 (1), 52-59 Research Information Network (2008). Researchers’ Use of Academic Libraries and their Services A report commissioned by the Research Information Network and the Consortium of Research Libraries http://www.rin.ac.uk/researchers-use-libraries Tattersall A (2008) 'Blogging in an Academic Health Library Setting. Libraries for Nursing Bulletin ; June 2008.

550. Library Thing: friend or foe? Using its potential to update the Core Collections series Helene Gorring and Helen Buckley Woods Information Specialists ScHARR, University of Sheffield

551. Introduction - what is this about? What are the Core Collections? Challenges Solutions using Web 2.0 Medical Core Collection - live demo

552. Background to the Core Collections History - Medical Information Working Party Medical Core Collection has been acquisition tool for health librarians for nearly 20 years. Collections were developed independently: Medical Nursing & Midwifery Mental Health Allied Health Process had become very time consuming...

553. Issues to Address: Three challenges: Communication Limited opportunity to contribute Format of finished product

554. Methodological Solutions Challenge Limited opportunity to contribute Established methods email, email lists, liaison with clinicians and information professionals Web 2.0 opportunity

555. Piloting Library Thing - Mental Health Core Collection First edition published in 1999 A briefing on the project and instructions on how to contribute using LT was sent out in mid-may 2009

559. Outcomes and Feedback Wider range of contributors 14 Librarians (10 NHS, 1 Higher Education, 2 from Special Libraries, BMA & RCPsych) 8 Clinicians 2 Lecturers &quot; Library Thing has revolutionised the production of this core list. It's an ideal application and has allowed us to consult much more widely and efficiently.&quot; &quot; It was quite a time consuming exercise as I tried to look at every area, but it flagged up new editions that I had missed so was directly useful for our service&quot; &quot; I think this was a very effective way of working together to produce the list.&quot;

560. Where are we at? Updated Core Collections

562. Contributing to the 6th edition of the Medical Core Collection - volunteers needed! Live Demonstration using Library Thing

570. POST GRADUATE TAUGHT COURSES

571. Courses MSc Health Economics and Decision Modelling MSc International Health Technology Assessment, Pricing and Reimbursement 27/10/11 © The University of Sheffield

572. MSc Health Economics and Decision Modelling 27/10/11 © The University of Sheffield

573. Why choose this course? Only Masters programme in the UK dedicated to the practical application of mathematical modelling to inform healthcare decision making The programme is suitable for those who wish to pursue careers as applied economists in the health sector or as the basis for doctoral research 27/10/11 © The University of Sheffield

574. Principal aims of the course Equip students with an analytical skill-base required to evaluate and improve the efficiency and effectiveness of healthcare systems to a professional level. Provide an in-depth understanding of current state-of-the-art methods for mathematical modelling to support and inform evidence-based healthcare decision problems. Provide a firm grounding for students wishing to pursue a career in health economics and mathematical modelling. 27/10/11 © The University of Sheffield

575. Core modules Cost-effectiveness Modelling for Health Technology Assessment Introduction to Health Economics Health Research Methods Economic Evaluation Advanced Topics in Economic Evaluation Medical Statistics and Evidence Synthesis Operational Research Techniques in Health Resource Allocation Advanced Simulation Methods Dissertation 27/10/11 © The University of Sheffield

576. Duration and Mode of Study One year-full time Two year-part time A full Masters course consists of units of study totalling 120 credits and a dissertation 27/10/11 © The University of Sheffield

577. Career Opportunities Ideal for students wishing to pursue a career as a professional health economist/modeller Graduates will be suitably prepared to carry out analytical work at a professional level in multidisciplinary teams for a range of organisations including: Pharmaceutical companies Health insurance/sickness funds Healthcare organisations Department of health /Ministries of health and associated agencies Municipalities and Public Health departments Universities Management consultancies

578. Who is eligible for the course? A first degree (minimum 2.1) from a recognised university in a numerate subject such as Economics, Operational Research, Mathematics, Statistics, Industrial Engineering, Management Science, Physics, Systems Control A good level of written language, both written and oral Willingness to engage with staff and students for mutual benefit Motivation to establish an excellent career in Health Economics and Decision Modelling

579. Further information For further information, visit us on: http://www.sheffield.ac.uk/scharr/prospective_students/masters/hedm 27/10/11 © The University of Sheffield

580. MSc in International Health Technology Assessment, Pricing and Reimbursement 27/10/11 © The University of Sheffield

581. Why choose this Masters course? Only graduate programme offering the entire range of knowledge and skills needed by those developing health technologies for market, or involved in commissioning or evaluating health technologies Suitable for those with busy work commitments in these fields Meets the needs of those involved in health technology policy formulation, management and evidence-based commissioning and purchasing of health technologies for governments or health services

582. Programme content Developing disease management pathways Comparative healthcare systems Economic evaluation in HTA Searching for evidence to support HTA Evidence synthesis The application of modelling to HTA Utilities and patient-reported outcomes Design of trials Analysing randomised and non-randomised trial data Epidemiology for HTA Systematic review methods Financing health technologies Marketing health technologies Strategy for health technology development

583. Duration and mode of Study One year-full time Two/three years-part time You can choose to study for a postgraduate Certificate over one year, or a Diploma over one or two years. It is also possible to study individual modules to meet specific learning needs

584. Career Opportunities The course will help you progress in your chosen career in market access, trial design and other aspects of health technology assessment, in the private or public sector The course will provide comprehensive training needed to gain a post in industry or working as a health technology evaluator or commissioner 27/10/11 © The University of Sheffield

585. Who is eligible for the course? A first degree (minimum 2.2), an equivalent professional qualification or relevant work experience An adequate level of English, both written and oral (IELTS 7.0) Enthusiasm to work hard and benefit from the course environment 27/10/11 © The University of Sheffield

586. Further information For further information visit us on: http://www.shef.ac.uk/scharr/prospective_students/masters/ihtapr/index 27/10/11 © The University of Sheffield

587. SHORT COURSES 27/10/11 © The University of Sheffield

588. Courses on: Using Utility Data for Health Technology Assessment Cluster Randomised Trials Evidence synthesis of qualitative research in Europe Issues and Challenges for Qualitative Research in Evidence Synthesis A guide to doing mixed methods studies in the health sciences 27/10/11 © The University of Sheffield

589. Using Utility Data on Health Technology Assessment - Two days course 27/10/11 © The University of Sheffield

590. Background NICE and similar decision-making bodies around the world are increasingly using QALYs in assessing cost-effectiveness of healthcare interventions. This raises the usual questions about the measurement of health, the valuation of health and whose values to use 27/10/11 © The University of Sheffield

591. What does the course deliver? The course outlines the practical requirements of measuring utility and obtaining utility data for health technology assessment agencies such as NICE 27/10/11 © The University of Sheffield

592. Who will benefit from the course? Academics Government agencies Pharmacoeconomics Outcomes experts in industry and consultancies with an interest in the use of health state utility data 27/10/11 © The University of Sheffield

593. Course Content Key issues in obtaining health state utility value Does it matter which instruments are used? What is the current NICE reference case? When are EQ-5D and other generic measures not appropriate? What are the latest developments in EQ-5D including EQ-5D-5L valuation? What should be done when EQ-5D and other generics are not appropriate? What should be done when (the generic) reference case data are not available? How can social value QALY weights be used? E.g. for Value Based Pricing methods. How should utility values be incorporated into cost effectiveness models? The course will consist of a mixture of presentations, group work, discussions and individual exercises. 27/10/11 © The University of Sheffield

594. Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 27/10/11 © The University of Sheffield

595. Cluster Randomised Trials- One day course 27/10/11 © The University of Sheffield

596. Background Cluster Randomised Trials (CRTs) are trials which randomise groups of patients rather than individual patients. For example: randomising communities to different public health interventions, or randomsing practitioners to different methods of treating patients. 27/10/11 © The University of Sheffield

597. What does the course deliver? Understand issues such as contamination and recruitment bias Be able to decide how big a study should be Read and critique the literature on cluster trials Understand how to analyse CRTs using simple methods and also regression as implemented in a variety of packages such as SPSS, R and Stata. The participants will have the opportunity to analyse data from a cluster randomised trial. 27/10/11 © The University of Sheffield

598. Who will benefit from the course? Clinicians who need to design CRTs New researchers who want to appreciate the benefits and disadvantages of CRTs Statisticians who want to understand how to design and analyse CRTs 27/10/11 © The University of Sheffield

599. Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 27/10/11 © The University of Sheffield

600. Evidence synthesis of qualitative research in Europe- Two and a half day course 27/10/11 © The University of Sheffield

601. Background The course will follow the systematic review process as it applies, and is adapted to, qualitative evidence syntheses. 27/10/11 © The University of Sheffield

602. What does the course deliver? The Programme Faculty will take the participants through the stages of defining and exploring scope, conducting the literature searches, quality assessing studies for inclusion, synthesising data and writing up and presenting the product of the synthesis. 27/10/11 © The University of Sheffield

603. Who will benefit from the course? Qualitative Researchers who want to learn how to synthesise qualitative research Systematic Reviewers who want to learn how to translate synthesis methods to qualitative data PhD Students in topic areas that require substantive review of qualitative research New researchers who want to develop evidence synthesis/systematic review Qualitative systematic reviewers who want to update and extend their skills and knowledge 27/10/11 © The University of Sheffield

604. Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 27/10/11 © The University of Sheffield

605. Issues and Challenges for Qualitative Research in Evidence Synthesis- One day course 27/10/11 © The University of Sheffield

606. Background This methodological update will rehearse the major issues and challenges for the main steps of the systematic review process as it applies, and is adapted to, qualitative evidence syntheses. 27/10/11 © The University of Sheffield

607. What does the course deliver? The Presenters, drawn from the Cochrane Qualitative Research Methods Group and involved in producing methodological guidance, will examine the specific challenges as they take participants through the stages of defining and exploring scope conducting the literature searches Quality assessing studies for inclusion synthesising data and writing up and presenting the product of the synthesis. 27/10/11 © The University of Sheffield

608. Who will benefit from the course? Qualitative Researchers wishing to extend their knowledge of how to synthesise qualitative research Systematic Reviewers who wish to advance translation of synthesis methods to qualitative data PhD Students in topic areas who are undertaking a substantive review of qualitative research Qualitative systematic reviewers who want to update and extend their skills and knowledge 27/10/11 © The University of Sheffield

609. Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 27/10/11 © The University of Sheffield

610. A guide to doing mixed methods studies in the health sciences- One day course 27/10/11 © The University of Sheffield

611. Background The course will follow the process of undertaking a mixed method study from developing research questions, selecting an appropriate design, integration of data and findings, through to reporting in peer reviewed publications. 27/10/11 © The University of Sheffield

612. What does the course deliver? The course will cover key issues such as: relevant paradigms quality assessment team work and structural issues affecting mixed methods research. The focus will be on health research, particularly health care research. 27/10/11 © The University of Sheffield

613. Who will benefit from the course? Researchers who have undertaken, are undertaking, or are planning to undertake a mixed methods study in the health sciences. 27/10/11 © The University of Sheffield

614. Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 27/10/11 © The University of Sheffield