Cost utility analysis of interventions to return employees to work following long term sickness absence
Upcoming SlideShare
Loading in...5
×
 

Cost utility analysis of interventions to return employees to work following long term sickness absence

on

  • 223 views

 

Statistics

Views

Total Views
223
Views on SlideShare
223
Embed Views
0

Actions

Likes
1
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cost utility analysis of interventions to return employees to work following long term sickness absence Cost utility analysis of interventions to return employees to work following long term sickness absence Presentation Transcript

  • School of Health and Related Research (ScHARR) The University of Sheffield Regent Court 30 Regent Street Sheffield, S1 4DA Website: www.sheffield.ac.uk/scharr
  • Health Economics and Decision Science (HEDS) The purpose of HEDS section within ScHARR is to promote excellence in national and international healthcare resource allocation decisions, through applied and theoretical research funded by the public and private sector; and supporting the effective implementation of the results of such research through education, training and management interventions 14/05/2014 © The University of Sheffield
  • Portfolio of specialities of HEDS 1. Research 2. Post-graduate taught programmes 3. Short-term courses 14/05/2014 © The University of Sheffield
  • RESEARCH
  • Information resources Bayesian Statistics in Health Economics Evidence review and synthesis Health Economics and Outcomes Research Cost-effectiveness modelling to support Healthcare Decision Making Main Focus of Research Areas
  • 14/05/2014 © The University of Sheffield Consultancy within HEDS Modelling alongside clinical trials Health related quality of life Clinical Trial Simulation Health Economics Modelling Evidence Synthesis Costing studies Systematic reviewing
  • Disease Areas • Acute coronary syndrome • Age-related macular degeneration • Alcohol • Alzheimer’s • Ankylosing spondylitis • Asthma • Back pain • Breast cancer • Cardiovascular conditions • Colon cancer • Colorectal cancer • Dementia • Depression • Dyskaryosis • Epilepsy • Flushing • Foot ulcers • Immune thrombocytopenic purpura • Influenza • Irritable bowel syndrome • Leg ulcers • Lymph node metastases • Menopause • Mental health problems • Multiple sclerosis • Overactive bladder • Prostate cancer • Psoriatic arthritis • Pulmonary hypertension • Renal disease • Rheumatoid arthritis • Schizophrenia • Sexual health • Surgical procedures in sex reassignment • Type 1 and Type 2 diabetes • Vaccination scheduling • Venous ulcers
  • Innovation & Knowledge Transfer Contact details Further information about consultancy projects and research areas is available at: http://www.sheffield.ac.uk/scharr/consultancy For an informal chat, contact the HEDS programme director, Roberta Ara: Email:r.m.ara@sheffield.ac.uk
  • Cost-effectiveness modelling to support Healthcare Decision Making 14/05/2014 © The University of Sheffield
  • The Decision Analytic Modelling team is a leading healthcare modelling group internationally, with a wide ranging, high impact programme of methodological and applied research. Research focuses on Operational Research modelling including both health technology assessment modelling and supporting service organisation and delivery.
  • In particular: • Cost-effectiveness modelling to support HTA submissions • Budget impact modelling • Systematic reviews & critique of cost- effectiveness literature • Critique / validate existing model • Country adaptations • Model adaptations
  • Examples of cost- effectiveness studies conducted in HEDS 14/05/2014 © The University of Sheffield
  • Evaluating the cost- effectiveness of diagnostic tests Dr Matt Stevenson Reader in Health Technology Assessment; NICE Appraisal Committee Member; Contributor to the NICE Diagnostic Assessment Programme manual
  • Overview • A brief description of how (and why) cost- effectiveness analyses are applied • A brief overview of the analyses required to model diagnostic tests in sepsis • A very brief description of the reasons why diagnostic evaluations are more difficult than pharmaceutical evaluations
  • Cost-effectiveness analyses In the last 10 years there has a been a considerable increase in the importance of cost- effectiveness analyses. This was due to the relatively fixed budget and a combination of ageing populations and emerging expensive interventions. This has led to the formation of funding agencies in England and Wales, Scotland, Australia and Canada.
  • Does a diagnostic test represent value for money? Diagnostic tests with high prices may be cost- effective (i.e. a worthwhile use of a limited budget). Conversely, diagnostic tests with low prices may not be cost-effective. The „gold standard‟ approach for determining whether the price of a diagnostic test is justified is through an economic evaluation, or cost- effectiveness analysis.
  • Cost-effectiveness analyses The goal of funding agencies is to provide the greatest amount of health for society within the budget, and thus opportunity cost is a key principle. That is, what health would be lost if money was diverted from one intervention in order to fund another. The process is typically to estimate the cost- effectiveness of an intervention through modelling, and comparing this result with a value assumed to represent opportunity cost.
  • Previous Diagnostic evaluations Whilst the majority of evaluations undertaken relate to pharmaceuticals, evaluations of diagnostic tests have been conducted: •Carotid Artery Imaging - Wardlow et al. HTA •Thrombophilia Testing - Simpson et al. HTA • BMD Scanning (Strontium Ranelate) – Stevenson et al HTA •Diagnostic strategies for DVT – Goodacre et al HTA • Diagnostic pathways for minor head injuries – Pandor et al. HTA • Non-Invasive Liver Testing – Stevenson et al HTA (in press)
  • Methods for evaluating diagnostic tests There have been, for some time, clear methods guide for undertaking evaluation of pharmaceutical interventions. Recently NICE has set up a Diagnostic Assessment Programme which has issues an interim statement of the methods it expects to be followed in evaluating diagnostics. http://www.nice.org.uk/media/164/3C/DAPInterim MethodsStatementProgramme.pdf
  • Simplified Overview of the modelling required The following slides discuss the steps that would be required to generate an estimate of the cost- effectiveness of a diagnostic test (or series of diagnostic tests). The overview is a simplification. More detailed discussion is provided in the previously listed HTA reports (all free to download) and the Diagnostic Methods statement
  • Estimating Test Accuracy The sensitivity and specificity of a diagnostic test must be estimated. These values would be combined with the estimated prevalence of the condition being tested for, to form an expectation of the number of true positives, true negatives, false positives and false negatives generated by the diagnostic test.
  • Modelling the patient experience For each of the four groups defined, an estimation of the events that would occur to the patient must be modelled. These may differ due to underlying risks and the chosen medical management. The modelling would include factors such as the risks of mortality, risk of morbidity, length of stay within hospital, costs for initial and subsequent care, treatment-related adverse-events and the quality of life for patients in each potential health state.
  • Modelling the patient experience Ultimately, an estimation of the life years, quality adjusted life years (QALYs*) and costs can be attributed to each of the four groups. These can be weighted by the proportions in each group to form a total cost and total QALY for patients post diagnosis. The costs of the diagnostic tests performed are then added. * The QALY is a combination of life years and patient utility. A person living for 10 years at a utility of 0.5 would gain 5 QALYs; a person living for 4 years at a utility of 0.75 would gain 3 QALYs
  • Calculating an ICER* Assume that post diagnosis, an average patient was expected to gain 10 QALYs at a cost of £20,000 under current best practice. These values became 11 QALYs at a cost of £18,000 following a new diagnostic test, which costs £4,000 per patient. In this instance the increase in cost is £2,000 (£18,000 - £20,000 + £4,000) The increase in QALYs is 1. (11 – 10) * An Incremental Cost Effectiveness Ratio.
  • Calculating an ICER In this example, the ICER would be £2,000 per QALY gained (£2,000 / 1) This would be compared with an estimation of the cost of gaining a QALY in interventions that are likely to be replaced.* Thus if this were the result from a real technology appraisal the diagnostic test would be likely to be recommended for use. * NICE has estimated this to be in the region of £20,000-£30,000
  • Implications for diagnostic pricing Where a new diagnostic test has a large impact on mortality or on the utility of a patient, then the QALY gained over the current diagnostic will be greater. ICER = Δ Cost / Δ QALY Thus, for a constant ICER, such a test would be able to command a higher price than a test with a smaller QALY gain.
  • Sequences and subgroups Note that sequences of tests and only incorporating tests on a subgroup of the population are possible. The following slide shows the predicted optimal strategy for diagnosing whether a patient has deep vein thrombosis. The costs of diagnostic tests, the risks of death, morbidity, recurrence, treatment-related adverse-events and the costs of treating future events were all considered in the model.
  • Example of diagnostic algorithm Taken from Goodacre et al. QJM 2006; 99:377–388
  • Returning to the example of sepsis Current gold standard is blood culture, but this has poor sensitivity. A new test is available (SeptiFast©) which has higher accuracy than blood culture tests, but is relatively expensive. The price may preclude use in all patients with suspected sepsis. A decision support system is also available, (Treat ©) which can categorise patients into high, medium and low risk. This may allow SeptiFast© to be used more efficiently.
  • Estimating the cost- effectiveness of a Treat© and SeptiFast© diagnostic strategy For each Treat© category, the patient experience must be modelled taking into account true positives, false positives, true negatives and false negatives. This will incorporate (amongst others) •Risks of mortality •Risks of post-infection sequalae •Length of stay in hospital These variables are expected to be lower where there is appropriate management of a patient with sepsis.
  • Estimating the cost- effectiveness of a Treat© and SeptiFast© diagnostic strategy Thus there will be a QALY gain (and cost reduction) associated with the new diagnostic tests. It is expected that these will be greatest in those patients denoted high risk. Factoring in the costs of the diagnostic (Treat© for all patients, SeptiFast© for the groups being evalauted) will allow the cost-effectiveness of diagnostic strategies to be evaluated.
  • Additional complications with evaluating diagnostic tests There are reasons why evaluating diagnostic tests are more difficult than evaluating pharmaceuticals. Due to time restrictions these will be mentioned very briefly under broad headings.
  • Complicating Issues 1) The need to understand the patient pathway 2) Data reporting 3) Missing and unobtainable data 4) Meta-analyses 5) Correlation between tests 6) Imperfect gold standards 7) Required operator skill 8) Spectrum bias 9) Incidental findings 10) Estimating the costs of diagnostic tests
  • NICE Decision Support Unit Dr Paul Tappenden Senior Research Fellow Acting DSU Director http://www.nicedsu.org.uk/ 14/05/2014 © The University of Sheffield
  • Background to DSU • Established in 2002. • Formal collaboration between the Universities of Sheffield (lead), York and Leicester. • Peripheral members at LSHTM, Brunel and Bristol. • May also involve highly specialist expertise from elsewhere. • Expertise covers all key areas of HTA. • Commissioned by NICE to provide a research and training resource to support the Institute's Technology Appraisal Programme.
  • Main strands of work 1. Appraisal-specific work • Rapid evaluation of technical issues/analysis around health economic evaluation/modelling problems within specific technology appraisals • An independent yet highly qualified voice 2. Methods development • Ongoing role in developing Methods guidance for NICE • DSU Technical Support Documents • Training for industry and NICE Appraisal Committee members • Other methods work
  • Appraisal-specific work • ~7 appraisals per year • Analysis of additional evidence submitted as part of the appraisal process • Cetuximab for 1st line metastatic colorectal cancer • Lapatinib for metastatic breast cancer • Provision of modelling expertise unavailable elsewhere (software) • Donepezil for Alzheimer‟s Disease • Evaluating Patient access schemes (PAS) • Tocilizumab for Rheumatoid Arthritis
  • The NICE Methods Guide for Technology Appraisal • Setting the agenda for, organising and facilitating the NICE methods guide for technology appraisal • Recent work includes managing the update to the 2008 NICE methods guide – 6 workshops covering issues around evidence synthesis, utilities, uncertainty analysis, subgroups etc. • Update due in 2011/2012 • Conduit for leading academic centres in HTA
  • DSU Technical Support Documents (TSDs) • Series of independent, non-prescriptive reports commissioned to support NICE Methods Guide • 14 TSDs completed or near completion • Main areas: • Health utilities (x5) • Evidence synthesis (x7) • Model development and reviewing evidence for informing model parameters • Survival modelling using individual patient data • Use of regression methods in HTA models
  • Other DSU methods research • Report and related publications demonstrating the feasibility of applying Value of Information methods. • Feasibility projects relating to computerised decision support systems and orphan drugs • Equity arguments applied to orphan drugs • Discounting of costs and benefits • The role of patient valuations of health states • The incorporation of uncertainty into cost-effectiveness models • The incorporation of equity weights into cost effectiveness models and decision making
  • Methods training • Bridge between NICE and the pharmaceutical industry in providing methods training • “Masterclasses” held: • Measurement and valuation of health-related quality of life • Network meta-analysis & indirect comparisons • NICE Appraisal Process, modelling, what makes a good submission, uncertainty analysis
  • Future updates • Join our mailing list at : http://www.nicedsu.org.uk
  • Health economic modelling in bowel cancer Dr Paul Tappenden Senior Research Fellow ScHARR, University of Sheffield p.tappenden@sheffield.ac.uk 14/05/2014 © The University of Sheffield
  • Health economics and cancer • Key area of methodological expertise in ScHARR for >15 years • Key areas include breast, bowel, cervical, CML, kidney, prostate, lu ng • Range of decision-makers/clients including: – NICE – NETSCC / NHS R&D Programme – NHS Cancer Screening Programmes – Department of Health – Local decision-makers
  • Key areas of application 1. Modelling interventions for the early detection / prevention of cancer 2. Modelling interventions for the treatment of diagnosed cancer 3. Modelling whole disease and treatment pathways (Whole Disease Modelling)
  • 1. Modelling interventions for the early detection / prevention of cancer • Methodological development in modelling natural history disease progression – Handling competing risks – Length / lead-time biases – Consideration of full trade-off between costs and benefits – Calibration of unobservable parameters (disease progression, presentation behaviours etc) • Evaluation of screening programmes in breast, bowel, cervical and prostate cancer • Evaluation of early awareness campaigns
  • An example – screening for CRC 14/05/2014 © The University of Sheffield
  • 2. Modelling interventions for the treatment of diagnosed cancer • Curative / palliative treatments for diagnosed cancer • Numerous appraisals for NICE - main areas covered include bowel and breast • Key issues around methods for handling • Extrapolation beyond trial duration • Handling treatment crossover • Treatment sequences • Evidence networks across multiple trials
  • An example – bevacizumab for MCRC 14/05/2014 © The University of Sheffield
  • 3. Whole Disease Modelling • Usefulness of models is in part determined by the scope of the decision it is intended to inform. • Single isolated point versus whole pathway model. • “Modelling the bigger picture” – development of models which can represent whole disease and treatment pathways
  • A lot of effort so why bother? • Consistent basis for economic evaluation across the pathway • Structurally capable of evaluating any intervention at any point in the pathway • Capturing upstream and downstream knock-on impacts • Shift to potentially more useful economic decision rules • Methodological challenges • Obtaining agreement regarding pathways • Handling geographical variability • Programming / model run time • Calibration of unobservable parameters • Non-trivial investment of time at outset but payoff may be considerable
  • An example – Colorectal Cancer Whole Disease Model
  • From piecewise CPQ to constrained maximisation 14/05/2014 © The University of Sheffield
  • Colorectal cancer screening : using mathematical modelling to inform policy decisions Dr Sophie Whyte Research Fellow ScHARR, University of Sheffield Email: s.whyte@sheffield.ac.uk 14/05/2014 © The University of Sheffield
  • Contents Background • The current NHS Bowel Cancer Screening Programme • The flexible sigmoidoscopy screening trial • Should the screening programme be changed? Methods • Possible screening strategies to consider • The value of modelling in this context • Modelling challenges • Modelling solutions Results and conclusions • Results • Policy implications 14/05/2014 © The University of Sheffield
  • The current NHS Bowel Cancer Screening Programme • Roll out commenced in 2006 and now screening covers the whole of England. • Persons aged 60-69 (being increased to age 74) are offered biennial screening. • The guaiac faecal occult blood (gFOB) test is used for screening and positives(≈2%) receive follow up with colonoscopy. • Uptake is 52% 14/05/2014 © The University of Sheffield
  • Flexible sigmoidoscopy(FS) screening trial* • Trial reported in 2010 • 170,432 persons aged 55-64 were randomised to FS screening or to a control group (no screening). • 40,674 persons underwent FS screening • After 10 years of follow-up the incidence of CRC was reduced by 33% and mortality was reduced by 43% in the FS group. *Atkin et al 2010, Once-only flexible sigmoidoscopy screening in prevention of colorectal cancer: a multicentre randomised controlled trial, The Lancet 14/05/2014 © The University of Sheffield
  • Should the screening programme be changed in light of new evidence? • Project for NHS cancer screening “Reappraisal of options for CRC screening” • Update to “Appraisal of options for CRC screening” completed by Tappenden et al in 2004. • The reappraisal uses: (1) Data from existing screening programme, (2) Data from FS trial, (3) Data on newer immunochemical FOBTs 14/05/2014 © The University of Sheffield
  • Methods: Possible screening scenarios • Biennial screening with guaiac or immunochemical FOBT for ages 60-74 • One-off FS screening (for different ages) • Combination screening strategies e.g. FS at age 55 then biennial FOBT at ages 60-74 14/05/2014 © The University of Sheffield
  • Methods: Advantages of modelling The number of possible screening strategies is very large To demonstrate a significant impact on incidence and mortality rates screening trials require: • very large numbers of participants • a long follow-up period Hence modelling is very useful for the evaluation of screening strategies 14/05/2014 © The University of Sheffield
  • Methods: Modelling challenges Considerable uncertainty surrounding values of some parameters essential for colorectal cancer screening modelling. • Uncertainty in underlying prevalence of precancerous conditions: 14/05/2014 © The University of Sheffield 0% 10% 20% 30% 40% 50% 60% 70% 80% 30 50 70 90 Polypprevalence Age Polyp prevalence from autopsy studies
  • Methods: Modelling challenges • Uncertainty in screening test characteristics Sensitivity to CRC of the Haemoccult gFOBT : range 0.25-0.96* • Uncertainty(very little data) on growth rates of colorectal cancer and precancerous conditions as: (1) difficult to observe and (2) cancers are removed if found. 14/05/2014 © The University of Sheffield * Burch JA, Soares-Weiser K, St John DJ, et al. Diagnostic accuracy of faecal occult blood tests used in screening for colorectal cancer: a systematic review. J Med Screen 2007;14(3):132-7.
  • CRC natural history model structure 14/05/2014 © The University of Sheffield Normal Epithelium Lowrisk adenomas High risk adenomas Dukes’A CRC Dukes’B CRC Dukes’C CRC Stage D CRC Dead (CRC) Dukes’A CRC clinical Dukes’C CRC clinical Stage D CRC clinical Dukes’B CRC clinical Dead (non-CRC) Transition estimated within model calibration Transition estimated directly from mortality data
  • Methods: Modelling solutions 14/05/2014 © The University of Sheffield 0% 10% 20% 30% 40% 50% 60% 70% 80% 30 50 70 90 Polypprevalence Age Polyp prevalence from autopsy studies 0% 10% 20% 30% 40% 50% 60% 70% 30 40 50 60 70 80Age Polyp detection rate at FS screeningPolyp prevalence Sensitivity of screening test to polyps
  • Methods: Modelling solutions 14/05/2014 © The University of Sheffield 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Age Statedistribution Normal epithelium CRC High risk polyp Lowrisk polyp 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 30 40 50 60 70 80 90 Incidence Age Probability of diagnosis of CRC (due to symptoms or chance detection) State allocation diagram CRC incidence in absence of screening
  • Methods: Modelling solutions 14/05/2014 © The University of Sheffield Parameter values are estimated by using a Bayesian approach in which sets of parameter values for which the model predictions have a good fit to observed data are generated.*
  • Results and conclusions CRC screening is generally cost-effective: Tappenden et al found FS screening to be cost saving and FOBT screening the have an ICER<£3K. “Reappraisal Project” results will be available by the end of October. DOH Press Release 2nd October 2010 “New Action on Cancer to Save Thousands More Lives Every Year Increase detection through a revolutionary new bowel cancer screening technology flexible sigmoidoscopy – a £60 million investment over the next four years to incorporate the latest breakthrough in bowel cancer screening into our existing national programme – saving 3,000 lives a year.” 14/05/2014 © The University of Sheffield
  • Potential impact on policy decisions Results of work will allow us to advise NHS cancer screening on various aspects of FS screening implementation. • At what age will a one-off FS screen provide the greatest health benefits? • How cost effective are combined screening strategies? • Would it be cost effective to create a national polyp centre to process all pathology generated through screening? • What level of FS uptake is required to achieve the same health benefits as the current screening programme? 14/05/2014 © The University of Sheffield
  • Calibration method The Metropolis-Hastings algorithm explores the multi- dimensional parameter space to generate multiple sets of parameters for which the model predictions have a good fit to observed data. 14/05/2014 © The University of Sheffield Parameter1 Parameter2 Difference between model predictions and observed data
  • Cost utility analysis of interventions to return employees to work following long term sickness absence Hazel Squires Research Fellow ScHARR, University of Sheffield Email: h.pilgrim@sheffield.ac.uk 14/05/2014 © The University of Sheffield
  • Introduction • Background • Project process • Modelling methodology • Results • Conclusions • Limitations/ further research
  • Background • People can apply for Incapacity Benefit after they have been on sick leave for >6 months. • Sickness absence in the UK costs the country around 1% of the annual GDP. • There is a need to prevent people from going onto Incapacity Benefit.
  • Process • Effectiveness literature review • Very limited evidence identified – only 3 UK papers • Cost-effectiveness literature review • No studies identified which consider the implications of RTW within the UK setting over a sufficient time frame to capture all impacts on costs and benefits of the interventions • Health economic model
  • Model scope Population: • A cohort of employed men & women that have been on between 1 week and 6 months of sick leave due to musculoskeletal disorders. Interventions: • Workplace intervention • Physical activity and education intervention • Physical activity, education and workplace visit Comparator: • Usual care for musculoskeletal disorders Outcomes: • Cost per quality-adjusted life year (QALY) gained • Cost per day on sick leave avoided Perspective: • NHS and PSS • Societal • Employer
  • Modelling methodology (1) At work On sick leave for 1 week -6 months On sick leave for 6-12 months On sick leave for 12-18 months On sick leave for 18+ months Retirement or death • Markov model which simulates the experience of a hypothetical cohort of employees who are currently on long term sick leave over a working lifetime.
  • Modelling methodology (2) • Costs and QALYs have been calculated for both cohorts starting at age 41 (average age of sickness absence from evidence) through to retirement at age 66. • Discount rate of 3.5% for both costs and utilities. • One-way sensitivity analysis and threshold analysis carried out.
  • Quality of life Utility at work Utility on sick leave Age<35 0.83 0.66 Age 35-45 0.8 0.59 Age 45-55 0.76 0.61 Age>55 0.76 0.61 • Utility scores estimated based on data from the British Household Survey Panel (BHSP) • Based on whether the individual is at work or on sick leave •Sf-36 SF-6D Transformed using SPSS • Impact of potential confounding variables assessed using regression
  • Costs State in the model Perspective NHS & PSS Societal Employer At work £0 £0 £0 1 wk to 6 months sick leave Cost of usual care and intervention incurred by NHS NHS & PSS costs + Employer costs - Transfer costs Cost of intervention incurred by employer + Cost of replacing employee + Production loss over friction period + Salary of replacement employee after friction period + Occupational sick pay + Employer‟s NI contribution 6-12 months sick leave Cost of usual care Cost of usual care Occupational sick pay + Employer‟s NI contribution 12 months+ sick leave Cost of usual care Cost of usual care
  • Key model assumptions (1) • If an employee has not returned to work within 6 months given the intervention they are subsequently no more likely to RTW than an employee who is given usual care • The intervention is only given the first time that the person goes onto LTS • The probability of going onto further episodes of LTS is the same for the intervention cohort and the usual care cohort
  • Key model assumptions (2) • The employee is assumed to receive 15 weeks on full pay and 16.4 weeks on half pay • Wages and productivity of a replacement worker are assumed to be the same as the employee on LTS • The probability of dying is assumed to be no different for people on LTS to people who are at work
  • Results: NHS/ Societal perspective (1) Workplace Intervention Physical activity & Education Physical activity, Education & Workplace visit -£500 -£400 -£300 -£200 -£100 £0 £100 £200 £300 £400 £500 -0.50 -0.40 -0.30 -0.20 -0.10 0.00 0.10 0.20 0.30 0.40 0.50 Difference in QALYs Differenceincosts Cost per QALY gained = £2,758
  • Results: NHS/ Societal perspective (2) -£5,000 £0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 £0 £1,000 £2,000 £3,000 £4,000 £5,000 Additional cost of intervention per employee CostperQALYgained relative risk of RTW of 1.05 (32/1000 additional employees RTW) relative risk of RTW of 1.1 (65/1000 additional employees RTW) relative risk of RTW of 1.2 (130/1000 additional employees RTW) relative risk of RTW of 1.3 (194/1000 additional employees RTW) relative risk of RTW of 1.4 (259/1000 additional employees RTW)
  • Results: Employer perspective Workplace Intervention Physical activity & Education Physical activity, Education & Workplace visit -£200 -£150 -£100 -£50 £0 £50 £100 £150 £200 -1,500 -1,000 -500 0 500 1,000 1,500 Days on sick leave avoided Differenceincosts Cost per day on sick leave avoided = £0.34
  • One way sensitivity analysis • If the difference in health utility between being at work and being on LTS is ≤0.02 then cost per QALY >£20,000. • If the person is aged 55 when going onto LTS then cost per QALY >£8,000. • All other assumptions tested within the model had a limited impact on model results
  • Conclusions • The model suggests that interventions to return employees to work do not have to be very effective to be considered cost-effective in comparison to other technologies routinely funded by the NHS. • This can be explained by the high costs of productivity loss and Incapacity Benefit in comparison to the relatively low cost of the intervention.
  • Model limitations • Effectiveness data based on generally poor evidence carried out in non-UK countries • Costs of interventions are uncertain • Lack of long term follow up data • Only able to consider interventions for musculoskeletal disorders due to limited data on interventions for other sickness absence • Does not take into account different employee/ employer characteristics (although some tested in sensitivity analyses)
  • Further research Further research is required that: • Is within the UK setting; • Provides follow up data beyond 12 months; • Reports comparable return to work outcomes between studies; • Reports quality of life data of the employees who are both at work and on sick leave.
  • Public health modelling: lessons learned from a contraception case study Hazel Squires, Jim Chilcott, Nick Payne, Lindsay Blank, Monica Hernandez, Louise Guillaume ScHARR, University of Sheffield
  • Introduction • Outline methods & results of contraception case study • Discuss key issues in public health modelling based upon this work
  • Aim of NICE contraceptive services project • To assess the effectiveness and cost- effectiveness of interventions to encourage young people to use contraceptives and contraceptive services 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield Model scope • Population: Young people aged between 14 & 16 years within secondary school who have not previously been a parent. • Interventions: • School-based dispensing of hormonal contraceptives to 14-16 year olds • School-based „dispensing‟ of condoms to 14-16 year olds • Comparator: School nurse only • Outcomes: • Cost per age 14 – 16 Pregnancy Averted • Cost per Abortion Averted • Perspective: Public sector perspective (incl. & excl. Benefit payments)
  • 14/05/2014 © The University of Sheffield Conceptual model Impact on „unintended‟ pregnancies Intervention to encourage contraceptive use Sexually transmitted infection (STI) rate Treatment of STIs Impact upon contraceptive services Abortion Miscarriage/ ectopic Pregnancy/ stilbirth Birth Mistimed UnwantedLow birth weight baby Maternity care Social care Mental health problems (eg. depression) Child health problems Crime Government- funded Benefits Education/ employment impacts
  • Methods • Markov model developed within Excel. • Cohort of 100,000 14-year olds who have not had a baby followed over lifetime. • Each year there is a probability of becoming pregnant. • Following conception, dependent upon age, there is a probability of birth, abortion, miscarriage, ectopic pregnancy or stillbirth. • Costs are associated with each of these outcomes. 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield Family, societal and individual characteristics Long term outcomes of child Immediate birth outcomes Long term outcomes of parent(s) Low social class Teenage birth Poor education Poor employment More claims for Means- tested Benefits Teenage pregnancy Crime Low birth weight Foetal death Poorly educated mother Poor behaviour/ education as child Other observable or unobservable characteristics A B C Outcomes of teenage birth
  • Literature review: long term outcomes • Econometric literature review assessing long term consequences of a teenage birth in the UK • Controlling for observable & unobservable characteristics (eg. personality) which might predispose a young woman to teenage motherhood • Eg. Comparing those who‟ve had a teenage miscarriage with those that have had a baby as a teenager 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield Model schematic t t+1 t+1 t Concept ion (may be mistime d or unwante d) No conceptio n (may Aborti on Birth Miscarriage/ ectopic pregnancy/ stillbirth Additional proportion of low birth weight babies Additional Benefit claims of teenage parents Cohort of young people ST I No STI t+ 1 t t t+ 1
  • 14/05/2014 © The University of Sheffield Key model assumptions • The negative impacts of a teenage birth include: • 4% more likely to receive Income Support & associated Benefits until age 35 years; • 90% will receive Income Support immediately, hence the earlier the birth, the more years it is received for; • <1% more babies will be low birth weight. • 50% of averted teenage births are mistimed & 50% are unwanted. • STIs are assumed to be transmitted to 1 person only.
  • Health outcomes • Usually increasing life years is considered to be a good thing. In this case, contraceptives aim to prevent life. • There is no long term quality of life evidence associated with motherhood by age, adjusted for socioeconomic status • Similarly, there is no quality of life evidence associated with the child by age of motherhood 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield Cost-effectiveness decision rules More expensive Less effective More expensive More effective Less expensive Less effective Less expensive More effective Difference in effectivenessDifferenceincosts NO YES
  • 14/05/2014 © The University of Sheffield Results (excl. Benefits) Cohort of 100,000 14-year olds followed over a lifetime -£1,000 -£800 -£600 -£400 -£200 £0 £200 £400 £600 £800 £1,000 0 200 400 600 800 1000 1200 Thousands Age 14 - 16 pregnancies averted Differenceincosts Dispensing condoms within schools Dispensing hormonal contraceptives within schools Expected cost per age 14 – 16 pregnancy averted = £37 (condoms) & £110 (hormonal) compared with no intervention Approx. 50% probability that cost-saving
  • 14/05/2014 © The University of Sheffield Results (incl. Benefits) Cohort of 100,000 14-year olds followed over a lifetime -£30 -£25 -£20 -£15 -£10 -£5 £0 0 100 200 300 400 500 600 700 800 900 1000 Millions Age 14 - 16 pregnancies averted Differenceincosts Dispensing condoms within schools Dispensing hormonal contraceptives within schools Dominates no intervention
  • Key issues in public health modelling • Model scoping & granularity • Extrapolating outcomes over the long term • Valuing outcomes • Handling uncertainty 14/05/2014 © The University of Sheffield
  • Model scoping & granularity (1) • Public health interventions & their settings are complex. • Often need to capture potentially adaptive behaviour (eg. sexual behaviour). • Often several health areas to be modelled (in this case, pregnancies & STIs). • Intersectoral impacts (not just health) 14/05/2014 © The University of Sheffield
  • Model scoping & granularity (2) • Formal OR problem structuring methods such as cognitive mapping could be used: 1. To determine the model scope 2. To make sure that all important costs & outcomes associated with the intervention are captured • Further research around the use of these methods applied to public health modelling is required 14/05/2014 © The University of Sheffield
  • Extrapolating outcomes over the long term • Trials generally collect intermediate outcomes. • The relationship between these outcomes & final outcomes needs to be understood. • Long term outcomes (eg. employment) are dependent upon many factors; hence econometric techniques will generally be required to disentangle the impacts of the intervention. 14/05/2014 © The University of Sheffield
  • Valuing outcomes • Within health economics, benefits are generally measured in terms of QALYs gained • Within public health economic modelling, the QALY measure may not be sufficiently broad eg. intersectoral costs/ consequences • There is also generally limited evidence around utilities • Further research is required around alternative outcome measures 14/05/2014 © The University of Sheffield
  • Handling uncertainty • Within health economics, uncertainty within model parameters is formally assessed using probabilistic sensitivity analysis (PSA) • The structure of public health models is also often highly uncertain eg. modelling mistimed pregnancies • It is important to parameterise structural uncertainties & include these within the PSA as outlined by Bojke et al. (2006). 14/05/2014 © The University of Sheffield
  • Conclusions • Dispensing contraceptives within schools to 14 – 16 yr olds is likely to be cost-effective • Key areas for methodological development within public health modelling: • Model scoping & level of granularity • Extrapolating outcomes over the long term • Valuing outcomes • Handling uncertainty
  • High dose lipid-lowering therapy Is this strategy cost- effective? Roberta Ara Senior Research Fellow ScHARR, University of Sheffield Email: r.m.ara@sheffield.ac.uk
  • Background Statin therapy for secondary CVD should “usually be initiated with a drug with a low acquisition cost (taking into account required daily dose and product price per dose)” (www.nice.org.uk/TA094, 2006) Savings: £1 billion over 5 years {Moon BMJ, 2006} 69% PCT switch to generic statins → NHS savings ≥ £85m/year (www.institute.nhs.uk, 2007)
  • The decision problem To evaluate the cost-effectiveness of high dose statins (atorvastatin 80mg/d, rosuvastatin 40mg/d & simvastatin 80mg/d) versus simvastatin 40mg/d in individuals with acute coronary syndrome.
  • Markov health states Post HS Post HS Post HS Post HS Post HS Post HS QE: Hospitalisation unstable angina QE: New non-fatal MI QE: Revascularisation New non fatal MI New hospitalisation for unstable angina New non fatal stroke CVD Mortality Transitions from all health states Non CVD Mortality
  • Clinical Data Literature review: 28 RCTs > 12 week duration Benefit: LDL-c Synthesis: MTC Relationship: LDL-c & CV events
  • Mean Relative Risks (95% CI) Atorvastatin 80mg/d Rosuvastatin 40mg/d Simvastatin 40mg/d Simvastatin 80mg/d Non-fatal MI 0.425 (0.302 - 0.544) 0.378 (0.241 - 0.51) 0.588 (0.500 - 0.675) 0.510 (0.404 - 0.613) Non fatal stroke 0.623 (0.508 - 0.737) 0.593 (0.465 - 0.717) 0.730 (0.647 - 0.813) 0.679 (0.580 - 0.777) Stroke death 0.809 (0.429 - 1.242) 0.794 (0.384 - 1.261) 0.863 (0.593 - 1.173) 0.837 (0.516 - 1.206) CHD death 0.580 (0.445 - 0.715) 0.545 (0.397 - 0.692) 0.699 (0.601 - 0.796) 0.642 (0.527 - 0.758)
  • Adherence to therapies Scenario A Scenario B Yr 1 Yr 5 Yr 1 Yr 5 S40 ITT ITT ITT ITT A80 ITT ITT 95% 85% R40 ITT ITT 95% 85% S80 ITT ITT 95% 85%
  • Treatment costs Annual cost Sensitivity analyses Atorvastatin 80mg/d £368 £92 Rosuvastatin 40mg/d £387 Simvastatin 40mg/d £17 Simvastatin 80mg/d £34 Monitoring cost £76
  • Results Comparing with simvastatin 40mg/d: Atorvastatin 80mg/d Rosuvastatin 40mg/d Simvastatin 80mg/d Scenario A £17,469 £12,484 £5,319 Scenario B £17,217 £12,277 £5,226
  • Results – reduced cost for Atorvastatin simvastatin 40mg/d simvastatin 80mg/d atorvastatin 80mg/d rosuvastatin 40mg/d 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 £0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 Threshold ratio Probabilitycosteffective
  • Summary & Conclusion • Rosuvastatin 40mg/d is currently the most cost- effective alternative • Atorvastatin 80mg/d is the most cost-effective (@ 25% of current price) • Simvastatin 80mg/d is not recommended Current PCT policies intended to minimise primary care drug acquisition costs result in sub-optimal treatment for patients with ACS
  • A cost-effectiveness model of prostate cancer screening Matthew Mildred, Jim Chilcott, Silvia Hummel ScHARR, University of Sheffield
  • 05/04/2011 © The University of Sheffield Contents • Introduction to the project and topic • Disease natural history model • Data and model calibration • Validation • Results • Conclusions
  • 05/04/2011 © The University of Sheffield The project • Client: UK National Screening Committee • Purpose: Help determine IF a national prostate cancer screening programme should occur AND which screening strategy is best. • Objectives: Estimate costs, benefits and resource implications of alternative screening options.
  • 05/04/2011 © The University of Sheffield Introduction to prostate cancer The prostate is a small gland in men behind the bladder. The most common cancer in men in UK (excluding non-melanoma skin cancer) In 2008: Over 37,000 men diagnosed Over 10,000 men died from prostate cancer
  • 05/04/2011 © The University of Sheffield Aim of screening: Reduce cancer mortality, morbidity and treatment costs through early diagnosis and intervention. Current evidence: In 2009 two large RCTs reported apparently inconsistent results in terms of the death rate ratio: • ERSPC – significant reduction in PCa death rate • PLCO – no statistically significant reduction
  • 05/04/2011 © The University of Sheffield Challenges: • Effectiveness of different screening programmes unknown. • Scarce data around disease process due to its unobservable nature. • Multiple unknown parameters in cancer screening model.
  • 05/04/2011 © The University of Sheffield Solution: • Develop loosely parameterised cancer screening simulation model. • Calibrate unobservable model parameters to observed data. • Estimate impact of prostate cancer screening using calibrated model.
  • 05/04/2011 © The University of Sheffield About the model: • Disease natural history model (Simul8) • Calibration module (Excel, Visual Basic) • Simulation model of prostate cancer screening (Simul8) • Resource impact model (Excel)
  • 05/04/2011 © The University of Sheffield Screening strategies investigated No. Screens Screening Age (years) Screening Interval (years) Single 50 N/A 55 60 65 70 Repeat 50-70 2, 4 50-74 1, 2, 4 55-70 2, 4 55-74 2, 4
  • 05/04/2011 © The University of Sheffield Outputs: • Age-specific incidence • Age-specific mortality • Prostate cancer stage distributions • Over-detection rate • Lead time • Life years gained, QALYs gained • Probability of developing prostate cancer • Etc...
  • 05/04/2011 © The University of Sheffield PCa Onset Screen Detection PCa Mortality Clinical Diagnosis Other Cause Mortality PCa Onset Screen Detection PCa Mortality Clinical Diagnosis Other Cause Mortality Lead-time Lead-time Over-detection: Relevant: Definitions & terms used
  • 05/04/2011 © The University of Sheffield Disease natural history model
  • 05/04/2011 © The University of Sheffield Data Data Source Age specific cancer incidence Office of National Statistics Cancer stage distributions ProtecT RCT UK Cancer Registry (ERIC) Gleason score distributions ProtecT RCT UK Cancer Registry (ERIC) PSA/biopsy test characteristics ERSPC RCT (Rotterdam section) Progression Free Survival ERSPC RCT (Rotterdam section) Overall Survival ERSPC RCT (Rotterdam section)
  • 05/04/2011 © The University of Sheffield Calibration process
  • 05/04/2011 © The University of Sheffield Total SSE during calibration
  • 05/04/2011 © The University of Sheffield Validation: Incidence
  • 05/04/2011 © The University of Sheffield Validation: PCa mortality
  • 05/04/2011 © The University of Sheffield Validation: BAUS
  • 05/04/2011 © The University of Sheffield Results: Incidence 0 2 4 6 8 10 12 14 16 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85+ PCaincidence(/1000yrs) Age band Noscreening Once at 50 50-74every 4 years 50-74every 2 years 50-74every year
  • 05/04/2011 © The University of Sheffield Results: Mortality 0 1 2 3 4 5 6 7 8 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85+ Age band PCamortality(/1000yrs) No screening Once at 50 50-74 every4 years 50-74 every2 years 50-74 everyyear
  • 05/04/2011 © The University of Sheffield Over-detection & Lead time: Once at 50 50-74 every 4 years 50-74 every 2 years 50-74 every year Over- detection rate 18% 44% 45% 46% Lead time (for over- detected cases) 15.2 yrs 11.6 yrs 12.5 yrs 13.0 yrs
  • 05/04/2011 © The University of Sheffield Conclusions: A minimal life gain is offset by the high levels of disease management and over-diagnosis: • One off screening: life gain of 0.004 years (1.2 days) with 36 years of additional disease management • Repeat screening: life gain of 0.03 years (10-11 days) with 67-84 years of additional disease management
  • 05/04/2011 © The University of Sheffield Have you heard our findings? BBC News 06/12/2010 http://www.bbc.co.uk/news/health-11930979
  • 05/04/2011 © The University of Sheffield Acknowledgements: • Dr Anne Mackie and Prof Julietta Patnick at the UK National Screening Committee • The South West Public Health Observatory • The British Association of Urological Surgeons • The ProtecT team
  • Healthcare costs alone do not describe the total costs directly attributable to Ankylosing Spondylitis R. Ara1, R. Rafia1, J. C. Packham2,3, K L Haywood4, E.L. Healey2,4 1University of Sheffield, 2Staffordshire Rheum Centre, 3University of Warwick, 4Arthritis Research Campaign, Keele. 14/05/2014 © The University of Sheffield
  • Introduction Ankylosing Spondylitis (AS) is a chronic, progressive disease and prognosis is often poor. In addition to direct health care costs, AS is associated with substantial indirect costs due to privately funded health care, absenteeism from work and early retirement. 14/05/2014 © The University of Sheffield
  • Objectives • To explore the total costs directly attributable to AS patients attending rheumatological centres in the UK. • To explore the relationship between total costs and disease severity (classified using the Bath AS Disease Activity Index (BASDAI) and the Bath AS Functional Index (BASFI) variables) 14/05/2014 © The University of Sheffield
  • Methods Health care costs included: medications, inpatient, GP/ outpatient, physiotherapy, hydrotherapy Non medical indirect costs included: productivity losses (unemployment, early retirement, absenteeism, presenteeism) Productivity losses were assessed using human capital approach Absenteeism = number of days off sick (average 5 working days per week) Presenteeism = relative to age/gender adjusted gross pay & WLQ-16 Two-step model used to explore relationship between disease severity and costs using data collected at baseline 1: logistic regression for probability incurring cost 2: generalised linear model conditional on incurring any cost Performance of the estimated model tested on data collected at 6 months
  • Results (1) Healthcare Utilisation: 41% reported no health care resources 45% reported no NHS funded healthcare resources 26% reported ≥ 1 medication 71/162 received anti-TNF treatment 10 hospitalisations due to AS (mean duration 11 days) 215 reported ≥ 1 GP consultation Number consultations > for greater disease activity (1.7 vs 2.6) Number consultations > for non workers (2.8 vs 1.9) 19% ≥ 1 physiotherapy session (33% of these self funded) 11% ≥ 1 hydrotherapy session (46% of these self-funded) Number sessions > non workers (7.1 vs 13.1) Number sessions > for greater disease activity (7.0 vs 12.3) 14/05/2014 © The University of Sheffield
  • Results (2) Total Costs: The distribution of costs is heavily skewed with small number incurring high costs and substantial proportion incurring none. Average 3 month direct health care costs = £420 £988 (median = £27) Average 3 month total costs = £2,837 £3,358 (median = £1,228) £0 £1,000 £2,000 £3,000 £4,000 £5,000 £6,000 Moderate (score <4) Severe (4 = score < 6) Very severe (6 = score = 10) Patients subgroupedby BASDAI(BASFI) bands 3monthtotalcost BASFI mean cost BASDAI mean cost BASFI median cost BASDAI median cost Mean 3 month total costs sub-grouped by disease severity bands
  • Results (3) Absenteeism/ presenteeism 25% of working age reported not working or early retirement due to AS 25% (85/339) of workers reported time off sick 93% (315/339) of workers reported a reduction in productivity while at work Mean number of days off sick due to AS = 5.7 (range 1 – 62) Mean reduction in productivity at work = 20% (range = 1 – 90%) 14/05/2014 © The University of Sheffield
  • Results (4) Using BASDAI & BASFI to predict AS disease costs Model 1: Probability of incurring costs = 3.26106 +0.13308*BASFI + 0.27695*BASDAI – 0.01323*BASFI*BASDAI + 0.30472*male – 0.04869*age – 0.01171*disease duration Model 2: 3 month total costs, conditional on costs incurred = 6.89107 + 0.26335*BASFI + 0.12048*BASDAI - 0.01411*BASFI*BASDAI + 0.46456*male – 0.01679*age + 0.00364*disease duration £0 £1,000 £2,000 £3,000 £4,000 £5,000 £6,000 £7,000 - 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 BASDAI Score Indirectcosts Observed mean cost Predicted cost Actual and predicted total 3 month costs
  • Conclusion This study shows that direct healthcare costs alone do not describe the total costs associated with AS and that productivity losses associated with AS are considerable. Additional research examining interventions and measures that improve both presenteeism and absenteeism will have the largest affect on the cost to society. 14/05/2014 © The University of Sheffield
  • The direct healthcare resource costs associated with Ankylosing Spondylitis patients attending a UK secondary care rheumatology unit RM Ara1, JC Packham2, KL Haywood3 1University of Sheffield; 2Staffordshire Rheumatology Centre, Stoke on Trent; 3Royal College of Nursing Institute, Oxford, UK 14/05/2014 © The University of Sheffield
  • Introduction • Anti-TNF inhibitors such as etanercept (ETN), are now licensed for treating patients with severe Ankylosing Spondylitis (AS). • These treatments have larger cost implications than previously available therapies and comprehensive economic evaluations of these novel treatments in individual disease areas are increasingly requested by policy decision makers to inform reimbursement decisions. 14/05/2014 © The University of Sheffield
  • Objectives • To explore the direct healthcare resources utilised by AS patients attending a UK secondary care rheumatology unit to inform an economic evaluation of ETN in AS patients. • To establish if resources, and thus health care costs, vary by disease severity (as classified using the Bath AS Disease Activity Index (BASDAI) and the Bath AS Functional Index (BASFI) variables).
  • Methods Costs: • Costs were assessed using a micro-costing approach starting with a detailed inventory and measurement of resources consumed by the patient. • Unit cost multipliers were applied to the quantity of each type of resource consumed. • The mean cost per patient is estimated using the total cost divided by the total number of patients included. Resources included: • NHS physiotherapy, outpatient appointments; inpatient days, laboratory based tests (FBR, ESR, LFT, U&E), X- rays, scans, prescribed medications. 14/05/2014 © The University of Sheffield
  • Results (1) Mean Annual Disease Costs • The mean total annual costs is estimated to be £1,837 (s.d. = £2,764; median = £772). • The total annual costs range from £101 to £18,012 with an asymmetric distribution (just 11/147 patients have annual costs concentrated beyond £7,000). • The mean annual cost per patient is correlated with both measures of disease activity (BASDAI) and functional disability (BASFI) £0 £500 £1,000 £1,500 £2,000 £2,500 £3,000 £3,500 £4,000 <30 30-39 40-49 50-59 60-69 70+ Disease severity (disease activity (BASDAI)orfunctionaldisability (BASFI)) Meanannualcost £0 £500 £1,000 £1,500 £2,000 £2,500 £3,000 £3,500 £4,000 Medianannualcost BASDAImean annualcost BASFImean annualcost BASDAImedian annualcost BASFImedian annualcost BASDAI < 60 BASDAI > 60 BASFI < 60 BASFI > 60 Physiotherapy £281; 22% £1,575;45 % £123; 13% £1,547; 43% Hospitalisation £303; 23% £785; 23% £243; 26% £779; 22% Medication £315; 24% £479; 14% £225; 24% £615; 17% Outpatient/tests £397; 31% £668; 19% £351; 37% £688; 19% The mean annual costs by levels of disease activity and functional disability Mean (proportion) annual costs by resource type and disease severity
  • Results (2) Using BASDAI & BASFI to predict AS disease costs • Both the BASDAI and BASFI measurements were moderately correlated with the log transformed annual costs (Pearson correlation = 0.40 and 0.48 respectively; p < 0.001). • The model# with the best predictive ability was Annual direct costs = exp(0.006*BASDAI + 0.016*BASFI + 5.862) #Adj R2=0.24 hence BASDAI and BASFI only partially account for the variability in costs. • A 10 unit increase in both BASDAI and BASFI measurements incurs an increment of approximately £130 for scores of 40 and below; and an increment of approximately £480 for scores of 70 and above.
  • Discussion/Future Research • The results of this costing study give an indication of the range of direct healthcare costs associated with patients who have AS and are eligible for anti-TNF treatments in the UK. • Due to time constraints only readily accessible data such as clinical visits; inpatient care, technical procedures including radiographic examinations, prescribed medications, and physiotherapy appointments were included. • The study would benefit from inclusion of additional information such as the number of GP visits. • Further research into the most (cost)-effective provision of physiotherapy to patients with AS would be constructive. • As AS is a progressive debilitating disease, it may be appropriate to include non-medical resources such as aids and appliances or formal household care and paid productivity losses such as absence from paid work.
  • A treatment option for patients with severe active Ankylosing Spondylitis: the costs and benefits associated with etanercept RM Ara1, A Reynolds2, P Conway2 1University of Sheffield; 2Wyeth Pharmaceuticals, UK 14/05/2014 © The University of Sheffield
  • Introduction • Ankylosing Spondylitis (AS) is a progressive inflammatory disease that can cause irreversible skeletal damage. Typically presenting in young males, the long term prognosis is poor. High levels of pain and severe loss of physical function can have a large effect on health related quality of life (HRQoL). • The Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) and the Bath Ankylosing Spondylitis Functional Index (BASFI) are validated and established measures in AS patients.
  • Objective To explore the costs and benefits associated with etanercept (ETN) in patients with severe AS in the UK under the BSR criteria. 14/05/2014 © The University of Sheffield
  • Methods • The model (written in Excel 2003) estimates the costs and benefits associated with ETN (2x 25mg/week) plus NSAIDS compared with NSAIDs alone. • Effectiveness of ETN is based on patient level data obtained from two RCTs • Placebo data are used to inform the comparator (88% of patients received NSAIDS) • Effectiveness of treatment are measured by changes in BASDAI and BASFI at weeks 12 and 24 • Initial response is supported by three year open label data [6,7] • Direct healthcare costs associated with AS are obtained from a UK costing study and are estimated using the relationship [8] • Annual direct costs = exp(0.006*BASDAI + 0.016*BASFI + 5.862) • QoL data (EQ-5D) collected during the RCT are used to estimate quality adjusted life years (QALYS) using the relationship • Utility = 0.923 – 0.004*BASFI – 0.004*BASDAI • A 25 year time horizon is used to reflect the chronic progressive nature of AS and results are also presented for shorter horizons • Univariate sensitivity analyses are used to explore the impact of varying individual key parameters • Costs and benefits are discounted at 3.5% per annum
  • Assumptions • Natural disease progression: BASFI increases at 0.7 units per annum • For responders to ETN: BASDAI & BASFI measurements remain constant at values observed in RCTS • BASDAI & BASFI measurements revert to baseline values and follow natural disease progression after withdrawal from ETN • 10% annual withdrawal from ETN due to lack of efficacy/ adverse effects • Age and sex related life expectancy is adjusted using ratio of 1.5 and assumed equal in both arms 14/05/2014 © The University of Sheffield
  • Base case results 14/05/2014 © The University of Sheffield Time horizon (years) 2 5 15 25 ETN 1,185 2,646 5,739 7,285 Comp 817 1,831 4,286 5,700 Incremental 368 815 1,453 1,585 ETN £13,042 £26,390 £51,415 £62,517 Comp £2,890 £7,109 £18,596 £26,538 Incremental £10,152 £19,281 £32,819 £35,978 £27,594 £23,649 £22,580 £22,704 Total discounted QALYS Total discounted costs (£,000) Discounted incremental cost per QALY
  • Univariate sensitivity analyses The tornado diagram shows that the three variables that have the largest impact of the 25 year results are: • the values used to represent HRQoL • the annual withdrawal rates • the health care costs directly attributable to AS 14/05/2014 © The University of Sheffield £10,000 £15,000 £20,000 £25,000 £30,000 £35,000 Discount BASDAI/BASFI capped at 80 BASFI progression rate Disease costs Annual withdrawal Quality of Life CI Cost per QALY Tornado diagram showing the parameters which have the largest impact on the results
  • Monte Carlo results • Using a 25 year horizon and a £30k per QALY threshold, ETN would be considered a cost-effective treatment when compared to NSAIDs in individuals with severe AS. The corresponding cost effectiveness acceptability curve shows that ETN is 93% likely to be cost effective when using a £25k per QALY threshold. 14/05/2014 © The University of Sheffield £0 £10,000 £20,000 £30,000 £40,000 0.0 0.5 1.0 1.5 2.0 Incrementaleffectiveness Incrementalcosts year 25 yaer 15 yaer 5 year 2 £20,000 per QALYgained £30,000 per QALYgained Cost effectiveness plane ETN plus NSAIDs compared with NSAIDs
  • Areas for future research • Potential impact of radiological progression on long term disability • Efficacy of ETN in individuals with BASDAI < 40 • Strengthen relationship between BASDAI/BASFI and both costs and utilities • BASDAI & BASFI further explore the relationship to disease costs • Effect of ETN on early retirement • Develop prognostic algorithms to identify patients who would benefit the most from ETN • Quantify natural disease progression and disease progression for individuals responding to ETN
  • An economic evaluation of etanercept in the treatment of patients with psoriatic arthritis RM Ara and R Rafia ScHARR, University of Sheffield 14/05/2014 © The University of Sheffield
  • Introduction • Psoriatic Arthritis (PsA) is a chronic, progressive disease and prognosis is often poor. • Anti_TNF agents such as etanercept are effective alternatives for patients whose current treatment is symptomatic.
  • Objective • To explore the costs and benefits associated with etanercept in the treatment for patients with psoriatic arthritis to inform a NICE submission.
  • Methods • The model compares ciclosporin to anti-TNF agents (etanercept, adalimumab, infliximab) and it is assumed that patients have failed methotrexate and sulphasalazine prior entering the model. • Relationship between HAQ and costs associated with PsA and HAQ and health related quality of life are then used to estimate the long term costs and benefits accrued. • Treatment effectiveness was derived from a Mixed Treatment Comparison published in a previous STA. • Treatment discontinuation was modelled using PsARC response rate at 12 and/or 24 weeks using results from the MTC. Long term withdrawal rate was derived from data from a recent observational study conducted by the BSRBR. • A set of 27 univariate sensitivity analysis was performed to test the robustness of the model to main assumptions. • The overall uncertainty was examined using a Monte Carlo approach. • Costs and quality of life benefits were discounted at 3.5% per annum as per NICE recommandations for economic evaluation • The model was extended beyond the trial duration to a 50 years time horizon
  • Relationship between HAQ and HrQoL Relationship estimated from PRESTA trial for the base case: Utility (EQ-5D) = 0.8996 – 0.4559 * HAQ – 0.0010 * Age + 0.0201 * male + 0.0031 *Age * HAQ – 0.0388 * male * HAQ
  • Relationship between HAQ and Direct health care Costs • Health care costs include both primary and secondary care resources. • The relationship between HAQ and costs was examined using a Generalised Linear Model assuming a poison distribution and a log link: Cost = 3.5367 + 2.0484 * HAQ + 0.0260 * Age – 0.0119 * Age * HAQ £0 £1,000 £2,000 £3,000 £4,000 £5,000 £6,000 £7,000 <=1.2 1.2<=1.4 1.4<=1.6 1.6<=1.8 1.8<=2.0 2.0<=2.2 2.2<=2.4 2.4<=2.6 2.6<=2.8 HAQ band Costs Observed cost Predicted cost Actual and predicted annual cost by HAQ
  • Base case results MEAN QALY MEAN COST (£) ICER RESULTS AT 50YEARS (LIFETIME) ETN 6.90 £65,650 £12,480 ADALIMUMAB 6.54 £61,381 EXTENDEDLY DOMINATED BY ETN INFLIXIMAB 6.39 £66,867 DOMINATED BY ADL CICLOSPORINE 5.96 £53,860
  • Uncertainty in results • The model results were sensitive to both the magnitude of the rebound of HAQ after withdrawal from treatment, and annual HAQ progression rates, relationship between HAQ and QoL and discount rates. • Results from the PSA, demonstrate one is 65% confident that etanercept is a cost-effective strategy when using a threshold of £20,000 per QALY. 14/05/2014 © The University of Sheffield 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 20000 40000 60000 80000 100000 120000 Willingness To Pay (£) Probabilitycosteffective Etanercept Cyclosporine Infliximab Adalimumab Cost-Effectiveness- Acceptability Curve
  • Conclusion While this evaluation provides evidence on the potential cost-effectiveness of etanercept compared to other anti-TNF agents or ciclosporin for adults with PsA, there are limitations with the evidence based used to populate the model. 14/05/2014 © The University of Sheffield
  • Evidence Review and Synthesis 14/05/2014 © The University of Sheffield
  • Extensive experience of systematic review and evidence synthesis work for • National Institute for Health and Clinical Excellence (NICE), • NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) • NICE Public Health Collaborating Centre (PHCC) • Multiple clients in industry. 14/05/2014 © The University of Sheffield
  • Systematic review of clinical evidence High quality systematic reviews for submissions to key reimbursement bodies or for publication • Clinical reviews • Critical appraisal of reviews • Review updates Other review methods • Mapping reviews • Reviews of model parameters • Rapid reviews • Research reports 14/05/2014 © The University of Sheffield
  • Generalised evidence synthesis Classical and Bayesian meta analysis, including network meta analysis • RCTs • Observational studies • Diagnostic studies • Qualitative studies (meta synthesis) • Mixed methods
  • Examples of Systematic reviews conducted in HEDS 14/05/2014 © The University of Sheffield
  • PET and MRI for the assessment of axillary lymph node metastases in early stage breast cancer A Systematic review Susan Harnan Research Associate ScHARR, University of Sheffield Email: s.harnan@sheffield.ac.uk 14/05/2014 © The University of Sheffield
  • Background • Axillary staging is important for breast cancer staging and treatment planning • Current techniques include • Axillary lymph node dissection (ALND) • Sentinel lymph node biopsy (SLNB) • Sampling techniques such as four node sampling (4- NS) • All these procedures (ALND, SLNB and 4-NS) have short and long term adverse effects such as lymphoedema 14/05/2014 © The University of Sheffield
  • Diagnostic pathway • NICE diagnostic pathway *Either fine needle aspiration cytology (FNAC) or core biopsy. • Positron emission tomography (PET) and magnetic resonance imaging (MRI) may offer alternatives with fewer adverse events • Could replace SLNB/4-NS • Or be added to the pathway 14/05/2014 © The University of Sheffield
  • HTA report • National Institute for Health Research Health Technology Assessment (HTA) programme commissioned research to evaluate PET and MRI: • Diagnostic accuracy • Cost-effectiveness • Patient outcomes • Work carried out by ScHARR: Katy Cooper, Yang Meng, Sue Harnan, Sue Ward, Patrick Fitzgerald, Diana Papaioannou • With clinical advisers: Lynda Wyld, Christine Ingram, Iain Wilkinson, Eleanor Lorenz 14/05/2014 © The University of Sheffield
  • Methods • Searched eleven databases • Inclusion criteria: • Assessed diagnostic accuracy of PET or MRI for assessment of axillary metastases in patients newly diagnosed with early-stage invasive primary breast cancer • PET studies only included if ≥ 20 patients • Studies with > 20% non-early stage, non- newly diagnosed or DCIS were excluded. • Quality Assessment of Diagnostic Accuracy Studies (QUADAS) checklist used. 14/05/2014 © The University of Sheffield
  • Diagnostic accuracy studies • Patients are classified against a reference standard (eg ALND or SLNB) as true positive, true negative, false positive or false negative • Sensitivity is % of positive patients correctly identified • Specificity is % of negative patients correctly identified • Studies only included if TP, TN, FP and FN were reported or could be calculated. • Bivariate random effects model used to meta-analyse data as sensitivity and specificity are linked.
  • Results 14/05/2014 © The University of Sheffield
  • PET results • Quality generally acceptable, though some problems with patient spectrum, blinding, availability of relevant clinical information and reporting of uninterpretable results. • 26 studies of PET or PET/CT 14/05/2014 © The University of Sheffield
  • PET results – forest plots PET/CT Study Chae 2009 Heusner 2009 Kim 2009 Taira 2009 Fuster 2008 Ueda 2008 Veronesi 2007 TP 16 13 27 13 14 34 38 FP 12 3 0 5 0 6 5 FN 17 9 8 14 6 25 65 TN 63 29 102 60 32 118 128 Sensitivity 0.48 [0.31, 0.66] 0.59 [0.36, 0.79] 0.77 [0.60, 0.90] 0.48 [0.29, 0.68] 0.70 [0.46, 0.88] 0.58 [0.44, 0.70] 0.37 [0.28, 0.47] Specificity 0.84 [0.74, 0.91] 0.91 [0.75, 0.98] 1.00 [0.96, 1.00] 0.92 [0.83, 0.97] 1.00 [0.89, 1.00] 0.95 [0.90, 0.98] 0.96 [0.91, 0.99] Sensitivity 0 0.2 0.4 0.6 0.8 1 Specificity 0 0.2 0.4 0.6 0.8 1 PET only Study Cermik 2008 Gil-Rendo 2006 Weir 2005 Agresti 2004 Fehr 2004 Inoue 2004 Lovrics 2004 Wahl 2004 Barranger 2003 Guller 2002 Nakamoto 2002 Rieber 2002 van der Hoeven 2002 Greco 2001 Noh 1998 Smith 1998 Adler 1997 Avril 1996 Utech 1996 TP 34 120 5 20 2 21 9 66 3 6 6 16 8 68 14 13 16 19 44 FP 11 2 3 3 1 2 2 40 0 1 1 1 1 13 0 1 6 0 20 FN 39 22 13 11 8 14 16 43 12 8 7 4 24 4 1 2 4 5 0 TN 104 131 19 37 13 44 63 159 17 16 16 19 37 82 12 22 26 17 60 Sensitivity 0.47 [0.35, 0.59] 0.85 [0.77, 0.90] 0.28 [0.10, 0.53] 0.65 [0.45, 0.81] 0.20 [0.03, 0.56] 0.60 [0.42, 0.76] 0.36 [0.18, 0.57] 0.61 [0.51, 0.70] 0.20 [0.04, 0.48] 0.43 [0.18, 0.71] 0.46 [0.19, 0.75] 0.80 [0.56, 0.94] 0.25 [0.11, 0.43] 0.94 [0.86, 0.98] 0.93 [0.68, 1.00] 0.87 [0.60, 0.98] 0.80 [0.56, 0.94] 0.79 [0.58, 0.93] 1.00 [0.92, 1.00] Specificity 0.90 [0.84, 0.95] 0.98 [0.95, 1.00] 0.86 [0.65, 0.97] 0.93 [0.80, 0.98] 0.93 [0.66, 1.00] 0.96 [0.85, 0.99] 0.97 [0.89, 1.00] 0.80 [0.74, 0.85] 1.00 [0.80, 1.00] 0.94 [0.71, 1.00] 0.94 [0.71, 1.00] 0.95 [0.75, 1.00] 0.97 [0.86, 1.00] 0.86 [0.78, 0.93] 1.00 [0.74, 1.00] 0.96 [0.78, 1.00] 0.81 [0.64, 0.93] 1.00 [0.80, 1.00] 0.75 [0.64, 0.84] Sensitivity 0 0.2 0.4 0.6 0.8 1 Specificity 0 0.2 0.4 0.6 0.8 1
  • PET sensitivity analyses
  • MRI results • Quality generally good. Some problems with patient spectrum, availability of relevant clinical information and uninterpretable results. • Small numbers, different methods • Nine studies of MRI, meta-analysed using highest reported sensitivity and specificity from each study
  • MRI results – forest plot 14/05/2014 © The University of Sheffield USPIO-enhanced MRI Study Kimura 2009 Harada 2007 Memarsadeghi 2006 Stadnik 2006 Michel 2002 TP 2 23 6 5 9 FP 0 2 0 1 0 FN 0 0 0 0 2 TN 8 8 16 4 7 MRI criteria USPIO uptake USPIO uptake USPIO uptake USPIO uptake USPIO + >10mm + round Sensitivity 1.00 [0.16, 1.00] 1.00 [0.85, 1.00] 1.00 [0.54, 1.00] 1.00 [0.48, 1.00] 0.82 [0.48, 0.98] Specificity 1.00 [0.63, 1.00] 0.80 [0.44, 0.97] 1.00 [0.79, 1.00] 0.80 [0.28, 0.99] 1.00 [0.59, 1.00] Sensitivity 0 0.2 0.4 0.6 0.8 1 Specificity 0 0.2 0.4 0.6 0.8 1 Gadolinium-enhanced MRI Study Mumtaz 1997 TP 36 FP 6 FN 4 TN 29 MRI criteria Gd uptake + >5mm Sensitivity 0.90 [0.76, 0.97] Specificity 0.83 [0.66, 0.93] Sensitivity 0 0.2 0.4 0.6 0.8 1 Specificity 0 0.2 0.4 0.6 0.8 1 Dynamic gadolinium-enhanced MRI Study Murray 2002 Kvistad 2000 TP 10 20 FP 17 4 FN 0 4 TN 20 37 MRI criteria Dynamic Gd + >4sq-mm Dynamic Gd Sensitivity 1.00 [0.69, 1.00] 0.83 [0.63, 0.95] Specificity 0.54 [0.37, 0.71] 0.90 [0.77, 0.97] Sensitivity 0 0.2 0.4 0.6 0.8 1 Specificity 0 0.2 0.4 0.6 0.8 1 MR spectroscopy (in vivo) Study Yeung 2002 TP 11 FP 0 FN 6 TN 10 MRI criteria MR spectroscopy (in vivo) Sensitivity 0.65 [0.38, 0.86] Specificity 1.00 [0.69, 1.00] Sensitivity 0 0.2 0.4 0.6 0.8 1 Specificity 0 0.2 0.4 0.6 0.8 1
  • MRI results • Criteria for positivity varied across studies, including variables of size, morphology and uptake. Uptake pattern appeared to give better combined sensitivity and specificity. Sensitivity analyses • Sensitivity analyses for size and nodal status not possible as these were not reported • Lower specificity and a trend towards higher sensitivity was seen for studies with only early-stage, newly- diagnosed non-DCIS patients, but wide range. • Study quality did not affect estimates (note small sample, little variation in scores) 14/05/2014 © The University of Sheffield
  • Adverse effects • No adverse effects were reported for PET • Mild to moderate adverse effects reported for MRI include mild rash following USPIO administration, claustrophobia and back pain. • Cautions and contraindications exist for both including pregnancy (PET), allergy to contrast agents, renal and liver dysfunction, pacemakers, metallic implants (MRI). 14/05/2014 © The University of Sheffield
  • Discussion • PET, PET-CT and MRI have lower sensitivity and specificity than ALND, SLNB and 4-NS • PET and PET-CT are similar to ultrasound in terms of sensitivity and specificity, MRI is slightly higher. • MRI has higher sensitivity than PET or PET-CT, but similar specificity. • USPIO-enhanced MRI gave highest estimates of sensitivity and specificity, but these are based on a small number of studies • All results vary widely between studies and caution should be taken when interpreting results 14/05/2014 © The University of Sheffield
  • Discussion (2) • Based on these estimates, if PET or MRI are used to replace SLNB/4-NS in the diagnostic pathway, more women will be at greater risk of recurrence or metastatic spread (false negatives) and more women will undergo ALND unnecessarily (false positives) • However PET and MRI have few adverse effects and much fewer women would undergo SLNB/4-NS and be at risk of the long term problems associated with them. • Decision modeling is needed for overall evaluation of benefits and harms. • Decision modeling will help evaluate the effects if PET or MRI are used in addition to SLNB/4-NS in the diagnostic pathway. 14/05/2014 © The University of Sheffield
  • PET results – ROC plot 14/05/2014 © The University of Sheffield All PET studies, showing ROC curve (solid line), mean sensitivity/specificity (black spot) and 95% confidence region (dashed ellipse)
  • MRI results – ROC plot 14/05/2014 © The University of Sheffield All MRI studies, showing ROC curve (solid line), mean sensitivity/specificity (black spot) and 95% confidence region (dashed ellipse)
  • Systematic reviews of relevant data Myfanwy Lloyd Jones Senior Research Fellow ScHARR, University of Sheffield Email: m.lloydjones@sheffield.ac.uk
  • More specifically • Drawing on the experience of the pilot diagnostics project, how do systematic reviews of diagnostic interventions differ from systematic reviews of therapeutic interventions, when both are undertaken to inform NICE decision-making? • Review question • PICO(S) components • Meta-analysis 14/05/2014 © The University of Sheffield
  • The review question • Is there a bigger difference between the overall project question and the question that forms the focus of the systematic review of clinical effectiveness in an assessment of a diagnostic intervention than in an assessment of a therapeutic intervention? • Therapeutic intervention: strontium ranelate TAR (2005) 14/05/2014 © The University of Sheffield
  • Therapeutic intervention Research question defined in the protocol: • To establish the clinical and cost effectiveness of strontium ranelate for the prevention of osteoporotic fractures in postmenopausal women with osteoporosis Systematic review of clinical effectiveness: • What is the clinical effectiveness of strontium ranelate for the prevention of osteoporotic fractures in postmenopausal women with osteoporosis, with or without prior fracture? 14/05/2014 © The University of Sheffield
  • In PICOS terms • Population: postmenopausal women with osteoporosis, with or without prior fracture • Intervention: strontium ranelate • Comparator: placebo/no treatment; specified active treatments if evidence available • Outcomes: • Principal: incident fractures (vertebral & nonvert) • Secondary: adverse effects, health-related quality of life etc • Study design: RCTs 14/05/2014 © The University of Sheffield
  • Diagnostic intervention Decision problem defined in the protocol: • Will using non-invasive liver assessment tools in patients with suspected alcohol-related liver fibrosis who might otherwise be candidates for biopsy or referral to specialist care reduce the number of referrals or biopsies and improve the health outcomes and quality of life of those patients? Systematic review of clinical effectiveness: • The subject of some debate 14/05/2014 © The University of Sheffield
  • In PICOS terms: population & intervention • Population: patients suspected of having liver fibrosis related to alcohol consumption • Intervention: the scope and protocol specified 4 non-invasive liver assessment tools (NILTs) • ? A fifth – how binding is the protocol? 14/05/2014 © The University of Sheffield
  • In PICOS terms: the comparator • The current UK decision-making process? • What the NICE team wanted: - A relevant comparator for the model - Problematic given the lack of consensus among clinicians about the potential role of NILTs - A problematic comparator for the systematic review because no study is likely to have been set up with that comparator • A specific diagnostic test (reference standard)? • Feasible in terms of the evidence base • What many clinicians are likely to want to know is how a NILT compares with the reference standard (at least if the reference standard is what they generally use) 14/05/2014 © The University of Sheffield
  • In PICOS terms: outcomes • Test accuracy compared with the reference standard • Number of test failures or other withdrawals • Adverse events • Number of referrals to specialist care • Number of biopsies needed (presumably because of test failures or borderline results) • Longer-term health outcomes • Quality of life Which of these should be the primary outcome? 14/05/2014 © The University of Sheffield
  • In PICOS terms: study design • Protocol: “The best available level of evidence will be included, with priority given to controlled studies if available” 14/05/2014 © The University of Sheffield
  • Population (1) • Therapeutic intervention: • reasonable match between review question and study populations (main exclusions related to conditions which could interfere with bone metabolism, and recent treatment with antiosteoporotic agents; 1 study excluded women aged >78) • Diagnostic intervention: • mismatch introducing spectrum bias 14/05/2014 © The University of Sheffield
  • Population (2) In diagnostic studies, if the reference standard is invasive (eg liver biopsy) or particularly expensive, then for ethical or cost reasons the study populations are likely not to be representative of the full range of people eligible for the index test • Either the whole study population is unrepresentative because of disease severity (eg already scheduled for liver biopsy because of suspected severe fibrosis) • Or the whole population is more representative, but liver biopsy is only performed in the subset whose index test results suggest more severe disease
  • Intervention Therapeutic intervention: • Description and other useful data from EMEA and BNF Diagnostic intervention: • No structured descriptions of the technology – not even sponsor submissions: • Data pulled together from assorted articles and manufacturers‟ websites 14/05/2014 © The University of Sheffield
  • Comparator: what is it? Therapeutic intervention: • No active intervention (placebo or no treatment) • Active intervention likely to have a direct effect on relevant clinical outcomes Diagnostic intervention: • Subject of debate: test or care pathway? • Primary comparator: liver biopsy • Secondary comparators: tests used to identify conditions associated with liver fibrosis
  • Comparator: issues • Studies of diagnostic test accuracy assume that the reference standard has 100% sensitivity and specificity, which is clearly unlikely • Some reference standards (eg liver biopsy) are known to be imperfect • Discordance between the results of the index test and reference standard may result from error in either test • Some studies may try to determine, for each individual discordant result, whether the reference standard or the new test is more likely to be correct, but will never identify cases where the results are concordant but both wrong 14/05/2014 © The University of Sheffield
  • Outcomes: therapeutic intervention • Primary outcome: fracture • Clinical relevance clear 14/05/2014 © The University of Sheffield
  • Outcomes: diagnostic intervention • Primary outcome: subject of debate • Test accuracy? • Fundamental: if the new test is inaccurate, no point in going further • Problematic, and often poorly reported • Patient outcomes? • Clinical outcomes: obviously relevant, but may depend heavily on clinician or patient choices subsequent to the test result (eg, for NILTs, would a positive test result make the patient more or less likely to stop drinking?) • Safety/ patient acceptability relative to current practice 14/05/2014 © The University of Sheffield
  • Reporting of outcomes: therapeutic intervention (1) • The Consort statement says that “for each outcome, study results should be reported as a summary of the outcome in each group (for example, the number of participants with or without the event and the denominators, or the mean and standard deviation of measurements), together with the contrast between the groups, known as the effect size” • For binary outcomes, the effect size could be the relative risk, odds ratio, or risk difference 14/05/2014 © The University of Sheffield
  • Reporting of outcomes: therapeutic intervention (2) • 1 14/05/2014 © The University of Sheffield Number (%) Relative risk (95% CI) Endpoint Strontium ranelate (n=719) Placebo (n=723) Nonvertebral fracture 112 (15.6) 122 (16.9) 0.90 (0.69 to 1.17)
  • Reporting of outcomes: therapeutic intervention (3) • Vertebral fracture: • SOTI trial reported %ages plus RR; numbers of participants in each treatment group with fracture obtained from investigator • The STRATOS trial and, at the time of our review, the TROPOS trial only reported %ages plus RR • Nonvertebral fracture: • SOTI and TROPOS reported numbers of participants in each treatment group with fracture, with RR (and %ages for TROPOS) • STRATOS just reported %ages, as adverse events 14/05/2014 © The University of Sheffield
  • Reporting of outcomes: diagnostic intervention (1) • The STARD (Standards for Reporting of Diagnostic Accuracy) guidelines require “a cross tabulation of the results of the index tests (including indeterminate and missing results) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard” 14/05/2014 © The University of Sheffield
  • Reporting of outcomes: diagnostic intervention (2) Reference standard positive Reference standard negative Index test positive True positive (TP) False positive (FP) Index test uninterpretable Uninterpretable (U1) Uninterpretable (U2) Index test negative False negative (FN) True negative (TN) Sensitivity = (TP/(TP+U1+FN))x100 Specificity = (TN/(TN+U2+FP))x100 14/05/2014 © The University of Sheffield
  • Reporting of outcomes: diagnostic intervention (3) 14/05/2014 © The University of Sheffield
  • Reporting of outcomes: diagnostic intervention (4) • NILTs produce results on a continuous scale, but liver biopsy results are expressed in terms of ordinal scoring systems, so when NILTs are compared with liver biopsy the results are also generally presented as ordinal data (fibrosis stages) • Most of the studies in the pilot project identified the threshold values for those stages from the ROC curve, rather than validating prospectively identified values • Only 1/11 studies in the pilot project which compared a NILT with liver biopsy published a reasonably clear tabulation of test accuracy 14/05/2014 © The University of Sheffield
  • Study design Therapeutic intervention: • Only RCTs included Diagnostic intervention: • No relevant RCTs identified: • Where RCTs of diagnostic test accuracy exist, they are generally underpowered to detect a difference between 2 tests • 15 cross-sectional studies, 2 of which were extended to form cohort studies looking at longer-term outcomes • Non-RCTs carry the risk of double-counting: often not clear whether different publications by the same research group include some of the same patients 14/05/2014 © The University of Sheffield
  • Meta-analysis (1) Therapeutic intervention: • Generally straightforward using Review Manager • In the strontium ranelate example, not possible because the relevant data for all 3 included studies were not published and could not be obtained from the original investigators Diagnostic intervention: • Complex, only partially possible in Rev Man • Data often not published in suitable format 14/05/2014 © The University of Sheffield
  • Meta-analysis (2) • Not appropriate if studies have heterogeneous populations, and in diagnostic studies that heterogeneity includes different prevalences of the condition of interest • How similar do populations need to be before the data can be combined? • Requires special techniques if studies use different diagnostic thresholds to define a positive result • Is it generally better to use the results of the best quality study which has the greatest clinical relevance to the study question? 14/05/2014 © The University of Sheffield
  • Systematic reviews of relevant data Myfanwy Lloyd Jones Senior Research Fellow ScHARR, University of Sheffield Email: m.lloydjones@sheffield.ac.uk
  • The aim of the assessment To answer the research question: Will using the specified non-invasive liver assessment tools in patients with suspected alcohol-related liver fibrosis who might otherwise be candidates for biopsy or referral to specialist care reduce the number of referrals or biopsies and improve the health outcomes and quality of life of those patients?
  • The specified tests  Three patented blood tests: • The Enhanced Liver Fibrosis (ELF) test • FibroTest • FibroMAX  Transient elastography • FibroScan  Test results are affected by alcohol consumption
  • Outcomes of interest:  Diagnostic test accuracy (including numbers of test failures)  Number of patients tested in primary care requiring referral to secondary care for further investigation or treatment  Number of patients requiring liver biopsy  Number of patients giving up alcohol, or significantly reducing alcohol consumption, because of test result  Long-term patient outcomes (disease progression, complications related to liver disease, need for liver transplantation, mortality)  Adverse effects of testing  Health-related quality of life  Cost-effectiveness
  • Reference standard tests  Liver biopsy  Also used in some studies • Hepatic venous pressure gradient (HVPG) measurement to identify portal hypertension • Upper intestinal endoscopy to identify oesophageal varices
  • Systematic review of clinical effectiveness  Population: patients with suspected alcohol- related liver fibrosis  Intervention: any of the 4 specified tests  Comparators: • Primary: liver biopsy • Secondary: HVPG measurement; upper endoscopy  Outcomes: previously listed  Study design: cohort or cross-sectional studies (prospective or retrospective)
  • Included studies: ELF test  Rosenberg 2004: patients with chronic liver disease (subgroup with ALD); test accuracy vs liver biopsy • Parkes in press: follows cohort of English patients from Rosenberg study; survival (median follow-up 6.86 years)
  • Included studies: FibroTest (1)  2 studies specifically in patients with known or suspected alcohol-related liver disease, both of test accuracy vs liver biopsy: • Naveau 2005 • Nguyen-Khac 2008 • Naveau 2009: 5 and 10 year survival of patients included in Naveau 2005
  • Included studies: FibroTest (2)  3 studies in mixed aetiology liver disease: • 1 study in patients with chronic liver disease: - Thabut 2003: test accuracy vs endoscopy • 1 study in patients undergoing transjugular liver biopsy for clinical reasons: - Thabut 2007a: test accuracy vs HPVG • 1 study in patients with severe cirrhosis: - Thabut 2007b: survival at 2 and 6 months
  • Included studies: FibroMAX  No relevant studies identified
  • Included studies: FibroScan (1)  6 studies specifically in patients with known or suspected alcohol–related liver disease, all looking at test accuracy vs liver biopsy:  4 biopsied all patients • Kim 2009, Mueller 2010, Nahon 2008, Nguyen-Khac 2008  2 biopsied subset only: • Janssens 2010: biopsy only in those with FS score >9.6, indicating severe fibrosis (>F3) (this study also looked at test accuracy vs HPVG) • Melin 2005: biopsy only in patients with FS score >13 kPa (chosen as threshold for cirrhosis in patients with hepatitis C)
  • Included studies: FibroScan (2)  3 studies in mixed aetiology liver disease: • 1 study in patients undergoing transjugular liver biopsy for clinical reasons: • Bureau 2008: test accuracy vs HVPG  2 studies in patients with cirrhosis: • Lemoine 2008: test accuracy vs HVPG • Nguyen-Khac 2009: test accuracy vs endoscopy
  • AUROCs from key studies: test vs liver biopsy Test Degree of fibrosis Study AUROC (95% CI) ELF „Moderate/severe‟ Rosenberg 0.94 (0.84-1.00) FibroTest F2-F4 Naveau 0.83 (0.81-0.87) Nguyen-Khac 0.79 (0.69-0.90) Cirrhosis (F4) Naveau 0.95 (0.94-0.96) Nguyen-Khac 0.84 (0.72-0.97) FibroScan Severe fibrosis (F3-F4) Kim 0.98 (0.94-1.02) Mueller 0.91+0.03 Nahon 0.94 (90-0.97) Nguyen-Khac 0.90 (0.82-0.97) Cirrhosis Kim 0.97 (0.93-1.01) Mueller 0.92 (0.87-0.97) Nahon 0.87 (0.81-0.93) Nguyen-Khac 0.94 (0.87-0.98)
  • Sensitivity and specificity: ELF Test (subgroup with ALD) Degree of fibrosis Study Threshold score Sensitivity Specificity „Moderate/severe‟ Rosenberg 0.087 100% 16.7% 0.431 93.3% 100%
  • Sensitivity and specificity: FibroTest (all patients ALD) Degree of fibrosis Study Threshold score Sensitivity Specificity Moderate-severe (F2-F4) Naveau 0.30 84% 66% 0.70 55% 93% Cirrhosis (F4) Naveau 0.30 100% 50% 0.70 91% 87%
  • Sensitivity and specificity: FibroScan (all patients ALD, all biopsied) Degree of fibrosis Study Threshold score Sensitivity Specificity Moderate-severe (F2- F4) Nguyen-Khac 7.8 kPa 80% 90.5% Severe (F3-F4) Mueller 8.0 kPa 91% 75% Nguyen-Khac 11 kPa 86.7% 80.5% Nahon 11.6 kPa 87% 89% Cirrhosis (F4) Mueller 11.5 kPa 100% 77% 12.5 kPa 96% 80% Nguyen-Khac 12.8 kPa 100% 75.4% 19.5 kPa 85.7% 84.2% Nahon 22.7 kPa 84% 83% Kim 28.5 kPa 90% 87%
  • Sensitivity and specificity: FibroScan (all patients ALD, only subset biopsied) Degree of fibrosis Study Threshold score Sensitivity Specificity Severe (F3-F4) Janssens 13.9 kPa 81% 59% 15.8 kPa 75% 70% 16.5 kPa 72% 70% 17.0 kPa 72% 76.5% 17.3 kPa 69% 76.5% Cirrhosis (F4) Janssens 19.6 kPa 80% 76% 21.1 kPa 75% 80% 23.5 kPa 65% 83%
  • FibroScan: exclusion of patients with laboratory signs of ASH Degree of fibrosis No of patients AUROC Threshold score Sensitivity Specificity F4 101 0.921 11.5 100% 77% 12.5 96% 80% F4 without GOT >100 U/L 86 0.944 11.5 100% 84% 12.5 95% 90% F3-F4 101 0.914 8.0 91% 75% F3-F4 without GOT >100 U/L 80 0.922 8.0 87% 87%
  • Evidence re outcomes of interest (1)  Diagnostic test accuracy: NILTs appear to have reasonable accuracy, but the evidence is not drawn from full spectrum of patients  Number of patients tested in primary care requiring referral to secondary care: no data  Number of patients requiring liver biopsy following NILT: no evidence of what this would be in clinical practice rather than study setting
  • Evidence re outcomes of interest (2)  Numbers of patients giving up or significantly reducing alcohol consumption following test result: only reported in one study, and not subdivided by test results.  Long-term patient outcomes: ELF test and FibroTest may have some prognostic value  Adverse effects of NILTs: no specific AEs, but tests which use a blood sample have AEs associated with venepuncture
  • Liver biopsy: an imperfect reference standard  Diagnostic studies assume that the reference standard has 100% sensitivity and specificity  Not true of liver biopsy: • Sampling error (may affect >30% of samples) • „Significant‟ inter- and intra-observer variation in interpretation of samples  Some studies try to determine whether, in discordant cases, the index test or the biopsy is more likely to be correct  What if the index test and the biopsy are concordant but both wrong?
  • Liver biopsy: ethical aspects affect study design  Adverse events: • Minor adverse events: 9.39% • Severe adverse events (including death): 0.86% • Death: 0.08%  Consequently, study populations not representative of full range of people with suspected ALD: • Either whole study population is unrepresentative because of disease severity (eg already scheduled for biopsy) • Or whole population is more representative, but biopsy is only performed in subset with NILT result suggesting more severe disease
  • How much homogeneity is required for meta- analysis?  The studies of FibroTest and FibroScan included in this review are heterogeneous in terms of: • Prevalence of the condition of interest • Diagnostic threshold for identifying that condition  Because of this heterogeneity, it seems inappropriate to combine their results using meta-analysis
  • Health Economics and Outcomes Research 14/05/2014 © The University of Sheffield
  • Areas of Particular Strength • Quality of life and outcomes measurement, wellbeing and equity • Primary economic evaluation in clinical studies, costing studies and service evaluation • Statistical methods in health economics
  • Research centre Centre for Well-being in Public Policy (CWiPP) • The goals of the centre are to consider as to how people‟s health and wellbeing can be defined, measured and improved in ways that help policy-makers determine the best use of scarce resources, and to investigate the determinants of well-being insofar as these are relevant to policy formulation 14/05/2014 © The University of Sheffield
  • For example: HEDS in collaboration of the Department of Economics, University of Sheffield run a number of joint research projects in the area of health and well-being, including the Independent Review of the Effects of Alcohol Pricing and Promotion project funded by the Department of Health 14/05/2014 © The University of Sheffield
  • Examples of health outcome studies conducted in ScHARR 14/05/2014 © The University of Sheffield
  • Developing and testing condition-specific preference-based measures: Lessons learnt and policy implications John Brazier, Donna Rowen, Ifigeneia Mavranezouli, Aki Tsuchiya , Tracey Young, Yaling Yang, Michael Barkham
  • Acknowledgements • COSMeQ team: John Brazier1, Aki Tsuchiya1, Ifigeneia Mavrazenouli1,2, Tracey Young1, Yaling Yang3, Rachel Ibbotson4, Michael Barkham1 1University of Sheffield 2University College London 3Brunel University 4Sheffield Hallam University • COSMeQ project, MRC-NIHR Methodology Research Programme project number 06/97/04
  • Economic evaluation • Economic evaluation using cost utility analysis (CUA) increasingly used to inform resource allocation decisions • CUA measures health outcomes using the Quality Adjusted Life Year (QALY) • QALY combines quantity and quality of life into single measure of health outcome by adjusting life years using a quality weight • QALY can be used across all health care interventions for all patient groups
  • Quality-adjusted life years Source: Drummond et al, 1997
  • Health utility measurement • Purpose of health utility measurement is to provide the quality adjustment weight, „Q‟, for the QALY • Typically obtained using generic preference- based measures • Condition-specific preference-based measures (CSPBM) are an alternative, and more commonly used when • generic measure is inappropriate • generic measure is unavailable
  • Health utility measurement • Preference-based measure has two components: • Health state description • Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead)
  • Health utility measurement • Preference-based measure has two components: • Health state description • Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead) Condition label? Narrow focus? Multi-dimensional? Side effects, comorbidities?
  • Health utility measurement • Preference-based measure has two components: • Health state description • Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead) Condition label? Narrow focus? Multi-dimensional? Side effects, comorbidities?
  • Health utility measurement • Preference-based measure has two components: • Health state description • Corresponding utility value measured on an interval scale: upper anchor at one for full health lower anchor at zero (assuming it is equivalent to dead) Condition label? Narrow focus? Multi-dimensional? Side effects? Comorbidities? Performance? Comparability?
  • Outline of presentation • Developing CSPBM from existing measure • impact of condition labels on health state utility values • impact of adding a generic dimension to a condition-specific preference-based measure to capture co-morbidities or side-effects • Testing CSPBM • performance in comparison to measure derived from and generic EQ-5D
  • Impact of condition labels on health state utility values
  • Impact of labelling: rationale • Generic measures typically valued with no condition label • Often CSPBMs include condition label as is embedded in original measure it is derived from e.g. „experience asthma symptoms as a result of air pollution‟ • No consensus in previous studies examining impact of labelling • Small sample size • Within-subject design • Small severity range of states
  • Study design • Between-subject study • Respondents valued: • same 8 states produced using EORTC-8D, non-labelled CSPBM cancer derived from EORTC QLQ-C30 • range of health states of differing severity • one of three versions: no label, irritable bowel syndrome label, cancer label • using MVH TTO protocol • Sampling strategy to ensure representativeness across label groups and to UK general population
  • Example health state (51224434) (no label) / Due to having irritable bowel syndrome / Due to having cancer •You have very much trouble taking a short walk outside of the house You are not limited in pursuing your hobbies or other leisure time activities Your physical condition or medical treatment interferes a little with your social activities Pain interferes a little with your daily activities You feel depressed very much You are tired very much You are constipated and/or have diarrhoea quite a bit You feel nauseated very much
  • Analysis • ANOVA using generalized linear model to determine significant differences in respondent characteristics across groups • t tests to determine significant differences in health state values across groups • Regression analysis (RE GLS model) to determine impact on elicited utility values due to condition label, state severity, interaction of label and severity, respondent characteristics, experience of condition
  • Respondent characteristics No label (n=81) IBS label (n=80) Cancer label (n=80) ANOVA P-value Mean age (s.d.) 48.08 (21.57) 51.78 (18.93) 47.1 (18.85) 0.30 Female 62% 63% 56% 0.68 Married/partner 63% 53% 56% 0.40 Employed or self-employed 36% 39% 41% 0.78 Unemployed 3% 3% 6% 0.35 Long-term sick 7% 4% 6% 0.60 Full-time student 12% 3% 6% 0.05 Housework 9% 14% 15% 0.44 Retired 30% 33% 23% 0.35 Experience of labelled condition 40% 78% in themselves 19% 13% in caring for others 14% 36%
  • Descriptive statistics Original study (n=344) No label (n=81) IBS label (n=79-80) Cancer label (n=79-80) Health state Modelled utility value Mean (s.d.) Mean (s.d.) Mean (s.d.) 11111111 1 0.96 (0.13) 0.99 (0.06) 0.96 (0.12) 31212241 0.75 0.74 (0.32) 0.81 (0.23) 0.80 (0.22) 13423411 0.72 0.67 (0.30) 0.71 (0.37) 0.64 (0.36) 44321321 0.65 0.66 (0.35) 0.68 (0.37) 0.56 (0.50) 23141224 0.64 0.63 (0.36) 0.69 (0.36) 0.57 (0.45) 24432411 0.64 0.66 (0.33) 0.65 (0.40) 0.54 (0.44) 51224434 0.51 0.49 (0.41) 0.53 (0.42) 0.41 (0.49) 54444444 0.29 0.20 (0.49) 0.17 (0.49) -0.03 (0.50)
  • Regression results
  • Regression results
  • Regression results States Cancer interaction terms Labelling 11111111 x Cancer -0.041 IBS 0.008 31212241 -0.197*** 31212241 x Cancer -0.007 13423411 -0.284*** 13423411 x Cancer -0.079 Experience of condition 44321321 -0.304*** 44321321 x Cancer -0.147** Cancer in themselves -0.157* 23141224 -0.313*** 23141224 x Cancer -0.128** Cancer in caring for others 0.134** 24432411 -0.317*** 24432411 x Cancer -0.148** IBS in themselves -0.036 51224434 -0.456*** 51224434 x Cancer -0.136** IBS in caring for others 0.064 54444444 -0.785*** 54444444 x Cancer -0.254*** Constant 0.967*** Sociodemographic variables: female, married, retired, unemployed, student, housework, long-term sick, secondary school is highest level of education
  • Regression results States Cancer interaction terms Labelling 11111111 x Cancer -0.041 IBS 0.008 31212241 -0.197*** 31212241 x Cancer -0.007 13423411 -0.284*** 13423411 x Cancer -0.079 Experience of condition 44321321 -0.304*** 44321321 x Cancer -0.147** Cancer in themselves -0.157* 23141224 -0.313*** 23141224 x Cancer -0.128** Cancer in caring for others 0.134** 24432411 -0.317*** 24432411 x Cancer -0.148** IBS in themselves -0.036 51224434 -0.456*** 51224434 x Cancer -0.136** IBS in caring for others 0.064 54444444 -0.785*** 54444444 x Cancer -0.254*** Constant 0.967*** Sociodemographic variables: female, married, retired, unemployed, student, housework, long-term sick, secondary school is highest level of education
  • Regression results States Cancer interaction terms Labelling 11111111 x Cancer -0.041 IBS 0.008 31212241 -0.197*** 31212241 x Cancer -0.007 13423411 -0.284*** 13423411 x Cancer -0.079 Experience of condition 44321321 -0.304*** 44321321 x Cancer -0.147** Cancer in themselves -0.157* 23141224 -0.313*** 23141224 x Cancer -0.128** Cancer in caring for others 0.134** 24432411 -0.317*** 24432411 x Cancer -0.148** IBS in themselves -0.036 51224434 -0.456*** 51224434 x Cancer -0.136** IBS in caring for others 0.064 54444444 -0.785*** 54444444 x Cancer -0.254*** Constant 0.967*** Sociodemographic variables: female, married, retired, unemployed, student, housework, long-term sick, secondary school is highest level of education
  • Discussion • Inclusion of condition label can affect health state values, dependent upon • specific condition • severity of state • Experience of condition affects values • Why does this occur? Greater precision? • Should health state values reflect this difference? • We argue not on grounds of comparability, to avoid distortion of values based on prior knowledge or preconceptions
  • Impact of adding a generic dimension
  • Impact of adding generic dimension: rationale • Condition-specific measures criticised for inability to capture side-effects or co-morbidities • One option is to include additional dimensions in classification system to capture this. i.e. „add-on‟ • What is the impact on utility values? • What is the impact on preference weights for existing dimensions? (i.e. in value set or „tariff‟) • What is the impact when using the add-on in the patient population?
  • Study design • Add-on pain/discomfort dimension (taken from EQ- 5D) to AQL-5D, asthma-specific condition specific measure to produce AQL-6D • Valuation studies for 3 level versions of AQL-5D and AQL-6D (states selected using orthogonal array) • Respondents valued: • 8 health states of differing severity • with descriptions produced using AQL-5D or AQL-6D • using MVH TTO protocol • Sampling strategy to ensure representativeness across measure and to UK general population
  • Example health states •Feel concerned about having asthma none of the time •Feel short of breath as a result of asthma some of the time •Experience asthma symptoms as a result of air pollution none of the time •Asthma interferes with getting a good night's sleep all of the time •Overall, moderate or some limitation in every activity done due to asthma •Feel concerned about having asthma none of the time •Feel short of breath as a result of asthma some of the time •Experience asthma symptoms as a result of air pollution none of the time •Asthma interferes with getting a good night's sleep all of the time •Overall, moderate or some limitation in every activity done due to asthma •Have extreme pain or discomfort
  • Analysis • t tests to determine significant differences in health state values across groups • Regression analysis to determine preference weights • OLS models on mean level data • RE GLS model and RE Tobit • Compared coefficients across measures for non- pain variables using the z score test • Applied AQL-5D and AQL-6D weights to clinical trial (COGENT examining computerized decision support systems) to examine impact
  • Respondent characteristics AQL-5D (n=91) AQL-6D (n=91) ANOVA p-value Mean age (s.d.) 52.13 (17.54) 50.01 (17.27) 0.41 Female 62% 59% 0.76 Married/partner 70% 76% 0.40 Employed or self-employed 31% 35% 0.53 Unemployed 11% 7% 0.30 Long-term sick 3% 9% 0.12 Full-time student 3% 1% 0.31 Housework 10% 15% 0.27 Retired 36% 29% 0.27 Have asthma 25% 23% Have moderate pain or discomfort 40% 31% Have extreme pain or discomfort 4% 9%
  • Descriptive statistics AQL-5D health state Mean (s.d.) n AQL-6D health state Mean (s.d.) n T test 11111 0.97 (0.14) 60 111111 0.98 (0.07) 61 0.492 12132 0.70 (0.41) 62 121323 0.56 (0.40) 60 0.061 23131 0.64 (0.44) 60 231311 0.78 (0.24) 61 0.034 33333 0.26 (0.53) 91 333333 0.30 (0.48) 91 0.576
  • AQL-5D AQL-6D Z score Concern2 0.032 0.034 -0.052 Concern3 0.041 0.047** -0.165 Breath2 0.019 -0.001 0.502 Breath3 0.167*** 0.047* 3.177*** Weather2 0.024 -0.016 1.035 Weather3 0.057** 0.033 0.734 Sleep2 0.016 -0.001 0.471 Sleep3 0.106*** 0.091*** 0.431 Activities2 0.074*** 0.040* 0.978 Activities3 0.307*** 0.150*** 4.213*** Pain2 0.071*** Pain3 0.301*** Constant 0.061 0.023 Regression results
  • Application to trial data AQL-5D AQL-6D Mean (s.d.) Mean (s.d.) n All patients 0.733 (0.188) 0.833 (0.144) 2791 Asthma symptoms score 0<NASS≤20 (least severe) 0.884 (0.087) 0.933 (0.069) 625 20<NASS≤40 0.808 (0.083) 0.893 (0.072) 528 40<NASS≤60 0.759 (0.114) 0.859 (0.090) 583 60<NASS≤80 0.697 (0.149) 0.805 (0.122) 478 80<NASS≤100 (most severe) 0.489 (0.189) 0.655 (0.157) 539 No pain/discomfort 0.817 (0.119) 0.930 (0.060) 1245 Moderate pain/discomfort 0.677 (0.198) 0.787 (0.108) 1373 Extreme pain/discomfort 0.563 (0.226) 0.495 (0.126) 173
  • Discussion • Add-on pain/discomfort dimension had significant level coefficients and affected level coefficients of other dimensions • Inclusion of add-on affected utilities in trial – utility values higher unless pain/discomfort was at level 3 • Why did this occur? • Comparability between measures requires that impact of different dimensions on preferences is additive, whether or not they are included in the classification • These results cast doubt on this
  • Performance of CSPBMs
  • Examining performance: rationale • Increased development and usage of condition- specific preference-based measures (CSPBM) • How does CSPBM compare to original measure (CSM) in terms of performance? • How does CSPBM compare to generic measure(s) in terms of performance? • Original rationale for using CSM is due to its greater relevance and sensitivity, is this true for CSPBM?
  • Analysis • Validity: ability to discriminate between patients with different levels of severity defined in terms of their specific condition • t tests, effect sizes • Responsiveness: sensitivity to change in trial data before and after treatment • t tests, effect sizes, standardised response mean, floor and ceiling effects • Agreement between measures • Pearson correlation coefficient
  • Measures Condition Original measure Preference-based measure Generic EQ-5D Asthma AQLQ AQL-5D Common mental health problems CORE-OM CORE-6D Cancer EORTC QLQ-C30 EORTC-8D
  • Brief summary of results • Little evidence of information loss moving from original measure to CSPBM, performed similarly • Validity: asthma and cancer measures similar to EQ-5D, mental health measure better • Responsiveness: mental health measure better than EQ- 5D, reverse for cancer measure • All show consistently high agreement • Less ceiling effects for CSPBM than EQ-5D, suggesting greater refinement at upper end of health • Mean change in utility scores before and after treatment lower for condition-specific measures than EQ-5D
  • Discussion • Large amount of resources to develop CSPBM, but may not offer much improvement • Mean change actually smaller using CSPBM than EQ-5D • Research suggests this is associated with MVH value set • Side effects and comorbidities? • Comparability with EQ-5D may actually be an advantage if only condition-specific data available
  • Overall conclusions • Utility values affected by inclusion of condition labels, we recommend classification systems should avoid labels • to avoid distortions by prior knowledge or preconceptions • to ensure comparability in economic evaluations across patient groups • The role of the classification system is important, as the relationship between dimensions may not be additive • Implications for condition-specific measures with narrow focus • Implications for generic measures with „absent‟ dimensions • Condition-specific preference-based measures seem to perform comparably to the original measure and to generic EQ-5D • Implications for development of future measures • Implications for usage if only condition-specific data is available
  • Policy implications • The original condition-specific measure used to derive the PBM: • must have desirable psychometric properties • must include all important dimensions • must offer advantage over generic measures • Care should be taken as the classification system can affect health state utility values • Contrary to what may be expected, CSPBM perform similarly to EQ-5D, with lower mean change but greater refinement at upper end of health
  • Publications • Brazier J, Rowen D, Tsuchiya A, Yang Y, Young T (2011) The impact of adding an extra dimension to a preference-based measure. Social Science and Medicine, 73(2), 245-253. • Brazier J, Rowen D, Mavranezouli I, Tsuchiya A, Young T, Yang Y, Barkham M, Ibbotson R. Developing and testing methods for deriving preference-based measures of health from condition specific measures (and other patient based measures of outcome). Health Technology Assessment. Forthcoming 2011. • Mavranezouli I, Brazier J, Rowen D, Barkham M. The development of a condition-specific preference-based measure for common mental disorders from the Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM) using Rasch analysis. HESG Paper presented at Cork, 23-25 July 2010 • Mavranezouli I, Brazier J, Young A, Barkham M. Using Rasch analysis to form plausible health states amenable to valuation: the development of the CORE-6D from a measure of common mental health problems (CORE-OM). Quality of Life Research 2011;20(3):321-33.
  • Publications • Rowen D, Brazier J, Tsuchiya A, Young T, Ibbotson R. It's all in the name, or is it? The impact of labelling on health state values. Medical Decision Making 2011;Forthcoming. • Rowen D, Brazier JE, Young TA, Gaugrist S, Craig BM, King MT, et al. Deriving a preference-based measure for cancer using the EORTC QLQ-C30. Value in Health, forthcoming. • Yang Y, Brazier J, Tsuchiya A, Young T. Estimating a Preference- Based Index for a 5-Dimensional Health State Classification for Asthma Derived From the Asthma Quality of Life Questionnaire. Medical Decision Making 2010;Forthcoming. • Young T, Yang Y, Brazier J, Tsuchiya A. The Use of Rasch Analysis in Reducing a Large Condition-Specific Instrument for Preference Valuation: The Case of Moving from AQLQ to AQL-5D. Medical Decision Making 2010;Forthcoming.
  • Deriving a preference- based measure for cancer using the EORTC QLQ-C30 Donna Rowen, John Brazier, Tracey Young, Sabine Gaugris, Benjamin Craig , Madeline King, Galina Velikova
  • Background • Cost effectiveness arguments are increasingly important in influencing reimbursement decisions • Generic measures are not valid or responsive for many cancer conditions (Garau et al, 2009) • Frequently used condition-specific measures, such as EORTC QLQ-C30, have great clinical utility but cannot be used to estimate QALYs • To maximise the value of QLQ-C30 data sets (past and future) it is useful to have the capacity to calculate a QALY
  • EORTC QLQ-C30 • QLQ-C30 is one of the most commonly used measures in cancer • Validity well established for many conditions in cancer • 30 questions/items covering: • the most common cancer symptoms (such as pain, fatigue, nausea and vomiting) • various aspects of function (including physical, role, social, emotional and cognitive functioning) • QLQ-C30 is summarised using fourteen scales, plus one global quality of life scale
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage IV Validation –stages I to IV repeated on another time period
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage V Valuation study to elicit general population TTO values by interview for a sample of health states Stage IV Validation –stages I to IV repeated on another time period
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage V Valuation study to elicit general population TTO values by interview for a sample of health states Stage VI Model TTO values to produce utility values for all health states defined by classification system Stage IV Validation –stages I to IV repeated on another time period
  • 14/05/2014 © The University of Sheffield Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors
  • Stage I – Establishing factors • Dataset used is screening phase of Multiple Myeloma trial, n=655 • Used factor analysis to identify structurally independent factors across 27 items (excluding financial impact and global QoL) • We used expert judgement alongside these factor and other psychometric analyses • Findings were analysed to establish whether they were in accordance with the QLQ-C30 scaling
  • Stage I – Establishing factors The 27 items were grouped into the 4 identified factors: 1.Physical functioning, role functioning, social functioning, pain and items „need a rest‟, „felt weak‟ and „difficulty concentrating‟ 2.Emotional functioning plus „trouble sleeping‟ 3.All items covering eating and digestion 4.Items „short of breath‟, „were you tired‟ and „difficulty remembering‟ Undertook second factor analysis on factors 1 and 4 (subset of 18 items)
  • Factors used for Stage II • Factors included: • Physical functioning, role functioning and pain • Social functioning • Emotional functioning • Nausea, vomiting, constipation, diarrhea • Fatigue and trouble sleeping • Factors excluded: • Cognitive functioning • Shortness of breath Excluded due to necessary prioritisation of factors showing greater change, greater problems and greater responsiveness with symptoms that were unlikely to be captured through the other dimensions
  • 14/05/2014 © The University of Sheffield Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor
  • Stage II – Selecting items and dimensions • Used psychometric analysis, Rasch analysis and expert judgement to select items and dimensions • Psychometric analysis: missing data, ceiling effects, floor effects, correlation with domain score, distribution of responses and responsiveness over time (using SRM) • Rasch analysis: item level ordering, differential item functioning, model fit, item range and spread across latent space
  • Stage II - Considerations • All within the overall constraint of producing a classification that generates states amenable to valuation • To make the valuation easier factors have been separated into their separate attributes these form the dimensions for the classification system
  • Chosen dimensions and items Dimension Items Question Physical functioning 2+3 Trouble taking a long walk with extra level added from item 3, trouble taking a short walk Role functioning 7 Limited in pursuing hobbies or other leisure time activities Pain 19 Pain interfere with daily activities Social functioning 27 Physical condition or medical treatment interfered with social life Emotional functioning 24 Felt depressed Nausea 14 Felt nauseated Constipation and diarrhea 16+17 Constipated and diarrhoea merged Fatigue and trouble sleeping 18 Tired
  • Health state classification system Physical functioning: You had no / a little / quite a bit / very much trouble taking a long walk / very much trouble taking a short walk outside of the house Role functioning: You were not limited / limited a little / quite a bit / very much in pursuing your hobbies or other leisure time activities Pain: Pain did not interfere / interfered a little / quite a bit / very much with your daily activities Social functioning: Your physical condition or medical treatment did not interfere / interfered a little / quite a bit / very much with your social activities Emotional functioning: You did not feel depressed / felt depressed a little / quite a bit / very much Nausea: You did not feel nauseated / felt nauseated a little / quite a bit / very much Constipation and diarrhea: You were not constipated and did not have diarrhea / were constipated and/or had diarrhea a little / quite a bit / very much Fatigue and sleep disturbance: You were not tired / tired a little / quite a bit / very much
  • 14/05/2014 © The University of Sheffield Stage IV Validation –stages I to II repeated on another time period
  • Stage IV - Validation • Selections were validated using data at cycle 5 of treatment (n=471) in the trial • Cycle was chosen as: • responses have changed (standardised response mean across 27 EORTC items = 0.104) • some respondents have side effects • problem of attrition is reduced in comparison to other treatment cycles
  • 14/05/2014 © The University of Sheffield Stage V Valuation study to elicit general population TTO values by interview for a sample of health states
  • Valuation survey • 350 interviews were conducted using members of the UK general population • A health state is made up of 8 sentences across 5 dimensions, generating a total of 81,920 possible health states • Each respondent valued 8 health states using time-trade off (TTO) and ranking (in accordance with the NICE reference case) • 85 states were valued (81 states generated using orthogonal array and 4 extra states added)
  • 14/05/2014 © The University of Sheffield Stage VI Model TTO values to produce utility values for all health states defined by classification system
  • Modelling the TTO data • Elicited values are only for a subset of states – we need values for every state • Data was analysed using a variety of models: OLS, RE MLE, Mean model, Episodic RUM • Standard model defined as: • Dependent variable, y, is TTO disvalue (1–TTO value) for health state i=1,2 …n valued by respondent j=1,2…m • X is a vector of dummy explanatory variables for each level λ of dimension δ of the health state classification xxx ijij fy )(βx
  • Preference weights – ERUM OLS • pf2 0.052 • pf3 0.077 • pf4 0.103 • pf5 0.104 • rf2 0.044 • rf3 0.050 • rf4 0.076 • pain2 0.054 • pain3 0.064 • pain4 0.070 • ef2 0.032 • ef3 0.053 • ef4 0.132 • sf2 0.029 • sf3 0.046 • sf4 0.132 • fat2 0.038 • fat3 0.052 • fat4 0.084 • • nau3 0.025 • nau3 0.027 • nau4 0.052 • cd2 0.011 • cd3 0.035 • cd4 0.059 R-squared = 0.56 Inconsistencies = 0 Significant coefficients = 22/25 MAE = 0.046 MAE>0.05 = 33 MAE>-0.10 = 6
  • Examples You had a little trouble taking a long walk You were limited a little in pursuing your hobbies or other leisure time activities Your physical condition or medical treatment interfered a little with your social activities Pain interfered a little with your daily activities You felt a little depressed You were a little tired You were constipated and/or had diarrhoea a little You felt a little nauseated You had very much trouble taking a short walk outside of the house You were limited very much in pursuing your hobbies or other leisure time activities Your physical condition or medical treatment interfered very much with your social activities Pain interfered very much with your daily activities You felt depressed very much You were tired very much You were constipated and/or had diarrhoea very much You felt nauseated very much Utility value = 0.715 Utility value = 0.291
  • Stage I Establish dimensions: factor analysis, established QLQ-C30 dimensionality, standard psychometric analysis and expert opinion used to establish factors Stage II Rasch and other psychometric analysis used to select items and dimensions to represent each factor Stage III Exploration of item level reducing per dimension Stage V Valuation study to elicit general population TTO values by interview for a sample of health states Stage VI Model TTO values to produce utility values for all health states defined by classification system 24 items across 5 factors Classification system of 8 dimensions using 10 QLQ-C30 items 27 EORTC QLQ- C30 items Stage IV Validation –stages I to IV repeated on another time period No item level reduction required
  • Discussion • This study has shown the feasibility of applying this approach to the QLQ-C30, but: • Will the health state classification look the same across all cancer conditions? • Does this approach need to be extended to disease specific modules of the EORTC? • Will the valuations be the same across countries? • Can the preference-based QLQ-C30 be used to inform cross programme resource allocation decisions? • Study forms part of a wider cross-country study that will examine the use of PBMs from the QLQ-C30 in a variety of countries and different patient groups
  • References • Garau M, Shah K, Towse A, et al. Assessment and appraisal of oncology medicines: does NICE's approach include all relevant elements? What can be learnt from international HTA experiences? Report for the Pharmaceutical Oncology Initiative (POI) 2009. • Rowen D, Brazier JE, Young TA, Gaugris S, Craig BM, King MT, Velikova, G. Deriving a preference-based measure for cancer using the EORTC QLQ-C30. Value in Health, 14(5), 721-731. For further details see:
  • Deriving preference based indices for dementia for use in economic evaluation Brendan Mulhern1, Donna Rowen1, John Brazier1, Sarah Smith2, Martin Knapp34, Donna Lamping2, Vanessa Loftus4, Tracey Young1, Robert Howard4, Sube Banerjee4 1 Health Economics and Decision Science, University of Sheffield 2 London School of Hygiene and Tropical Medicine 3 London School of Economics and Political Science 4 Institute of Psychiatry, London Contact: b.mulhern@sheffield.ac.uk
  • 14/05/2014 © The University of Sheffield Contents • Introduction • Development of dementia specific health state classification systems for patient and carer report • Deriving utility values for each classification system • Discussion and limitations
  • Measurement in dementia • Total care and health expenditure in UK £50 billion by 2050 • Economic evaluation of emerging treatments in dementia requires valid measurement tools • Debate about validity of generic preference based measures (e.g. EQ-5D) in dementia • Differences in patient and proxy report • Generics missing relevant HRQL dimensions (e.g. Cognition) 14/05/2014 © The University of Sheffield
  • Dementia specific utility measures • Dementia-specific utility measures developed for both self and proxy report from DEMQOL system • Provide dementia specific utility values • Address concerns about using generic measures • DEMQOL system (Smith et al., 2007) • DEMQOL (28 items + global health item) • DEMQOL-Proxy (31 items + global health item) • Investigates HRQL related to dementia • Interviewer administered • Reliable and valid for use across all dementia severity levels 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield Step I Establish dimensions using Exploratory Factor Analysis Step II Eliminate items per dimension using Rasch analysis Step III Select items per dimension using Rasch analysis and input from clinicians and DEMQOL system developers Step IV Exploration of item level reducing per dimension Step V Valuation exercise to elicit TTO health state values Step VI Model TTO valuation results to produce utility values Methodology and development process
  • Step I: DEMQOL dimensionality • Exploratory factor analysis applying a range of models with 2-10 factors • Mild/moderate patient data collected in clinical service (n=644) • Five factor model used for classification system • Explains 45.5% of the variance • 4 items do not load on any factor so removed from classification system at this stage 14/05/2014 © The University of Sheffield
  • Step I: DEMQOL-Proxy dimensionality • Exploratory factor analysis applying a range of models with 2-10 factors • Mild/moderate carer data collected in clinical service (n=683) • Five factor model used for classification system • Explains 49.3% of the variance • 8 items do not load on any factor so removed from classification system at this stage 14/05/2014 © The University of Sheffield
  • Step II: DEMQOL item elimination methods • Item elimination criteria: • Response ordering • Differential item functioning Uniform Non uniform • Fit to Rasch model Fit residuals Chi squared statistic 14/05/2014 © The University of Sheffield Item with disordered response levels Item displaying DIF
  • Step II/III: DEMQOL item selection methods • Item selection criteria: • Spread at logit 0 • Item range (severity) • Fit to the Rasch model Overall dimension Item fit Person fit 14/05/2014 © The University of Sheffield Item with ordered response levels Graph displaying spread at logit 0
  • 14/05/2014 © The University of Sheffield Step II/III: DEMQOL item elimination and selection Dimension Eliminated Item selected Fit residual Chi Severity range Spread at logit 0 Cognition Fit: 1 item Worry about forgetting things that happened recently -0.158 0.740 -0.769 to 2.037 0.12 to 0.68 Negative emotion DIF: 1 item Felt frustrated -0.626 0.155 -0.753 to 1.348 0.21 to 0.68 Positive emotion No items Felt cheerful 1.208 0.165 -3.104 to 2.427 0.08 to 0.96 Social relationships Order: 3 items Fit: 1 item Worry about making yourself understood -0.087 0.472 -0.205 to 1.280 0.22 to 0.55 Loneliness No items Felt lonely 0.293 0.256 -1.044 to 2.309 0.09 to 0.74
  • 14/05/2014 © The University of Sheffield Step II/III: DEMQOL-Proxy item elimination and selection Dimension Eliminated Item selected Fit residual Chi Severity range Spread at logit 0 Cognition DIF: 1 item Fit: 4 items Worry about forgetting people’s names -0.628 0.211 -0.947 to 0.324 0.42 to 0.72 Negative emotion DIF: 1 item Felt frustrated 1.126 0.695 -1.007 to 1.923 0.15 to 0.76 Daily activities Order: 2 items Fit: 1 item No item selected n/a n/a n/a n/a Positive emotion Fit: 1 item Felt lively 0.629 0.735 -3.243 to 3.276 0.04 to 0.96 Appearance No items Worry about keeping yourself looking nice 0.903 0.938 -1.630 to 1.452 0.19 to 0.84
  • Overall goodness of fit of dimensions 14/05/2014 © The University of Sheffield
  • Step IV: Item level reduction • Item level reduction investigated for selected items • Examine: • Threshold probability curves • Distribution of responses across item levels • Original four level response categories used 14/05/2014 © The University of Sheffield
  • DEMQOL-U classification system 14/05/2014 © The University of Sheffield COGNITION 1. I do not worry at all about forgetting things that happened recently 2. I worry a little about forgetting things that happened recently 3. I worry quite a bit about forgetting things that happened recently 4. I worry a lot about forgetting things that happened recently NEGATIVE EMOTION 1. I do not feel frustrated at all 2. I feel frustrated a little 3. I feel frustrated quite a bit 4. I feel frustrated a lot POSITIVE EMOTION 1. I feel cheerful a lot 2. I feel cheerful quite a bit 3. I feel cheerful a little 4. I do not feel cheerful at all RELATIONSHIPS 1. I do not worry at all about making myself understood 2. I worry a little about making myself understood 3. I worry quite a bit about making myself understood 4. I worry a lot about making myself understood LONELINESS 1. I do not feel lonely at all 2. I feel lonely a little 3. I feel lonely quite a bit 4. I feel lonely a lot
  • DEMQOL-Proxy-U classification system COGNITION 1. I do not worry at all about forgetting what day it is 2. I worry a little about forgetting what day it is 3. I worry quite a bit about forgetting what day it is 4. I worry a lot about forgetting what day it is NEGATIVE EMOTION 1. I do not feel frustrated at all 2. I feel frustrated a little 3. I feel frustrated quite a bit 4. I feel frustrated a lot POSITIVE EMOTION 1. I feel lively a lot 2. I feel lively quite a bit 3. I feel lively a little 4. I do not feel lively at all APPEARANCE 1. I do not worry at all about keeping myself looking nice 2. I worry a little about keeping myself looking nice 3. I worry quite a bit about keeping myself looking nice 4. I worry a lot about keeping myself looking nice 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield Step I Establish dimensions using Exploratory Factor Analysis Step II Eliminate items per dimension using Rasch analysis Step III Select items per dimension using Rasch analysis and input from clinicians and DEMQOL system developers Step IV Exploration of item level reducing per dimension Step V Valuation exercise to elicit TTO health state values Step VI Model TTO valuation results to produce utility values Methodology and development process
  • Step V: Valuation study (defining health states) • Health states generated using DEMQOL-U and DEMQOL-Proxy-U classification • Generated using one level of each dimension. 14/05/2014 © The University of Sheffield
  • Step V: Valuation study (state selection) • DEMQOL-U: 1024 states • DEMQOL-Proxy-U: 256 states • Each respondent valued 8 states using Time Trade Off (TTO) • 7 mixed states and worst state • States selected using simulation • DEMQOL: 87 unique states across 13 blocks • DEMQOL-Proxy: 70 unique states across 12 blocks 14/05/2014 © The University of Sheffield
  • Step V: Valuation study (sample) • Representative sample of UK general population. • NICE and Washington Panel recommend general population values for QALY estimates • Sample • 310 for DEMQOL-U • 290 for DEMQOL-Proxy-U • Obtained by sampling 600 households in urban and rural areas in North England 14/05/2014 © The University of Sheffield
  • Step V: Valuation study (procedure) • Interview process: • EQ-5D and DEMQOL/DEMQOL-Proxy classification system completed for own health • 8 health states with „full health‟ and „dead‟ ranked in order from best to worst. • Practice TTO task • Value 8 health states • Matched methods with UK valuation of EQ-5D • Respondents not informed that health states dementia specific 14/05/2014 © The University of Sheffield
  • Step V: Valuation study (DEMQOL-U results) Mean values for each state Histogram of observed values 14/05/2014 © The University of Sheffield Observed TTO value 1.000.500.00-0.50-1.00 Frequency 800 600 400 200 0
  • Step V: Valuation study (DEMQOL- Proxy-U results) Mean values for each state Histogram of observed values 14/05/2014 © The University of Sheffield Observed TTO value 1.000.500.00-0.50-1.00 Frequency 800 600 400 200 0
  • Step VI: Modelling to produce utility values (methods) • Mean and individual level multivariate regression used to estimate utility decrements • Ordinary least squares (individual/mean level) • Random and fixed effects to account of repeated observations • Random effects Tobit model to take account of TTO observations at 1 • Performance indicators (model): • No. of inconsistent coefficients • No. of significant coefficients • Performance indicators (predictions): • Root mean squared error (RMSE) and mean absolute error (MAE) at state level • No of states with absolute error (AE) greater than 5% and 10% • Plots of actual and predicted health state values 14/05/2014 © The University of Sheffield
  • Step VI: Modelling to produce utility values (DEMQOL-U results) 14/05/2014 © The University of Sheffield • Model 2 selected • 12 significant variables • No inconsistent coefficients • Good predictive ability • low RMSE • Lowest percentage of absolute errors at the state level greater than 0.05 and 0.10. • No systematic bias in the predictions by severity, but some health states had large prediction errors. • Range 0.986 (best state) to 0.243 (worst state),
  • Step VI: Modelling to produce utility values (DEMQOL-Proxy-U results) • „Appearance‟ inconsistent in all models • Health worsens but predicted utility increases • Resolved by merging adjacent inconsistent levels and re-estimating the model to • Model 6 selected due to higher number of significant coefficients. • Range 0.937 (best state) to 0.363 (worst state) 14/05/2014 © The University of Sheffield
  • Discussion (1) • Possible to generate dementia specific utility scores when DEMQOL/DEMQOL-Proxy completed • Utility scores used as the „Q‟ quality adjustment weight of the QALY • DEMQOL-U and DEMQOL-Proxy-U enable utilities to be generated across full dementia severity range • Dementia-specific measures and utility scores designed for a cognitively impaired population • Offer potential advantage over generic measures • Address some of the concerns around using generic measures in dementia • First attempt to derive a PBM for proxy-report • Complementary but key differences • Use both measures in tandem 14/05/2014 © The University of Sheffield
  • Discussion (2) • DEMQOL has interpretable domains amenable to the generation of health state classification systems representing HRQL in dementia • Psychometric and Rasch techniques valid method of generating health states for a specific mental disorder • Do different methods select different items and if so which are most representative of the condition-specific HRQL issues? • Areas of HRQL are not represented in the classification system (e.g. daily activities) • The results have not been validated on another dataset and item responsiveness has not been tested • TTO used but other valuation techniques (standard gamble, discrete choice) may produce different utility values • Comparability with EQ-5D? 14/05/2014 © The University of Sheffield
  • Mapping generic quality of life measurements to disease specific variables in patients with Ankylosing Spondylitis R.M. Ara, J.Brazier University of Sheffield 14/05/2014 © The University of Sheffield
  • Introduction The disease specific instruments the Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) and Bath Ankylosing Spondylitis Functional Index (BASFI) are frequently used to measure disease progression and response to treatment in patients with ankylosing spondylitis (AS). However, economic evaluations demand health related benefits to be measured in terms of quality adjusted lifeyears. These require utility measurements which can be generated using generic instruments such as the EQ- 5D (EuroQol) or the SF-6D 14/05/2014 © The University of Sheffield
  • Objective To identify which QoL instrument (EuroQol or SF- 36) is the most appropriate to measure health state utility values in patients with AS. To derive a relationship between BASDAI and BASFI variables and the preference based index.
  • Methods • Data on the disease specific (BASDAI and BASFI) and HRQoL (EQ- 5D and SF-36) variables were collected during a multi-centre European etanercept (ETN) RCT (n=356) • Differences in the utility scores resulting from the two utility measures were explored through graphical measures (box and scatter plots examining the data at the extremes of the distributions). • Data was analysed using STATA (version 8( and SPSS (version 12) with significance set at P < 0.001. • Linear regressions were used to explore a possible relationship between BASDAI, BASFI and health state utility values. 14/05/2014 © The University of Sheffield
  • Exploration of data Table 1: Correlations between the SF-6D, EQ-5D, BASDAI and BASFI 14/05/2014 © The University of Sheffield
  • Direct comparison of the SF-6D and EQ-5D scores • The pattern of correlation between SF-6D and EQ-5D preference based indices are not linear • A ceiling effect in the EQ-5D data where a wide range in SF-6D indices correspond to EQ-5D = 1 • A floor effect in the SF-6D where a wide range in EQ-5D indices correspond to low SF-6D indices ) • The EQ-5D indices cluster with gaps between EQ-5D = 1 and those < 1, and around EQ-5D = 0.45 (caused by the N3 term) 14/05/2014 © The University of Sheffield Scatter plot of paired SF-6D and EQ-5D utility scores
  • Comparing dimensional scores • There is evidence of correlation between similar dimensions: mobility with physical function; and pain/discomfort with body pain. • The lowest correlations are observed between self care with both mental health and vitality. • All patients reported 1 for the anxiety/depression question on the EQ-5D questionnaire. 14/05/2014 © The University of Sheffield Correlations between the 8 overall dimensional scores in the SF-36 (scale 0-100 where 0 is worse) and the 5 scores from the EQ-5D (scale 1-3 where 3 is worse)
  • Predicting health state utility values • The correlations between the BASDAI and BASFI and EQ-5D utility measurements are reasonably strong at -0.69 and -0.70 respectively. • Using the BASDAI and BASFI as the independent variables and the EQ-5D as the dependent variable in a linear regression; the disease specific measurements explained 52% of the variance in utility. • The model predicts health state utility values over the range 0.09 to 0.92, corresponding to BASDAI and BASFI measurements of 100 and 0 respectively. • Utility = 0.920 - 0.004*BASFI - 0.004*BASDAI (p<0.001; MAE=0.13) 14/05/2014 © The University of Sheffield 0 0.2 0.4 0.6 0.8 1 0 20 40 60 80 100 BASDAI & BASFI (scale 0-100) ... Utility(scale0-1)... Using BASDAI & BASFI to predicted health state utility values
  • Discussion/Conclusion • There are large differences in the EQ-5D and SF-6D health state utility values • The differences in the health state utility values are comparable with other results [7] • The results supports evidence which suggests utility values derived using different instruments in spine patients are not directly comparable [6]. • The EQ-5D health state utility values differ from those observed by Maetzel et al. [8] but are similar to those reported by Kobelt et al. [9] and Boonen et al.[10] • The SF-6D is likely to underestimate the magnitude of improvement in quality of life for patients starting at the lower end of the scale • The EQ-5D may be more sensitive to changes in patients with severe AS and thus may be the most appropriate instrument for measuring HRQoL in patients with severe AS
  • Deriving an algorithm to convert the 8 mean SF-36 dimension scores into a mean EQ-5D preference-based score from published studies when patient level data are not available Roberta Ara and John Brazier ScHARR, University of Sheffield 14/05/2014 © The University of Sheffield
  • Introduction Policy decision makers like NICE recommend generic preference-based measures such as EQ-5D and HUI are used to value health related quality of life (HRQoL) in economic evaluations. Many important research studies do not use a generic preference-based measures. One approach to tackle this is to map non-preference based measure onto a generic preference-based measure, such as the EQ-5D. Existing mapping functions have been estimated and tested in individual level data, but researchers may not have access to individual level data. This presentation presents a mapping function designed and tested for use in sub-groups of patients.
  • Objective To derive a method to predict a mean sub- group EQ-5D preference-based index score using published mean statistics of the eight dimension scores describing the SF-36 health profile.
  • Methodology DATASETS: Individual patient level data (n=6,350) pooled from 12 studies covering a wide range of health conditions are used to explore relationships between the 8 dimension scores of the SF-36 and the preference-based EQ-5D index. STATISTICAL METHODS: The EQ-5D index is regressed onto the eight health dimension scores using ordinary least square (OLS) regressions. The general model is defined as: EQ = + 1xi + 2di + 3ri + whereby EQ represents the EQ-5D preference-based index, x the vector of main effects (PF, RP BP, GH, VT, SF, RE, MH), d the vector of demographs (age and gender), r the vector of main effect squares, i individual respondents and the stochastic error term of the regression (the residual).
  • Results
  • Out-of Sample Cohort Results
  • Goodness of fit using out-of sample summary statistics
  • Discussion/Conclusion • The models provide analysts with a mechanism to estimate EQ-5D utility data from published mean SF-36 dimension scores for use in economic analyses. • The errors observed at the individual level still exist when applying the algorithms to summary data, but the magnitude of the errors is much smaller. • We advocate Model EQ(1) as the first choice for predicting mean EQ-5D scores from mean dimension SF-36 scores when patient level data is not available. • If comparing differences between study arms or changes over time, Model EQ(2) is the preferred choice. • Caution should be taken when applying the algorithm to conditions: - likely to have very low mean utility scores - where energy is an important symptom • Further research is required to validate the results in other health conditions.
  • Predicting the SF-6D preference- based index using mean SF-6D health dimension scores: approximating health related utilities when patient level data are not available John Brazier and Roberta Ara ScHARR, University of Sheffield 14/05/2014 © The University of Sheffield
  • Introduction • The SF-6D preference-based index can be derived from individual level SF-36 data to calculate QALYs for use in economic evaluation. • However, SF-36 results are usually published in terms of profile of 8 mean dimension scores. • As patient level data is required to derive the SF-6D (i.e. 11 items), a substantial proportion of this evidence cannot be used in cost-effectiveness models that rely on a synthesis of published evidence. 14/05/2014 © The University of Sheffield
  • Objective To derive an algorithm to predict the preference- based SF-6D index using the eight mean health dimension scores from the SF-36 (v2) when patient level data are not available. 14/05/2014 © The University of Sheffield
  • Methodology DATASETS: Individual patient level data (n=6,890) is pooled from 15 studies covering a wide range of health conditions STATISTICAL METHODS: The SF-6D index is regressed onto the eight health dimension scores using ordinary least square (OLS) regressions. The general model is defined as: SF = + 1xi + 2di + 3ri + whereby SF represents the SF-6D preference-based index; x the vector of main effects (the 8 dimension scores PF, RP, BP, GH, VT, SF, RE, MH); d the vector of demographic characteristics (age and gender); r the vector of first order interactions between the main effects (i.e. the eight squares and 28 products of the main effects); vectors of unknown parameters; i individual respondents, and the stochastic error term of the regression (the residual). 14/05/2014 © The University of Sheffield
  • Regression Results 14/05/2014 © The University of Sheffield
  • Validation: Using mean scores to predict subgroup values 14/05/2014 © The University of Sheffield
  • Conclusion • This study presents a mechanism to estimate preference-based SF-6D scores from the eight mean dimension scores derived from the SF-36 questionnaire. • This study is unique in that it looks at predicting a preference- based SF-6D score using published dimension scores. This will be very useful to those seeking to populate cost effectiveness models from a synthesis of published evidence (e.g. as required for submissions to NICE). • • Further research is required to validate the results in different health conditions. 14/05/2014 © The University of Sheffield
  • Health and well-being research in ScHARR John Brazier Professor of Health Economics ScHARR, University of Sheffield Email: j.e.brazier@sheffield.ac.uk
  • Health technology assessment HTA is concerned with the cost-effectiveness of different healthcare treatments • Does the technology work? • If so, who for? • At what cost? • How does it compare with alternative treatments? © The
  • Health technology assessment • HTA is concerned with the cost- effectiveness of different healthcare treatments • Does the technology work? • If so, who for? • At what cost? • How does it compare with alternative treatments? © The Measure of outcome that is comparable across treatments and patient groups QALY
  • QALY • QALY combines both quantity and quality of life into a single measure • QALY can be used across all health care interventions for all patient groups
  • Health utility measurement • Health utility measurement provides the quality adjustment weight, „Q‟, for the QALY with two components: • Health state description • Corresponding utility value Example: SF-6D (others include EQ-5D and HUI3)
  • SF-6D Physical Functioning Your health does not limit you in vigorous activities……. Your health limits you a little in vigorous activities…….... Your health limits you a little in moderate activities…...... Your health limits you a lot in moderate activities……..... Your health limits you a little in bathing and dressing...... Your health limits you a lot in bathing and dressing……... Role Limitations You have no problems with your work or other regular daily activities as a result of your physical health or any emotional problems.......………….. You are limited in the kind of work or other activities as a result of your physical health…… You accomplish less than you would like as a result of emotional problems……...…........... You are limited in the kind of work or other activities as a result of your physical health and accomplish less than you would like as a result of emotional problems…… Social Functioning Your health limits your social activities none of the time…... Your health limits your social activities a little of the time..... Your health limits your social activities some of the time….. Your health limits your social activities most of the time…... Your health limits your social activities all of the time…….... Pain You have no pain……...………………………… You have pain but it does not interfere with your normal work (both outside the home and housework) …….. You have pain that interferes with your normal work (both outside the home and housework) a little bit………… You have pain that interferes with your normal work (both outside the home and housework) moderately…........ You have pain that interferes with your normal work (both outside the home and housework) quite a bit………... You have pain that interferes with your normal work (both outside the home and housework) extremely………... Mental Health You feel tense or downhearted and low none of the time...... You feel tense or downhearted and low a little of the time..... You feel tense or downhearted and low some of the time….. You feel tense or downhearted and low most of the time…... You feel tense or downhearted and low all of the time……… Vitality You have a lot of energy all of the time……..... You have a lot of energy most of the time........ You have a lot of energy some of the time…... You have a lot of energy a little of the time….. You have a lot of energy none of the time…… The following questions ask about different aspects of your health. There are six groups of statements, each covering a different aspect of health. Please tick one statement in each group to show the statement which best describes your health over the past 4 weeks.
  • SF-6D scoring algorithm Physical functioning PF23 -0.035 PF4 -0.044 PF5 -0.056 PF6 -0.117 Social functioning SF2 -0.057 SF3 -0.059 SF4 -0.072 SF5 -0.087 Pain PN23 -0.042 PN4 -0.065 PN5 -0.102 PN6 -0.171 Role limitation RL 234 -0.053 Mental health MH 23 -0.042 MH4 -0.100 MH5 -0.118 Vitality V234 -0.071 V5 -0.092 Interaction term MOST -0.061 (if at 3 most serious levels for PF or at 2 most serious levels for any other dimension)
  • Development of preference- based measures • SF-6D from SF-36 • Valuation studies conducted in Australia, Brazil, Hong Kong, Japan, Portugal, Singapore, UK • Generic child-specific measure, CHU-9D • Condition-specific QALY measures for asthma, overactive bladder, cancer, sexual functioning, paediatric atopic dermatitis, flushing • Enables data collected in study to be used to populate cost- effectiveness models • „Bolt-on‟ measures for EQ-5D (and SF-6D) for when a gap is identified the generic • Finally broader „wellbeing‟ measures 38
  • Valuation methods • Currently time trade-off method is complex and biased by time preference • Developing simpler ordinal methods (e.g. Pair wise tasks – do you prefer state A or B?) • Use of willingness to pay to capture non-health benefits • NICE currently uses general population values, but open to criticisms • Current research examining use of patient and carer values vs. general population © The
  • Use of preference-based measures • What to do when EQ-5D is not used? Estimate regression models between measure used in trial and generic measure like EQ-5D (i.e. „Mapping‟ or „cross walking‟) • When EQ-5D is inappropriate (i.e. Not sensitive or relevant to population)? Examples include in vision, hearing, cancer, mental health, skin • Literature reviews and synthesis of utility values • Undertaken in vision, breast cancer, osteoporosis • Use in cost effectiveness models 38
  • Beyond health effects • Estimation of utility impact on carers and family of children e.g. Study in ADHD for Shire • Can involve primary data collection • Productivity effects – the impact of health on the productivity • Carer effects – impact of carer costs, leisure time and productivity • Is a QALY and QALY regardless of who gets it – work for DH on weighting QALYs for the Burden of Illness (for UK DH Policy makers) • Well-being QALY 38
  • Key recent studies Use of generic and condition specific measures in NICE decision making Medical research Council Assessing the Impact of Children with ADHD on the health and wellbeing of their families Shire Pharmaceuticals Inc. Validating generic preference-based measures of health in mental health populations and estimating mapping functions for widely used specific measures Medical Research Council Estimation of a preference-based single index measure of health form the EORTC QLQ-C30 for use in multiple myeloma patients Janesson-Cilag Ltd Development and testing of methods for deriving preference-based measures from condition specific measures National Institute of Health Research (NIHR) Health Technology Assessment (HTA) Methods Programme A review and meta analysis of utility values in breast cancer Pfizer Ltd. An update of a systematic review of health state utility values for Osteoporosis-related conditions Lilly UK Developing a broader measure for economic evaluation MRC For further information see http://www.sheffield.ac.uk/scharr/sections/heds/mvh
  • The Development of the Child Health Utility 9D (CHU9D) and other Paediatric Preference Based Measures Dr Katherine Stevens School of Health and Related Research The University of Sheffield, UK
  • Overview • Background – economic evaluation and QALYs • Issues in child QALY measures • Current options • Development of the CHU9D • Discussion
  • Economic evaluation Quality adjusted life years (QALYS)
  • Economic evaluation • Scarce resources and competing needs • Health care resource allocation decisions • Economic evaluation can help • Compare the difference in costs and benefits of alternatives
  • Economic evaluation • Different types of economic evaluation • Main difference is benefit measurement • In cost utility analysis benefits are typically measured in quality adjusted life years - QALYs
  • Quality adjusted life years (QALYs) • Combine length of life and quality of life into a single measure • Increased survival rates • e.g. cancer, premature birth, heart conditions • Impact on HRQoL not just length of life • Increase either or both • Most useful type for resource allocation decisions
  • How do we calculate QALYs? • Collect information on length of life • Quality adjustment where 0 =dead and 1=perfect health • E.g. 3 years at 0.8 = 3*0.8=2.4 QALYs • How do we get this quality adjustment?
  • Off the shelf measures • Off the shelf measures are commonly used in clinical trials and health care research studies • They consist of a descriptive system and a set of preference weights which assign a preference weight to each health state • These preferences are often based on the adult general population • Adult and child QALY measures
  • Issues in child QALY measures
  • What should be included in the descriptive system? • Dimensions that are relevant to the purpose of the measure • Health care resource allocation decisions • Quality of life vs health related quality of life • e.g. Items on toys, housing, environment • Dimensions that are relevant to children
  • Why are children different?
  • Why are children different? • Differences not just in language but at the conceptual level • Dimensions that are relevant for children but not for adults e.g. school • Dimensions that are not relevant for children but are relevant for adults e.g. work, sexual relationships
  • Age range • Teenagers, younger children, babies • Confounding with development in younger children, particularly under 5 years of age • HRQoL measures may demonstrate an improvement but this could be due to natural development, not to an improvement in QoL
  • Taken from the HUI2 Unable to walk at all Required the help of another person to eat, bathe, dress or use the toilet
  • Who provides the information? • Growing recognition children have their own unique views and a right to express them in matters affecting them Increasingly recognized in clinical trials and HSR that descriptions of the experience of a health state should be elicited from the patients – to reflect the actual experience of the disease and its treatment
  • Whose preferences? • Publicly funded health care system • Tax payers • Adults as rational and informed individuals • Adult general population vs patients • Children?
  • Current options available
  • What exists? • Currently 3 existing preference based measures available and 1 in development • The Health Utilities Index 2 (HUI2) • The AQoL 6D • The EQ 5D Y •The Child Health Utility 9D (CHU9D)
  • The HUI2 • The Health Utilities Index 2 • 6 dimensions,(sensation, mobility, emotion, cognition, self care and pain) each with 4-5 levels • Application: generic • Age range: 5 + years • Mode of completion: self, proxy, administered • Recall period: 1,2, 4 weeks or usual self
  • HUI2 example health state • Sees, hears, or speaks with limitations even with equipment • Requires mechanical equipment (such as canes, crutches, braces, or wheelchair) to walk or get around independently. • Occasionally fretful, angry, irritable, anxious, depressed, or suffering “night terrors”. • Learns and remembers very slowly and usually requires special educational assistance. • Requires mechanical equipment to eat, bathe, dress, or use the toilet independently. • Occasional pain. Discomfort relieved by non prescription drugs or self-control activity without disruption of normal activities.
  • The AQoL 6D • 6 dimensions, (Independent Living, Mental Health, Coping, Relationships, Pain, Senses) • Application: generic • Age range: adolescents • Mode of completion: self completion • Recall period: current
  • AQoL 6D Example Health State Physical ability Excellent Social and family My close friendships make me generally relationships unhappy. There are some group activities I am not involved in because of my health. Mental health I usually feel sad, I often feel worried, I am sometimes calm and sometimes agitated Coping Excellent Pain I suffer from severe pain. Pain often interferes with my usual activities. Vision, hearing & Excellent Communication
  • The EQ-5D Y • Same dimensions as the original adult EQ 5D • Empirical work done with children to alter language but assumption that concepts remain the same • Further research needed to determine if descriptive system complete • No preference weights (yet)
  • Development of the CHU9D
  • The Child Health Utility 9D • Other instruments - Content generated from a mixture of literature reviews, expert opinion and interviews with relevant populations -parents and paediatricians • Adaptation from adult instruments • Evidence that parent‟s views are affected by their own health status, knowledge, experience and expectations • Development of a child centred measure
  • Development of the descriptive system • Aim – to find out how health affects children‟s lives • 1 to 1 interviews with 70 children age 7-11 years with a wide range of acute and chronic health problems to find out how they say health affects their lives. • interviews with children at 2 primary schools in Sheffield • Sampled purposively on basis of health and balanced for gender, age and ethnicity
  • List of Health Problems Covered in Interviews 7-9 years Headache Allergies - various Feeling sick, being sick Nose bleeds Hearing, glue ear – grommets Ear ache Poor vision Muscle not growing properly in stomach Tummy ache Fever Eye infection No feeling in legs Dyslexia Badly cut nose Asthma Nausea Broken arm Verruca Tooth decay Chicken pox Leaky ear Heat rashes Flu Sensitive to food colouring/hyperactivity Pneumonia Tonsillitis Hay fever Sticky/lumpy eyes Cough Twisted ankle Spots/rash Fits Sore throat Eczema Broken toe Itchy eyes
  • Interviews • Topic guide • Tell me about your health/health problems. • How does this affect you (probing for school, home…)
  • Forming dimensions WORRYING • What was going to happen to them • Health would get worse • Would always have the health problem • Physical symptoms I felt like really, really worried, worry me, like, um it felt really scratchy and itchy just really worried like it‟s never gonna stop and it‟s never gonna go away. (Female, 8, fair health)
  • SLEEP • Symptoms – coughing, sick, itching • Worrying • Medication – broken sleep Yeah I couldn‟t sleep cos it really hurt my throat when I slept, so I couldn‟t sleep at all. (B110, female, 11, good health) In the night I wake up because I‟m scratching it. And how does that feel? Horrible because I can‟t get back to sleep. (B28, male, 9, excellent health)
  • Dimension 7-9 Years 9-11 Years 1 Worried Scared Worried 2 Sad Upset Sad Upset Unhappy Miserable 3 Annoyed Frustrated Annoyed Frustrated Angry 4 Hurt Pain Hurt Pain 5 School work Learning 6 Daily Routine Daily Routine 7 Tired Weak Drowsy Tired Weak Energy Weary 8 Joining in activities that want to Joining in activities that want to 9 Sleep Sleep 10 Embarrassed 11 Jealous
  • Developing levels/scales • Development of levels/scales using the data • It‟s just really frustrating • I just feel really frustrated • It was really annoying • I was really annoyed • it‟s quite annoying • really annoys me • they just kind of annoy me a bit. • a bit annoyed. • it still gets a bit annoying • it was really annoying
  • Testing the measure • Study 1: 150 children in 2 schools • 7-11 years • Balance of age, gender, ethnicity • 1 to 1 administration • Study 2: 97 children in hospital • 7 – 11 years • Balance of age, gender, ethnicity • 1 to 1 administration • Mix of chronic and acute conditions. Surgical and medical patients, inpatient and day care.
  • Psychometric Performance • Practicality – excellent – response and completion rates, low time to complete • Majority able to self complete • Good face and construct validity • Reduction of dimensions and levels • Final descriptive system
  • The Child Health Utility 9D (CHU9D) • 9 dimensions, (worried, sad, pain, tired, annoyed, schoolwork, sleep, daily routine, activities) each with 5 levels • Application: generic • Age range: 7-11 years • Mode of completion: self completion • Recall period: today/last night
  • CHU9D HUI2 AQoL 6D EQ 5D Y Worried Emotion Mental Health Anxiety and depression Sad Annoyed Tired Pain Pain Pain Pain/discomfort Sleep Daily Routine Self Care Independent Living Self care School work Cognition Usual activities Joining in activities Mobility Mobility Usual activities Sensation Senses Relationships Coping
  • Example health state I feel very worried I feel quite sad I feel a little bit annoyed I have a lot of pain I feel a little bit tired I have many problems sleeping I have many problems with my daily routine I have a few problems with my schoolwork I can join in with some activities
  • Obtaining preference weights • Valuation interviews were undertaken with 300 members of the UK adult general population • Modelling • Every health state defined by the descriptive system has a preference weight • This can be used to calculate QALYs
  • Obtaining preference weights • 9 dimensions, 5 levels = 1,953,125 states • Valuation interviews were undertaken with 300 members of the UK adult general population • Standard gamble and ranking • Sample of health states
  • Obtaining preference weights • Orthogonal design for main effects only • 64 health states in the design • Each respondent: • Completed descriptive system • Ranking of 9 states (including the worst state) • SG of 9 states (ping pong version using chance board)
  • Obtaining preference weights • Regression modelling to estimate models that could predict a value for every health state defined by the system. • A range of models were tested and were evaluated based on their predictive performance
  • Types of Model • Standard OLS • Fixed effects • Random effects • Mean, median • Rank • Data set not large enough to test all possible interactions
  • Assessing Models • Significance of coefficients and expected sign • Coefficients increasing in absolute size • Adjusted R2 • MAE and RMSE • % predicted within 0.1 and 0.05 of mean value • Graph of observed and predicted • Test for bias in predictions
  • Obtaining preference weights • 2478 observations from 282 respondents (6% of respondents were excluded. • Mean health state values range 0.388 - 0.932 • 23 negative valuations (0.93%) • Interactions made things worse • Best 3 models were OLS, RE and mean
  • Further Modelling • All models had a few inconsistencies or insignificant levels • Further modelling to estimate a parsimonious consistent regression model using the general to specific approach, by combining inconsistent levels and removing non significant levels • This was done on the two best performing models (the mean and OLS restricted)
  • Further Modelling Test OLS Mean Inconsistencies 0 0 % within +/-0.1 98.41 98.41 % within +/-0.05 73.02 76.19 MAE 0.0343 0.0349 RMSE 0.0426 0.0431 T test -0.770 -0.336
  • Comparison of measures How was the content generated? Who completes it? Whose values? CHU9D Interviews with children Child or proxy UK adult general population HUI2 Literature review followed by views of parent/child Child or proxy •Parents (Canada) •Adult general population (UK) AQoL 6D Adaptation of adult instrument Adolescent Adolescent or adult EQ 5D Y Adaptation of adult instrument Child/Adolescent None yet
  • Application • Range of paediatric health care research studies in the UK, US and Australia, including economic evaluations alongside clinical trials, observational studies, methodological studies and routine data collection in clinics • Clinical areas include: obesity; diabetes; ADHD; intensive care; dentistry; sleep; heart defects; mental health and clinical genetics
  • Application in adolescents • Validation in other age groups • Recent research to test the validity of using the CHU9D in an adolescent population (11-17 years) • Community based sample, web based • Demonstrated good construct validity and practicality • Research into obtaining adolescent preferences
  • Discussion • Content of descriptive system – children only • Use in other age groups and testing in clinical populations • Adult preference weights • Further research is needed to investigate the impact of children‟s preferences • Methods that could be used to obtain child preferences.
  • Summary • Issues with child QALY measurement • There are options available • Many issues outstanding for further research www.chu9d.org
  • Bayesian Statistics in Health Economics 14/05/2014 © The University of Sheffield
  • The Centre for Bayesian Statistics in Health Economics (CHEBS), in collaboration with the School of Mathematics and Statistics is internationally respected for its research in Bayesian statistics. CHEBS disseminates its research by publishing papers in scientific journals and presentations at scientific meetings, and provides training in the application of Bayesian methods in health economics. 14/05/2014 © The University of Sheffield
  • Research Themes • Evidence synthesis • Modelling health-state preference data • Elicitation of expert‟s probabilities • Meta-modelling • Bayesian clinical trial simulation 14/05/2014 © The University of Sheffield
  • Information Resources 14/05/2014 © The University of Sheffield
  • The Information Resource Group has established itself as a national key player in providing information support to health technology assessment and health services research. 14/05/2014 © The University of Sheffield
  • Research areas • Support to health and social services research, including systematic reviews • Education in systematic reviews methods and health informatics • Library and Enquiry services • E-learning and Continuing Professional Development • Creation and evaluation of electronic information resources • Training in information skills for health and information professionals • Evidence Based Information Practice and Health Information Project Evaluations • Development of information syntheses and summaries in health and social care
  • Courses provided • Evidence Synthesis of Qualitative Research in Europe (ESQUIRE) • Issues and Challenges for Qualitative Research in Evidence Synthesis (InCQuiRES) • Folio Australia • Folio Shall • Systematic Reviews
  • Examples of information studies conducted in HEDS 14/05/2014 © The University of Sheffield
  • The anatomy of e-learning Lynda Ayiku and Anthea Sutton Information Specialists School of Health and Related Research (ScHARR), University of Sheffield 14/05/2014 © The University of Sheffield
  • Introduction CPD needs of health information professionals because… Demands on NHS knowledge services have become increasingly sophisticated, requiring that health information professionals‟ roles evolve to include: • knowledge management • training in information and evidence-seeking skills • involvement in clinical decision making • implementation of policies 14/05/2014 © The University of Sheffield
  • Barriers to traditional face-to- face CPD training… Despite their requirements for training in these areas, health information professionals often find it difficult to get away from the workplace to attend training due to: • budgetary constraints • shortage of time • small numbers of staff and lack of appropriate staff cover 14/05/2014 © The University of Sheffield
  • FOLIO was created to overcome these difficulties… • The online courses enable participants to study without the need to leave the workplace • The courses are free of charge and are available for all information professionals who support staff working within the UK National Health Service (NHS) 14/05/2014 © The University of Sheffield
  • What is FOLIO? (1) FOLIO is the…. Facilitated Online Learning as an Interactive Opportunity programme 14/05/2014 © The University of Sheffield
  • What is FOLIO? (2) • FOLIO is a programme of continuing professional development (CPD) e-learning courses for health information professionals • Part of the National Library for Health Librarian Development Programme (NLH LDP) • NLH LDP provides support and training for librarians moving into new roles • Course topics are based on a Training Needs Analysis (TNA) that was carried out in 2003. • TNA revealed that technical skills, management skills, professional skills and contextual knowledge were required 14/05/2014 © The University of Sheffield
  • Completed FOLIO Courses Up to yet, 13 FOLIO courses have been delivered… Management skills • Making your case successfully (MACHIAVEL) • Managing change for health information professionals (MCHIP) • Managing for service quality (MSQ) Contextual knowledge • Information for social care (I4SC) • Understanding the business of clinical care (CLINICOS) 14/05/2014 © The University of Sheffield
  • Completed FOLIO Courses Professional skills • The FOLIO customer care course (FRONTIER) • Planning and conducting an information needs analysis (PACINA) • Maximising the impact of your service (MAXIM) • Getting to grips with knowledge management (G2G) • Designing, conducting and analysing surveys and questionnaires (ASQ) Learning and teaching skills • Designing and delivering information skills training courses (INFOSKILLS) • Evaluating information skills training courses (INFOSKILLS2) • Introduction to e-learning (E-FOLIO)
  • FOLIO Team The FOLIO courses are delivered by a team at ScHARR. The FOLIO Team comprises: • Andrew Booth (Programme Director) • Anthea Sutton and Lynda Ayiku (Learning Resource Co- ordinators) Plus external input from experts in the field • In addition, clerical support is provided by the Course Administrator, Tricia Campsell 14/05/2014 © The University of Sheffield
  • Aims of the FOLIO programme 14/05/2014 © The University of Sheffield
  • (1) Informing • Providing participants with an overview of topics relevant to their work • Teaching is provided via course materials such as: - Briefings - Guest lectures - Interactive PowerPoint presentations - etc 14/05/2014 © The University of Sheffield
  • (2) Engaging • We provide a mixture of formal and informal teaching methods to: 1.engage and motivate participants 2.increase the potential to meet a range of preferential learning styles among participants 14/05/2014 © The University of Sheffield
  • (3) Socialising • Online learners often feel isolated when undertaking e-learning courses • This can lead to de-motivation and participant withdrawal from e-learning courses • The FOLIO courses include a „buddy‟ system to help lessen feelings of isolation • Buddies are required to collaborate with each other on specific FOLIO exercises and activities 14/05/2014 © The University of Sheffield
  • Planning FOLIO courses: Lessons learnt 14/05/2014 © The University of Sheffield
  • Lessons learnt… Workload • Fewer, evenly spread exercises and tasks • Timescales giving participants time to interact with buddies and complete tasks etc. • Participants now receive a clear indication of time needed on the FOLIO courses (2-4 hrs per week) 14/05/2014 © The University of Sheffield
  • Lessons learnt… Buddy system • Participants given the opportunity to follow the buddy route or the self-directed route • Larger buddy groups- overcomes problem of „orphan‟ buddies 14/05/2014 © The University of Sheffield
  • Current and upcoming FOLIO courses Current course (Jun-Jul 2006): • Promoting and marketing library and information services (ProMISe) [Sept-Oct 2006] Future courses: • Moving into supervision: Supervisory skills for paraprofessionals and/or newly qualified professionals (MOVES) [Sep-Oct 2006] • Breaking out of the Box: Extending the health LIS professional role- skills and strategies (BREAKOUT) [Nov-Dec 2006] 14/05/2014 © The University of Sheffield
  • Any Questions? • Contact the FOLIO course team at: folio@sheffield.ac.uk • Keep an eye on the FOLIO website for future courses and developments: www.nelh.nhs.uk/folio 14/05/2014 © The University of Sheffield
  • Portals and Pitfalls: Developing web-based research portals for the NHS and University of Sheffield Andy Tattersall, Claire Beecroft, Anna Cantrell, Information Specialists, ScHARR, University of Sheffield 14/05/2014 © The University of Sheffield
  • Rationale for Developing the Portals (1) • Establishment of the NIHR RDS • Lack of knowledge about information management among NHS researchers • Need to simplify gathering of information from a wide range of multi-media sources • Changing nature of research- information resources are becoming more diverse 14/05/2014 © The University of Sheffield
  • Rationale for Developing the Portals (2) • ScHARR, University of Sheffield- key UK Public Health research school • Research is conducted among teams with members from throughout the school • ScHARR is keen to innovate- hence recruitment of a specialist in web technologies 14/05/2014 © The University of Sheffield
  • How portals can help you • Keep you up to date with what interests you • Provide a point of reference • Help you find, share and collate information • Help you interact • Combine text, links, images audio and video • Be accessible anywhere • Provide a snapshot on a topic, organisation, country, the world • Provide entertainment • Portals are adaptable and moderately easy to master • Be automated (for the most part) • Make your life simpler? 14/05/2014 © The University of Sheffield
  • How portals can hinder you • They are not as automated as we would like • RSS feeds can break • Web pages can go out of date or just disappear • There is always new information, links and people to add • Multiple moderation is needed for specialist topics • Not all content is applicable to everyone – UK/US angles • Information overload • Pages can be slow loading • Need for a decent Internet connection • Sponsored links • They could make your working life more complicated 14/05/2014 © The University of Sheffield
  • Choosing a portal provider 14/05/2014 © The University of Sheffield
  • Choosing a portal provider (2) • Initial evaluation focused on: Pageflakes, Netvibes and iGoogle • Eventually chose Netvibes for a number of reasons: • Comprehensive and adaptable • Content can be spread over a number of tabs • Many widgets available: rss/Atom feeds, calendars, search engine boxes, notes, bookmarks, Flickr photos, Facebook, YouTube, Twitter, email and user-created modules • Netvibes provides reliable support to users. 14/05/2014 © The University of Sheffield
  • Widgets “A web widget is a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation” 14/05/2014 © The University of Sheffield Wikipedia (2009) http://en.wikipedia.org/wiki/Web_widget
  • The ScHARR Portal 14/05/2014 © The University of Sheffield
  • Journal feeds and podcasts 14/05/2014 © The University of Sheffield
  • Tailoring Your Information • Health News • Special Research Topics and Interest Groups • Library Journals • Videos • Funding Feeds • Maps • Customised Search Engines 14/05/2014 © The University of Sheffield
  • Supporting Specialists (1) 14/05/2014 © The University of Sheffield
  • Supporting Specialists (2) 14/05/2014 © The University of Sheffield
  • What next ? (1) • Evaluation of the portals • Horizon scanning for new portal providers • Horizon scanning for new web 2.0 (and web 3.0) tools 14/05/2014 © The University of Sheffield
  • What next ? (2) • Working with NHS to overcome access issues • Increasing the number of portals • Sharing with other RDS • Investigating commercial potential 14/05/2014 © The University of Sheffield
  • Bite Size Technology Sessions to Support Research, Teaching and Collaboration Andy Tattersall Information Specialist ScHARR, University of Sheffield Email: a.tattersall@sheffield.ac.uk 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • ScHARR • The School of Health and Related Research (ScHARR) specialises in health services and public health research • ScHARR concentrates on postgraduate teaching and delivers a teaching and learning portfolio based on research-based, international, multi-disciplinary and world-class curricula 14/05/2014 © The University of Sheffield
  • A growing problem • Too much choice – which one is best? • Too little time – too much time wasted • Lack of awareness – lack of application 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • The Ingredients • Informal promotion via email, blogs and in house posters Informal interactive presentations from a mixture of academics, technical, clerical and professional staff Staff, PGR and PGT students welcome 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • Structure • 20 minute session 10 minutes for questions • Guest speakers from across the University • Slides made available and recordings uploaded to Intranet, YouTube and Vimeo 14/05/2014 © The University of Sheffield
  • The story so far • Started in Autumn 2010 • 29 previous Bite Size sessions - including Google Docs, Prezi, PowerPoint tools, professional social networks, uSpace, Echo360, Pubget, Research Net Contribution, Assessment Methods, Screencasting, Wikis, Electronic Voting Systems, Senate teaching awards, Google Scholar, Mendeley, rss, social media, Google Apps, voice works, data copyright and the Cloud, MOLE 2, mobile phone apps, video capture. 14/05/2014 © The University of Sheffield
  • To come • The Research Excellence Framework, supervising PhD students, what the Teaching Support Unit can do for you, Scirus scientific search engine, Google Maps for Research, Plagiarism, Medline, Cinahl, what ScHARR Library can do for you, Interactive whiteboards, creating effective posters, how to give a memorable presentation, research costing 14/05/2014 © The University of Sheffield
  • 14/05/2014 © The University of Sheffield
  • Opening the Information Literacy Toolbox Helen Buckley Woods Information Specialist ScHARR, University of Sheffield Email: h.b.woods@sheffield.ac.uk
  • Housekeeping • permission slip and recorder
  • Outline of session 14/05/2014 © The University of Sheffield • Introduction • Jigsaw activity • Break • Reflection and feed back on groupwork • Summary of session and evaluation • Closing comments
  • Aim • An opportunity to explore the relationship between theory and practice in information literacy teaching.
  • Learning Outcomes • By this end of this session participants should be able to: • Analyse four areas of interest in learning and teaching • Evaluate the effectiveness of active learning techniques • Describe how they will apply what they have learnt in their practice.
  • Active learning • What is it? • Methods • Evidence - effectiveness • Seen as most appropriate method • Explore through Jigsaw Atherton J S (2011) Teaching and Learning; Lectures [On-line: UK] retrieved 3 July 2011 from http://www.learningandteaching.info/teaching/lecture.htm
  • Active Learning • “Learning is not a spectator sport. Students do not learn much just by sitting in classes listening to teachers, memorizing pre-packaged assignments, and spitting out answers. They must talk about what they are learning, write about it, relate it to past experiences and apply it to their daily lives. They must make what they learn part of themselves.” Chickering, Arthur W. and Gamson, Zelda F. (1987) Seven Principles for good practice in undergraduate education. American Association for Higher Education & Accreditation Bulletin [online], 39 (7) 3-76. Available from: http://www.aahea.org/buletins/bulletins.htm [3 July 2011]
  • Question • Write down any challenges you have experienced when supporting learners. • Discuss with a partner • Share with the group
  • Jigsaw Activity Aim: acquire lots of information in short space of time Analyse four areas of interest in learning and teaching 1. Planning learning 2. Working with small groups 3. Working with large groups 4. Motivating learners
  • Jigsaw Group Activity Aim: acquire lots of information in short space of time •2 groups •4 different areas – you will become the expert in 1 of these areas •Review and digest the information •Meet with fellow experts •Return to your original group and present your findings
  • Reflection • How did it feel to be a learner? • Was it effective? • How useful would it be for you as a facilitator? • How would you adapt the session? • Consider the four themes – what was notable for you about the material we looked at?
  • Conclusion and Evaluation • In this session • Listening, note taking, discussion, reflection, working alone, group work, peer teaching, feedback, answering/asking questions. • Engaging with different materials • Explicit or mediated theory
  • Supporting the Health Researcher of the Future Andrew Booth & Andrew Tattersall, Information Resources Group ScHARR, University of Sheffield
  • Presentation Plan • The Context of Research Support (AB) • The Potential of Web 2.0 (AB)  • The Way Forward? (AB) • Some Examples of Good Practice (AT) • What we are doing in ScHARR/Yorkshire & Humber (AT) Questions (AB/AT)
  • The Context of Research Support Andrew Booth
  • From Gamekeeper….
  • Researchers‟ Use of Academic Libraries and their Services • Significant differences of perceptions and views between researchers and librarians • Communication channels need to be improved. How? • Research community uses social networking to exchange and share research-based information. • Role of libraries presently ill-defined. • Researchers don’t readily recognize content on their desktop is provided through library. Researchers’ Use of Academic Libraries and their Services A report commissioned by the Research Information Network and the Consortium of Research Libraries
  • Research habits Users "power-browse" or skim material, using "horizontal" (shallow) research. Most spend only a few minutes looking at academic journal articles and few return to them. "It almost seems that they go online to avoid reading in the traditional sense,” Not just "screenagers". “Undergraduates to professors….exhibit a strong tendency towards shallow, horizontal, flicking behaviour in digital libraries. Factors specific to the individual, personality and background are much more significant than generation." INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE
  • Researchers information literacy training should focus on information management, not information retrieval – 1 (Booth, 2007) • Inappropriate to meet IL needs of researchers using instruction methods based on undergraduates; • Researchers do not follow neat stepwise progression from state of unknowing (“information need”) to knowing that underpins most IL instruction. • Information management, rather than information retrieval, should be focus of IL instruction for researchers.
  • Researchers information literacy training should focus on information management, not information retrieval – 2 (Booth, 2007) • Information retrieval should focus on “area scanning”, footnote chasing and known author searching rather than keyword searching • IL training should be “socialised” through formal collaboration …..and integration with existing research programmes or research groups. • Training should focus on practically based outcomes e.g. production of log book or portfolio. • Training should optimally be tailored to individual and delivered at time of need.
  • RIP - Library as Place • “the library has changed from being the place for researchers to visit for help with information searching and for picking up the actual information, to being the “living room” for undergraduate students, making the researchers who visit the library feel outnumbered, and sometimes unwelcome.” Haglund and Olsson (2008)
  • Rethinking the Library Web Site • “Libraries spend huge amounts of time and money to work on the structure and content of the library Web page, while few researchers use it as a starting point for information searching. Many researchers….used the Web of their own department as a starting point, and this is where the library should establish a presence with direct links targeted to that particular group”. Haglund and Olsson (2008).
  • The Potential of Web 2.0 Andrew Booth
  • The Health Researcher of Today: • Expects service that optimizes benefits of technology: • anywhere, • anytime, • personalized, • at the point of need, • instantly. • Requires services that are seamless, integrated and open access
  • Web 2.0 • Includes specific applications that actively engage users (advanced searches for information and production of information). • Young predominant early users of social networking and user-generated content. • Social networking sites mainly used for simple social interactions but sometimes linked to information searches. • Not only Google Generation display advanced online behaviour. Other “Generations” also able and willing to engage in complex online activities. INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE - A British Library / JISC Study
  • Impact of Web 2.0 • Blogs, wikis, podcasts, social networks/other online features offer new educational opportunities. • Application of e-learning technologies is evolving. More than publishing content online - dynamic learning environment in which to actively share knowledge and…experiences. • Blogs, wikis and social networking stimulate dynamic and proactive engagement in learning (or indeed investigation!) process. • INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE
  • Some Candidate Web 2.0 Applications • One stop shops – Wikis & Portals • Project-based Blogs/Wikis/Discussion Fora • Collaborative tools – Shared Slides/ Documents/Bookmarking/Tagging • “How To”s – Instructional Resources (Wikis)/Videos • The Business of Research – e.g. Publication/Research Assessment/Funding Opportunities (RSS feeds)
  • Some Examples of Good Practice Andrew Tattersall
  • Dublin Public Library Portal
  • Tropika.Net
  • http://greylit.pbworks.com/
  • Central Medical Library, University Medical Center Groningen (UMCG)
  • Central Medical Library, University Medical Center Groningen (UMCG)
  • What we are doing in ScHARR/Yorkshire & Humber Andrew Tattersall
  • Context • Independent University-based library serving health services researchers and students and providing tertiary information services for NHS researchers (Trent RDSU up to March 2010) • Move away from physical use to use of virtual services • New Yorkshire & Humber Research Design Support Service (from October 2008)
  • Personal Homepages and Web Portals
  • How could it help us? • Keep you up to date with what interests you • Entertain you • Be a point of reference • Be formal and informal • Be automated (for the most part) • Help you find/share/collate information • Help you interact • Be accessible anywhere • Combine text, links, images audio and video • Give you snapshot on topic, organisation, country, the world • Adaptable and moderately easy to master • Make your life simpler?
  • How could it hinder us? • Not as automated as we would like • RSS feeds can break • Web pages go out of date or just disappear • Always new information, links and people to add • Multiple moderation needed for specialist topics • Not all content is applicable to everyone – UK/US angles • Information overload • Pages can be slow loading • Need for decent Internet connection • Could make your working life more complicated • Sponsored links
  • Widgets “A web widget is a portable chunk of code that can be installed and executed within any separate HTML-based web page by an end user without requiring additional compilation” Wikipedia (2009) http://en.wikipedia.org/wiki/Web_widget
  • The ScHARR Portal
  • Journal feeds and podcasts
  • Tailoring Your Information •Health News •Special Research Topics and Interest Groups •Library Journals •Videos •Funding Feeds •Maps •Customised Search Engines
  • Evaluation of The ScHARR Portal • “Easy access to resources - saving time” • “Could give quick access to interesting info and share what's going on around ScHARR” • “I value the links to anything to do with my current project and report writing” • “Will be a handy one-stop place for lots of information” • “I can get the right feeds for me on there”
  • The Way Forward? Andrew Booth
  • At the moment…… We are at the “If we Build them a Portal will they come?” phase of development
  • However….. • Evidence both locally and internationally that health researchers have an appetite….. • Not for the technology itself, • But for the “What‟s In It for Me” of new technologies • They will need Guides……and Architects
  • Where will become less important than How!
  • Above All…… • Health researchers will want us to open our toolbox of “make your job easier/more effective tools” • Will include both free Web 2.0 tools and conventional products (e.g. reference management; citation tools etc)
  • Welcome to the era of Web Tool Point Zero!
  • References - 1 • Booth A (2008) Google: It's all at the Co-op now! Health Info Internet; 62: 3-4. • Booth A (2007) Researchers require tailored information literacy training focusing on information management, not simply information retrieval. Report for Research Information Network Consultative Group on Librarianship and Information Science. http://www.rin.ac.uk/training-research-info- spec • CIBER. Information behaviour of the researcher of the future – (A British Library/JISC Study) http://www. bl.uk/news/pdf/googlegen.pdf
  • References - 2 • Haglund L and Olsson P (2008). The Impact on University Libraries of Changes in Information Behavior Among Academic Researchers: A Multiple Case Study. Journal of Academic Librarianship 34 (1), 52-59 • Research Information Network (2008). Researchers’ Use of Academic Libraries and their Services A report commissioned by the Research Information Network and the Consortium of Research Libraries http://www.rin.ac.uk/researchers-use-libraries • Tattersall A (2008) 'Blogging in an Academic Health Library Setting. Libraries for Nursing Bulletin; June 2008.
  • Library Thing: friend or foe? Using its potential to update the Core Collections series Helene Gorring and Helen Buckley Woods Information Specialists ScHARR, University of Sheffield
  • Introduction - what is this about? • What are the Core Collections? • Challenges • Solutions using Web 2.0 • Medical Core Collection - live demo
  • Background to the Core Collections History - Medical Information Working Party Medical Core Collection has been acquisition tool for health librarians for nearly 20 years. Collections were developed independently: • Medical • Nursing & Midwifery • Mental Health • Allied Health Process had become very time consuming...
  • Issues to Address: Three challenges: • Communication • Limited opportunity to contribute • Format of finished product
  • Methodological Solutions Challenge Limited opportunity to contribute Established methods email, email lists, liaison with clinicians and information professionals Web 2.0 opportunity
  • Piloting Library Thing - Mental Health Core Collection First edition published in 1999 A briefing on the project and instructions on how to contribute using LT was sent out in mid-may 2009
  • Outcomes and Feedback Wider range of contributors • 14 Librarians (10 NHS, 1 Higher Education, 2 from Special Libraries, BMA & RCPsych) • 8 Clinicians • 2 Lecturers " Library Thing has revolutionised the production of this core list. It's an ideal application and has allowed us to consult much more widely and efficiently." " It was quite a time consuming exercise as I tried to look at every area, but it flagged up new editions that I had missed so was directly useful for our service" " I think this was a very effective way of working together to produce the list."
  • Where are we at? Updated Core Collections
  • Contributing to the 6th edition of the Medical Core Collection - volunteers needed! Live Demonstration using Library Thing
  • POST GRADUATE TAUGHT COURSES
  • Courses • MSc Health Economics and Decision Modelling • MSc International Health Technology Assessment, Pricing and Reimbursement 14/05/2014 © The University of Sheffield
  • MSc Health Economics and Decision Modelling 14/05/2014 © The University of Sheffield
  • Why choose this course? • Only Masters programme in the UK dedicated to the practical application of mathematical modelling to inform healthcare decision making • The programme is suitable for those who wish to pursue careers as applied economists in the health sector or as the basis for doctoral research 14/05/2014 © The University of Sheffield
  • Principal aims of the course • Equip students with an analytical skill-base required to evaluate and improve the efficiency and effectiveness of healthcare systems to a professional level. • Provide an in-depth understanding of current state-of- the-art methods for mathematical modelling to support and inform evidence-based healthcare decision problems. • Provide a firm grounding for students wishing to pursue a career in health economics and mathematical modelling. 14/05/2014 © The University of Sheffield
  • Core modules • Cost-effectiveness Modelling for Health Technology Assessment • Introduction to Health Economics • Health Research Methods • Economic Evaluation • Advanced Topics in Economic Evaluation • Medical Statistics and Evidence Synthesis • Operational Research Techniques in Health Resource Allocation • Advanced Simulation Methods • Dissertation 14/05/2014 © The University of Sheffield
  • Duration and Mode of Study • One year-full time • Two year-part time A full Masters course consists of units of study totalling 120 credits and a dissertation 14/05/2014 © The University of Sheffield
  • Career Opportunities • Ideal for students wishing to pursue a career as a professional health economist/modeller • Graduates will be suitably prepared to carry out analytical work at a professional level in multidisciplinary teams for a range of organisations including: • Pharmaceutical companies • Health insurance/sickness funds • Healthcare organisations • Department of health /Ministries of health and associated agencies • Municipalities and Public Health departments • Universities • Management consultancies
  • Who is eligible for the course? • A first degree (minimum 2.1) from a recognised university in a numerate subject such as Economics, Operational Research, Mathematics, Statistics, Industrial Engineering, Management Science, Physics, Systems Control • A good level of written language, both written and oral • Willingness to engage with staff and students for mutual benefit • Motivation to establish an excellent career in Health Economics and Decision Modelling
  • Further information For further information, visit us on: http://www.sheffield.ac.uk/scharr/prospective _students/masters/hedm 14/05/2014 © The University of Sheffield
  • MSc in International Health Technology Assessment, Pricing and Reimbursement 14/05/2014 © The University of Sheffield
  • Why choose this Masters course? • Only graduate programme offering the entire range of knowledge and skills needed by those developing health technologies for market, or involved in commissioning or evaluating health technologies • Suitable for those with busy work commitments in these fields • Meets the needs of those involved in health technology policy formulation, management and evidence-based commissioning and purchasing of health technologies for governments or health services
  • Programme content • Developing disease management pathways • Comparative healthcare systems • Economic evaluation in HTA • Searching for evidence to support HTA • Evidence synthesis • The application of modelling to HTA • Utilities and patient-reported outcomes • Design of trials • Analysing randomised and non-randomised trial data • Epidemiology for HTA • Systematic review methods • Financing health technologies • Marketing health technologies • Strategy for health technology development
  • Duration and mode of Study • One year-full time • Two/three years-part time You can choose to study for a postgraduate Certificate over one year, or a Diploma over one or two years. It is also possible to study individual modules to meet specific learning needs
  • Career Opportunities • The course will help you progress in your chosen career in market access, trial design and other aspects of health technology assessment, in the private or public sector • The course will provide comprehensive training needed to gain a post in industry or working as a health technology evaluator or commissioner 14/05/2014 © The University of Sheffield
  • Who is eligible for the course? • A first degree (minimum 2.2), an equivalent professional qualification or relevant work experience • An adequate level of English, both written and oral (IELTS 7.0) • Enthusiasm to work hard and benefit from the course environment 14/05/2014 © The University of Sheffield
  • Further information For further information visit us on: http://www.shef.ac.uk/scharr/prospective_st udents/masters/ihtapr/index 14/05/2014 © The University of Sheffield
  • SHORT COURSES 14/05/2014 © The University of Sheffield
  • Courses on: • Using Utility Data for Health Technology Assessment • Cluster Randomised Trials • Evidence synthesis of qualitative research in Europe • Issues and Challenges for Qualitative Research in Evidence Synthesis • A guide to doing mixed methods studies in the health sciences 14/05/2014 © The University of Sheffield
  • Using Utility Data on Health Technology Assessment - Two days course 14/05/2014 © The University of Sheffield
  • Background • NICE and similar decision-making bodies around the world are increasingly using QALYs in assessing cost-effectiveness of healthcare interventions. • This raises the usual questions about the measurement of health, the valuation of health and whose values to use 14/05/2014 © The University of Sheffield
  • What does the course deliver? • The course outlines the practical requirements of measuring utility and obtaining utility data for health technology assessment agencies such as NICE 14/05/2014 © The University of Sheffield
  • Who will benefit from the course? • Academics • Government agencies • Pharmacoeconomics • Outcomes experts in industry and consultancies with an interest in the use of health state utility data 14/05/2014 © The University of Sheffield
  • Course Content Key issues in obtaining health state utility value • Does it matter which instruments are used? • What is the current NICE reference case? • When are EQ-5D and other generic measures not appropriate? • What are the latest developments in EQ-5D including EQ-5D-5L valuation? • What should be done when EQ-5D and other generics are not appropriate? • What should be done when (the generic) reference case data are not available? • How can social value QALY weights be used? E.g. for Value Based Pricing methods. • How should utility values be incorporated into cost effectiveness models? The course will consist of a mixture of presentations, group work, discussions and individual exercises. 14/05/2014 © The University of Sheffield
  • Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 14/05/2014 © The University of Sheffield
  • Cluster Randomised Trials- One day course 14/05/2014 © The University of Sheffield
  • Background • Cluster Randomised Trials (CRTs) are trials which randomise groups of patients rather than individual patients. For example: randomising communities to different public health interventions, or randomsing practitioners to different methods of treating patients. 14/05/2014 © The University of Sheffield
  • What does the course deliver? • Understand issues such as contamination and recruitment bias • Be able to decide how big a study should be • Read and critique the literature on cluster trials • Understand how to analyse CRTs using simple methods and also regression as implemented in a variety of packages such as SPSS, R and Stata. The participants will have the opportunity to analyse data from a cluster randomised trial. 14/05/2014 © The University of Sheffield
  • Who will benefit from the course? • Clinicians who need to design CRTs • New researchers who want to appreciate the benefits and disadvantages of CRTs • Statisticians who want to understand how to design and analyse CRTs 14/05/2014 © The University of Sheffield
  • Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 14/05/2014 © The University of Sheffield
  • Evidence synthesis of qualitative research in Europe- Two and a half day course 14/05/2014 © The University of Sheffield
  • Background • The course will follow the systematic review process as it applies, and is adapted to, qualitative evidence syntheses. 14/05/2014 © The University of Sheffield
  • What does the course deliver? The Programme Faculty will take the participants through the stages of defining and exploring scope, conducting the literature searches, quality assessing studies for inclusion, synthesising data and writing up and presenting the product of the synthesis. 14/05/2014 © The University of Sheffield
  • Who will benefit from the course? • Qualitative Researchers who want to learn how to synthesise qualitative research • Systematic Reviewers who want to learn how to translate synthesis methods to qualitative data • PhD Students in topic areas that require substantive review of qualitative research • New researchers who want to develop evidence synthesis/systematic review • Qualitative systematic reviewers who want to update and extend their skills and knowledge 14/05/2014 © The University of Sheffield
  • Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 14/05/2014 © The University of Sheffield
  • Issues and Challenges for Qualitative Research in Evidence Synthesis- One day course 14/05/2014 © The University of Sheffield
  • Background • This methodological update will rehearse the major issues and challenges for the main steps of the systematic review process as it applies, and is adapted to, qualitative evidence syntheses. 14/05/2014 © The University of Sheffield
  • What does the course deliver? The Presenters, drawn from the Cochrane Qualitative Research Methods Group and involved in producing methodological guidance, will examine the specific challenges as they take participants through the stages of • defining and exploring scope • conducting the literature searches • Quality assessing studies for inclusion • synthesising data and • writing up and presenting the product of the synthesis. 14/05/2014 © The University of Sheffield
  • Who will benefit from the course? • Qualitative Researchers wishing to extend their knowledge of how to synthesise qualitative research • Systematic Reviewers who wish to advance translation of synthesis methods to qualitative data • PhD Students in topic areas who are undertaking a substantive review of qualitative research • Qualitative systematic reviewers who want to update and extend their skills and knowledge 14/05/2014 © The University of Sheffield
  • Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 14/05/2014 © The University of Sheffield
  • A guide to doing mixed methods studies in the health sciences- One day course 14/05/2014 © The University of Sheffield
  • Background • The course will follow the process of undertaking a mixed method study from developing research questions, selecting an appropriate design, integration of data and findings, through to reporting in peer reviewed publications. 14/05/2014 © The University of Sheffield
  • What does the course deliver? The course will cover key issues such as: • relevant paradigms • quality assessment • team work and • structural issues affecting mixed methods research. The focus will be on health research, particularly health care research. 14/05/2014 © The University of Sheffield
  • Who will benefit from the course? • Researchers who have undertaken, are undertaking, or are planning to undertake a mixed methods study in the health sciences. 14/05/2014 © The University of Sheffield
  • Further information If you have any queries, please contact Jacquie Bennett. Email: jacquie.bennett@sheffield.ac.uk or telephone + 44 (0)114 222 2968. 14/05/2014 © The University of Sheffield