Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

J.1756 5391.2010.01087.x


Published on

Published in: Education, Health & Medicine
  • Be the first to comment

  • Be the first to like this

J.1756 5391.2010.01087.x

  1. 1. Journal of Evidence-Based Medicine ISSN 1756-5391METHODOLOGYA methodological framework of preparing economicevidence for selection of medicines in the Chinese settingXin Sun1 and Thomas Alured Faunce21 Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, China2 Associate Professor and Australian Research Council Future Fellow, College of Medicine, Biology and the Environment and College of Law(joint appointment), Australian National University, AustraliaCorrespondence AbstractXin Sun, Chinese Evidence-Based MedicineCenter, West China Hospital, Sichuan Medicines are becoming a major component of health expenditure in China. Se-University, No. 37 GuoXueXiang, Chengdu lection of effective and cost-effective medicines represents an important effort to610041, China. improve medicines use. A guideline on cost-effectiveness studies has been avail-Tel: +86-28-85423458 able in China. This guideline, however, fails to be a practical tool to prepare andFax: +86-28-85422253Email: critically appraise economic evidence. This article discusses, in the Chinese con- text, the approach to integrating economic component into the medicines selection, and elaborates the methods of producing economic evidence, including conducingReceived 22 June 2010; accepted for economic reviews and primary economic studies.publication 5 July 2010.doi: 10.1111/j.1756-5391.2010.01087.x Medicines are an increasingly significant component of tical tools for preparing and using economic evidence forhealth expenditure in China. The medicines expenditure selection of medicines. This article offers a methodologi-of hospitalized patients, for example, usually accounts for cal framework to address issues about economic evidence inaround 45% of their total medical bill (1). Rapid increases the context of medicines selection for a Chinese Essentialin price are rendering medicines unaffordable to many Medicines List.Chinese patients. The demand for more efficient, quality useof medicines has become increasingly strident, and the im- Integrating economic evidence intoplementation of an Essential Medicines List in patient care the selection of medicinesrepresents one of the efforts to improve medicines use. Selection of effective and cost-effective medicines is the Integrating economic evidence in the process of medicineskey to implementing an efficient Essential Medicines List. selection for an Essential Medicines List typically starts withIn the last decade, extensive methodological work has been formulation of a focused economic question that reflects pol-undertaken in China to improve the assessment of effective- icy maker’s perspective. It is followed with a systematic re-ness of medicines (2, 3). However, little was done to explore view of existing studies that assesses the strength of eco-best approaches to addressing cost-effectiveness. Despite nomic evidence. The subsequent decisions will then be madeincreasing publication of cost-effectiveness studies in the on the basis of strength of economic evidence, which can beChinese medical literature (4, 5). the methodological quality categorized into three levels.varies substantially. r Strong evidence without uncertainty. The existing evi- In an effort to improve the conduct of cost-effectiveness dence provides precise and unbiased estimates of compar-studies, China has developed a guideline on economic evalua- ative cost effectiveness, and the study settings—includingtion of medicines (6). This endeavor—which follows a stream patients, interventions, outcomes, and clinical practiceof similar attempts in other countries (7–11)—offers method- patterns—are applicable to the question being evaluated. Inological standards of conducting economic studies. However, such case, a decision about acceptance or refutation is ableit fails to provide policy makers and researchers with prac- to be made.156 JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University
  2. 2. X Sun and TA Faunce Economic evidence for selection of medicines Develop the research question Corresponding to the research questions, eligibility criteria Target population, interventions, and outcomes should be established. Eligibility criteria may be wide or Search the economic studies Foreign and Chinese medical databases, narrow, however, should be reflected with policy makers’ Specialized database, health economics databases perspective. That is, the patients and interventions defined, Screen the economic studies and outcomes of interest should be relevant to the policy Primary inclusion, and justify the exclusion making. For example, the finding that a new drug gives a Assess the methodological quality Adequate – secondary inclusion, and move to patient increased quality adjusted life years (QALYs) may the next step; Inadequate – exclusion be more important than the finding it alters physiological Assess the transferability parameters (like blood pressure). Acceptable – final inclusion In defining the criteria, the systematic review should in- Poor – exclusion Extract and adjust data clude all types of study designs (eg, modeling, trial-based Adjustable – results applicable for drug studies, database study, and observational cohort), and all decision-making Analyze and summarize data Not adjustable – justify the results forms of evaluation (eg, cost-effectiveness analyses, cost- utility analyses, cost-consequence analyses, and cost analy-Figure 1 The flowchart of systematic review of economic evidence. ses), unless otherwise indicated by policy makers. Methodological qualityr Moderate evidence suggesting cost-effectiveness pro- Methodological quality, which provides the extent to whichfiles. The estimates of comparative cost-effectiveness are the results are biased, is subject to study design, data collec-either imprecise, involve moderate likelihood of bias, or tion, and data analysis, and reporting (15–17). Tools address-the study settings differ in part of components affecting ing methodological quality of economic studies differ (18–transferability of results. In such case, a decision will in- 20), and two checklists—Drummond’s checklist (28). andvolve uncertainty, and conduct of new economic studies is Chiou’s checklist (21)—are commonly used, both consistingdesirable.r Week evidence involving substantial uncertainty. The of similar items. Chiou’s instrument also weighs items, and is validated externally (22). A modified Drummond’s check-estimates of results involve significant uncertainty (either the list incorporating three set of weights (23). demonstrated itsestimates is imprecise, the methodological quality is very ability in rating methodological quality, while not validated.low, or both), and/or the study settings are substantially dif- We recommend using both checklists to examine consis-ferent from question being evaluated. New economic studies tency of results; in the presence of discrepancy, one mayare required in such circumstances. consider exploring details. In presenting the risk of bias of estimates, the overall methodological quality may be ratedSystematic review of economic and categorized into three levels (ie, high, moderate, andevidence low).The framework for conducting systematic review ofeconomic evidence on a medicine’s comparative cost- Transferabilityeffectiveness is presented in Figure 1. Empirical studies areundertaken elsewhere (12–14). We discuss points with par- The extent to which the study is applicable (or transferrable)ticular interest to the selection of medicines for subsidy under to the question of interest depends on a number of potentiala Chinese Essential Medicines List. factors (24–27). Currently, neither consensus nor instrument is available to rate the impact of on the transferability of results. Here, we suggest key factors and propose methodsDevelop the research question and of assessment (Table 1).eligibility criteria Methodological characteristics are likely to be adjustable,A single most important step of an economic review is to de- including currency, base year, and discount rate, Methods arevelop the research question. Typically, this question has three available for adjusting these characteristics, and the adjust-minimal components, including study population, interven- ment requires availability of sufficient primary data. The im-tion strategies, and health outcomes of interest. Collaboration pact of patient characteristics on transferability is subtle, andbetween researchers and policy makers is crucial in devel- empirical evidence manifesting this does not exist. Healthoping a well-defined research question. An example might preference is among the major factor influencing transfer-be a study comparing the cost-effectiveness of a new heart ability. In assessing the potential impact, patients character-failure medication against an existing marketed product at istics should be listed, and the judgment about the impactthe standard dose and price in rural or urban populations. should be justified.JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 157
  3. 3. Economic evidence for selection of medicines X Sun and TA FaunceTable 1 Factors influencing transferability of economic evidenceFactors Appraisal guideMethodological characteristics Perspective Applicable if conducted from the healthcare sector perspective, or if disaggregated data were available from the societal perspective. Possibly inapplicable if applying a narrower perspective, or if aggregated data are used. Clinical setting Desirable if including study population of diverse socio-economic conditions Potentially problem for a single healthcare centre study and international study. Discount rate Applicable if using the same discount rate or similar range. Costing methods Questionable if micro-costing methods are not used.Patient characteristics Demographics Minor impact on the transferability in the presence of age or sex differences only. Case mix May significantly influence transferability of results. Potentially inapplicable in the presence of differential case mix groups across countries/regions. Comorbidity Small to moderate effect on transferability, particularly when its contribution to total costs and effects are minor. Health preferences Its impact on transferability varies. Sensitivity analyses may be considered to examine its impact. Compliance Particular relevance to trial-based studies. Its impact depends on the extent of compliance. Small to moderate influence on transferability.Healthcare system characteristics Unit costs/prices Large impact on transferability. Adequate adjustment might be appropriate to transfer the published results to the Chinese setting. Resources use Its impact depends on clinical practice variations and patient characteristics. Questionable transferability if resources use pattern differ across regions. Adjustment is usually not sufficient. Transferable if only a small part of resources use differs and have minimal impact on the total costs. Clinical practice variations Unlikely transferable if clinical practice pattern differs substantially. Unit costs, resources use, and clinical practice patterns Audiences and perspectiverepresent the key components affecting transferability. Unit Typically, policy makers form the primary audience of eco-costs usually differ across regions, and may be adjustable. nomic studies, and health care sector perspective may beClinical practice patterns, which also underlie resources use, used as the primary perspective of study, supplemented withhave the great potential of influencing transferability, and an analysis with the societal perspective (37).may be least adjustable. In the presence of differences ofclinical practice patterns, a summary detailing the differencesand potential impact is desirable. Indications Ideally, the study population should cover all that are pre- scribed with major and minor indications. Most often,Data Analysis and Presentation economics studies focus on major registered drug indica-Economic studies are often heterogeneous, and may differ tions. Nevertheless, the selection of indications needs to bequalitatively (28). Four approaches are available to ana- justified.lyze results in systematic reviews (19, 29–31), and narra-tive summaries and permutation matrix represent two most Comparatorswidely used methods. However, all these methods have dif-ficulties with handling discrepant results that equally spread In the context of competing medications, three treatmentin the matrix. A possible solution is to use the single best strategies—including most widely existed practice, mini-evidence. mal practice/or doing-nothing practice, and the drugs it might replace—may be included as comparators. In the pres- ence of large number of competing medicines, compara- tors should at least include a reference drug listed in theConducting primary studies formulary, and the most widely used alternatives (includ-In the context of medicines selection, the absence of reliable ing “do nothing” or “minimal practice”). The decision andand transferable economic studies necessitates the conduct use of reference drug is usually based on decision-makers’of new studies. perspective.158 JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University
  4. 4. X Sun and TA Faunce Economic evidence for selection of medicinesTime horizon Four approaches are used to measure resources use, the level of measurement ranging from primary data to expertThe choice of time horizon is typically made on the basis of opinions. The choice of approach is based on study per-question of interest. A desirable time horizon is able to cap- spective, contribution to total costs, availability of data, andture important health outcomes and costs. Surrogate markers balance between internal and external (40). We recommendare often used in clinical studies, in which case modeling that that resources use of significant contributions to the totalrepresents an alternative to acquire outcomes important to costs should be measured. Expert opinions are the last resortpatients. of measuring resources use. Unit costs can be obtained through national healthcareStudy design price reports, charges or tariffs, published data, and calcu- lation of unit costs. National Healthcare Price Report rep-Five study designs can inform cost-effectiveness studies resents a common source of unit costs, and may be good(33). Pragmatic trials provide reliable elements for assess- representation of price from societal perspective. Chargeing economic efficiency of medicines, and offer advantages and tariff is potentially useful to value price from health-of achieving desirable internal and external validity, while it care sector perspective. Naive charge is likely to over rep-is associated with substantial costs and time. Piggy-backed resent unit costs, and an adjustment for charge-to-pricestudies enjoy the high internal validity, but are susceptible to ratio may be desirable. Published data, although readilylow external validity. Probably, multicenter trial offers an al- accessible, may be invalid. In any case, the choice ofternative to addressing the drawback. Observational studies unit costs is associated with measurement of resourcesoften yield “real-world” data, but are more prone to bias. Such use (41).design is time and cost consuming. Retrospective databaseanalysis provides a relatively inexpensive and expedient ap-proach. Its major advantages include a wide and large pop- Discountingulation over long time, and coverage of current practice. Its Most analysts agree both the health outcomes and costs bekey limitation is low internal validity. Modeling technique discounted at the same rate. A discount rate of 5% can beoffers a flexible and timely framework. However, integra- used as the reference case, and a range of 3%–6% used fortion of data from various studies and populations mandates sensitivity analysis.sophisticated expertise, and may render biased results. Dealing with uncertaintyHealth outcomes The approaches to handling uncertainty depends on the na-Study outcomes of cost-effectiveness studies include natural ture of data. A CI around cost-effectiveness ratios may beclinical outcomes and health-related adjusted quality of life. estimated, using one of the following methods including theThe single best natural outcomes may be based on a synthe- Fiedler’s theorem method, the Taylor series or delta method,sis of multicenter trials, secondary to observational studies nonparametric bootstrapping estimation techniques. A sen-and expert opinions (34). In the case of cost-utility analyses, sitivity analysis is desirable, and the choice of types of sen-the approaches to valuing outcomes differ (35, 36). Standard sitivity analyses depends. One rule of thumb is to conductgamble and time trade-off may be desirable through a deliv- sensitivity analysis on those variables that have importantery of patient survey. Quality of life instruments may also be influence on results. The development of cost-effectivenessused for valuing utility (37,38). ratios comparing cost with equality of life years gained will become an important factor in government price negotiation over what it will pay manufacturers whose drugs are listedValuing costs on the Essential Medicines List.The approaches to valuing costs differ (39), and can be con-ducted in four steps, including identification of cost compo- Data analysis and presentationnents, measurement of resources use, identification of unitcosts, and valuing resources use. Identification of cost com- Information about resources use, unit costs, total costs, andponents is usually associated with the choice of perspective. health outcomes should be fully reported. In the presence ofIn the context of health care sector, direct medical costs are a dominant strategy, there is no need to report incrementalmeasured, and resources use is collected associated with dis- ratio. The incremental ratio is reported, otherwise. The resultsease management. The impact of each cost-generating event of analyses should be presented in disaggregated manner, soon the total costs differs. We suggest documenting details of that the results could be recalculated and examined whenthose events that have significant contributions. adapting to different perspectives.JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 159
  5. 5. Economic evidence for selection of medicines X Sun and TA FaunceConclusion 11. Canadian Coordinating Office for Health Technology Assessment. A guidance document for the costing processThe development of a Chinese Essential Medicines List in (Version 1.0), Canadian Coordinating Office for Healthwhich the cost of listed medicines is subsidized is an impor- Technology Assessment, Ottawa, 1996.tant public health initiative. Value for government expendi- 12. Brady BK, McGahan L, Skidmore B. Systematic review ofture on such a list will be enhanced if pharmaco-economic economic evidence on stroke rehabilitation services.experts recommend listing after rigorous cost-effectiveness International Journal of Technology Assessment in Health Careanalysis of new medications against existing products and 2005; 21(1): 15–21.initiate a related price negotiation with the manufacturer. 13. Holloway RG, Benesch CG, Rahilly CR, Courtright CE. A systematic review of cost-effectiveness research of strokeAcknowledgment evaluation and treatment. Stroke 1999; 30(7): 1340–9. 14. Siebert U, Bornschein B, Walbert T, Dodel RC. SystematicThis study is funded by NSFC (70703025). assessment of decision models in Parkinson’s disease. Value Health 2004; 7(5): 610–26.References 15. Jefferson T, Demicheli V. Quality of economic evaluations in 1. Health statistics from the Ministry of Health, China. Available health care: it is time for action to ensure higher methodological from: URL: and quality. British Medical Journal 2002; 324(7333): 313–4. (accessed 20 16. Udvarhelyi IS, Colditz GA, Rai A, Epstein AM. December 2005). Cost-effectiveness and cost-benefit analyses in the medical 2. Li SL, Cao LY, Xiao AL,Yan L, Lan F. Evidence-based literature. Are methods being used correctly?. Annals of Internal evaluation of essential medicines. Chinese Pharmaceutical Medicine 1992; 116(3): 238–44. Issues 2003; 17(6): 337–9 (in Chinese). 17. Jefferson T, Demicheli V. Is vaccination against hepatitis B 3. The Centre for Drug Re-evaluation, State Food and Drug efficient? A review of world literature. Health Economics 1994; Administration. The evaluation process of the essential 3(1): 25–37. medicines. Available from: URL: 18. Drummond MF, Jefferson TO. Guidelines for authors and peer (accessed 11 March reviewers of economic submissions to the BMJ. British Medical 2006). Journal 1996; 313(7052): 275–83. 4. Sun X, Cao LY, Chen XZ, Lan F, Li X, Xiao AL, et al. Four 19. Jefferson T, Demicheli V, Vale L. Quality of systematic reviews non-sedating antihistamines for allergic diseases: health of economic evaluations in health care. Journal of the American technology assessment (protocol). Chinese Journal of Medical Association 2002; 287: 2809–12. Evidence-based Medicine 2003; 3(1): 35–40. 20. Evers S, Goossens M, de Vet H, van Tulder M, Ament A. 5. Li YP, Sun X. An evidence-based approach to evaluation and Criteria list for assessment of methodological quality of selection of post-marketing medicines. In:Zheng XY, Liao QJ, economic evaluations: consensus on Health Economic Criteria. eds. Annual Advance of Pharmacy. Beijing: Chinese Medical International Journal of Technology Assessment in Health Care Science and Technology Press, 2004. 2005; 21(2): 240–5. 6. ISPOR Chinese Medical Doctor Association 21. Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ, Pharmacoeconomics Chapter (ISPOR Chinese MDA-PE Sullivan SD, et al. Development and validation of a grading Chapter). China Pharmacoeconomic Evaluation Guidelines (the system for the quality of cost-effectiveness studies. Medical Solicitation Draft), Beijing, China, 2006. Care 2003; 41(1): 32–44. 7. Commonwealth Department of Human Services and Health. 22. Spiegel BM, Targownik LE, Kanwal F, Derosa V, Dulai GS, Guidelines for the Pharmaceutical Industry on Preparation of Gralnek IM, et al. The quality of published health economic Submissions to the Pharmaceutical Benefits Advisory analyses in digestive diseases: a systematic review and Committee. Commonwealth Department of Human Services and quantitative appraisal. Gastroenterology 2004; 127(2): Health, Canberra, 1995. 403–11. 8. Garattini L, Grilli R, Scopelliti D, Mantovani L. A proposal for 23. Gonzalez-Perez JG. Developing a scoring system to quality Italian guidelines in pharmacoeconomics. Pharmacoeconomics assess economic evaluations. European Journal of Health 1995; 7(1): 1–6. Economics 2002; 3(2): 131–6. 9. Canadian Coordinating Office for Health Technology 24. Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S, Assessment. Guidelines for Economic Evaluation of Urdahl H, et al. Generalisability in economic evaluation studies Pharmaceuticals, Canadian Coordinating Office for Health in healthcare: a review and case studies. Health Technology Technology Assessment, Ottawa, 1997. Assessment 2004; 8(49): iii–iv, 1–192.10. Oostenbrink JB, KoopmanschapMA, Rutten FFH. 25. Welte R, Feenstra T, Jager H, Leidl R. A decision chart for Standardization of Costs: The Dutch Manual for Costing in assessing and improving the transferability of economic Economic Evaluations. Pharmacoeconomics 2002; 20(7): evaluation results between countries. Pharmacoeconomics 2004; 443–54. 22(13): 857–76.160 JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University
  6. 6. X Sun and TA Faunce Economic evidence for selection of medicines26. Barbieri M, Drummond M, Willke R, Chancellor J, Jolain B, effectiveness in health and medicine. New York: Oxford Towse A. Variability of cost-effectiveness estimates for University Press, 1996. pharmaceuticals in Western Europe: lessons for inferring 34. Levels of Evidence – March 2002. Evidence-Based on Call. generalizability. Value Health 2005; 8(1): 10–23. Available from: URL: Greiner W, Schofski O, Graf v. d., Schulenburg JM, et al. The levels.html transferability of international economic health-economic results 35. Arnesen T, Trommald M. Are QALYs based on time trade-off to national study questions. HEPAC 2000; 1: 94–102. comparable? A systematic review of TTO methodologies.28. Nixon J, Khan KS, Kleijnen J. Summarising economic Health Economics 2005; 14(1): 39–53 evaluations in systematic reviews: a new approach. British 36. Dolan P. Valuing health-related quality of life: issues and Medical Journal 2001; 322(7302): 1596–8. controversies. Pharmacoeconomics 1999; 15(2): 119–27.29. NHS Centre for Reviews and Dissemination. Undertaking 37. Bala MV, Wood LL, Zarkin GA, Norton EC, Gafni A, O’Brien systematic reviews of research on effectiveness. CRD’s B. Valuing outcomes in health care: a comparison of willingness Guidance for Carrying Out or Commissioning Reviews. 2nd ed. to pay and quality-adjusted Life-years. Journal of Clinical York: NHS Centre for Reviews and Dissemination, 2000. Epidemiology 1998; 51(8): 667–76.30. Black WC. The CE plane: a graphic representation of 38. Perez DJ, McGee R, Campbell AV, Christensen EA, Williams S. costeffectiveness. Medical Decision Making 1990; 10(3): A comparison of time trade-off and quality of life measures in 212–4. patients with advanced cancer. Quality of Life Research 1997;31. Birch S, Gaffni A. Cost-effectiveness and cost utility analyses: 6(2): 133–8. methods for the non-economic evaluation of healthcare 39. Jacobs P, Ohinmaa A, Brady B. Providing systematic guidance programs and how we can do better. In:Geisler E, Heller O, eds. in pharmacoeconomic guidelines for analysing costs. Managing Technology in Healthcare. Norwell, MA: Kluwer Pharmacoeconomics 2005; 23(2): 143–53. Academic, 1996. 40. Jacobs P, Baladi J-F. Biases in cost measurement for economic32. Drummond MF, O’Brien B, Stoddart GL. Methods for the evaluation studies in health care. Health Economics 1996; 5(6): economic evaluation of health care programs. 2nd ed. Oxford: 525–9. Oxford University Press, 1997. 41. Horngren CT. Cost accounting. 5th ed. Englewood Cliffs, NJ:33. Gold M, Siegel J, Russell L, Weinstein M, editors. Cost- Prentice-Hall, 1987.JEBM 3 (2010) 156–161 c 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 161