Abstract
Cost-effectiveness analysis has gained status over the last 15 years as an important tool for assisting resource allocation decisions in a budget-limited environment such as healthcare. Randomised (multicentre) multinational controlled trials are often the main vehicle for collecting primary patient-level information on resource use, cost and clinical effectiveness associated with alternative treatment strategies. However, trial-wide cost effectiveness results may not be directly applicable to any one of the countries that participate in a multinational trial, requiring some form of additional modelling to customise the results to the country of interest.
This article proposes an algorithm to assist with the choice of the appropriate analytical strategy when facing the task of adapting the study results from one country to another. The algorithm considers different scenarios characterised by: (a) whether the country of interest participated in the trial; and (b) whether individual patient-level data (IPD) from the trial are available.
The analytical options available range from the use of regression-based techniques to the application of decision-analytic models. Decision models are typically used when the evidence base is available exclusively in summary format whereas regression-based methods are used mainly when the country of interest actively recruited patients into the trial and there is access to IPD (or at least country-specific summary data).
Whichever method is used to reflect between-country variability in cost-effectiveness data, it is important to be transparent regarding the assumptions made in the analysis and (where possible) assess their impact on the study results.
Similar content being viewed by others
Notes
The issue is the mismatch between the source of the data and the location of the decision maker.Thus,it could be argued that we should be referring to the ‘jurisdiction of interest ’,where the term ‘jurisdiction’ encompasses both within-country (e.g.regions,provinces)and country-level decision makers.
Note that v j and u j are assumed to follow a bivariate Normal distribution,to reflect the fact that each country ’s mean cost in the control arm is correlated to the differential mean cost.In the analysis of the clinical data,this assumption would reflect the fact that the baseline events are correlated with the relative treatment effects.
This type of bias occurs when the relationship of relative treatment effect and patient averages across countries is not the same as the relationship for patients within countries.It may happen,for instance,that at a trial level a given treatment is clinically more effective (and cost effective)in younger patients.However,using average age in each country as a covariate for subgroup analysis will show no relationship between mean age and relative treatment effect if the distribution of the covariate age is similar across countries.[85,86]
An analysis similar to what would be carried out if one were to explore the generalisability of the absolute treatment effect identified in trials across a range of clinically defined patient subgroups where the same separation of baseline risks and relative treatment effect can be employed.
There are various dimensions in the extrapolation problem. This can relate to beyond-trial extrapolation (that is from short-term to long-term outcomes), from intermediate endpoints to final outcomes, and from intermediate endpoints or final clinical outcomes to health-related QOL.
References
National Institute for Health and Clinical Excellence (NICE). Guide to the methods of technology appraisal. London: National Institute for Health and Clinical Excellence (NICE), 2004
AMCP. The American Managed Care Pharmacy Format for Formulary Submissions. Alexandria (VA): The Foundation for Managed Care Pharmacy, 2005 Apr 2005
CADTH. Guidelines for the economic evaluation of health technologies: Canada. 3rd ed. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health, 2006
Commonwealth Department of Health Housing and Community Services. Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee. Canberra: AGPS, 1992
Gricar JA, Langley PC, Luce B, et al. AMCP’s format for formulary submissions: a format for submissions of clinical and economic evaluation data in support of formulary consideration by managed health care systems in the United States. Alexandria (VA): Academy of Managed Care Pharmacy (AMCP), 2002
SMC. New product assessment form. Glasgow: Scottish Medicines Consortium, 2005
Sculpher MJ, Pang FS, Manca A, et al. Generalisability in economic evaluation studies in health care: a review and case-studies. Health Technol Assess 2004; 8 (49): 1–20
Goeree R, Burke B, Manca A, et al. Generalizability of economic evaluations: using results from other geographic areas or from multinational trials to help inform health care decision making in Canada. CCOHTA HTA Capacity Building Grants Program. Toronto (ON): Canadian Coordination Office for Health Technology Assessment, 2005. Grant N. 67
Drummond MF, Manca A, Sculpher MJ. Increasing the generalisability of economic evaluations: recommendations for the design, analysis and reporting of studies. Int J Technol Assess Health Care 2005; 21 (2): 165–171
Buxton MJ, Drummond MF, Van Hout BA, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ 1997; 6 (3): 217–227
O’Brien BJ. A tale of two (or more) cities: geographic transferability of pharmacoeconomic data. Am J Manag Care 1997; 3: S33–S39
Reed SD, Anstrom KJ, Bakhai A, et al. Conducting economic evaluations alongside multinational clinical trials: toward a research consensus. Am Heart J 2005; 149 (3): 434–443
Weinstein MC. Recent developments in decision-analytic modelling for economic evaluation. Pharmacoeconomics 2006; 24 (11): 1043–1053
Claxton KP, Sculpher MJ. Using value of information analysis to prioritise health research: some lessons from recent UK experience. Pharmacoeconomics 2006; 24 (11): 1055–1068
Torrance GW. Utility measurement in healthcare: the things I never got to. Pharmacoeconomics 2006; 24 (11): 1069–1078
Briggs AH, Levy AR. Pharmacoeconomics and pharmacoepidemiology: curious bedfellows or a match made in heaven? Pharmacoeconomics 2006; 24 (11): 1079–1086
Birch S, Gafni A. Information created to evade reality (ICER): things we should not look to for answers. Pharmacoeconomics 2006; 24 (11): 1121–1131
Buxton MJ. Economic evaluation and decision making in the UK. Pharmacoeconomics 2006; 24 (11): 1133–1143
Goeree R, Levin L. Building bridges between academic research and policy formulation: the PRUFE framework — an integral part of Ontario’s evidence-based HTPA process. Pharmacoeconomics 2006; 24 (11): 1145–1158
Laupacis A. Economic evaluations in the Canadian common drug review. Pharmacoeconomics 2006; 24 (11): 1159–1164
Neumann PJ, Sullivan SD. Economic evaluation in the US: what is the missing link? Pharmacoeconomics 2006; 24 (11): 1165–1170
Sculpher MJ, Drummond MF. Analysis sans frontières: can we ever make economic evaluations generalisable across jurisdictions? Pharmacoeconomics 2006; 24 (11): 1087–1099
O’Shea JC, DeMets DL. Statistical issues relating to international differences in clinical trials. Am Heart J 2001; 142 (1): 21–28
Lewis JA. Statistical principles for clinical trials (ICH E9): an introductory note on an international guideline. Stat Med 1999; 18 (15): 1903–1904
Lewis J, Louv W, Rockhold F, et al. The impact of the international guideline entitled statistical principles for clinical trials (ICH E9). Stat Med 2001; 20 (17–18): 2549–2560
International Conference on Harmonisation; guidance on statistical principles for clinical trials; availability: FDA Notice. Fed Regist 1998; 63 (179): 49583-49598
International Conference on Harmonisation; choice of control group and related issues in clinical trials; availability. Notice. Fed Regist 2001; 66 (93): 24390-1
Chang WC, Midodzi WK, Westerhout CM, et al. Are international differences in the outcomes of acute coronary syndromes apparent or real? A multilevel analysis. J Epidemiol Community Health 2005; 59 (5): 427–433
O’Shea JC, Califf RM. International differences in treatment effects in cardiovascular clinical trials. Am Heart J 2001; 141 (5): 875–880
O’Shea JC, Califf RM. International differences in cardiovascular clinical trials. Am Heart J 2001; 141 (5): 866–874
Gupta M, Chang W-C, Van de Werf F, et al. International differences in in-hospital revascularization and outcomes following acute myocardial infarction: a multilevel analysis of patients in ASSENT-2. Eur Heart J 2003; 24 (18): 1640–1650
Mark DB, Naylor CD, Hlatky MA, et al. Use of medical resources and quality of life after acute myocardial infarction in Canada and the United States. N Engl J Med 1994; 331 (17): 1130–1135
Grieve R, Hutton J, Bhalla A, et al. A comparison of the costs and survival of hospital-admitted stroke patients across Europe. Stroke 2001; 32 (7): 1684–1691
Weir NU, Sandercock PAG, Lewis SC, et al. Variations between countries in outcome after stroke in the International Stroke Trial (IST). Stroke 2001; 32 (6): 1370–1377
Van de Werf F, Topol EJ, Lee KL, et al. Variations in patient management and outcomes for acute myocardial infarction in the United States and other countries: results from the GUSTO trial. JAMA 1995; 273 20): 1586–1591
Barbash GI, Modan M, Goldbourt U, et al. Comparative case fatality analysis of the International Tissue Plasminogen Activator/Streptokinase Mortality Trial: variation by country beyond predictive profile. J Am Coll Cardiol 1993; 21 (2): 281–286
Pilote L, Califf RM, Sapp S, et al. Regional variation across the United States in the management of acute myocardial infarction. N Engl J Med 1995; 333 (9): 565–572
Postma MJ, Leidl R, Downs AM, et al. Economic impact of the AIDS epidemic in the European Community: towards multinational scenarios on hospital care and costs. AIDS 1993; 7 (4): 541–553
Rhodes G, Wiley M. Comparing EU hospital efficiency using diagnostic-related groups. Eur J Public Health 1997; 7 Suppl. 3: 42–50
Cohen MG, Pacchiana CM, Corbala R, et al. Variation in patient management and outcomes for acute coronary syndromes in Latin America and North America: results from the platelet IIb/IIIa in unstable angina: receptor suppression using integrilin therapy (PURSUIT) trial. Am Heart J 2001; 141 (3): 391–401
Lingard EA, Berven S, Katz JN, et al. Management and care of patients undergoing total knee arthroplasty: variations across different health care settings. Arthritis Care Res 2000; 13 (3): 129–136
Stason WB. Cost-effectiveness analysis in health care: opportunities and challenges to international comparisons. In: Lasser U, Roccella EJ, Rosenfeld JB, et al., editors. Costs and benefits in health care and prevention: an international approach to priorities in medicine. Berlin: Springer-Verlag, 1990: 20–26
Baker AM, Goldberg A, Arnold RJ, et al. Considerations in measuring resource use in clinical trials. Drug Inf J 1995; 29: 1421–1428
Baltussen R, Ament A, Leidl R. Making cost assessments based on RCTs more useful to decision-makers. Health Policy 1996; 37 (3): 163–183
Drummond MF. Comparing cost-effectiveness across countries: the model of acid-related disease. Pharmacoeconomics 1994; 5 (S3): 60–67
Revicki DA, Frank L. Pharmacoeconomic evaluation in the real world: effectiveness versus efficacy studies. Pharmacoeconomics 1999; 15 (5): 423–434
Drummond MF. The future of pharmacoeconomics: bridging science and practice. Clin Ther 1996; 18 (5): 969–978
Bailey KR. Generalising the results of randomized clinical trials. Control Clin Trials 1994; 15: 15–23
Bryan S, Brown J. Extrapolation of cost-effectiveness information to local settings. J Health Serv Res Policy 1998; 3: 108–112
Spiegelhalter DJ. Surgical audit: statistical lessons from Nightingale and Codman. J R Stat Soc Ser A Stat Soc 1999; 162 (1): 45–58
Normand S-LT, Glickman ME, Gatsonis CA. Statistical methods for profiling providers of medical care: Issues and applications. J Am Stat Assoc 1997; 92 (439): 803–814
Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc Ser A Stat Soc 1996; 159 (3): 385–443
Roberts C. The implication of variation in outcome between health care professionals for the design and analysis of randomised controlled trials. Stat Med 1999; 18: 2605–2615
Hall BL, Hamilton BH. New information technology systems and a Bayesian hierarchical bivariate probit model for profiling surgeon quality at a large hospital. Q Rev Econ Financ 2004; 44: 410–429
Drummond MF, O’Brien BJ, Stoddart G, et al. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press, 1997
Mason J. The generalisability of pharmacoeconomic studies. Pharmacoeconomics 1997; 11: 503–514
Greiner W, Schoffski O, Graf VD, et al. The transferability of international economic health results to national study questions. HEPAC Health Econ Prev Care 2000; 1: 94–102
Bonsel GJ, Rutten FF, Uyl de Groot CA. Economic evaluation alongside cancer trials: methodological and practical aspects. Eur J Cancer 1993; 29A Suppl. 7: S10–S14
Mason J, Drummond MF, Torrance GW. Some of the guidelines on the use of cost effectiveness league tables. BMJ 1993; 306: 570–572
Bennett CL, Armitage JL, LeSage S, et al. Economic analyses of clinical trials in cancer: are they helpful to policy makers? Stem Cells 1994; 12 (4): 424–429
Briggs A, Sculpher M, Buxton M. Uncertainty in the economic evaluation of health care technologies: the role of sensitivity analysis. Health Econ 1994; 3: 95–104
Haycox A. Pharmacoeconomics: integrating economic evaluation into clinical trials. Br J Clin Pharmacol 1997; 43 (6): 559–562
Rizzo JDPNR. Methodological hurdles in conducting pharmacoeconomic analyses. Pharmacoeconomics 1999; 15 (4): 339–355
Drummond M, Brandt A, Luce BC, et al. Standardising methodologies for economic evaluation in health care. Int J Health Technol Assess Health Care 1993; 9 (1): 26–36
Fayers PM, Hand DJ. Generalisation from phase III clinical trials: survival, quality of life, and health economics. Lancet 1997; 350: 1025–1027
Goeree R, Gafni A, Hannah M, et al. Hospital selection for unit cost estimates in multicentre economic evaluations: does the choice of hospitals make a difference? Pharmacoeconomics 1999; 15 (6): 561–572
Carr Hill RA. The evaluation of health care. Soc Sci Med 1985; 21 (4): 367–375
Neymark N, Kiebert W, Torfs K, et al. Methodological and statistical issues of quality of life (QoL) and economic evaluation in cancer clinical trials: report of a workshop. Eur J Cancer 1998; 34 (9): 1317–1333
O’Connell D, Glasziou P, Hill S, et al. Results of clinical trials and systematic trials: to whom do they apply? In: Stevens A, Abrams K, Brazier R, et al., editors. The advanced handbook of methods in evidence based healthcare. London: Sage, 2001: 57–72
Caro JJ, Huybrechts KF, De Backer G, et al. Are the WOSCOPS clinical and economic findings generalizable to other populations? A case study for Belgium. Acta Cardiol 2000; 55 (4): 239–246
Caro J, Klittich W, McGuire A, et al. The West of Scotland coronary prevention study: economic benefit analysis of primary prevention with pravastatin. BMJ 1997; 315 (7122): 1577–1582
Caro JJ, Klittich W, McGuire A, et al. International economic analysis of primary prevention of cardiovascular disease with pravastatin in WOSCOP. Eur Heart J 1999; 20 (4): 263–268
Shepherd J, Cobbe SM, Ford I, et al. Prevention of coronary heart disease with pravastatin in men with hypercholesterolemia. N Engl J Med 1995; 333 (20): 1301–1307
McAlister FA. Commentary: relative treatment effects are consistent across the spectrum of underlying risks... usually. Int J Epidemiol 2002; 31: 76–77
Gail MH, Simon R. Testing for qualitative interaction between treatment effects and patient subsets. Biometrics 1985; 41: 361–372
Cook JR, Drummond MF, Glick H, et al. Assessing the appropriateness of combining economic data from multinational clinical trials. Stat Med 2003; 22 (12): 1955–1976
Willke RJ, Glick HA, Polsky D, et al. Estimating country-specific cost-effectiveness from multinational clinical trials. Health Econ 1998; 7: 481–493
Snijders TAB, Bosker RJ. Multilevel analysis: an introduction to basic and advanced multilevel modeling. London: Sage Publications, 1999
Willan AR, Pinto EM, O’Brien BJ, et al. Country specific cost comparisons from multinational clinical trials using empirical Bayesian shrinkage estimation: the Canadian ASSENT-3 economic analysis. Health Econ 2005; 14: 327–338
Thompson SG, Nixon R, Grieve R. Addressing the issues that arise in analysing multicentre cost data, with application to a multinational study. J Health Econ. Epub 2006 Mar 13
Manca A, Rice N, Sculpher MJ, et al. Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models. Health Econ 2005; 14 (5): 471–485
Manca A, Lambert PC, Sculpher MJ, et al. Cost effectiveness analysis using data from multinational trials: the use of bivariate hierarchical modelling. Med Decis Making. In press
Grieve R, Nixon R, Thompson SG, et al. Using multilevel models for assessing the variability of multinational resource use and cost data. Health Econ 2005; 14 (2): 185–196
Pinto EM, Willan AR, O’Brien BJ. Cost-effectiveness analysis for multinational clinical trials. Stat Med 2005; 24: 1965–1982
Rice N, Leyland A. Multilevel models: applications to health data. J Health Serv Res Policy 1996; 1: 154–164
Berlin JA, Santanna J, Schmid CH, et al. A-LAITS. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med 2002; 21 (3): 371–387
Sculpher MJ, Poole L, Cleland J, et al. Low doses vs. high doses of the angiotensin converting-enzyme inhibitor lisinopril in chronic heart failure: a cost-effectiveness analysis based on the Assessment of Treatment with Lisinopril and Survival (ATLAS) study. Eur J Heart Fail 2000; 2: 447–454
Paker M, Poole-Wilson PA, Armstrong PW, et al. Comparative effects of low and high doses of the angiotensin converting-enzyme inhibitor, lisinopril, on morbidity and mortality in chronic heart failure. Circulation 1999; 100: 2312–2318
Gelman A, Carlin JB, Stern HS, et al. Bayesian data analysis. New York: Chapman & Hall/CRC, 2004
Palmer S, Sculpher M, Philips Z, et al. Management of non-ST-elevation acute coronary syndromes: how cost-effective are glycoprotein IIb/IIIa antagonists in the UK National Health Service? Int J Cardiol 2005; 100: 229–240
Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004; 23 (20): 3105–3124
Sharp SJ, Thompson SG. Analysing the relationship between treatment effect and underlying risk in meta-analysis: comparison and development of approaches. Stat Med 2000; 19: 3251–3274
Sharp SJ, Thompson SG, Altman DG. The relationship between treatment benefit an underlying risk in meta-analysis. BMJ 1996; 313: 735–738
Thompson SG, Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat Med 2002; 21 (11): 1559–1573
Thompson SG, Higgins JP. Treating individuals. 4: can meta-analysis help target interventions at individuals most likely to benefit? Lancet 2005; 365 (9456): 341–346
Higgins JPT, Thompson SG. Controlling the risk of spurious findings from meta-analysis. Stat Med 2004; 23 (11): 1663–1682
Collinson J, Flather MD, Fox KA, et al. Clinical outcomes, risk stratification and practice patterns of unstable angina and myocardial infarction without ST elevation: Prospective Registry of Acute Ischaemic Syndromes in the UK (PRAIS-UK). Eur Heart J 2000; 21: 1450–1457
Brown N, Young T, Gray D, et al. Inpatient deaths from acute myocardial infarction, 1982–1992: analysis of data in the Nottinham Heart Attack Register. BMJ 1997; 315: 159–164
Robinson M, Ginnelly L, Sculpher M, et al. A systematic review update of the clinical effectiveness and cost-effectiveness of glycoprotein IIb/IIIa antagonists. Health Technol Assess 2002; 6 (25): 1–160
Sculpher MJ, Claxton K, Drummond MF, et al. Whither trial-based economic evaluation for health care decision making? Health Econ 2006; 15: 677–687
Ades AE, Claxton K, Sculpher MJ. Evidence synthesis, parameter correlation and probabilistic sensitivity analysis. Health Econ 2006; 15 (4): 373–381
Ades AE, Sculpher MJ, Sutton A, et al. Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics 2006; 24 (1): 1–19
Sculpher MJ, Claxton K, Akerhurst R. It’s just evaluation for decision making: recent developments in, and challenges for, cost-effectiveness research. In: Smith PC, Ginnelly L, Sculpher MJ, editors. Health policy and economics: opportunities and challenges. Maidenhead, Berkshire: Oxford University Press, 2005
Cooper NJ, Abrams KR, Sutton AJ, et al. Use of Bayesian methods for Markov modelling in cost-effectiveness analysis: an application to taxane use in advanced breast cancer. J R Stat Soc [Ser A] 2003; 166 (3): 389–405
Cooper NJ, Sutton AJ, Abrams KR. Decision analytical economic modelling within a Bayesian framework: application to prophylactic antibiotics use for caesarean section. Stat Methods Med Res 2002; 11 (6): 491–512
Cooper NJ, Sutton AJ, Abrams KR, et al. Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach. Health Econ 2004; 13 (3): 203–226
Glenny AM, Altman DG, Song F, et al. Indirect comparisons of competing interventions. Health Technol Assess 2005; 9 (26): 1–148
Lim E, Ali Z, Ali A, et al. Indirect comparison meta-analysis of aspirin therapy after coronary surgery [published erratum appears in BMJ 2004 Jan 17; 328 (7432): 147]. BMJ 2003; 327 (7427): 1309
Song F, Glenny AM, Altman DG. Indirect comparison in evaluating relative efficacy illustrated by antimicrobial prophylaxis in colorectal surgery. Control Clin Trials 2000; 221 (5): 488–497
Caldwell DM, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direst and indirect evidence. BMJ 2005; 331: 897–900
Ades AE. A chain of evidence with mixed comparisons: models for multi-parameter synthesis and consistency of evidence. Stat Med 2003; 22 (19): 2995–3016
Urdahl H, Manca A, Sculpher MJ. Assessing generalisability in model-based economic evaluation studies: a structured review in osteoporosis. Pharmacoeconomics. In press
Acknowledgements
A. Manca is recipient of a Wellcome Trust funded post-doctoral Training Fellowship in Health Services Research (grant number GR071304MA). A.R. Willan is funded through the Discovery Grant Program of the Natural Sciences and Engineering Research Council of Canada (grant number 44868-03). The authors are grateful to Mark Sculpher for his permission to use the GPAs example from the NHS R&D funded project on generalisability, and to Stefano Conti for his help with some of the graphs in this paper. The views and opinions expressed therein are the authors’ and do not necessarily reflect those of the funding institutions.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Manca, A., Willan, A.R. ‘Lost in Translation’. Pharmacoeconomics 24, 1101–1119 (2006). https://doi.org/10.2165/00019053-200624110-00007
Published:
Issue Date:
DOI: https://doi.org/10.2165/00019053-200624110-00007