Economic evaluation of healthcare technologies using primary research
Clinicians and policy makers are often required to make difficult decisions regarding which health technologies they use. Increasingly, government bodies, such as the National Institute of Health and Clinical Excellence (NICE) in the UK, produce national clinical guidelines and technology assessments to direct practice.1 These assessments often consider not only the benefits conferred by alternate treatment choices but also the costs.
It is vital that clinical practitioners have a clear understanding of why economic evaluations are needed, what the results of such evaluations mean, and how these findings can inform allocation decisions. We will begin by introducing key ideas and then illustrate these ideas with examples.
Central government or federally funded services must deliver health care within fixed and limited budgets (see box). Decisions made to spend money in one way will incur an opportunity cost in terms of investment in other health services that are foregone. For example, investment in a new cancer centre may mean that a new spinal rehabilitation centre is not set up. Alternatively, the delivery of an expensive treatment to one patient may mean that 10 other patients cannot be treated with a less expensive intervention.
What is economics?
Economics is the science of allocating scarce or limited resources (ie, labour, land, raw material, or capital). Whatever the type and volume of resource available, society will always have more wants than can be met. Economic decisions aim to maximise benefits for the costs incurred between alternative interventions. However, for every decision made regarding resource use, there is an opportunity cost—the cost of failing to use a resource for the next best alternative.
In general, economic evaluation is the “comparative analysis of alternative courses of action in terms of both their costs and consequences.”2 In other words, an economic evaluation compares 2 or more interventions in terms of the costs (resources) incurred with the resulting benefits (consequences). The aim of such evaluations is to promote understanding about the return on money spent. This informs the decisions of health policy makers about which healthcare interventions to fund.3 Economic evaluations can be conducted in several different ways (Table 1). Of these, the most widely used economic evaluations are cost-effectiveness and cost–utility analyses.
The relevance of an economic evaluation to different decision makers is influenced by the perspective from which it is conducted. If the cost perspective is that of the healthcare provider, all resources incurred by this provider must be measured and included. These resources could include staff time, equipment used, drugs prescribed, length of hospital stay, transport costs, and overheads incurred. If relevant, the perspective can also include resources utilised from wider state care services, such as social care services. In the UK, the NICE guidelines advocate a National Health Service (NHS) and personal social services perspective for costs, as its interest is predominantly in costs to the NHS.5 In other countries, a wider societal perspective is taken, which may include indirect costs that impact on wider society (eg, time away from paid employment as well as direct costs to patients, such as private treatment).
Once the perspective is set, the relevant data can be collected using primary or secondary research. Increasingly, primary research studies such as RCTs capture resource use details from staff and participants at each stage of treatment delivery and follow-up. Resources used can then be converted to costs by applying unit costs (Table 2), giving a cost for each individual and a mean overall cost per trial arm.
In terms of perspectives, common guidance dictates that all health effects should be assessed in an economic evaluation.5 From Table 1, we see that health benefits can be measured in several ways depending on the evaluative approach being used. The more common methods are the use of effectiveness measures (cost-effectiveness analysis) and the use of utilities (cost-utility analysis). Effectiveness measures include natural units, such as life-years, disease-free days, or even surrogate outcomes (eg, cancer biomarkers or reduction in wound area). Although such measures are useful when assessing the cost-effectiveness of 2 or more treatments for a particular disease, they do not allow for comparison between treatments or diseases when the natural units are different.
Economic evaluations aim to help decision makers allocate resources to maximise patient benefit across the whole of society. Thus, the outcome measures most useful for decision makers are those inviting comparisons between conditions and treatments. Mortality is the most generic (and very common) measure, but for many diseases, improvement in morbidity is the main treatment aim. Having measures that capture both quantity and quality of life in a single index is thus desirable.
A common index used in economic evaluation is quality adjusted life-years (QALYs) (Figure 1). This measure adjusts the amount of time lived with a measure of utility (utility weight), which expresses the impact of morbidity. Data required for the calculation of QALYs can be collected from participants in primary research using questionnaires such as the EuroQol-5D (EQ-5D). The EQ-5D (http://www.euroqol.org/) is a widely recognised and validated descriptive system of health-related quality of life.9 It has 5 questions, each relating to a different health dimension: mobility, self-care, ability to undertake usual activity, pain, and anxiety/depression. Each question has 3 possible response levels: no problems, moderate problems, and severe problems. Based on their combined answers to the EQ-5D questionnaire, participants can be classified as being in 1 of 243 possible health states. Each of these health states has an associated utility weight that denotes the impact of this state on health-related quality of life. Thus, perfect health has a weight of 1, and this weight decreases as health becomes impaired. There are 2 gold standard methods of evaluating the impact of quality of lived life in different health states to generate utility weights, and both rely on elicitation of individual’s preferences.10 The first is a hypothetical time trade-off between quality and quantity of life, whereby individuals record the amount of life lived they would sacrifice to live in perfect health. The second is a game in which individuals record the risk of death they would accept in return for restoration of perfect health (standard gamble). The social tariff for the EQ-5D questionnaire was based on a time trade-off exercise in a sample of the UK population.11 Examples of other questionnaires used to generate utility indices are the Health Utilities Index12 and SF-36 (through the SF-6D).13
OUTCOME OF ECONOMIC EVALUATION
The results of cost-effectiveness or cost-utility studies can lead to 1 of 4 scenarios (Figure 2).
The graph shown in Figure 2 is often described using compass points. Of the 4 scenarios, the north west (NW) quadrant of the cost-effectiveness plane implies that a new intervention is dominated (ie, not as good) because its cost is higher and its benefits lower than the comparison treatment. In this case, the decision regarding the new treatment is straightforward: one should not adopt it. The decision is also straightforward when the cost-effectiveness measure is positioned in the south east (SE) quadrant. The new intervention is dominant (ie, better) because it is less costly and more beneficial than the comparator and should therefore be implemented. However, it is more difficult to make a decision for the scenarios positioned in the north east (NE) or south west (SW) quadrants. Here we must evaluate whether the increased cost of the new intervention is worth the increased benefit (NE scenario) or if the reduced benefit conferred by the new treatment is justified by the reduced costs (SW). To do this, a measure combining the 2 components (costs and benefits) is required so that a decision rule can be applied.
The measure commonly used is the incremental cost-effectiveness ratio (ICER), which combines costs and benefits as a ratio of the 2 different mean costs (between the 2 comparison arms) and the 2 different mean effectiveness measures (between the 2 comparison arms). Although an economic evaluation study is not the only information source considered for allocation decisions, proponents of new health technologies tend to compare the ICER value against the (approximate) maximum amount the decision maker is willing to pay for an additional unit of benefit. In the absence of other considerations, a treatment strategy can be considered to be cost-effective only if the decision maker’s willingness to pay for an additional QALY (or other effectiveness measure) is greater than (or equal to) the ICER. Willingness to pay thresholds are not defined explicitly, and the methods for determining such thresholds are the subject of much discussion.14 15
Additionally, it is important to recognise that uncertainty exists around the cost-effectiveness/utility estimates (ie, ICER). That is, in the evaluation, we have calculated point estimates, and we are uncertain how well these estimates reflect the true value since they were taken from only a sample of the population of interest. In clinical research, uncertainty is usually presented using confidence intervals, but in economic evaluation, the cost-effectiveness acceptability curve (CEAC) is the approach commonly used. The CEAC expresses the likelihood that the cost-effectiveness estimate reflects a cost-effective intervention, based on the existing evidence.16 For every value of willingness to pay threshold, the CEAC summarises the evidence in favour of the intervention being cost-effective. The CEAC is usually built by simulating several potentially real scenarios (based on the data) and producing an estimate of the ICER for each of these scenarios. We can then evaluate the proportion of scenarios that identify the intervention as cost-effective in relation to the comparator, over different willingness to pay values. The CEAC is equivalent to other measures of uncertainty and assumes no distributional shape for the ICER distribution (ie, is a non-parametric approach).
Example 1: Cost-effectiveness study
The prevention of pressure ulcers is often the responsibility of nurses, who must make decisions about which pressure-relieving surfaces to use. Such decisions should be informed by clinical and cost-effectiveness information where possible. In terms of cost-effectiveness, a recent RCT compared alternating pressure mattresses with alternating pressure overlays from the perspective of the NHS.17 Trial participants were people > 55 years of age who were admitted to hospital without a pressure ulcer. Patients were randomised to receive either a mattress or overlay and assessed to see whether a pressure ulcer (grade 2 or more) developed. Patients were followed up for a maximum of 60 days. Clinically, the mattress and overlay groups did not differ in the proportion of patients who developed pressure ulcers.19 In terms of cost-effectiveness, the main resource use of interest was length of hospital stay and the costs of the mattresses and overlays, which were all measured during the trial. Benefit was assessed in natural units—time to development of a pressure ulcer. Mean cost and mean number of ulcer-free days were calculated for each group (Table 3a).
In this example, the mean cost of treating a patient with the mattress was £283.60 less than treating a patient with the overlay, whereas patients allocated to mattresses had, on average, 10.6 more ulcer free-days than a those allocated to overlays. The point estimate of the relative cost-effectiveness of the mattress falls into the SE corner of Figure 1. The mattress dominates the overlay, which appears to be more expensive and less effective. In this case, the decision is easy, and estimation of the ICER is superfluous.
Figure 3a is a CEAC for this analysis. The point estimate suggests that the use of mattresses covers is cost-effective, and the uncertainty around this suggests that we can be 80–90% sure that mattresses are cost-effective for any willingness to pay value between 0 and £30 000 per pressure-ulcer-free day. Assuming that the decision maker would be willing to pay £30 000 per pressure-ulcer-free day, then the economic evaluation study based on the PRESSURE trial suggests the adoption of mattresses over overlays as a more cost-effective intervention in the prevention of pressure ulcers.
Example 2: Cost-utility analysis
Musculoskeletal back and neck pain are common conditions that are expensive to treat. Physiotherapy is a common treatment for these musculoskeletal conditions, although several different approaches can be used. A recent RCT compared the clinical and cost-effectiveness of a new brief physiotherapy pain management approach using cognitive–behavioural principles (Solution-Finding approach) with a commonly used traditional method of physical therapy (McKenzie approach) in participants with back or neck pain.20 The cost-utility analysis18 was conducted from an NHS perspective. Costs were measured in UK pounds sterling. The main resource use of interest was number of physiotherapy sessions and general practitioner and nurse visits; these were measured during the trial. Benefit was measured as health-related quality of life using the EQ-5D, which allows the estimation of patient-specific QALYs. Patients were followed up for 12 months.
We can see that patients using the Solution-Finding approach incurred, on average, fewer costs than those using the McKenzie approach; nevertheless, Solution Finding also led to reduced benefits for patients. This falls into the SW corner of Figure 1, and a decision rule is required. In the absence of other data, decision makers need to decide whether to save healthcare resources, adopting the less expensive treatment (Solution Finding), with the disadvantage of losing health benefits. Relating the incremental mean costs and incremental mean QALYs gives an ICER of £1220 (−24.4/−0.020) (Table 3b).
We can investigate the uncertainty around this decision as before. Figure 3b shows the CEAC for this analysis. We can see that if the willingness to pay (threshold value) per QALY was 0 (ie, the decision maker had no budget to pay for health), Solution Finding would be cost-effective as it offers a cost-saving treatment. However, as soon as willingness to pay per QALY increases, the probability that the Solution-Finding approach is cost-effective drops markedly. This is because the cost of the MacKenzie approach is deemed to be worthwhile in terms of the benefit obtained even though it is the more expensive treatment.
Finally, like all research, economic evaluations can be of variable quality, and so the onus is always on the reader to make a considered judgment of the validity of the findings. Good critical appraisal ensures not only the consistency of the methodology used in an economic evaluation study but also that robust data are applied as evidence for practice. The critical appraisal of economic evaluations will be discussed in a future Notebook.