Economic evaluation guide

The aim of this guide is to introduce economic evaluation concepts and show how they can be used to demonstrate the impact of healthcare interventions. The guide presents ways in which the value of an intervention can be explored and interpreted. Please note that this should not be viewed as a how-to guide to build a health economic model, but an overview of how health economics can be used to illustrate the value of an intervention and what key elements should be considered.

What is health economics?

Health economics is a branch of economics which evaluates issues that relate to effectiveness, efficiency and value as a whole within health and social care. Health economics informs decision-making by prioritising resources to help improve efficiency in the healthcare system and help patients to live longer and healthier lives.

Why is health economics important?

Health economics can be used in evaluations to understand interventions on the basis on their costs and outcomes. It can inform decision-making by qualifying the expected health benefits and costs of an intervention, and the uncertainty around this.

It looks at the interplay of:

To answer questions such as:

• What interventions to commission?
• How to implement them?
• Who should receive them?

This section presents some defining factors to consider when conducting a heath economic evaluation and will help you to understand the scale of the intervention. The scale of the intervention will influence the best approach for the economic evaluation.

Priority

Is the intervention aligned with a local or national priority?

Timeline

What is the timeline for the implementation project?

Population Impact

What proportion of the population will be impacted by the innovation?

Existing data or research

Is there a large volume of existing data or research already available that is relevant to the innovation?

Environmental considerations

Are there additional environmental considerations to intervention?

Population/Region size

How many providers/sites are involved in the evaluation?

Implementation stage

Have previous trials of the intervention been conducted?

Different factors will influence the scale of the project; evaluations including a large population and a large number of participating sites will likely be a large-scale evaluation for example.

Example: Small scale 6-month project

This type of evaluation may be more appropriate for an early-stage intervention. A cost benefit analysis approach may be suitable to understand whether the benefits of the intervention outweigh the costs using a combination of any previous trial data or literature. The analysis will include financial benefits and social benefits.

Example: Large scale 12-month project

This type of evaluation may be suitable for an intervention that has already been trialed at a smaller scale. It should be ensured prior to the roll out that there are sufficient resources to support the modelling. Robust data collection methods should be explored to ascertain what data can be collected from the participating sites and what data will be reliant on literature. A budget impact modelling approach may be more suitable as the results can be used to highlight to commissioners their potential return on investment based on their given demographic and assumptions.

After determining the approach for the evaluation there are several considerations to be made prior to the roll-out to ensure success.

Data collection methods

Large scale evaluations will require more robust data collection methods. This is to ensure that, firstly, reliable baseline data is available to be fully representative of the system without the intervention. Secondly, data collection during the intervention will need to be appropriate and reliable, involving all healthcare staff partaking in the pilot or trial. For smaller scale projects, it may be possible to use literature and academic sources where data is not available and there is not enough time to establish a new data collection method.

Time

Large scale evaluations will require more time prior to ensure successful roll out of the intervention. This will involve engagement with a large number of stakeholders across providers and all clinical teams. During the evaluation, more time will be required to ensure consistency in data collection between different sites.

Budget

Ultimately, ensuring robust data collection methods are established and continued engagement with a large number of stakeholders will require additional time. Not only will the extended timeline of large-scale evaluations require a larger budget, but the additional considerations of a large-scale evaluation will be a key factor and should be considered in budgeting.

Potential risks

Retrospectively capturing data

This may lead to a lack of data points or small sample of data from the evaluation. Additionally, baseline data may not be available and would have to rely on historic data or a forecasting approach. Determining data collection methods significantly before the implementation of an intervention will ensure robust baseline data is captured, and that there is an appropriate data collection method established for during the intervention.

Capturing time savings

Evaluations will commonly look to ascertain the impact of an intervention on staff time and whether efficiencies develop, for example the workforce impact of delivering rehabilitation using a digital platform in place of face-to-face delivery. As such, capturing accurate time savings is essential. There may be difficulties in capturing true time savings on tasks, and will rely on accurate input from staff. Confirming prior to the evaluation whether clinical systems, for example EMIS or Rio, capture staff time will ensure that if current systems do not capture the required data then bespoke time trackers can be developed and distributed to the participating providers.

Generalising results

Evaluations may be focused on a certain pathway or practice with results that may not be applicable to the wider healthcare system. Considering in the initial planning of the evaluation whether the chosen pathway is representative of standard care will help to ascertain whether the findings will be able to be appropriately generalised for additional pathways or practices.

Accounting for uncertainty

There may be uncertainty in evaluations with data collection methods or overinflating benefits and underestimating costs. An optimism bias is applied to health economic analysis to account for any optimistic estimates. This adjustment reduces the benefits and increases the costs compared to figures achieved through calculations on raw data. The more reliable the data is, the lower the optimism bias. For example, costings based on local figures will carry a lower optimism bias that costings based on national figures.

Top tips

• Early engagement with all stakeholders
• Early identification of data sources and whether data sharing agreements are required
• Choice of evaluator (External/In house)
  1. In house evaluators will have local knowledge of the evaluation and the practices involved
  2. External evaluators will ensure impartiality to the evaluation, and bring expertise in methodology and establishing data collection methods/data sharing agreements
• Ensure the overall methodological approach is appropriate
• Identify all the important health effects and resource costs and include in the modelling
• Acknowledge the limitations of the modelling and the approach taken and ensure the results of the analysis are interpreted appropriately.

Further advice & guidance

Visit the evaluation toolkit planning page for further advice and guidance on planning your evaluation.

Benefits

In order to transform the intended outcomes of a service, intervention or project into economic, health or social impacts, each outcome needs to first be identified as a benefit, and then monetised, where possible. There are a number of different types of benefits as described below:

NHS cash releasing benefits

Benefits that provide immediate cashable savings to a provider. Examples may be:

• Reduction in medical equipment purchases
• Decommissioning of services

NHS non-cash releasing benefits

Benefits that help to reduce the demand and strain on services but for which a fiscal value cannot be realised without the decommissioning of services. Examples may be:

• Reduction in appointments
• Saving clinician time

Social Quality-Adjusted Life Years

Benefits that relate to quality of life. They use a Quality Adjusted Life Year (QALY) calculation, which provides a valuation for a particular health state by calculating the number of years of life spent in that state, multiplied by a health state utility-based weighting. Examples of benefits relating to QALYs may be:

• Improvement in the quality of life of a patient, owing to reduced severity of a stroke
• Improvement in the quality of life of a patient from avoiding an amputation

Social Non-Quality-Adjusted Life Years

Benefits that relate to the overall benefit to the public including, but not limited to, improved health and wellbeing. Examples may be:

• Reduction in patient travel costs

Environmental

Benefits relating to environmental impact of an intervention. Examples may be:

• Reduction in carbon emissions
• Reduction in packaging
• Reduction in toxic disposal

Costs

All incurred costs as a direct result should be considered as well as potential indirect impacts on the healthcare system. Costs of the project team, implementation, equipment, training as well as retraining or technical faults should be understood when conducting a health economic evaluation.

Baseline or comparator

When conducting a health economic evaluation, the solution or intervention should be assessed against a comparator. This can be the service provided before the intervention or the service running in parallel without the solution (current standard of care). Collection of comparator data is essential prior to the start of the implementation to establish the counterfactual.

Further advice & guidance

Visit the Evaluation Toolkit page for collecting and analysing data for further advice and guidance

There are a number of different health economic models with examples described below.

Cost Benefit Analysis

A cost-benefit analysis (CBA) aims to determine whether the economic value of an intervention can justify its costs, by comparing the costs of two or more alternatives and reviewing the return on investment. For example, all the benefits of an intervention that reduces the complications of a stroke.

Cost Utility Analysis

A cost utility analysis (CUA) aims to determine cost in terms of utilities, to say in quantity and quality of life. The incremental cost of a programme from a particular point of view is compared to the incremental health improvement expressed in the unit of quality adjusted life years (QALYs). For example, the cost of cancer treatment for an increase in life expectancy of 1 year.

Budget Impact Analysis (or Model)

A budget impact analysis (BIM) aims to estimate the change in expenditure to a specific budget holder and primarily assesses financial affordability of an intervention. For example, the costs to a provider of online consultations in place of face-to-face.

Cost Effectiveness Analysis

A cost effectiveness analysis (CEA) aims to examine the costs of various approaches to achieving a specific health outcome. The analysis measures outcomes in ‘natural units’. For example, the cost of an intervention for diabetes per kg lost.