3Plan

Steps

To develop your evaluation plan you will need to identify the:

1. Purpose of the evaluation
2. Aims and objectives
3. Approach you are taking and data requirements
4. Resource requirements

What is the purpose of the evaluation?

It’s important to be clear right at the beginning:

• Why are we conducting an evaluation?
• Who is it for?

These are really important questions to get right as they will inform the design of your evaluation. We recommend that you work in partnership with your stakeholders to clearly identify and agree the purpose of the evaluation, along with its aims and objectives. Working in partnership with your stakeholders allows you to explore the different views, values and priorities of each stakeholder and what they would like to gain from the evaluation.

What are aims and objectives?

The aim will be the overall question you are trying to answer through the evaluation. The objectives are specific statements of what you need to find out in order to answer the overall aim. We recommend that your objectives should be SMART, this means that they are specific, measurable, achievable, relevant and time bound.

This will help keep your evaluation realistic, achievable and easy to communicate. Your aim and objectives for the evaluation will guide the approach you take, what information you need to collect and the tools to do this, so it’s important to get them right.

What approach should I take?

To help with this aspect we recommend a focus on the following questions:

• What information (data) do I need to help me to answer my evaluation’s aims and objectives?
• What has happened before and where are we now? Baseline data
• What information do I already have? Routine data, secondary data (eg hospital admissions, waiting times)
• What are the gaps and what information (data) do I need to collect to fill them? Primary data (eg survey of patients)
• Do we need to compare with another group or standard? Control/comparator/benchmarking
• Do we need information for an economic evaluation? Financial data on expected health benefits and costs and benefits

Consider using a mixture of methods, collecting both numerical (quantitative) and narrative (qualitative) data, as this will strengthen your evaluation and help you to triangulate your findings. This might mean using existing data (routine or secondary data) or it may be necessary to collect new data (primary data). This might also include using multiple methods – can you use clinical audit data for example?

There are lots of frameworks and approaches to evaluation which can be confusing. We have chosen a simple approach but there are other approaches or designs depending on the type and scale of the evaluation. For instance, it may be appropriate to use health economics in your evaluation to determine which interventions deliver value for money for the healthcare system. Check out the tools in the toolbox if you would like more information on evaluation approaches and design.

For further advice and guidance on planning an economic evaluation visit the economic evaluation guide.

What resources do I need?

It’s important that all the resources needed to undertake the evaluation are identified and estimated at the outset and costed into your business cases and plans. It’s very easy to underestimate the cost of an evaluation and there is very little specific external funding available for conducting service evaluations.

The level of resources you need will depend on the purpose of the evaluation. For example, if the purpose of your evaluation is to improve your services or product then an evaluation conducted internally (self-evaluation) using existing project resources would probably be sufficient. If you want to demonstrate your service or product has had an impact or is cost-effective you may need to consider an independent evaluation using more robust approaches.

If you’re going to budget for some externally funded support, think about what you need that support for. This might be for the whole evaluation or just for specific elements such as collecting some of the data or for analysing data. The cost for external support will depend on the organisation that’s providing the resources. For example, universities will charge different rates to independent organisations.

Creating a risk register

It is a good idea, and good practice, for larger scale evaluations to keep a list of risks that may affect the findings or timescales of your evaluation. It can be helpful to share this with the project sponsor or funder, particularly if their involvement in mitigating against these risks will improve the accuracy, timeliness or value of the final evaluation report. A risk register is a live document and needs to be reviewed and updated through the life of the evaluation. You can find an example risk register in the toolbox above.

Local experts

• Research, development or evaluation teams. If you work for BNSSG ICB the Clinical Effectiveness team can provide advice and guidance
• Your local Quality Improvement leads
• NIHR ARC West
• The Health Innovation West of England
• Your local universities