Assess tools - questions to ask before you start an evaluation

Do I need to do an evaluation?

Evaluation requires an investment of time, money and resources. It’s important to assess whether an evaluation is of value before making this investment.

The following questions will help you to start to consider whether you need to and can do an evaluation, we recommend that you consider these in partnership with your stakeholders. You may not be able to answer them all straight away but useful to keep in mind when planning your evaluation.

An Evaluability Assessment may be useful in determining whether the service is ready to be evaluated and what type of evaluation is required.

Question	Yes	No	?
Do I know what I want to evaluate and why? Consider drivers and audience
Do I already know the answer to my evaluation question? Consider evidence
Will I be able to use the results of the evaluation? Consider context Will these evaluation results provide me with useful information that will be considered “value for money”? Do benefits of conducting the evaluation out-weight costs and consequences.
Do I have the resources and/or skills available to undertake this evaluation? Consider how
Will the information I collect be able to answer my evaluation question (the evaluations aims and objectives)? Consider quality and accessibility of data
Is it the right time to conduct an evaluation of the service? Consider: • Stage of development and complexity of the service • Timescales of the service and the final evaluation report • Context the service is operating in
Are there other options to doing an evaluation? Consider: • Are current project processes such as monitoring enough? • Should I consider other options such as clinical audit, quality improvement or research?

Resources

It is important to consider what resources are required for the evaluation from the outset. This may need to be reviewed throughout the planning process. The resource requirements will be informed by the type of evaluation you need(and quite often impact on the design of your evaluation. This is an important step and should be considered early when planning and developing your intervention – service, innovation or other change.

Plan your evaluation early, alongside using your new service or innovation. This will enable you to ensure that you use the most appropriate methodology, collect the right data, allocate adequate resources and set appropriate timescales.

Who is responsible for overseeing the delivery of the evaluation?

There will usually be an identified lead for the evaluation who may be a project manager or the service manager, healthcare commissioner or innovator. You will also want to consider your reporting (governance) arrangements for the evaluation, this may be an established project group or committee.

Who should be involved in the evaluation? Who can help you deliver this evaluation?

Who are your key stakeholders? Are there local experts that can help? Consider involving citizens as potential users or discuss with your public and patient involvement (PPI) lead if you have one – service users and their families are often well placed to help evaluate service changes.

What resources are available to support the evaluation? Include financial resources.

This could be money and/or staff time depending on the evaluation. If you are conducting an in-house evaluation then you will need input from staff. If you need to commission an external evaluation then you will need to also identify a budget (approximately 5% of the project value). Utilise existing data and project resources where possible and appropriate.

Am I doing an evaluation or research?

Once you have identified the need to do an evaluation and planned your approach it is important that you check that you are doing a service evaluation and not research. To do this we recommend you use the Health Research Authorities tool to help you to assess this – it asks 3 questions. Alternatively contact local experts such as your research and development department.

Keep a record of the result as a part of your evaluation plan as this acts as an audit trail. It’s also a good idea to check with your own organisation to ensure that you comply with local arrangements when it comes to the governance and ethics of your evaluation.

Ethics and governance

If you are doing a service evaluation then you will need to comply with your local and/or organisational ethics and governance processes. These may vary and so it is important to ensure that you obtain the right advice and seek the right permissions. If you are unsure check with your local experts – a Research and Development lead, Clinical Governance lead, an Evaluation lead or a University Ethics department.

In contrast to research, evaluations do not need any formal ethical approval from an NHS Ethics Committee. However every evaluation should follow best practice when it comes to ethics, ensuring that adequate safeguards are put in place and that the benefits of conducting an evaluation outweigh the risks. This will involve reviewing your evaluation during its design, delivery and dissemination to ensure any ethical issues are identified and actions are put in place to address them. This will include ensuring appropriate informed consent is obtained from participants, confidentiality and anonymity are maintained and the study complies with your organisation‘s information governance and data protection policies and procedures. Particular consideration needs to made for any participants that may be considered vulnerable such as children and people who lack capacity.

Engage with your local research and information governance experts.

Case studies

Find inspiration for your own evaluation with these real life examples

Guidance

Guidance from a range of organisations for in-depth advice

Services and support

Knowledgeable organisations who may be able to help you

Training resources

Want to learn more? Our training resources are a good place to start