Assess Tools
Resource 1. Do I need to do an evaluation?
Evaluation requires an investment of time, money and resources and so before you begin you need to assess whether you an evaluation is of value before making this investment. The following questions will help you to start to consider whether you need to and can do an evaluation, we recommend that you consider these in partnership with your stakeholders. You may not be able to answer them all straight away but useful to keep in mind when planning your evaluation.
An Evaluability Assessment may be useful in determining whether the service is ready to be evaluated and what type of evaluation is required.
Question | Yes | No | ? |
|---|---|---|---|
Do I know what I want to evaluate and why? Consider drivers and audience | |||
Do I already know the answer to my evaluation question? Consider evidence | |||
Will I be able to use the results of the evaluation? Consider context Will these evaluation results provide me with useful information that will be considered “value for money”? Do benefits of conducting the evaluation out-weight costs and consequences. | |||
Do I have the resources and/or skills available to undertake this evaluation? Consider how | |||
Will the information I collect be able to answer my evaluation question (the evaluations aims and objectives)? Consider quality and accessibility of data | |||
Is it the right time to conduct an evaluation of the service? Consider: •Stage of development and complexity of the service •Timescales of the service and the final evaluation report •Context the service is operating in | |||
Are there other options to doing an evaluation? Consider: •Are current project processes such as monitoring enough? •Should I consider other options such as clinical audit, quality improvement or research? |
Resource 2. Resources
It is important to consider what resources are required for the evaluation from the outset. This may need to be reviewed throughout the planning process. The resource requirements will be informed by the type of evaluation you need(and quite often impact on the design of your evaluation. This is an important step and should be considered early when planning and developing your intervention – service, innovation or other change.
Plan your evaluation early, alongside using your new service or innovation. This will enable you to ensure that you use the most appropriate methodology, collect the right data, allocate adequate resources and set appropriate timescales.
Question | Description |
|---|---|
Who is responsible for overseeing the delivery of the evaluation? | There will usually be an identified lead for the evaluation who may be a project manager or the service manager, healthcare commissioner or innovator. You will also want to consider your reporting (governance) arrangements for the evaluation, this may be an established project group or committee. |
Who should be involved in the evaluation? Who can help you deliver this evaluation? | Who are your key stakeholders? Are there local experts that can help? Consider involving citizens as potential users or discuss with your public and patient involvement (PPI) lead if you have one - service users and their families are often well placed to help evaluate service changes. |
What resources are available to support the evaluation? Include financial resources. | This could be money and/or staff time depending on the evaluation. If you are conducting an in-house evaluation then you will need input from staff. If you need to commission an external evaluation then you will need to also identify a budget (approximately 5% of the project value). Utilise existing data and project resources where possible and appropriate. |
Who will conduct the evaluation? | Evaluations can be conducted in-house, externally by an independent organisation or a mixture of the two. What you decide will depend on the purpose of your evaluation: if it is to learn and improve then an in-house evaluation may be adequate; if it is to demonstrate objective effectiveness or accountability then consider an external evaluation, Your decision will also depend on the level of risk involved to the organisation, the participants and the skills and resources available. |
Resource 3. Am I doing an evaluation or research?
Once you have identified the need to do an evaluation and planned your approach it is important that you check that you are doing a service evaluation and not research. To do this we recommend you use the Health Research Authorities tool to help you to assess this – it asks 3 questions. Alternatively contact local experts such as your research and development department.
Keep a record of the result as a part of your evaluation plan as this acts as an audit trail. It’s also a good idea to check with your own organisation to ensure that you comply with local arrangements when it comes to the governance and ethics of your evaluation.
Ethics and Governance
If you are doing a service evaluation then you will need to comply with your local and/or organisational ethics and governance processes. These may vary and so it is important to ensure that you obtain the right advice and seek the right permissions. If you are unsure check with your local experts – a Research and Development lead, Clinical Governance lead, an Evaluation lead or a University Ethics department.
In contrast to research, evaluations do not need any formal ethical approval from an NHS Ethics Committee. However every evaluation should follow best practice when it comes to ethics, ensuring that adequate safeguards are put in place and that the benefits of conducting an evaluation outweigh the risks. This will involve reviewing your evaluation during its design, delivery and dissemination to ensure any ethical issues are identified and actions are put in place to address them. This will include ensuring appropriate informed consent is obtained from participants, confidentiality and anonymity are maintained and the study complies with your organisation‘s information governance and data protection policies and procedures. Particular consideration needs to made for any participants that may be considered vulnerable such as children and people who lack capacity.
Engage with your local research and information governance experts.