Quick guide to evaluation approaches

Which type of evaluation approach should I use?

There are a number of different types of evaluation, the most widely used are formative, process and summative (outcome) evaluation.  The following information provides you with a guide to the types pf evaluation, when they are used and the types of approaches and data collection methods that are associated with them. (Roberts, Cavill and Rutter 2009)

Formative

• A type of process evaluation that assesses how the intervention / programme can be improved as it is being implemented
• Takes place before the project starts and throughout pilot phase to inform implementation.
• Shows what worked well and why, and identify challenges to capture lessons learned (continually improving)
• Utilises evaluation theory e.g. Theory-of-Change; Realist Evaluation, Experience-Based Design, Critical Path etc.
• Helps to assess needs; clarify theory-of-change; to identify areas for shared learning and improvement
• Qualitative methods: Literature Review; Interviews; Focus Groups; Observation; Diaries; Questionnaires
• Service Improvement methods: Stakeholder Analysis; Logic Modelling; PDSA Cycles; Rapid Evaluation Cycles

Process

• Assesses implementation and delivery of the initiative, and whether this was carried out as planned
• Takes place throughout the project implementation and delivery phase.
• Demonstrates to stakeholders that project objectives are being met
• Qualitative methods: Interviews; Focus Groups; Questionnaires
• Quantitative methods: Metrics, Benchmarking, quantifiable Performance, Financial, Service Usage data
• Service Improvement methods: PDSA Cycles; Rapid Evaluation Cycles

Summative

• Was the project successful? Has the met its predefined aims and objectives?
• Looks at the outcomes (i.e. measures the changes that have occurred as a result of the programme) and/or impact (i.e. the longer, deeper, systemic changes).
• Demonstrates to stakeholders if project aims were met.
• Qualitative methods: Interviews; Focus Groups; Questionnaires
• Quantitative methods: Metrics, Benchmarking, quantifiable Performance, Financial, Service Usage data
• Can be Experimental (e.g. randomised controlled trial), Quasi-Experimental e.g. comparing before and after or use service Improvement methods: PDSA Cycles; Rapid Evaluation Cycles

Visit the Institute for Healthcare Improvement for more information.