Guide to data analysis

Evaluation is about collecting and analysing data to answer questions.

This section will help you understand what types of data can be collected and how to analyse different types of data.

This guide provides only a handful of examples of data collection and data analysis. There are countless other ways that data can be approached. Try not to be daunted by the prospect of statistics or interviews – more often than not it will be necessary to consult your experts for advice. Data collection and analysis are a very important part of evaluation. The data that you produce will help to inform not only your service or innovation, but also future services, projects and innovations.

The type of data you want to collect will depend on your outcome measures. Data can be categorised as being either quantitative or qualitative.

Qualitative data

Definition: qualitative methods measure human experience and produce rich data in the form of narrative.

Explanation: this type of data can be analysed using a variety of methods, depending on what you are trying to find out.

Examples of methods: unstructured/semi-structured interviews, telephone interviews, observations, document reviews.

Advantages

• Detailed findings
• Small samples
• Captures context

Disadvantages

• Resource-intensive
• Time-consuming to analyse
• Can be subjective

Qualitative data analysis

The most common way to analyse qualitative data in an evaluation is by observer impression. This is where yourself and your team examine the data and interpret it’s meaning. In practice, this often involves listening back to an interview recording as a group and forming an impression of what has been said. Alternatively, thematic analysis offers a more structured approach to analysing qualitative data from an interview. This is an iterative process that loosely follows five stages:

1. transcribe your interviews
2. read transcripts, note down initial codes/interesting features
3. sort similar codes into themes
4. review relevance of codes and themes
5. refine and name each theme (see Braun & Clarke, 2006 paper for more detail).

The BNSSG ICB Clinical Effectiveness Team have produced a quick guide to qualitative analysis using Excel for when qualitative data is collected using an online survey platform.

There are excellent resources available locally, nationally, and internationally that can help you with your evidence and/or evaluation. Below is a selection of the ones we find helpful.

• NHS England bite-size guide to building greater insight through qualitative research
• National Council for Voluntary Organisations (NCVO) advice on analysing qualitative data for evaluation
• Healthwatch guidance on how to analyse qualitative data

Quantitative data

Definition: quantitative methods measure variables that produce numeric outcomes and can be used to infer evidence for a theory.

Explanation: this form of data can be subject to a wide range of robust statistical tests which produce numerical findings to answer a question.

Examples of methods: questionnaires, monitoring data, performance data, structured interviews.

Advantages:

• Quick to gather a lot of data
• Easy to analyse and display
• More scientific and objective

Disadvantages:

• Lacks detail
• Often requires large samples
• Ignores context of data collection

Quantitative data analysis

Descriptive statistics

Once you have collected your data, it will need to be analysed for meaning to be gathered. Descriptive statistics refer to a simple process of counting the number of occurrences of a particular event and describing what is found. This can only be used when you have access to all the information you are interested in, known as your ‘population’. For example, if you collected data on the number of GP appointments made by a particular group of people, you may want to find out the mean per person (average). Data such as this can easily be displayed in a histogram or a pie chart as way to visually present the results.

Inferential statistics

Unlike descriptive statistics which are used when you have access to all of the information you are interested in, inferential statistics are used when you have a sample of participants that are being used to represent a larger population. For example, if you recruited a selection of patients who had undergone therapy to represent all the patients in the group, you would analyse the data by producing inferential statistics. This covers a wide range of tests of varying complexity that can be used to make inferences about the population from which the sample was taken. Luckily, many computer programmes such as excel, or SPSS can run these tests for you – if in doubt consult the data analysts in your team.

There are excellent resources available locally, nationally, and internationally that can help you with your evidence and/or evaluation. Below is a selection of the ones we find helpful.

• NIHR ARC West’s basic statistics and data interpretation online learning resource.
• US Centre for Disease Control’s evaluation brief on Analyzing Quantitative Data for Evaluation