Guide to Data Analysis
Undertaking an evaluation will normally include:
- Collecting data
- Analysing data
The type of data you want to collect will depend on your outcome measures. As a general guide, data can be categorised as being either quantitative or qualitative.
- Definition: quantitative methods measure variables that produce numeric outcomes and can be used to infer evidence for a theory.
- Explanation: this form of data can be subject to a wide range of robust statistical tests which produce numerical findings to answer a question.
- Examples of methods: questionnaires, monitoring data, performance data, structured interviews.
- Quick to gather a lot of data
- Easy to analyse and display
- More scientific and objective
- Lacks detail
- Often requires large samples
- Ignores context of data collection
Analysing Quantitative Data
Once you have collected your data, it will need to be analysed for meaning to be gathered. Descriptive statistics refer to a simple process of counting the number of occurrences of a particular event and describing what is found. This can only be used when you have access to all of the information you are interested in, known as your ‘population’. For example, if you collected data on the number of GP appointments made by a particular group of people, you may want to find out the mean per person (average). Data such as this can easily be displayed in a histogram or a pie chart as way to visually present the results.
Unlike descriptive statistics which are used when you have access to all of the information you are interested in, inferential statistics are used when you have a sample of participants that are being used to represent a larger population. For example, if you recruited a selection of patients who had undergone therapy to represent all of the patients in the group, you would analyse the data by producing inferential statistics. This covers a wide range of tests of varying complexity that can be used to make inferences about the population from which the sample was taken. Luckily, many computer programmes such as excel or SPSS can run these tests for you – if in doubt consult the data analysts in your team.
- Definition: qualitative methods measure human experience and produce rich data in the form of narrative.
- Explanation: this type of data can be analysed using a variety of methods, depending on what you are trying to find out.
- Examples of methods: unstructured/semi-structured interviews, telephone interviews, observations, document reviews.
- Small samples
- Captures context
- Time-consuming to analyse
- Can be subjective
Analysing Qualitative Data
The most common way to analyse qualitative data in an evaluation is by observer impression. This is where yourself and your team examine the data and interpret it’s meaning. In practice, this often involves listening back to an interview recording as a group and forming an impression of what has been said. Alternatively, thematic analysis offers a more structured approach to analysing qualitative data from an interview. This is an iterative process that loosely follows five stages:
1) transcribe your interviews
2) read transcripts, note down initial codes/interesting features
3) sort similar codes into themes
4) review relevance of codes and themes
5) refine and name each theme (see Braun & Clarke, 2006 paper for more detail).
- This guide provides only a handful of examples of data collection and data analysis.
- There are countless other ways that data can be approached.
- Try not to be daunted by the prospect of statistics or interviews – more often that not it will be necessary to consult your experts for advice.
- Data collection and analysis are a very important part of evaluation.
- The data that you produce will help to inform not only your service but also future services and projects.