What is meant by process evaluation?

Process evaluations studies may run alongside, or after, an RCT.2 3 A helpful definition is ‘a study which aims to understand the functioning of an intervention, by examining implementation, mechanisms of impact and contextual factors’.4 There are many examples of process evaluation in the literature, however, the studies are not always labelled as a process evaluation which may make them more difficult to identify.3 5 6

What areas of process are studied?

The processes that are of interest in any given study will vary depending on the phase of the research. For example, is the research developmental, evaluative or is its main focus implementation. The processes may be related to the trial itself (such as recruitment), the intervention that is being tested (such as whether recipients of the intervention find it acceptable) or the systems/contexts within which the intervention is delivered (such as how people work together to deliver the intervention).

Skivington et al 1 propose six core elements that should be considered at all stages of complex intervention development, testing and implementation. These are: context, development and refining of the programme theory, engaging stakeholders, identifying key uncertainties, refining the intervention and economic considerations. Considering which of these are the process evaluation foci will be helpful in structuring and justifying process evaluation design considerations. Another useful resource when considering process evaluation design is the MRC guidance.2 4 They propose three key functions of a process evaluation: context, implementation and mechanisms of impact, with the investigation of these functions informed by transparent intervention description and which informs how the trial outcomes are understood.

The place of context in understanding RCT processes and outcomes is important for onward implementation.1 2 7 Four parts of context (personal, organisational, trial and problem context) need to be understood to have clarity about how the intervention works, or not, and to enable assessment of internal and external validity.7 Moore et al 2 4 suggest context is explored in terms of how it may influence intervention implementation and links to outcomes, for example, understanding intervention mechanisms and factors that may influence implementation, such as the physical surroundings of where an intervention is delivered or the varied people who deliver it. As an example, one contextual feature that we found to influence adherence in one of our studies (the OPAL trial [Optimal PFMT for Adherence Long-term]) was the lack of time women had for themselves which made it difficult for them to adhere to their exercise programme.8

Implementation is about understanding what is delivered and how.2 A key focus of understanding implementation is learning about how the new evidence can be translated into practice.3 9 Moore et al 2 suggest that four features are considered to understand what is delivered:

• Fidelity (is the intervention delivered as planned).

• Dose (how much is delivered).

• Adaptations (eg, what is changed to support delivery in a particular context).

• Reach (does the delivery reach the intended recipients).

For clinical practice, adaptations might be considered an important process to research to enable understanding of changes made to an intervention to achieve person-centred care.

The programme theory, which includes the mechanisms of impact, hypothesises how an intervention leads to its outcomes and under what circumstances.1 2 In our process evaluations, we have developed the programme theories with data from the process evaluation. We have started the study with an often quite simplistic mechanism of action and built up to a more detailed programme theory which includes features of context learnt from the process evaluation.8

Even from this brief description, it is possible to see that process evaluations are variable in what they do, and therefore, also how they do it.

How to do a process evaluation?

How a process evaluation is undertaken will vary depending on the stage of the RCT the evaluation is targeting, the processes that are deemed most relevant and financial or pragmatic constraints.2 3 9

A decade or so ago process evaluations were criticised for mainly focusing on qualitative methods3 ; however, there has been a shift to mixed-methods designs often with a strong qualitative component.1 2 Many different research designs are used. Two that are used more commonly are Realist Evaluation and Case Study Design. Realist Evaluation10 is naturally appealing given the focus on the interaction between context, mechanisms and outcomes which is often an important part of a process evaluation.11 Case Study Design12 has a focus on understanding a problem in-depth and from multiple perspectives and hence also has value in a process focused design.13

Thus, there is no simple formula to follow to support design considerations in a process evaluation and researchers have to think about, and justify, which methods are needed to answer their process focused questions for their particular study.

Theory and process evaluation

Some argue that theory is not necessarily required for a process evaluation3 and others criticise them for not using theory.5 Given contemporary emphasis on understanding mechanisms that link the intervention with the outcomes,1 using theory would seem like a sensible choice within a process evaluation.

To date, many different theories have been used within process evaluations9; given the varied nature of the interventions tested, this is unsurprising. Theory may be drawn from methodological guidance (such as the MRC complex intervention guidance,1 from psychology (eg, behaviour change theory)14 or sociology (eg, normalisation process theory).15 As an example, our TOPSY study (Treatment Of Prolapse with Self-care pessarY)16 contained an RCT plus a concurrent process evaluation and focused on a self-management intervention for women who used a pessary to treat pelvic organ prolapse and compared it to clinic-based care. The study was about self-management and the hence the theory underpinning self-management formed the basis of the intervention.17 Research in other domains had suggested that improvement in self-efficacy18 was key to improving quality of life outcomes in self-management interventions, and therefore self-efficacy was part of the posited programme theory. The study was based on implementation in real-world clinical practice so it also drew on elements of normalisation process theory15 to support actions that may encourage implementation in practice. Thus, a combination of theories were brought together in our study.19

What is the benefit for understanding nursing care and practice?

Process evaluations are valued for adding to wider knowledge, for informing implementation in practice, for identifying intervention improvements and concerns and for providing reasons for the results of the trial.6 All of these features are valuable additions to the evidence-base for nursing. For nursing care to be person-centred, interventions need to be adaptable to a person and the context of their lives. Process evaluation supports understanding of these key features of complex interventions and allows consideration of how an intervention delivered in one location may be adaptable to another location through that understanding of context and adaptability. There are increasing numbers of examples of process evaluations linked to nursing relevant RCTs20 21 and these demonstrate the additional value that is offered over and above understanding whether or not an intervention is effective.