Article Text
PDF
Statistics from Altmetric.com
What is realist evaluation?
A frequent conundrum faced by nurses when implementing research findings and developing new interventions is that their patients, staff and sociocultural environment may be quite different from those in the original studies. Therefore, implementing and evaluating new interventions may need a carefully thought out and creative approach. One such approach is realist evaluation (RE). This paper will give an overview of RE and how this methodology can be used to support nurses when evaluating the implementation of evidence-based care.
Characteristics of realist evaluation
RE is a research methodology, otherwise known as a system of enquiry, that provides a framework for conducting an evaluation of a healthcare intervention.1 An RE approach enables researchers to develop theories about how their intervention works by asking ‘What works for whom, in what circumstances, in what respects, and how?’ (Pawson and Tilley, p2).2 This contrasts with traditional approaches to evaluations which simply measure and report the results without considering the environment, culture or what the human behaviours and emotional reactions between the intervention and outcome may be. This style of evaluation has been described as a ‘black box’, meaning that the inner workings are hidden from view.3 The RE approach is a ‘clear box’ evaluation where in addition to the outcomes, the processes of how the intervention has worked, in terms of peoples’ actions and reactions, and in which sociocultural context, are reported transparently (Kazi, p204 and Scriven3).
The theories that describe how the interventions work are constructed by recognising that Outcomes are the result of Mechanisms triggered in a specific Context and are written as Context, Mechanism, Outcome configurations (CMOc) (Pawson and Tilley, p60).1 An example of a CMOc is as follows:
Some people in prison are rebellious (context), so will refuse to engage in non-mandated activity (mechanism), so hepatitis C test uptake rates will not improve (outcome).
The process of undertaking RE research involves following a cyclical model (figure 1).
The realist evaluation cycle (Pawson and Tilley, p85).1
Contexts
The contexts in which an evaluation, or indeed any form of research, occurs will certainly include the location. However, Pawson (p212)5 clarifies that context may be a characteristic of any of the following ‘4Is’:
The individuals who participate in the programme under evaluation.
The interrelationships between all stakeholders.
The institution in which the programme is operating.
The wider infrastructure (societal, economic and cultural) of the programme’s setting.
A clear characterisation of the context in which a healthcare intervention is introduced will enable the distinction to be made between the intervention’s efficacy (results obtained in an ideal clinical environment) and effectiveness (results obtained in a real-world environment).6
Mechanisms
Mechanisms have three specific characteristics; they are the interactions and responses of people towards the intervention; they explain the way in which interventions can lead to the observed outcomes; and they are hidden, but still influenced by the context.7 Thus, they are not the components, interventions or any other variable that could be observed and included in a statistical regression model.
Table 1 shows the constructs of five specific types of mechanisms, with examples in different system level groups, namely, psychological, social groups, social institutions and material (Westhorp, p49).8 The examples of mechanisms in table 1 illustrate their highly varied nature and the creative thinking required to characterise them in the CMOc.
Examples of mechanisms at different system levels (Westhorp, p49)8
Outcomes
The outcomes observed are required to be precise and ideally quantitative in order for the context and mechanism to be clearly defined and measurable. A qualitative outcome can be valuable if the differences described are presented in a categorical format (Byrne, p97–98).9
Theory
A programme theory is the assumption(s) made by the intervention, or programme designers, that explains how, why and under what conditions they expect the intervention to work (Marchal et al, p83).10 The first first step is to construct programme theories by articulating the explanatory CMOc.
Hypothesis
Next, the researcher will review the existing literature, discuss the programme theory with other colleagues and stakeholders, and consider their own professional clinical experiences. This will enable them to construct a series of plausible alternative CMOc explanations for how the intervention is expected to work, or potentially not work. These are sometimes referred to as ‘folk theories’ in the RE discipline, but other methodologies may frame these ideas as hypotheses.
Observations
The next stage, the theories, or hypotheses, written as CMOc, are tested. RE uses mixed methods data collection to fully explore the contexts and unseen mechanisms. The research findings are then analysed in conjunction with the CMOc and evidence sought to confirm or reject the theories.
Programme specification
The goal of an RE is to identify a robust and transferable ‘Programme specification’ theory, as opposed to a generalisable result which would be the goal of formal quantitative research. This is achieved by the development of Middle-Range Theories that remain untied from any specific contexts or situations so they can be transferable (Emmel et al, p7).11 It may be necessary to repeat the RE cycle to retest the middle-range, or ‘programme specification’ theories, to confirm the accuracy. This process is explained further in a paper by Jack and Linsley.12
Strengths and weaknesses of RE
RE methodology is advantageous in its recognition that individuals’ actions, reactions and interpretations are likely to affect the outcome of a healthcare intervention and includes these factors when collecting data and examining the results. However, there are limitations of this approach.6 In the evidence-based practice hierarchy, systematic reviews and randomised controlled trials (RCTs) are considered to be the most robust and standardised methodological approaches that can confirm the causes of outcomes due to interventions. Thus, there are concerns that an RE study will not have the same degree of validity, reliability and generalisability that can be expected in an RCT.6
Conclusion
Although RE methodology has not been used widely, this approach is gaining traction among nurses who, following an interaction with a patient or a service, ask themselves ‘what is really going on here, beyond what I can see?’ Nursing, as a discipline, adopts a holistic approach to assessing patients and delivering care. RE is an approach to research with the same intrinsic understanding of the critical importance of including not just physical components of health, but the psychosocial, environmental and contextual domains too. RE methodologies can thus be viewed as the natural partner of research-curious nurses.
Ethics approval
Not applicable.
Footnotes
Twitter @KathrynJackNHS
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Commissioned; internally peer reviewed.