Article Text
PDF
Statistics from Altmetric.com
Introduction
Whenever developing training competencies, tools to support clinical practice or a response to a professional issue, seeking the opinion of experts is a common approach. By working to identify a consensus position, researchers can report findings on a specific question (or set of questions) that are based on the knowledge and experience of experts in their field.
However, there are challenges to this approach. For example, what should be done when consensus cannot be reached? How can experts be engaged in a way that allows them to consider objectively the views of others and—where appropriate—change their own opinions in response? One approach that attempts to provide a clear method for gathering expert opinion is the Delphi technique.
The Delphi technique was first developed in the 1950s by Norman Dalkey and Olaf Helmer in an attempt to gain reliable expert consensus. Specifically, they developed an approach—named after the Ancient Greek Oracle of Delphi, who could predict the future—which promoted anonymity and avoided direct confrontation between experts, so that the methods employed “…appear to be more conducive to independent thought on the part of the experts and to aid them in the gradual formation of a considered opinion”.1 Though the original Delphi study was linked to the defence industry, the technique has spread to other research areas, including nursing.2
Characteristics of Delphi studies
As with all research methods, the Delphi technique has evolved since it was first reported on in the 1960s. However, many of the fundamental characteristics of the approach still remain from Dalkey and Helmer’s original outline. First, the overarching approach is based on a series of ‘rounds’, where a set of experts are asked their opinions on a particular issue. The questions for each round are based in part of the findings of the previous one, allowing the study to evolve over time in response to earlier findings.
Second, participants are able to see the results of previous rounds—including their own responses—allowing them to reflect on the views of others and reposition their own opinions accordingly.2 This also gives them the opportunity to consider and feedback on what they perceive to be the strengths and weaknesses of other’s responses. Finally, the findings of each round are always shared with the broader group anonymously. This avoids any bias that might result from participants being concerned about their own views being viewed negatively or from their own opinions being biased by personal factors. This framework of expert opinion rounds, with each round built on previous findings and each allowing for responses to be reconsidered by participants, is designed to allow the development of a consensus view that answers the research question.
Within this broad approach, there can be variation in areas such as how many rounds there are, how the questions are delivered and responses collected, and how ‘consensus’ is judged. For example, a study of human factors that contributed to nursing errors used only two rounds. The first took the form of an online survey asking 25 experts to list all the ‘human’ causes of nursing errors that they could. Analysis of responses resulted in a list of 28 potential reasons—this list was sent back to the same group of experts for the second round, asking them to score each one for importance. Analysis of this scoring then allowed for consensus conclusions on the top 10 human factors that contributed to nursing errors (with fatigue, heavy workload and communication problems the top three).3
In another example, nurse practitioners (NPs) were recruited to participate in a Delphi study to achieve consensus related to NP advance care planning competencies. In round 1, draft competencies were developed from the findings of a survey of NP beliefs, knowledge and level of implementation of advance care planning. Round 2 included engagement with 29 NPs who evaluated the draft competencies and their components. Revisions were made based on the original feedback, and a third round was conducted where 15 of the original NP participants confirmed their consensus with the final document. The final document includes four competencies, each with several elements: Clinical Practice, Consultation and Communication, Advocacy and Therapeutic Management.4
Strengths and weaknesses of Delphi studies
The Delphi technique offers a flexible approach to gathering the views of experts on an area of interest. The ability for participants to reconsider their views in light of the contribution of others allows for an element of reflection that is missing from studies based on single interviews or focus groups. The anonymity among the expert groups that underpins Delphi studies promotes honesty among participants and reduces the risk of the ‘halo effect’ where views from dominant or high-profile members of the group are given extra credence.5
However, Delphi studies can—by their very nature—be complex and time consuming. The need for participants to complete multiple rounds can lead to high drop-out rates which impacts on validity of the study. The ability of participants to amend or alter their views at each round is also something of a double-edged sword. It provides those taking part with the opportunity to reflect and reconsider their position in response to additional information, which is an important part of nursing practice. Conversely though, there is a danger that this flexibility introduces bias, with participants altering their response to comply with what they view to be the majority view (sometime called the ‘bandwagon effect’).5
Delphi studies can be criticised due to a lack of clarity on what is meant by ‘consensus’. Even with the level of flexibility and reflexivity present in Delphi studies, it is still unlikely that a group of experts will demonstrate 100% agreement on issues. However, because consensus is a requirement of a Delphi study, there does need to be a judgement on when this point is reached. This is where there is inconsistency across studies and authors, with the suggested level of consensus ranging from 51% to 100%.2 In addition, it has been identified that in some areas, consensus is not predefined as part of the study method. For example, a review of Delphi studies in nurse education found that fewer than half of the papers appraised included a predefined level at which consensus was judged to have been achieved.6 In addition, the identification of an objective level consensus is only possible when gathering quantifiable data—the judgement on consensus being reached in some qualitative Delphi studies will always be rather more subjective on the part of the researcher, and therefore potentially open to bias.
By their nature, Delphi studies often rely purely on expert opinion to generate findings. A further limitation is therefore related to the quality of evidence, with expert opinion viewed as providing a poor basis for making judgements on healthcare interventions.7 This does not mean that the findings of Delphi studies are intrinsically unreliable or invalid; it does mean that researchers should consider whether their research question is one that can be answered through expert consensus or whether other approaches (such as a systematic review of research evidence) are more appropriate.
Conclusion
The Delphi technique is a well-established approach to answering a research question through the identification of a consensus view across subject experts. It allows for reflection among participants, who are able to nuance and reconsider their opinion based on the anonymised opinions of others. However, researchers must take steps to enhance robustness of the studies. It is important to try and prevent participants from simply resorting to agreeing with the majority view; studies must also predefine what is meant by ‘consensus’ and how it will be established.
With careful and clear design though, Delphi studies can make a valuable contribution to the nursing evidence base by tapping into the profession’s most precious resource—the knowledge and expertise of its practitioners.
Footnotes
Twitter @barrett1972, @robertaheale
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Commissioned; internally peer reviewed.