Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements

Abstract

Delphi is a scientific method to organize and structure an expert discussion aiming to generate insights on controversial topics with limited information. The technique has seen a rise in publication frequency in various disciplines, especially over the past decades. In April 2021, the term Delphi method yielded 28,200 search hits in Google Scholar for the past five years alone. Given the increasing level of uncertainty caused by rapid technological and social change around the globe, collective expert opinions and assessments are likely to gain even more importance. Therefore, the paper at hand presents technical recommendations derived from a Delphi study that was conducted amid the outbreak of the COVID-19 pandemic in 2020.•The paper comprehensively demonstrates how to prepare, conduct, and analyze a Delphi study. In this regard, it combines several methodological advancements of the recent past (e.g., dissent analyses, scenario analyses) with state-of-the-art impulses from other disciplines like strategic management (e.g., fuzzy clustering), psychology (e.g., sentiment analyses), or clinical trials (e.g., consensus measurement).•By offering insights on the variety of possibilities to exploit Delphi-based data, we aim to support researchers across all disciplines in conducting Delphi studies and potentially expand and improve the method's field of application.

Keywords: Clinical trials; Consensus method; Cross-impact analysis; Decision-making; Delphi method; Expert opinion; Foresight; Judgmental forecasting; Scenario analysis; Sentiment analysis.

Graphical abstract

Fig. 1

Three Phases of Delphi-based Research

Fig. 2

Goals and Time Estimates for Delphi Phases

Fig. 3

Visualization of Relationship between Confidence and Expected Probability. Note. This mosaic plot is based on the data of Beiderbeck et al. . It shows the relationship between confidence (measured with a five-point Likert scale from 1 = very low to 5 = very high) and expected probability (0 = statement will never occur; 1 = statement will occur long-term; 2 = statement will occur short-term). Size of the respective mosaic represents number of participants with respective confidence and expected probability assessment.