Developing DELPHI expert consensus rules for a digital twin model of acute stroke care in the neuro critical care unit

Abstract

Introduction: Digital twins, a form of artificial intelligence, are virtual representations of the physical world. In the past 20 years, digital twins have been utilized to track wind turbines' operations, monitor spacecraft's status, and even create a model of the Earth for climate research. While digital twins hold much promise for the neurocritical care unit, the question remains on how to best establish the rules that govern these models. This model will expand on our group's existing digital twin model for the treatment of sepsis.

Methods: The authors of this project collaborated to create a Direct Acyclic Graph (DAG) and an initial series of 20 DELPHI statements, each with six accompanying sub-statements that captured the pathophysiology surrounding the management of acute ischemic strokes in the practice of Neurocritical Care (NCC). Agreement from a panel of 18 experts in the field of NCC was collected through a 7-point Likert scale with consensus defined a-priori by ≥ 80% selection of a 6 ("agree") or 7 ("strongly agree"). The endpoint of the study was defined as the completion of three separate rounds of DELPHI consensus. DELPHI statements that had met consensus would not be included in subsequent rounds of DELPHI consensus. The authors refined DELPHI statements that did not reach consensus with the guidance of de-identified expert comments for subsequent rounds of DELPHI. All DELPHI statements that reached consensus by the end of three rounds of DELPHI consensus would go on to be used to inform the construction of the digital twin model.

Results: After the completion of three rounds of DELPHI, 93 (77.5%) statements reached consensus, 11 (9.2%) statements were excluded, and 16 (13.3%) statements did not reach a consensus of the original 120 DELPHI statements.

Conclusion: This descriptive study demonstrates the use of the DELPHI process to generate consensus among experts and establish a set of rules for the development of a digital twin model for use in the neurologic ICU. Compared to associative models of AI, which develop rules based on finding associations in datasets, digital twin AI created by the DELPHI process are easily interpretable models based on a current understanding of underlying physiology.

Keywords: AI; Acute Ischemic Stroke; DELPHI; Digital Twin; Expert Consensus; Neuro Critical Care.

Fig. 1

Directed Acyclic Graph (DAG) providing a visual representation of connections between different concepts and variables. Modifiable variables are represented in green, semi-modifiable variables in yellow, intermediary states in gray, and end states in red. These nodes are connected by unidirectional black arrows depicting the flow of processes from one condition to the subsequent state

Fig. 2

Flowchart providing an overview of the DELPHI consensus process. A foundational Directed Acyclic Graph (DAG) model is first constructed and refined. From this model, DELPHI statements are established, sent to Neurocritical Care (NCC) experts, and further refined before being deemed valid and sent to a programmers to incorporate into the Digital Twin AI Model

Fig. 3

Map of the geographic distribution of Neurocritical Care Experts

Fig. 4

Flow chart of the DELPHI consensus process. After three rounds of DELPHI consensus, 93 statements reached consensus (green), 11 statements were excluded (yellow), and 16 statements did not reach consensus (red)

Fig. 5

Stacked bar graph demonstrating how many rounds of DELPHI were needed to reach consensus by sub-statement type