Intricacies of human-AI interaction in dynamic decision-making for precision oncology

Abstract

AI decision support systems can assist clinicians in planning adaptive treatment strategies that can dynamically react to individuals' cancer progression for effective personalized care. However, AI's imperfections can lead to suboptimal therapeutics if clinicians over or under rely on AI. To investigate such collaborative decision-making process, we conducted a Human-AI interaction study on response-adaptive radiotherapy for non-small cell lung cancer and hepatocellular carcinoma. We investigated two levels of collaborative behavior: model-agnostic and model-specific; and found that Human-AI interaction is multifactorial and depends on the complex interrelationship between prior knowledge and preferences, patient's state, disease site, treatment modality, model transparency, and AI's learned behavior and biases. In summary, some clinicians may disregard AI recommendations due to skepticism; others will critically analyze AI recommendations on a case-by-case basis; clinicians will adjust their decisions if they find AI recommendations beneficial to patients; and clinician will disregard AI recommendations if deemed harmful or suboptimal and seek alternatives.

Fig. 1. ARCliDS evaluation module workflow.

The modules were deployed on cloud in shinyapps.io server and used google sheets as data storage. Welcome Page contains links to tutorial video hosted in YouTube and manuscript preprint; from which a new evaluator can create a new account and returning user can login back to complete their evaluation. Create Account page consists of a series of input prompts including a unique username and 8-digit PIN so that they could log back in if disconnected unexpectedly or if needed to step away. Data Frame hosted as a google sheet automatically saves login info and evaluation input. Additionally, it saves randomized list of patients, and the last patient evaluated for each user account to check if evaluation is completed. Unassisted Phase page presents patient’s relevant info, treatment plans including 3D dose distribution and structure, cumulative DVH, pre and mid treatment 3D PET scans for NSCLC, pre and mid treatment liver functions along with pre-treatment 3D MRI scans for HCC; and contains input prompts for dose decision, decision confidence, and a textbox for remark. AI-Assisted Phase page presents AI-recommendation, outcome estimation for a range of dose show in TCP vs NTCP outcome space, and Distribution plots for all the feature variables; and contains input prompts for dose decision and decision confidence, a series of multiple-choice questions to access the user trust level, and a textbox for remark. Exit Page marks the end of the evaluation.

Fig. 2. Unassisted vs AI-assisted decision analysis grouped by evaluators and patient number.

Plots a–d summarizes data from NSCLC and plots e–h from HCC. Density plots a, b, e, and f compare the unassisted ($un$) and AI-assisted ($aia$) distribution and includes the p-values from a matched pair randomization two-sided t-test with $un-aia\ne 0$ as the alternative hypothesis and standard significance code: *** ($p<0.001$), ** ($p<0.01$), *($p<0.05$),.($p<0.1$), N.S ($p\ge 0.1$). Bar plots c, d, g, and h shows the frequency of decision adjustment $\left(un\ne aia\right)$ after AI-assistance in percentages.

Fig. 3. Analysis of Decision adjustment with respect to AI recommendation.

The first, second, and third column of figures correspond to NSCLC, NSCLC excluding evaluators with zero decision adjustment, and HCC, respectively, and all the plots are colored by evaluators and marked coded patient number. All evaluators in HCC adjusted at least one of their decisions. Scatter plots a, b, and c shows relationship between the level of decision adjustment $\left(aia-un\right)$ and dissimilarity with AI recommendation $\left(ai-un\right)$. Scatter plots d–f show relationship between AI Trust Level (0-5, 5 being the highest level) and agreement with AI-recommendation ($-\mid aia-ai$|). The $aia-ai=0$ line corresponds to absolute agreement with the AI-recommendation. All plots include the Spearman correlation coefficients, p-values (two-sided t test), and co-variance ellipse (95 % confidence). Covariance ellipses are included for a visual insight about the data distribution.

Fig. 4. Analysis of evaluator’s decision confidence.

The first, second, and third column of figures correspond to NSCLC, NSCLC excluding Evaluators with zero decision adjustment, and HCC, respectively. All evaluators in HCC adjusted at least one of their decisions. 2D Scatter plots a, b, and c show the relationship between evaluators’ AI-assisted decision ($aia$) confidence (0-5, 5 being the highest level) and $AITrustlevel$(0-5, 5 being the highest level). 2D Scatter plots d, e, and f show the relationship between evaluators’ change in confidence level ($aiaconf-unconf$) and $AItrustlevel$. 2D Scatter plots g, h and i show the relationship between evaluators’ change in confidence level and level of decision adjustment with AI-assistance. All plots include the Spearman correlation coefficients, p-values (two-sided t test), and co-variance ellipse (95 % confidence). Covariance ellipses are included for a visual insight about the data distribution.

Fig. 5. Analysis of Evaluator’s decision confidence with respect to the Standard of Care Dose fractionation (2 Gy/fx for NSCLC RT; 10 Gy/fx for HCC SBRT).

The first, second, and third column of figures correspond to NSCLC, NSCLC excluding evaluators with zero decision adjustment, and HCC, respectively. All evaluators in HCC adjusted at least one of their decisions. 2D Scatter plots a–c show the relationship between unassisted decision ($un$) confidence (0-5, 5 being the highest level) and closeness of $un$ to the standard of care dose decision values (NSCLC:$-∣un-2∣$Gy/fx); HCC: $-∣un-10∣$Gy/fx). 2D Scatter plots d, e, and f show the relationship between AI-assisted decision ($aia$) confidence and closeness of $un$ to the standard of care dose decision values (NSCLC:$-∣aia-2∣$ Gy/fx); HCC: $-∣aia-10∣$Gy/fx). All plots include the Spearman correlation coefficients, p-values (two-sided t test), and co-variance ellipse (95 % confidence). Covariance ellipses are included for a visual insight about the data distribution.

Fig. 6. Reliability analysis via intraclass correlation coefficient (ICC).

Bar plots a and b compare McGraw and Wong’s ICC between unassisted decision and AI-assisted decision for NSCLC and HCC, respectively. ICC value along with 95% confidence interval, and p-value (one-sided F-test, $H_{\alpha }:icc>0$) for NSCLC and HCC are presented in Table 1. We applied two-way random effects model to calculate four types of ICC for $n\times k$ data structure where $n$ and $k$ are the number of patients and evaluators, respectively, which were both chosen randomly from a larger pool of patients and evaluators (NSCLC: $n=8$, $k=9$; HCC: $n=9$, $k=8$). ICC type Consistency (C) measures the symmetric differences between the decisions of the $k$ evaluators, whereas ICC type Absolute Agreement (A) measures the absolute differences. ICC unit Single rater corresponds to using the decision from a single evaluator as the basis for measurement and ICC unit Average corresponds to using the average decision from all evaluators.

Fig. 7. AI utility analysis of decision adjustment with respect to RT outcome estimate for NSCLC.

Scatter plots a and b, grouped by patient number, show the RT outcome estimate (RTOE) in the space spanned by tumor control probability ($TCP$) and normal tissue complication probability ($NTCP$) for Unassisted ($un$) and AI-assisted ($aia$) decision, respectively. Scatter plots c and d show the change in RTOE for adjusted decisions in $un$ vs $aiaTCP$ space and $un$ vs $aiaNTCP$ space, respectively, including the $45^{\circ }$ Null dashed line. Out of 41 decision adjustment, 32 (76%) increased both $TCP$ and $NTCP$ while 10 (24%) decreased $TCP$ and $NTCP$. Paired plot e and violin plot f, present analysis of adjusted decision based on RTOE scoring schema $TCP\left(1-NTCP\right)$ [1 for $\left(tcp,ntcp\right)=\left(\mathrm{1,\; 0}\right)$, 0 for $ntcp=1$]. Paired plots e compares the change in score for $un$ and $aia$ for each patient. Violin plot f presents the overall summary statistics for the pairwise difference in score between $aia$ and $un$: $mean\left(sd\right)=0.0011\left(0.0058\right);median\left(Q1\mid Q3\right)=-0.0011\left(-0.0016\mid 0.0022\right).$ Box plots include center line: median, box limits: upper and lower quartiles; whiskers: 1.5x interquartile range; and points: outliers.

Fig. 8. AI utility analysis of decision adjustment with respect to RT outcome estimate for HCC.

Scatter plots a and b, grouped by patient number, show the RT Outcome Estimate (RTOE) in the space spanned by tumor control probability ($TCP$) and normal tissue complication probability ($NTCP$) for Unassisted ($un$) and AI-assisted ($aia$) decision, respectively. Scatter plots c and d show the change in RTOE for adjusted decisions in $un$ vs $aiaTCP$ space and $un$ vs $aiaNTCP$ space, respectively, including the $45^{\circ }$ Null dashed line. Out of 34 decision adjustment, 9 (26%) increased both $TCP$ and $NTCP$ while 25 (74%) decreased $TCP$ and $NTCP$. Paired plots e and violin plot f, present analysis of adjusted decision based on RTOE scoring schema $TCP\left(1-NTCP\right)$ [1 for $\left(tcp,ntcp\right)=\left(\mathrm{1,\; 0}\right)$, 0 for $ntcp=1$]. Paired plots e compares the change in score for $un$ and $aia$ for each patient. Violin plot f presents the overall summary statistics for the pairwise difference in score between $aia$ and $un$: $mean\left(sd\right)=0.0059\left(0.0144\right);$ $median\left(Q1\mid Q3\right)=0.0034\left(-7E-4\mid 0.0086\right).$ Box plots include center line: median, box limits: upper and lower quartiles; whiskers: 1.5× interquartile range; and points: outliers.