Use and Usefulness of Risk Prediction Tools in Urologic Surgery: Current State and Path Forward

Abstract

Introduction: Although the enthusiasm for artificial intelligence (AI) to enhance surgical decision-making continues to grow, the preceding advance of risk prediction tools (RPTs) has had limited impact to date. To help inform the development of AI-powered tools, we evaluated the role of RPTs and prevailing attitudes among urologists.

Methods: We conducted a national mixed methods study using a sequential explanatory design. Through the 2019 AUA Census, we surveyed urologists on RPT use, helpfulness, and trust. Based on responses, we interviewed 25 participants on RPTs, risk evaluation, and surgical decision-making. Coding-based thematic analysis was applied and integrated with survey findings.

Results: Among 2081 urologic surgeons (weighted sample 12,366), 30.4% (95% CI, 28.2%-32.6%) routinely used RPTs and 34.3% (95% CI, 31.9%-36.6%) found them helpful while 47.0% (95% CI, 44.6%-49.5%) generally trusted their own assessment over RPT-generated estimates. More years in practice was negatively associated with RPT use, helpfulness, and trust (P < .001). Qualitatively, participants described relying on their intuition for surgical risks and benefits and using gist-based approximations rather than numerical information, which RPTs provide. RPT helpfulness centered on risk/benefit confirmation, calibration, and communication, but methodological (eg, individual vs group estimates and missing variables) and operational (eg, ease of use and clinical workflow) challenges limit greater RPT use.

Conclusions: Despite their wide availability, RPTs remain limited in their use and helpfulness. This reflects both the intuitive nature of surgical decision-making and implementation challenges. For AI to reach its promise and improve surgical care and outcomes, both types of barriers will need to be addressed.

Keywords: predictive tools; usability; user experience.