Developing PRISM: A Pragmatic Institutional Survey and Bench Marking Tool to Measure Digital Research Maturity of Cancer Centers

Abstract

Background: Multicenter precision oncology real-world evidence requires a substantial long-term investment by hospitals to prepare their data and align on common Clinical Research processes and medical definitions. Our team has developed a self-assessment framework to support hospitals and hospital networks to measure their digital maturity and better plan and coordinate those investments. From that framework, we developed PRISM for Cancer Outcomes: PR: agmatic I: nstitutional S: urvey and benchM: arking.

Objectives: The primary objective was to develop PRISM as a tool for self-assessment of digital maturity in oncology hospitals and research networks; a secondary objective was to create an initial benchmarking cohort of >25 hospitals using the tool as input for future development.

Methods: PRISM is a 25-question semiquantitative self-assessment survey developed iteratively from expert knowledge in oncology real-world study delivery. It covers four digital maturity dimensions: (1) Precision oncology, (2) Clinical digital data, (3) Routine outcomes, and (4) Information governance and delivery. These reflect the four main data types and critical enablers for precision oncology research from routine electronic health records.

Results: During piloting with 26 hospitals from 19 European countries, PRISM was found to be easy to use and its semiquantitative questions to be understood in a wide diversity of hospitals. Results within the initial benchmarking cohort aligned well with internal perspectives. We found statistically significant differences in digital maturity, with Precision oncology being the most mature dimension, and Information governance and delivery the least mature.

Conclusion: PRISM is a light footprint benchmarking tool to support the planning of large-scale real-world research networks. It can be used to (i) help an individual hospital identify areas most in need of investment and improvement, (ii) help a network of hospitals identify sources of best practice and expertise, and (iii) help research networks plan research. With further testing, policymakers could use PRISM to better plan digital investments around the Cancer Mission and European Digital Health Space.

Fig. 1

Digital maturity framework with Bronze, Silver, and Gold classification. eCRF, electronic case report form; EHR, electronic health record; GDPR, General Data Protection Regulation; IHC, immunohistochemistry; MDX, molecular diagnostics; NCCN, National Comprehensive Cancer Network; PRO, patient-reported outcome; RECIST, Response Evaluation Criteria In Solid Tumors; NSCLC, non-small cell lung cancer; SoC, standard of care.

Fig. 2

( A ) Privacy conserving visualization of the entire community by question showing median and interquartile ranges of responses to each question from the sample. ( B ) Box plot of the four survey dimensions. The first dimension is Precision oncology (blue, far left, average of survey questions Q1.1–Q1.7), the second dimension is Clinical digital data (orange, middle left Q2.1–Q2.5), the third dimension is Pragmatic outcomes (gray, middle right, Q3.1–Q3.7), and the last dimension is Information governance and delivery (yellow, far right, Q4.1–Q4.6). The original survey used a 5-point score in which 1 represents the least mature option while 5 represents the most mature. EHR, electronic health record; RECIST, Response Evaluation Criteria In Solid Tumors.

Fig. 3

Anonymous example of the individual results shared back to the hospitals.

Fig. 4

Cancer digital maturity snapshot. The color of the bubbles represents the degree of clinical digital data maturity of each center; the top quartile (green), between interquartile range (amber), and the bottom quartile (red). The size of the circle represents the number of annual patient diagnostics, 19 centers provided counts (solid outline) while the remaining centers were estimated using the average annual number of new cancer diagnostics across the 19 responding centers (dashed outline). EHR, electronic health record; RECIST, Response Evaluation Criteria In Solid Tumors.