Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jan 1;10(1):95-102.
doi: 10.1001/jamaoncol.2023.5120.

A Learning Program for Treatment Recommendations by Molecular Tumor Boards and Artificial Intelligence

Kuniko Sunami  1 , Yoichi Naito  2 , Yusuke Saigusa  3 , Toraji Amano  4 , Daisuke Ennishi  5 , Mitsuho Imai  6   7 , Hidenori Kage  8 , Masashi Kanai  9 , Hirotsugu Kenmotsu  10 , Keigo Komine  11 , Takafumi Koyama  12 , Takahiro Maeda  13 , Sachi Morita  14 , Daisuke Sakai  15 , Makoto Hirata  16 , Mamoru Ito  17 , Toshiyuki Kozuki  18 , Hiroyuki Sakashita  19 , Hidehito Horinouchi  20 , Yusuke Okuma  20 , Atsuo Takashima  21 , Toshio Kubo  22 , Shuichi Hironaka  23 , Yoshihiko Segawa  24 , Yoshihiro Yakushijin  25 , Hideaki Bando  6 , Akitaka Makiyama  26 , Tatsuya Suzuki  27 , Ichiro Kinoshita  4 , Shinji Kohsaka  28 , Yuichiro Ohe  20 , Chikashi Ishioka  11 , Kouji Yamamoto  3 , Katsuya Tsuchihara  29 , Takayuki Yoshino  30
Affiliations

A Learning Program for Treatment Recommendations by Molecular Tumor Boards and Artificial Intelligence

Kuniko Sunami et al. JAMA Oncol. .

Abstract

Importance: Substantial heterogeneity exists in treatment recommendations across molecular tumor boards (MTBs), especially for biomarkers with low evidence levels; therefore, the learning program is essential.

Objective: To determine whether a learning program sharing treatment recommendations for biomarkers with low evidence levels contributes to the standardization of MTBs and to investigate the efficacy of an artificial intelligence (AI)-based annotation system.

Design, setting, and participants: This prospective quality improvement study used 50 simulated cases to assess concordance of treatment recommendations between a central committee and participants. Forty-seven participants applied from April 7 to May 13, 2021. Fifty simulated cases were randomly divided into prelearning and postlearning evaluation groups to assess similar concordance based on previous investigations. Participants included MTBs at hub hospitals, treating physicians at core hospitals, and AI systems. Each participant made treatment recommendations for each prelearning case from registration to June 30, 2021; participated in the learning program on July 18, 2021; and made treatment recommendations for each postlearning case from August 3 to September 30, 2021. Data were analyzed from September 2 to December 10, 2021.

Exposures: The learning program shared the methodology of making appropriate treatment recommendations, especially for biomarkers with low evidence levels.

Main outcomes and measures: The primary end point was the proportion of MTBs that met prespecified accreditation criteria for postlearning evaluations (approximately 90% concordance with high evidence levels and approximately 40% with low evidence levels). Key secondary end points were chronological enhancements in the concordance of treatment recommendations on postlearning evaluations from prelearning evaluations. Concordance of treatment recommendations by an AI system was an exploratory end point.

Results: Of the 47 participants who applied, 42 were eligible. The accreditation rate of the MTBs was 55.6% (95% CI, 35.3%-74.5%; P < .001). Concordance in MTBs increased from 58.7% (95% CI, 52.8%-64.4%) to 67.9% (95% CI, 61.0%-74.1%) (odds ratio, 1.40 [95% CI, 1.06-1.86]; P = .02). In postlearning evaluations, the concordance of treatment recommendations by the AI system was significantly higher than that of MTBs (88.0% [95% CI, 68.7%-96.1%]; P = .03).

Conclusions and relevance: The findings of this quality improvement study suggest that use of a learning program improved the concordance of treatment recommendations provided by MTBs to central ones. Treatment recommendations made by an AI system showed higher concordance than that for MTBs, indicating the potential clinical utility of the AI system.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Sunami reported receiving personal fees from Chugai Pharmaceutical Co Ltd, Illumina Inc, Guardant Health Japan, and MDS KK and grant funding from Sysmex Corporation outside the submitted work. Dr Naito reported receiving grant funding from the Ministry of Health, Labour and Welfare and personal fees from the Genomedia Inc–National Cancer Center joint research fund during the conduct of the study; grant funding from AbbVie Inc, Ono Pharmaceutical Co Ltd, Daiichi Sankyo, Pfizer Inc, Boehringer Ingelheim, Eli Lilly and Company, Eisai Co Ltd, AstraZeneca, Chugai Pharmaceutical Co Ltd, Bayer AG, and Genomedia Inc outside the submitted work; and lecture fees from AstraZeneca, Eisai Co Ltd, Ono Pharmaceutical Co Ltd, Guardant Health Japan, Takeda Pharmaceuticals International, Eli Lilly and Company, Novartis AG, Pfizer Inc, Chugai Pharmaceutical Co Ltd, FUJIFILM Toyama Chemical Co Ltd, Taiho Pharmaceutical Co Ltd, Mundipharma International Limited, Bristol Myers Squibb, and Shionogi & Company, Ltd outside the submitted work. Dr Ennishi reported receiving personal fees from Nippon Shinyaku Co Ltd and honoraria for lectures and personal fees from Chugai Pharmaceutical Co Ltd, Eisai Co Ltd, Bristol Myers Squibb, and Kyowa Kirin Co Ltd outside the submitted work. Dr Imai reported receiving support from Sumitomo Corporation for participation in academic conferences outside the submitted work and having had a US patent provisional application pending. Dr Kage reported receiving grant funding from Konica Minolta Inc outside the submitted work. Dr Kanai reported receiving personal fees from Chugai Pharmaceutical Co Ltd, having stock ownership in Therabiopharma, receiving research expenses from Molecular Health GmbH outside the submitted work, and having US patents 11141529B2, 6923162, and r6931853 issued. Dr Kenmotsu reported receiving grant funding from Chugai Pharmaceutical Co Ltd and personal fees from Chugai Pharmaceutical Co Ltd during the conduct of the study and receiving grant funding from AstraZeneca, Eli Lilly and Company, Daiichi Sankyo, Ono Pharmaceutical Co Ltd, and Novartis AG and personal fees from AstraZeneca, Taiho Pharmaceutical Co Ltd, Eli Lilly and Company, Boehringer Ingelheim, Bristol Myers Squibb Japan, Kyowa Kirin Co Ltd, MSD KK, Daiichi Sankyo, Pfizer Inc, Merck & Co Inc, Nippon Kayaku, Bayer Yakuhin Ltd, Amgen Inc, Takeda Pharmaceuticals International, Ono Pharmaceutical Co Ltd, and Loxo Oncology Inc outside the submitted work. Dr Komine reported receiving personal fees from Chugai Pharmaceutical Co Ltd, Incyte Corporation, Kyowa Kirin Co Ltd, Merck & Co Inc, Taiho Pharmaceutical Co Ltd, Takeda Pharmaceuticals International, and Daiichi Sankyo, outside the submitted work. Dr Koyama reported receiving grant funding from PACT Pharma Inc, Chugai Pharmaceutical Co Ltd, Daiichi Sankyo, Novartis AG, Eli Lilly and Company, Pfizer Inc, Janssen Pharmaceuticals, Zymeworks Inc, Takeda Pharmaceuticals International, AstraZeneca, and Noile-Immune Biotech Inc and personal fees from Chugai Pharmaceutical Co Ltd, AstraZeneca, and Sysmex Corporation outside the submitted work. Dr Sakai reported receiving grants from Eli Lilly and Company, Chugai Pharmaceutical Co Ltd, and Daiichi Sankyo outside the submitted work. Dr Kozuki reported receiving personal fees from Chugai Pharmaceutical Co Ltd, AstraZeneca, Ono Pharmaceutical Co Ltd, Eli Lilly and Company Japan, Bristol Myers Squibb, MSD KK, Taiho Pharmaceutical Co Ltd, Nippon Boehringer Ingelheim, Nippon Kayaku, Daiichi Sankyo, Merck & Co Inc, Pfizer Inc Japan, Takeda Pharmaceuticals International, Amgen Inc, Novartis AG, Bayer AG, AbbVie Inc, Kyowa Kirin Co Ltd, and Sawai Pharmaceutical Co Ltd, and grant funding from Chugai Pharmaceutical Co Ltd, AstraZeneca, Eli Lilly and Company Japan, Taiho Pharmaceutical Co Ltd, Bristol Myers Squibb, Ono Pharmaceutical Co Ltd, MSD KK, Kyowa Kirin Co Ltd, Merck & Co Inc, Daiichi Sankyo, Amgen Inc, AbbVie Inc, Sanofi SA, Eisai Co Ltd, Labcorp Japan, IQVIA Services Japan KK, Gilead Sciences Inc, Pfizer Inc, and Bayer AG outside the submitted work. Dr Sakashita reported receiving grant funding from Ono Pharmaceutical Co Ltd, Chugai Pharmaceutical Co Ltd, AstraZeneca KK, Taiho Pharmaceutical Co Ltd, Boehringer Ingelheim Japan Inc, Novartis Pharma KK, Merck & Co Inc, and SBI Pharmaceuticals Co Ltd, and personal fees from Ono Pharmaceutical Co Ltd, Chugai Pharmaceutical Co Ltd, AstraZeneca KK, Taiho Pharmaceutical Co Ltd, Boehringer Ingelheim Japan Inc, Daiichi Sankyo, Nippon Kayaku, Novartis Pharma KK, Astellas Pharma Inc, and Merck & Co Inc outside the submitted work. Dr Takashima reported receiving grant funding from Daiichi Sankyo, Pfizer Inc, Ono Pharmaceutical Co Ltd, MSD KK, Bristol Myers Squibb, Isofol Medical, Hutchison Medipharma Ltd, and Incyte Corporation and personal fees from Eli Lilly and Company, Ono Pharmaceutical Co Ltd, Taiho Pharmaceutical Co Ltd, Chugai Pharmaceutical Co Ltd, Takeda Pharmaceuticals International, and Merck Serono outside the submitted work. Dr Kubo reported receiving personal fees from Bristol Myers Squibb, Taiho Pharmaceutical Co Ltd, Kyowa Kirin Co Ltd, AstraZeneca, Ono Pharmaceutical Co Ltd, Nippon Kayaku, Chugai Pharmaceutical Co Ltd, MSD KK, Pfizer Inc, Eli Lilly and Company, Novartis AG, Boehringer Ingelheim, and Towa Pharmaceutical Co Ltd outside the submitted work. Dr Segawa reported receiving honoraria for lectures from Taiho Pharmaceutical Co Ltd during the conduct of the study. Dr Bando reported receiving lecture fees from Eli Lilly and Company Japan, Taiho Pharmaceutical Co Ltd, and Ono Pharmaceutical Co Ltd and grant funding from Ono Pharmaceutical Co Ltd outside the submitted work. Dr Makiyama reported receiving personal fees from Eli Lilly and Company Japan, Taiho Pharmaceutical Co Ltd, Ono Pharmaceutical Co Ltd, Bristol Myers Squibb, and Daiichi Sankyo outside the submitted work. Dr Kinoshita reported receiving personal fees from Chugai Pharmaceutical Co Ltd, Takeda Pharmaceuticals International, MSD KK, Bayer AG, and AstraZeneca KK and nonfinancial support from Novartis Pharma KK outside the submitted work. Dr Ohe reported receiving grant funding from AstraZeneca, Chugai Pharmaceutical Co Ltd, Eli Lilly and Company, Ono Pharmaceutical Co Ltd, BMS, Takeda Pharmaceuticals International, Janssen Pharmaceuticals, and AnHeart Therapeutics Inc and personal fees from AstraZeneca, Celltrion Inc, Chugai Pharmaceutical Co Ltd, Ono Pharmaceutical Co Ltd, BMS, Pfizer Inc, Taiho Pharmaceutical Co Ltd, Takeda Pharmaceuticals International, Janssen Pharmaceuticals, AnHeart Therapeutics Inc, Novartis AG, and Amgen Inc outside the submitted work. Dr Ishioka reported receiving personal fees from Kyowa Kirin Co Ltd, Daiichi Sankyo, Eli Lilly and Company, Nippon Kayaku, Takeda Pharmaceuticals International, Taiho Pharmaceutical Co Ltd, Chugai Pharmaceutical Co Ltd, Incyte Corporation, AstraZeneca, Merck & Co Inc, BMS, Bayer AG, M3 Inc, MSD KK, Novartis AG, and Ono Pharmaceutical Co Ltd and grant funding from Hitachi Ltd, Daiichi Sankyo, Kyowa Kirin Co Ltd, Nippon Kayaku, Asahi Kasei, Taiho Pharmaceutical Co Ltd, and Chugai Pharmaceutical Co Ltd outside the submitted work. Dr Yamamoto reported receiving personal fees from Chugai Pharmaceutical Co Ltd, CMIC Holdings Co Ltd, J-Pharma Co Ltd, Craif Inc, Johokiko Co Ltd, Triceps, Kanagawa Prefectural Hospital Organization, and Kanagawa Medical Practitioners Association and grant funding from Taiho Pharmaceutical Co Ltd, Boehringer Ingelheim, Ono Pharmaceutical Co Ltd, Takeda Pharmaceuticals International, Bayer Yakuhin Ltd, Daiichi Sankyo, Astellas Pharma Inc, and Kyowa Kirin Co Ltd outside the submitted work. Dr Tsuchihara reported receiving personal fees from Takeda Pharmaceuticals International, Chugai Pharmaceutical Co Ltd, Eisai Co Ltd, Boehringer Ingelheim, Bayer AG, Miyarisan Pharmaceutical Co Ltd, and Illumina Inc outside the submitted work. Dr Yoshino reported receiving grant funding from the Ministry of Health, Labour and Welfare during the conduct of the study and receiving grant funding from Amgen KK, Chugai Pharmaceutical Co Ltd, Daiichi Sankyo, Eisai Co Ltd, FALCO Biosystems Ltd, Genomedia Inc, Molecular Health GmbH, Nippon Boehringer Ingelheim, Ono Pharmaceutical Co Ltd, Pfizer Inc Japan, Roche Diagnostics KK, Sanofi KK, Sysmex Corporation, Taiho Pharmaceutical Co Ltd, Parexel International, and MSD KK; honoraria from Bayer Yakuhin Ltd, Merck Biopharma, Chugai Pharmaceutical Co Ltd, MSD KK, Ono Pharmaceutical Co Ltd, and Takeda Pharmaceuticals International; and consulting fees from Sumitomo Corporation outside the submitted work. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Study Overview
A, Fifty simulated cases were randomly divided into prelearning and postlearning evaluations (25 cases in each), stratified by levels of evidence (high vs low) and presence or absence of TP53 alteration. B, From May 1 to June 30, 2021, each participant made treatment recommendations (TRs) for each simulated case in the prelearning evaluation. Then, they participated in the learning program on July 18, 2021. Finally, TRs were made for simulated cases in postlearning evaluations from August 1 to September 30, 2021. The central committee evaluated the concordance in the prelearning evaluations between TRs by the participant and the central TRs prepared in April 2021. In postlearning evaluations, TRs by participants were evaluated for concordance with the central TRs prepared in August 2021. AI indicates artificial intelligence; CGP, comprehensive genome profiling; MTBs, molecular tumor boards; and TCGA, The Cancer Genome Atlas.
Figure 2.
Figure 2.. Study Flow Diagram
AI indicates artificial intelligence; MTBs, molecular tumor boards.
Figure 3.
Figure 3.. Analysis of Concordance of Treatment Recommendations by Molecular Treatment Boards (MTBs) and an Artificial Intelligence System (AI) in Postlearning Evaluations
The concordance of treatment recommendations made by the AI system was significantly higher than that of treatment recommendations made by 27 MTBs in all postlearning evaluations, especially those cases with genomic alterations with low evidence levels (ELs). NA indicates not applicable.
See this image and copyright information in PMC

References

    1. Radovich M, Kiel PJ, Nance SM, et al. . Clinical benefit of a precision medicine based approach for guiding treatment of refractory cancers. Oncotarget. 2016;7(35):56491-56500. doi:10.18632/oncotarget.10606 - DOI - PMC - PubMed
    1. Olsen S, Liao J, Hayashi H. Real-world clinical outcomes after genomic profiling of circulating tumor DNA in patients with previously treated advanced non–small cell lung cancer. Curr Oncol. 2022;29(7):4811-4826. doi:10.3390/curroncol29070382 - DOI - PMC - PubMed
    1. André F, Bachelot T, Commo F, et al. . Comparative genomic andomizedon array and DNA sequencing to direct treatment of metastatic breast cancer: a multicentre, prospective trial (SAFIR01/UNICANCER). Lancet Oncol. 2014;15(3):267-274. doi:10.1016/S1470-2045(13)70611-9 - DOI - PubMed
    1. Le Tourneau C, Delord JP, Gonçalves A, et al. ; SHIVA investigators . Molecularly targeted therapy based on tumour molecular profiling versus conventional therapy for advanced cancer (SHIVA): a multicentre, open-label, proof-of-concept, andomized, controlled phase 2 trial. Lancet Oncol. 2015;16(13):1324-1334. doi:10.1016/S1470-2045(15)00188-6 - DOI - PubMed
    1. Eckhardt SG, Lieu C. Is precision medicine an oxymoron? JAMA Oncol. 2019;5(2):142-143. doi:10.1001/jamaoncol.2018.5099 - DOI - PubMed

Publication types