Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 May;134(5):1308-1317.
doi: 10.1016/j.bja.2025.01.028. Epub 2025 Mar 3.

Automated structured data extraction from intraoperative echocardiography reports using large language models

Emily J MacKay  1 , Shir Goldfinger  2 , Trevor J Chan  3 , Rachel H Grasfield  4 , Vikram J Eswar  5 , Kelly Li  6 , Quy Cao  7 , Alison M Pouch  3
Affiliations

Automated structured data extraction from intraoperative echocardiography reports using large language models

Emily J MacKay et al. Br J Anaesth. 2025 May.

Abstract

Background: Consensus-based large language model (LLM) ensembles might provide an automated solution for extracting structured data from unstructured text in echocardiography reports.

Methods: This cross-sectional study utilised 600 intraoperative transoesophageal reports (100 for prompt engineering; 500 for testing) randomly sampled from 7106 adult patients undergoing cardiac surgery at two hospitals within the University of Pennsylvania Healthcare System. Three echocardiographic parameters (left ventricular ejection fraction, right ventricular systolic function, and tricuspid regurgitation) were extracted from both the presurgical and postsurgical sections of the reports. LLM ensembles were generated using five open-source LLMs and four voting strategies: (1) unanimous (five out of five in agreement); (2) supermajority (four or more of five in agreement); (3) majority (three or more of five in agreement); and (4) plurality (two or more of five in agreement). Returned LLM ensemble responses were compared with the reference standard dataset to calculate raw accuracy, consensus accuracy, error rate, and yield.

Results: Of the four LLM ensembles, the unanimous LLM ensemble achieved the highest consensus accuracies (99.4% presurgical; 97.9% postsurgical) and the lowest error rates (0.6% presurgical; 2.1% postsurgical) but had the lowest data extraction yields (81.7% presurgical; 80.5% postsurgical) and the lowest raw accuracies (81.2% presurgical; 78.9% postsurgical). In contrast, the plurality LLM ensemble achieved the highest raw accuracies (96.1% presurgical; 93.7% postsurgical) and the highest data extraction yields (99.4% presurgical; 98.9% postsurgical) but had the lowest consensus accuracies (96.7% presurgical; 94.7% postsurgical) and highest error rates (3.3% presurgical; 5.3% postsurgical).

Conclusions: A consensus-based LLM ensemble successfully generated structured data from unstructured text contained in intraoperative transoesophageal reports.

Keywords: artificial intelligence (AI); cardiac surgery; echocardiography; large language models; perioperative medicine.

PubMed Disclaimer

Conflict of interest statement

Declaration of interest The authors declare that they have no conflicts of interest.

References

    1. Nicoara A., Skubas N., Ad N., et al. Guidelines for the use of transesophageal echocardiography to assist with surgical decision-making in the operating room: a surgery-based approach: from the American Society of Echocardiography in collaboration with the Society of Cardiovascular Anesthesiologists and the Society of Thoracic Surgeons. J Am Soc Echocardiogr. 2020;33:692–734. - PubMed
    1. Silva F., Arruda R., Nobre A., et al. Impact of intraoperative transesophageal echocardiography in cardiac surgery: retrospective analysis of a series of 850 examinations. Rev Port Cardiol. 2010;29:1363–1382. - PubMed
    1. Zoghbi W.A., Chambers J.B., Dumesnil J.G., et al. Recommendations for evaluation of prosthetic valves with echocardiography and Doppler ultrasound: a report from the American society of echocardiography's guidelines and standards committee and the task force on prosthetic valves, developed in conjunction with the American College of Cardiology cardiovascular imaging committee, cardiac imaging committee of the American heart association, the European association of echocardiography, a registered branch of the European society of Cardiology, the Japanese society of echocardiography and the Canadian society of echocardiography, endorsed by the American College of Cardiology foundation, American heart association, European association of echocardiography, a registered branch of the European society of Cardiology, the Japanese society of echocardiography, and Canadian society of echocardiography. J Am Soc Echocardiogr. 2009;22:975–1014. - PubMed
    1. MacKay E.J., Zhang B., Augoustides J.G., Groeneveld P.W., Desai N.D. Association of intraoperative transesophageal echocardiography and clinical outcomes after open cardiac valve or proximal aortic surgery. JAMA Netw Open. 2022;5 - PMC - PubMed
    1. MacKay E.J., Neuman M.D., Fleisher L.A., et al. Transesophageal echocardiography, mortality, and length of hospitalization after cardiac valve surgery. J Am Soc Echocardiogr. 2020;33:756–762.e1. - PMC - PubMed

LinkOut - more resources