Meta-Analysis

. 2022 Jun 13:377:e068424.

doi: 10.1136/bmj-2021-068424.

Development and validation of a decision support tool for the diagnosis of acute heart failure: systematic review, meta-analysis, and modelling study

Kuan Ken Lee^{1

2}, Dimitrios Doudesis^{1

3

2}, Mohamed Anwar^{1

2}, Federica Astengo¹, Camille Chenevier-Gobeaux⁴, Yann-Erick Claessens⁵, Desiree Wussler^{6

7}, Nikola Kozhuharov^{6

8}, Ivo Strebel⁶, Zaid Sabti⁶, Christopher deFilippi⁹, Stephen Seliger¹⁰, Gordon Moe¹¹, Carlos Fernando¹¹, Antoni Bayes-Genis¹², Roland R J van Kimmenade¹³, Yigal Pinto¹⁴, Hanna K Gaggin^{15

16}, Jan C Wiemer¹⁷, Martin Möckel¹⁸, Joost H W Rutten¹⁹, Anton H van den Meiracker²⁰, Luna Gargani²¹, Nicola R Pugliese²², Christopher Pemberton²³, Irwani Ibrahim²⁴, Alfons Gegenhuber²⁵, Thomas Mueller²⁶, Michael Neumaier²⁷, Michael Behnes²⁸, Ibrahim Akin²⁸, Michele Bombelli²⁹, Guido Grassi³⁰, Peiman Nazerian³¹, Giovanni Albano³¹, Philipp Bahrmann³², David E Newby¹, Alan G Japp¹, Athanasios Tsanas³, Anoop S V Shah^{1

33}, A Mark Richards^{23

34}, John J V McMurray³⁵, Christian Mueller⁶, James L Januzzi^{15

16}, Nicholas L Mills^{36

3}; CoDE-HF investigators

Collaborators, Affiliations

Collaborators

CoDE-HF investigators:
Adam Singer, Judd Hollander, Humberto Villacorta, Evandro Tinoco Mesquita, Joel Coste, Patrick Jourdain, Kimiaki Komukai, Michihiro Yoshimura, Olivier Hanon, Jean-Sébastien Vidal, Peter Cameron, Louisa Lam, Ben Freedman, Tommy Chung, Sean P Collins, Christopher John Lindsell

Affiliations

¹ British Heart Foundation Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK.
² Contributed equally.
³ Usher Institute, University of Edinburgh, Edinburgh, UK.
⁴ Department of Biochemistry, Cochin Hospital, Assistance Publique-Hopitaux de Paris, Paris, France.
⁵ Department of Emergency Medicine, Princess Grace Hospital Center, Monaco, Principality of Monaco.
⁶ Cardiovascular Research Institute of Basel, Department of Cardiology, University Hospital Basel, Basel, Switzerland.
⁷ Department of Internal Medicine, University Hospital Basel, University of Basel, Switzerland.
⁸ Liverpool Heart and Chest Hospital, Liverpool, UK.
⁹ Inova Heart and Vascular Institute, Falls Church, VA, USA.
¹⁰ Division of Nephrology, University of Maryland School of Medicine, Baltimore, MD, USA.
¹¹ University of Toronto, St Michael's Hospital, Toronto, ON, Canada.
¹² Heart Institute, Hospital Universitari Germans Trias i Pujol, Badalona, CIBERCV, Spain.
¹³ Department of Cardiology, Radboud University Medical Center, Nijmegen, Netherlands.
¹⁴ University of Amsterdam, Amsterdam, Netherlands.
¹⁵ Harvard Medical School, Boston, MA, USA.
¹⁶ Division of Cardiology, Massachusetts General Hospital, Boston, MA, USA.
¹⁷ BRAHMS, Thermo Fisher Scientific, Hennigsdorf, Germany.
¹⁸ Department of Emergency and Acute Medicine with Chest Pain Units, Charité - Universitätsmedizin Berlin, Campus Mitte and Virchow, Berlin, Germany.
¹⁹ Department of Internal Medicine, Radboud University Medical Center, Nijmegen, Netherlands.
²⁰ Department of Internal Medicine, Division of Pharmacology and Vascular Medicine, Erasmus Medical Center, Rotterdam, Netherlands.
²¹ Department of Surgical, Medical and Molecular Pathology and Critical Care Medicine, University of Pisa, Pisa, Italy.
²² Department of Clinical and Experimental Medicine, University of Pisa, Pisa, Italy.
²³ Christchurch Heart Institute, University of Otago, Christchurch, New Zealand.
²⁴ Emergency Medicine Department, National University Hospital, Singapore.
²⁵ Department of Internal Medicine, Krankenhaus Bad Ischl, Bad Ischl, Austria.
²⁶ Department of Laboratory Medicine, Hospital Voecklabruck, Voecklabruck, Austria.
²⁷ Institute for Clinical Chemistry, University Medical Centre Mannheim, Faculty of Medicine Mannheim, University of Heidelberg, Mannheim, Germany.
²⁸ First Department of Medicine, University Medical Centre Mannheim, Faculty of Medicine Mannheim, University of Heidelberg, Mannheim, Germany.
²⁹ University of Milan Bicocca, ASST-Brianza, Pio XI Hospital of Desio, Internal Medicine, Desio, Italy.
³⁰ Clinica Medica, University Milan Bicocca, Milan, Italy.
³¹ Department of Emergency Medicine, Azienda Ospedaliero-Universitaria Careggi, Florence, Italy.
³² Department of Internal Medicine III, Division of Cardiology, University Hospital of Heidelberg, Ruprecht-Karls University Heidelberg, Heidelberg, Germany.
³³ London School of Hygiene and Tropical Medicine, London, UK.
³⁴ Cardiovascular Research Institute, National University Heart Centre Singapore, Singapore.
³⁵ British Heart Foundation Cardiovascular Research Centre, University of Glasgow, Glasgow, UK.
³⁶ British Heart Foundation Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK nick.mills@ed.ac.uk.

PMID: 35697365
PMCID: PMC9189738
DOI: 10.1136/bmj-2021-068424

Meta-Analysis

Development and validation of a decision support tool for the diagnosis of acute heart failure: systematic review, meta-analysis, and modelling study

Kuan Ken Lee et al. BMJ. 2022.

. 2022 Jun 13:377:e068424.

doi: 10.1136/bmj-2021-068424.

Authors

Collaborators

CoDE-HF investigators:
Adam Singer, Judd Hollander, Humberto Villacorta, Evandro Tinoco Mesquita, Joel Coste, Patrick Jourdain, Kimiaki Komukai, Michihiro Yoshimura, Olivier Hanon, Jean-Sébastien Vidal, Peter Cameron, Louisa Lam, Ben Freedman, Tommy Chung, Sean P Collins, Christopher John Lindsell

Affiliations

¹ British Heart Foundation Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK.
² Contributed equally.
³ Usher Institute, University of Edinburgh, Edinburgh, UK.
⁴ Department of Biochemistry, Cochin Hospital, Assistance Publique-Hopitaux de Paris, Paris, France.
⁵ Department of Emergency Medicine, Princess Grace Hospital Center, Monaco, Principality of Monaco.
⁶ Cardiovascular Research Institute of Basel, Department of Cardiology, University Hospital Basel, Basel, Switzerland.
⁷ Department of Internal Medicine, University Hospital Basel, University of Basel, Switzerland.
⁸ Liverpool Heart and Chest Hospital, Liverpool, UK.
⁹ Inova Heart and Vascular Institute, Falls Church, VA, USA.
¹⁰ Division of Nephrology, University of Maryland School of Medicine, Baltimore, MD, USA.
¹¹ University of Toronto, St Michael's Hospital, Toronto, ON, Canada.
¹² Heart Institute, Hospital Universitari Germans Trias i Pujol, Badalona, CIBERCV, Spain.
¹³ Department of Cardiology, Radboud University Medical Center, Nijmegen, Netherlands.
¹⁴ University of Amsterdam, Amsterdam, Netherlands.
¹⁵ Harvard Medical School, Boston, MA, USA.
¹⁶ Division of Cardiology, Massachusetts General Hospital, Boston, MA, USA.
¹⁷ BRAHMS, Thermo Fisher Scientific, Hennigsdorf, Germany.
¹⁸ Department of Emergency and Acute Medicine with Chest Pain Units, Charité - Universitätsmedizin Berlin, Campus Mitte and Virchow, Berlin, Germany.
¹⁹ Department of Internal Medicine, Radboud University Medical Center, Nijmegen, Netherlands.
²⁰ Department of Internal Medicine, Division of Pharmacology and Vascular Medicine, Erasmus Medical Center, Rotterdam, Netherlands.
²¹ Department of Surgical, Medical and Molecular Pathology and Critical Care Medicine, University of Pisa, Pisa, Italy.
²² Department of Clinical and Experimental Medicine, University of Pisa, Pisa, Italy.
²³ Christchurch Heart Institute, University of Otago, Christchurch, New Zealand.
²⁴ Emergency Medicine Department, National University Hospital, Singapore.
²⁵ Department of Internal Medicine, Krankenhaus Bad Ischl, Bad Ischl, Austria.
²⁶ Department of Laboratory Medicine, Hospital Voecklabruck, Voecklabruck, Austria.
²⁷ Institute for Clinical Chemistry, University Medical Centre Mannheim, Faculty of Medicine Mannheim, University of Heidelberg, Mannheim, Germany.
²⁸ First Department of Medicine, University Medical Centre Mannheim, Faculty of Medicine Mannheim, University of Heidelberg, Mannheim, Germany.
²⁹ University of Milan Bicocca, ASST-Brianza, Pio XI Hospital of Desio, Internal Medicine, Desio, Italy.
³⁰ Clinica Medica, University Milan Bicocca, Milan, Italy.
³¹ Department of Emergency Medicine, Azienda Ospedaliero-Universitaria Careggi, Florence, Italy.
³² Department of Internal Medicine III, Division of Cardiology, University Hospital of Heidelberg, Ruprecht-Karls University Heidelberg, Heidelberg, Germany.
³³ London School of Hygiene and Tropical Medicine, London, UK.
³⁴ Cardiovascular Research Institute, National University Heart Centre Singapore, Singapore.
³⁵ British Heart Foundation Cardiovascular Research Centre, University of Glasgow, Glasgow, UK.
³⁶ British Heart Foundation Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK nick.mills@ed.ac.uk.

PMID: 35697365
PMCID: PMC9189738
DOI: 10.1136/bmj-2021-068424

Abstract

Objectives: To evaluate the diagnostic performance of N-terminal pro-B-type natriuretic peptide (NT-proBNP) thresholds for acute heart failure and to develop and validate a decision support tool that combines NT-proBNP concentrations with clinical characteristics.

Design: Individual patient level data meta-analysis and modelling study.

Setting: Fourteen studies from 13 countries, including randomised controlled trials and prospective observational studies.

Participants: Individual patient level data for 10 369 patients with suspected acute heart failure were pooled for the meta-analysis to evaluate NT-proBNP thresholds. A decision support tool (Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF)) that combines NT-proBNP with clinical variables to report the probability of acute heart failure for an individual patient was developed and validated.

Main outcome measure: Adjudicated diagnosis of acute heart failure.

Results: Overall, 43.9% (4549/10 369) of patients had an adjudicated diagnosis of acute heart failure (73.3% (2286/3119) and 29.0% (1802/6208) in those with and without previous heart failure, respectively). The negative predictive value of the guideline recommended rule-out threshold of 300 pg/mL was 94.6% (95% confidence interval 91.9% to 96.4%); despite use of age specific rule-in thresholds, the positive predictive value varied at 61.0% (55.3% to 66.4%), 73.5% (62.3% to 82.3%), and 80.2% (70.9% to 87.1%), in patients aged <50 years, 50-75 years, and >75 years, respectively. Performance varied in most subgroups, particularly patients with obesity, renal impairment, or previous heart failure. CoDE-HF was well calibrated, with excellent discrimination in patients with and without previous heart failure (area under the receiver operator curve 0.846 (0.830 to 0.862) and 0.925 (0.919 to 0.932) and Brier scores of 0.130 and 0.099, respectively). In patients without previous heart failure, the diagnostic performance was consistent across all subgroups, with 40.3% (2502/6208) identified at low probability (negative predictive value of 98.6%, 97.8% to 99.1%) and 28.0% (1737/6208) at high probability (positive predictive value of 75.0%, 65.7% to 82.5%) of having acute heart failure.

Conclusions: In an international, collaborative evaluation of the diagnostic performance of NT-proBNP, guideline recommended thresholds to diagnose acute heart failure varied substantially in important patient subgroups. The CoDE-HF decision support tool incorporating NT-proBNP as a continuous measure and other clinical variables provides a more consistent, accurate, and individualised approach.

Study registration: PROSPERO CRD42019159407.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the British Heart Foundation, Medical Research Council, and Chief Scientist Office; Y-EC has received honorariums for lectures and presentations from Biomérieux, Roche Diagnostics, and Thermo Fisher; CdF has received consulting fees from Fuji Rebio, Ortho Diagnostics, Quidel, and Roche Diagnostics and a patent entitled “Methods for assessing differential risk for developing heart failure” (patent number: PCT/US2015/029838); SS has received a grant from Roche Diagnostics and a patent entitled “Methods for assessing differential risk for developing heart failure” (patent number: PCT/US2015/029838); ABG has received personal fees and non-financial support from Roche Diagnostics during the conduct of the study and personal fees from Abbott and AstraZeneca, grants, personal fees, and non-financial support from Boehringer-Ingelheim, and personal fees and non-financial support from Novartis and Vifor outside the submitted work. YP has received consulting fees from Roche Diagnostics, Pfizer, and Forbion and honorariums from CVOI and Daiichi Sankyo; HKG has received grants from Roche Diagnostics, Jana Care, Ortho Clinical, Novartis, Pfizer, Alnylam, and Akcea (IONIS), consulting fees from Amgen, Eko, Merck, and Pfizer, and stock in Eko; JCW works as a biostatistician at the biotech company BRAHMS GmbH, part of Thermo Fisher Scientific; MM has received grants from Health Care Research Projects and Biomarker Research and personal fees from Consulting outside the submitted work; ASVS has received speaker fees from Abbott Diagnostics outside the submitted work; AMR has received grants, personal fees, and non-financial support from Roche Diagnostics outside the submitted work; JJVM has received consulting fees from Alnylam, AstraZeneca, Bayer, Boehringer Ingelheim, BMS, Cardurion, Dal-Cor, GSK, Ionis, KBP Biosciences, Novartis, Pfizer, and Theracos, payments for advisory boards, symposiums, or lectures from Abbott, Alkem Metabolics, Canadian Medical and Surgical Knowledge Translation Research Group, Eris Lifesciences, Hikma, Lupin, Sun Pharmaceuticals, Medscape/Heart.Org, ProAdWise Communications, Radcliffe Cardiology, Servier, and the Corpus, has participated on a data safety monitoring board or advisory board for Cardialysis (MONITOR study) and Merck (VICTORIA trial), and works as company director for Global Clinical Trial Partners Ltd (GCTP) outside the submitted work; CM has received grants and non-financial support from several diagnostic companies during the conduct of the study and grants, personal fees, and non-financial support from several diagnostic companies outside the submitted work; JJ has received grants from Abbott Diagnostics, Innolife, and Novartis and consulting fees from Abbott, Jana Care, Novartis, Roche Diagnostics, Bristol-Myers Squibb, Janssen, and Prevencio; NLM has received grants from Siemens Healthineers, consulting fees from Roche Diagnostics and LumiraDx, and speaker fees from Abbott Diagnostics and Siemens Healthineers outside the submitted work; KKL, DD, and NLM are employed by the University of Edinburgh, which has filed a patent on the CoDE-HF score (patent reference: PCT/GB2021/051470); no other relationships or activities that could appear to have influenced the submitted work.

Figures

**Fig 1**
N-terminal pro-B-type natriuretic peptide (NT-proBNP) thresholds for acute heart failure. Top left: negative predictive values of NT-proBNP concentrations to rule out diagnosis of acute heart failure. Bottom left: cumulative proportion of patients presenting with suspected acute heart failure with NT-proBNP concentrations below each threshold. Top right: positive predictive values of NT-proBNP concentrations to rule in diagnosis of acute heart failure. Bottom right: cumulative proportion of patients presenting with suspected acute heart failure with NT-proBNP concentrations above each threshold

**Fig 2**
Diagnostic performance of guideline recommended N-terminal pro-B-type natriuretic peptide thresholds across patient subgroups: negative predictive value of threshold of 300 pg/mL. COPD=chronic obstructive pulmonary disease; eGFR=estimated glomerular filtration rate

**Fig 3**
Diagnostic performance of guideline recommended NT-proBNP thresholds across patient subgroups: positive predictive value of age specific thresholds across patient subgroups (450, 900, and 1800 pg/mL for <50, 50-75, and >75 years, respectively). COPD=chronic obstructive pulmonary disease; eGFR=estimated glomerular filtration rate

**Fig 4**
Calibration of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score with observed proportion of patients with acute heart failure. Dashed line represents perfect calibration. Each point represents 100 patients. Top: calibration of CoDE-HF in patients with no previous heart failure. Bottom: calibration of CoDE-HF in patients with previous heart failure

**Fig 5**
Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score across patient subgroups: negative predictive value of CoDE-HF rule-out score of 4.7 in patients without previous heart failure across patient subgroups. CoDE-HF incorporates N-terminal pro-B-type natriuretic peptide concentrations as continuous measure and predefined simple objective clinical variables (age, estimated glomerular filtration rate (eGFR), haemoglobin, body mass index, heart rate, blood pressure, peripheral oedema, chronic obstructive pulmonary disease (COPD), and ischaemic heart disease) to provide individualised assessment of likelihood of diagnosis of acute heart failure

**Fig 6**
Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score across patient subgroups: positive predictive value of CoDE-HF rule-in score of 51.2 in patients without previous heart failure across patient subgroups. CoDE-HF incorporates NT-proBNP concentrations as continuous measure and predefined simple objective clinical variables (age, estimated glomerular filtration rate (eGFR), haemoglobin, body mass index, heart rate, blood pressure, peripheral oedema, chronic obstructive pulmonary disease (COPD), and ischaemic heart disease) to provide individualised assessment of likelihood of diagnosis of acute heart failure

**Fig 7**
Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score across patient subgroups: positive predictive value of CoDE-HF rule-in score of 84.5 in patients with previous heart failure across patient subgroups. CoDE-HF incorporates NT-proBNP concentrations as continuous measure and predefined simple objective clinical variables (age, estimated glomerular filtration rate (eGFR), haemoglobin, body mass index, heart rate, blood pressure, peripheral oedema, chronic obstructive pulmonary disease (COPD), and ischaemic heart disease) to provide individualised assessment of likelihood of diagnosis of acute heart failure

**Fig 8**
Diagnostic performance of Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) score in patients without previous heart failure. Top: negative and positive predictive values of CoDE-HF scores. Blue vertical dashed line represents target rule-out score of 4.7. Red vertical dashed line represents target rule-in score of 51.2. Bottom: density plot of CoDE-HF score in patients without previous heart failure. Target rule-out and rule-in scores identify 40.3% of patients as low probability and 28.0% as high probability respectively

**Fig 9**
Cumulative incidence of all cause mortality stratified by Collaboration for the Diagnosis and Evaluation of Heart Failure (CoDE-HF) probability group

See this image and copyright information in PMC

References

1. Conrad N, Judge A, Tran J, et al. . Temporal trends and patterns in heart failure incidence: a population-based study of 4 million individuals. Lancet 2018;391:572-80. 10.1016/S0140-6736(17)32520-5 - DOI - PMC - PubMed
1. National Institute for Cardiovascular Outcomes Research. National Heart Failure Audit 2019 Summary Report. 2019. https://www.nicor.org.uk/wp-content/uploads/2019/09/Heart-Failure-2019-R....
1. Roberts E, Ludman AJ, Dworzynski K, et al. NICE Guideline Development Group for Acute Heart Failure . The diagnostic accuracy of the natriuretic peptides in heart failure: systematic review and diagnostic meta-analysis in the acute care setting. BMJ 2015;350:h910. 10.1136/bmj.h910 - DOI - PMC - PubMed
1. Januzzi JL, Jr, Chen-Tournoux AA, Christenson RH, et al. ICON-RELOADED Investigators . N-Terminal Pro-B-Type Natriuretic Peptide in the Emergency Department: The ICON-RELOADED Study. J Am Coll Cardiol 2018;71:1191-200. 10.1016/j.jacc.2018.01.021 - DOI - PubMed
1. National Institute for Health and Care Excellence. Acute heart failure: diagnosis and management. Clinical Guideline 187. 2014. https://www.nice.org.uk/guidance/cg187. - PubMed

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Development and validation of a decision support tool for the diagnosis of acute heart failure: systematic review, meta-analysis, and modelling study

Collaborators

Affiliations

Development and validation of a decision support tool for the diagnosis of acute heart failure: systematic review, meta-analysis, and modelling study

Authors

Collaborators

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

Substances

Grants and funding

LinkOut - more resources

Full Text Sources

Medical

Research Materials

Miscellaneous