Clinical usefulness of scoring systems to predict severe acute pancreatitis: A systematic review and meta-analysis with pre and post-test probability assessment

Abstract

Background: Scoring systems for severe acute pancreatitis (SAP) prediction should be used in conjunction with pre-test probability to establish post-test probability of SAP, but data of this kind are lacking.

Objective: To investigate the predictive value of commonly employed scoring systems and their usefulness in modifying the pre-test probability of SAP.

Methods: Following PRISMA statement and MOOSE checklists after PROSPERO registration, PubMed was searched from inception until September 2022. Retrospective, prospective, cross-sectional studies or clinical trials on patients with acute pancreatitis defined as Revised Atlanta Criteria, reporting rate of SAP and using at least one score among Bedside Index for Severity in Acute Pancreatitis (BISAP), Acute Physiology and Chronic Health Examination (APACHE)-II, RANSON, and Systemic Inflammatory Response Syndrome (SIRS) with their sensitivity and specificity were included. Random effects model meta-analyses were performed. Pre-test probability and likelihood ratio (LR) were combined to estimate post-test probability on Fagan nomograms. Pooled severity rate was used as pre-test probability of SAP and pooled sensitivity and specificity to calculate LR and generate post-test probability. A priori hypotheses for heterogeneity were developed and sensitivity analyses planned.

Results: 43 studies yielding 14,116 acute pancreatitis patients were included: 42 with BISAP, 30 with APACHE-II, 27 with Ranson, 8 with SIRS. Pooled pre-test probability of SAP ranged 16.6%-25.3%. The post-test probability of SAP with positive/negative score was 47%/6% for BISAP, 43%/5% for APACHE-II, 48%/5% for Ranson, 40%/12% for SIRS. In 18 studies comparing BISAP, APACHE-II, and Ranson in 6740 patients with pooled pre-test probability of SAP of 18.7%, post-test probability when scores were positive was 48% for BISAP, 46% for APACHE-II, 50% for Ranson. When scores were negative, post-test probability dropped to 7% for BISAP, 6% for Ranson, 5% for APACHE-II. Quality, design, and country of origin of the studies did not explain the observed high heterogeneity.

Conclusions: The most commonly used scoring systems to predict SAP perform poorly and do not aid in decision-making.

Keywords: APACHE-II; BISAP; RANSON; Revised Atlanta Criteria; SIRS; acute pancreatitis; meta-analysis; prediction; scoring system; severe.

FIGURE 1

PRISMA 2020 flow diagram with included studies and reasons for exclusion.

FIGURE 2

Panel (a) performance of the Bedside Index for Severity in Acute Pancreatitis (BISAP) score in 42 studies with a pre‐test probability of 17.8% and positive and negative likelihood ratios of 4.09 and 0.31, respectively, and the post‐test probability of severe acute pancreatitis (SAP) is 47% when BISAP is positive and 6% when it is negative. With this performance, only 1847 of the 2482 patients who eventually developed SAP would have been correctly identified as true positives, with 635 false negatives; only 9376 of the 11,462 patients experiencing non‐severe AP would have been correctly classified, with 2086 having a false positive prediction of SAP. Panel (b) performance of the Acute Physiology and Chronic Health Examination (APACHE)‐II score in 30 studies with a pre‐test probability of 16.6% and positive and negative likelihood ratios of 3.76 and 0.28, respectively, and the post‐test probability of SAP is 43% when APACHE is positive and 5% when it is negative. With this performance, only 1207 of the 1551 patients who eventually developed SAP would have been correctly identified as true positives, with 344 false negatives, and only 6180 of the 7793 patients experiencing non‐severe AP would have been correctly classified, with 1613 having a false positive prediction of SAP. Panel (c) performance of the Ranson score in 27 studies with a pre‐test probability of 18.8% and positive and negative likelihood ratios of 4.06 and 0.25, respectively. The post‐test probability of SAP is 48% when Ranson is positive and 5% when it is negative. With this performance, only 1511 of the 1889 patients who eventually developed SAP would have been correctly identified as true positives, with 378 false negatives; only 6549 of the 8156 patients experiencing non‐severe AP would have been correctly classified, with 1607 having a false positive prediction of SAP. Panel (d) performance of the Systemic Inflammatory Response Syndrome (SIRS) score in eight studies with a pre‐test probability of 25.3% and positive and negative likelihood ratios of 1.99 and 0.41, respectively, with a post‐test probability of SAP of 40% when SISR was positive and 12% when it was negative. With this performance, only 384 of the 516 patients who eventually developed SAP would have been correctly identified as true positives, with 132 false negatives, and only 955 of the 1523 patients with non‐severe AP would have been correctly classified, with 568 having a false positive prediction of SAP.

FIGURE 3

Performance of the Bedside Index for Severity in Acute Pancreatitis (BISAP), Acute Physiology and Chronic Health Examination (APACHE)‐II, and Ranson scores in 18 studies with a pooled pre‐test probability of 18.7% for severe acute pancreatitis. The post‐test probabilities when the scores were positive were similar: 48% for BISAP, 46% for APACHE‐II, and 50% for Ranson. However, when the scores were negative, the post‐test probability dropped to 7% for BISAP, 6% for Ranson, and 5% for APACHE‐II.