The many roles of decision thresholds for primary research, evidence synthesis, and health decision-making

Abstract

Background: A decision threshold (DT) reflects the point at which a decision or judgment changes, leading to the selection of an action or a commitment for one of several alternatives. Thresholds have always played a role in decision-making. Very small effects may achieve statistical significance yet remain not important to patients or the public. Judgments shift, for instance, from "no or trivial effect" to "small, moderate, and large benefit" with direct implications for decision-making. However, in guideline panels and other clinical or policy decisions, these thresholds are often applied subconsciously when interpreting effect estimates from studies and likely to vary across panel members.

Study design and setting: In this commentary, inspired by the concepts leading to recent publications by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group and its members, we argue that the use of DTs has many advantages.

Results: DTs are the basis for an interpretation of results that is not centered on "statistical significance." In addition, DTs are useful for other aspects of evidence synthesis. The certainty of evidence ratings using the GRADE approach (https://book.gradepro.org/) are centered on DTs, including the determination of the target of the certainty rating, with advantages for transparency, objectivity, and simplicity. For example, judging imprecision is informed by DTs. Specifically, the number of DTs crossed by the plausible effect sizes, as indicated by its confidence interval, helps determine the degree of uncertainty assigned in a GRADE assessment of imprecision, including the number of levels of certainty a user rates down. DTs have also altered the way how users can transparently integrate bodies of evidence from both nonrandomized and randomized studies. Once determined, DTs can be used to validate automated judgments about the certainty of evidence. Beyond these developments, DTs can be useful for designing primary research. For example, sample size calculations could use standardized DTs for large effects when there are known harms that the intended benefits need to outweigh.

Conclusions: DTs have many roles in interpretation, certainty assessments and research planning and design.

Plain language summary: Decision thresholds are the points where a decision changes-for example, when evidence shifts our judgment from "moderate benefit" to "large benefit." Unlike statistical significance, decision thresholds focus on what matters to citizens and decision-makers. In health guidelines, these thresholds often influence judgments unconsciously, but making them explicit improves transparency and consistency. The GRADE approach uses decision thresholds to judge how certain we are about evidence, helping to make these judgments clearer and more objective. They can also guide research design, such as calculating sample sizes. Although identifying thresholds takes some effort, it ultimately makes evidence assessment and decision-making more efficient and structured.

Keywords: Decision-making; GRADE; Guidelines; Health technology assessment; Recommendations; Sample size.