Accumulating Evidence and Research Organization (AERO) model: a new tool for representing, analyzing, and planning a translational research program

Abstract

Background: Maximizing efficiency in drug development is important for drug developers, policymakers, and human subjects. Limited funds and the ethical imperative of risk minimization demand that researchers maximize the knowledge gained per patient-subject enrolled. Yet, despite a common perception that the current system of drug development is beset by inefficiencies, there remain few approaches for systematically representing, analyzing, and communicating the efficiency and coordination of the research enterprise. In this paper, we present the first steps toward developing such an approach: a graph-theoretic tool for representing the Accumulating Evidence and Research Organization (AERO) across a translational trajectory.

Methods: This initial version of the AERO model focuses on elucidating two dimensions of robustness: (1) the consistency of results among studies with an identical or similar outcome metric; and (2) the concordance of results among studies with qualitatively different outcome metrics. The visual structure of the model is a directed acyclic graph, designed to capture these two dimensions of robustness and their relationship to three basic questions that underlie the planning of a translational research program: What is the accumulating state of total evidence? What has been the translational trajectory? What studies should be done next?

Results: We demonstrate the utility of the AERO model with an application to a case study involving the antibacterial agent, moxifloxacin, for the treatment of drug-susceptible tuberculosis. We then consider some possible elaborations for the AERO model and propose a number of ways in which the tool could be used to enhance the planning, reporting, and analysis of clinical trials.

Conclusion: The AERO model provides an immediate visual representation of the number of studies done at any stage of research, depicting both the robustness of evidence and the relationship of each study to the larger translational trajectory. In so doing, it makes some of the invisible or inchoate properties of the research system explicit - helping to elucidate judgments about the accumulating state of evidence and supporting decision-making for future research.

Figure 1

Consistency of results. Studies are shown as vertices. The graph shows eight experiments across five years with some degree of consistency evident at every phase of research: Both of the in vitro studies, two of the three in vivo, and two of the three phase 1 studies were positive. There is also some inconsistency within in vivo (β₁) and phase 1 (γ₂); nevertheless, the accumulating state of evidence is largely positive and the transitions between phases appear relatively smooth.

Figure 2

Concordance in a trajectory. Edges show the intellectual lineage. The graph shows eleven experiments across six years. The edges between studies represent the intellectual lineage between them and illustrate the translational research trajectories. Some trajectories are perfectly concordant (for example, α₁→α₂→β₃→γ₃→δ₂), while others show discordance (for example, α₁→β₁ or α₁→α₂→β₂→γ₁→δ₁). The phase 2 studies are also highly inconsistent (that is, one positive, one negative, and one inconclusive study), indicating a relatively rough transition into this phase. Given that phase 2 was initiated after only one positive phase 1 trial, this may indicate that the threshold of evidence used to transition into phase 2 is too low and ought to require at least some degree of consistency.

Figure 3

Planning future studies. The graph shows twelve completed experiments across six years along with two contemplated future studies, a fourth phase 2 trial (δ₄) and a second phase 3 trial (ϵ₂). A phase 3 trial (ϵ₁) was initiated following the sole positive phase 2 study (δ₂). The result of this phase 3 trial was negative, discordant with the earlier phase 2 result. Now researchers must decide which study (or studies) to do next: Trust that the accumulated evidence is still sufficient to motivate another phase 3 or return to phase 2 in search of greater consistency and a potential explanation for the discordance between δ₂ and ϵ₁.

Figure 4

Complete AERO graph for moxifloxacin in an anti-tuberculosis regimen. The graph shows 19 completed experiments across 12 years along with four contemplated future studies. The overall trend of study results was positive until the transition into phase 2 (x₁), when significant discordance (for example, w₄→x₁ and w₆→x₄) and inconsistency (that is, negative results in x₁,x₄ vs. positive results in x₂,x₃,x₅) began to emerge. Researchers must now decide how to proceed in the face of an equivocal state of total evidence: Investigate mechanisms of discordance between animal models and human trials (A); investigate drug interactions (B); further investigate efficacy and evaluate predictivity of specific phase 2 trial designs (C); or proceed to a decisive phase 3 effectiveness trial (D).