Association between medical academic genealogy and publication outcome: impact of unconscious bias on scientific objectivity

Abstract

Background: Our previous studies suggest that the training history of an investigator, termed "medical academic genealogy", influences the outcomes of that investigator's research. Here, we use meta-analysis and quantitative statistical modeling to determine whether such effects contribute to systematic bias in published conclusions.

Methods: A total of 108 articles were identified through a comprehensive search of the high-grade glioma (HGG) surgical resection literature. Analysis was performed on the 70 articles with sufficient data for meta-analysis. Pooled estimates were generated for key academic genealogies. Monte Carlo simulations were performed to determine whether the effects attributed to genealogy alone can arise due to chance alone.

Results: Meta-analysis of the HGG literature without consideration for academic medical genealogy revealed that gross total resection (GTR) was associated with a significant decrease in the odds ratio (OR) for the hazard of death after surgery for both anaplastic astrocytoma (AA) and glioblastoma (AA: log [OR] = - 0.04, 95% CI [- 0.07 to - 0.01]; glioblastoma log [OR] = - 0.36, 95% CI [- 0.44 to - 0.29]). For the glioblastoma literature, meta-analysis of articles contributed by members of a genealogy consisting of mostly radiation oncologists revealed no reduction in the hazard of death after GTR [log [OR] = - 0.16, 95% CI [- 0.41 to 0.09]. In contrast, meta-analysis of published articles contributed by members of a genealogy consisting of mostly neurosurgeons revealed that GTR was associated with a significant reduction in the hazard of death [log [OR] = - 0.29, 95% CI [- 0.40 to 0.18]. Monte Carlo simulation revealed that the observed discrepancy between the articles contributed by the members of these two genealogies was unlikely to arise by chance alone (p < 0.006).

Conclusions: Meta-analysis of articles contributed by authors belonging to the different medical academic genealogies yielded distinct and contradictory pooled point-estimates, suggesting that genealogy contributes to systematic bias in the published literature.

Keywords: Brain tumor; Medical academic genealogy; Meta-analysis; Scientific objectivity.

Figure 1:

Forest plot demonstrating odds ratio for hazard of death after gross total resection (GTR) in anaplastic astrocytoma (AA). Squares and horizontal bars indicate point estimate and 95% confidence intervals of odds ratio in each study, respectively. Diamonds indicate the summary estimates that are calculated as per random effects model. All values are in logarithmic scale.

Figure 2:

Forest plot demonstrating odds ratio for hazard of death after GTR in glioblastoma. All values are in logarithmic scale.

Figure 3:

Studies demonstrating extent of surgical resection of malignant brain tumors and median survival benefit (in months). GTR: gross total resection; STR: subtotal resection. Genealogy A (studies contributed by the neurosurgeon genealogy); genealogy B (studies contributed by the radiation oncology genealogy).

Figure 4:

Forest plot demonstrating OR for hazard of death in glioblastoma patients who underwent GTR relative to those who underwent STR. Red: studies from the members of neurosurgeon genealogy; blue: studies from the members of radiation oncology genealogy; black: studies contributed by other independent groups. All values are in logarithmic scale.