Towards open, reliable, and transparent ecology and evolutionary biology

Abstract

Unreliable research programmes waste funds, time, and even the lives of the organisms we seek to help and understand. Reducing this waste and increasing the value of scientific evidence require changing the actions of both individual researchers and the institutions they depend on for employment and promotion. While ecologists and evolutionary biologists have somewhat improved research transparency over the past decade (e.g. more data sharing), major obstacles remain. In this commentary, we lift our gaze to the horizon to imagine how researchers and institutions can clear the path towards more credible and effective research programmes.

Fig. 1

. The strained researcher is tugged away from their ideals by the incentives of the institutions they rely upon for employment and promotion. Practices and behaviours on the left-hand side of the tug-of-war (shaded orange) depict problems of the status quo, where research is focussed more on publishing papers than answering questions. Preferred practices and behaviours on the right-hand side of the tug-of-war (shaded blue) depict a vision for efficient and collaborative science aimed at credibly answering questions. To shift research practices towards reliability, three types of institutional incentives could change, as shown by grey boxes underneath the tug-of-war. First, journals and funders could quickly encourage validation of original research by publishing and funding replication studies. Less likely, journals could publish fewer, more comprehensive and coherent research programmes (both long-term studies and collections of smaller studies on the same research topic), thereby relieving pressures to oversell the importance of small studies. Second, employers could hire individuals with specialised expertise (e.g. data stewards, empiricists, statisticians, and writers), whose employment does not depend on particular research outcomes. Reducing the pyramid structure of academic career paths might promote a more diverse workforce that—without the pressure to maintain professional brands—could be quicker to discard discredited beliefs. Third, funding agencies could curb the benefits of self-promotion and irreproducible results by funding diverse teams, science maintenance (e.g. validation and error detection) as much as innovation, and by selecting randomly from projects that pass particular thresholds (i.e. grant lotteries). Grant lotteries are already being trialled by multiple funding agencies (e.g. the Fetzer Franklin Fund, the Health Research Council of New Zealand, and the Swiss National Science Foundation), but their effects on the reliability of research will depend on which metrics are used to select entrants into the lottery

Fig. 2

Three areas for reform to relieve research strain, outstanding questions for meta-research, and possible answers. Error detection: researchers need to be able to distinguish between reliable and unreliable research. A better system of quality control (both prior to and post-publication) might discourage research practices that inflate the rate of false-positive findings in the research literature (e.g. selective reporting; p-hacking; HARKing). At the same time, there should be incentives for researchers to remedy mistakes in their previous work, for example, through ‘living’ papers that can be easily updated. A more drastic change would be to require self-contained studies to be replicated, and for published results from long-term field studies to be revisited in subsequent years (e.g. before funding is renewed). Theory development: research in ecology and evolutionary biology sometimes fails to traverse the space between speculation and theory. In addition to hypothesis testing, answering big questions requires space for descriptive and exploratory research [10]. Detailed descriptions of natural history help calibrate theoretical models, and predictions of models should be tested in natural settings. To specify conditions under which findings are expected to replicate, authors can include ‘constraints on generality’ statements alongside inferences. When un-expected results are attributed to ‘context dependence’, specific contexts can be tested with new data. For cumulative research, foundational studies can be validated with close replications, and their generality assessed in different settings. Human resources: education programmes could increase the ability of researchers to work transparently and reproducibly, but honing these skills and conducting rigorous research is too often unrewarded. Any change to evaluation metrics requires careful consideration and measurement of unintended consequences (e.g. how to ensure costs are not disproportionately borne by less well-resourced research groups and universities). Much published research represents independent projects conducted by trainees, but reliability might be increased by coordinating multiple trainees on the same projects (including replication projects) and providing secure employment to people with specialised expertise (who can be professionally indifferent to the outcome of a particular study)