Toward a new model of scientific publishing: discussion and a proposal

Abstract

The current system of publishing in the biological sciences is notable for its redundancy, inconsistency, sluggishness, and opacity. These problems persist, and grow worse, because the peer review system remains focused on deciding whether or not to publish a paper in a particular journal rather than providing (1) a high-quality evaluation of scientific merit and (2) the information necessary to organize and prioritize the literature. Online access has eliminated the need for journals as distribution channels, so their primary current role is to provide authors with feedback prior to publication and a quick way for other researchers to prioritize the literature based on which journal publishes a paper. However, the feedback provided by reviewers is not focused on scientific merit but on whether to publish in a particular journal, which is generally of little use to authors and an opaque and noisy basis for prioritizing the literature. Further, each submission of a rejected manuscript requires the entire machinery of peer review to creak to life anew. This redundancy incurs delays, inconsistency, and increased burdens on authors, reviewers, and editors. Finally, reviewers have no real incentive to review well or quickly, as their performance is not tracked, let alone rewarded. One of the consistent suggestions for modifying the current peer review system is the introduction of some form of post-publication reception, and the development of a marketplace where the priority of a paper rises and falls based on its reception from the field (see other articles in this special topics). However, the information that accompanies a paper into the marketplace is as important as the marketplace's mechanics. Beyond suggestions concerning the mechanisms of reception, we propose an update to the system of publishing in which publication is guaranteed, but pre-publication peer review still occurs, giving the authors the opportunity to revise their work following a mini pre-reception from the field. This step also provides a consistent set of rankings and reviews to the marketplace, allowing for early prioritization and stabilizing its early dynamics. We further propose to improve the general quality of reviewing by providing tangible rewards to those who do it well.

Keywords: neuroscience; peer review; publishing.

Figure 1

Diagram of the current publishing process. (A) Outline of the steps involved in the first submission of a paper to a journal. (1, 2) Authors prepare and submit the manuscript. (3) Journal editors decide whether to reject the paper or send it for review. If the decision is to review, reviews are solicited (4), written by the reviewers (5), and sent back to the editors (6). (7) Editors then decide whether to accept, reject, or allow revision and resubmission of the manuscript. In practice almost no paper is accepted in this first round of review. If the decision is to allow revision, the reviews are sent to the authors (8) and a theoretically unbounded revision loop (B) then proceeds. This loop can terminate in either acceptance or rejection. If the paper is accepted it proceeds to the proof stage, where it is exchanged between the editors and authors (16) until ready for publication in the journal (17). If the paper is rejected at any of the decision points (3, 7, 14) the authors will generally proceed to submit it to another journal, beginning a journal loop (C). (B) Details of the revision loop. In each iteration the authors prepare the revision (9), which is communicated to the reviewers by the editors (10, 11). The reviewers write re-reviews (12) that are sent back to the editors (13). (14) The editors then decide whether to accept, reject, or allow revision. If the decision is revision the loop begins again, and continues until an accept or reject decision is reached. (C) If a paper is rejected, a loop of repeated submissions to many journals begins until the paper is accepted. In practice, few papers are ever abandoned, so the loop generally continues until acceptance an publication. Each new submission has the same step as the original submission (A).

Figure 2

Prioritization of the literature by the current system. Histogram of the distribution of the current number of citations for every neuroscience paper published between 2000 and 2007 for six major journals (15 citation width bins). The x-axis is cutoff at 400 citations only for display purposes. There were a small proportion of papers that had more citations, and these papers were included in all analyses. There are three rough tiers of journals, based on their 2010 impact factors (to the right of the journal names in the legend). Note the large amount of overlap between the distributions; indicating the journal where a paper is published is not strongly predictive of the eventual number of citations it will acquire.

Figure 3

Diagram of the proposed publishing process. (1, 2) Authors prepare and submit the manuscript. As all papers are reviewed, the editors select a set of reviewers and solicit reviews (3). The reviewers write the reviews (4), which are automatically conveyed to the authors (5). Authors then have the choice of revising the manuscript or leaving it unchanged. They prepare the revision and their response to the reviews (6), which is conveyed to the editors (7). The editors select an appropriate editorial board member and convey the reviews, revision, and the author’s response (8). The board member uses these components to craft a high level summary of the work, its importance and context within the literature, and a set of initial numerical seeds representing the quality of the paper (9). This summary is conveyed to the editors (10), who begin the proof exchange with the authors (11). Once the proofs are approved, the editors publish the complete product (12), which is the final manuscript and board summary and seed values.

Figure 4

Post-publication system. (A) A paper in its complete form under the proposed publishing process. Every paper will be comprised first of the manuscript itself (green square). This manuscript will have a standard format comprised of the familiar components along with the editorial board summary and ratings. The manuscript will be stable over time and none of its components subject to change. Associated with this manuscript will be a set of statistics (yellow circle) that include any additional ratings from the field collected post-publication and continually updated counts of its citations and downloads. A forum will also be attached to the manuscript where members of the field can post their own detailed comments on the paper and authors have the opportunity to respond. Members of the field may also pose theoretical and methodological questions that can be answered either by the authors or any other member of the field. Finally, the manuscript also has a literature valence (large blue circle), comprised of its citations and those papers which have cited it as well as topics and uncited work deemed by either the board, the authors, or the field to be related. The valence also includes ID numbers for the authors, editorial board member who provided the summary, and the reviewers. (B) Currently, a literature search generates a relatively unstructured list of papers, organized by author or subject heading. The literature valence will allow for dynamic organizations of the literature based on the needs of a particular researcher. (C) For example, organizing the papers in (B) by citation and topic might reveal that there were actually two sub-fields within the list. One a small set of distinct closely related papers, while the other is a large complex set, centered on a single paper. (D) This organization might differ entirely when organized by the method used, revealing a different set of relationships amongst the papers.