Reporting checklists in neuroimaging: promoting transparency, replicability, and reproducibility

Abstract

Neuroimaging plays a crucial role in understanding brain structure and function, but the lack of transparency, reproducibility, and reliability of findings is a significant obstacle for the field. To address these challenges, there are ongoing efforts to develop reporting checklists for neuroimaging studies to improve the reporting of fundamental aspects of study design and execution. In this review, we first define what we mean by a neuroimaging reporting checklist and then discuss how a reporting checklist can be developed and implemented. We consider the core values that should inform checklist design, including transparency, repeatability, data sharing, diversity, and supporting innovations. We then share experiences with currently available neuroimaging checklists. We review the motivation for creating checklists and whether checklists achieve their intended objectives, before proposing a development cycle for neuroimaging reporting checklists and describing each implementation step. We emphasize the importance of reporting checklists in enhancing the quality of data repositories and consortia, how they can support education and best practices, and how emerging computational methods, like artificial intelligence, can help checklist development and adherence. We also highlight the role that funding agencies and global collaborations can play in supporting the adoption of neuroimaging reporting checklists. We hope this review will encourage better adherence to available checklists and promote the development of new ones, and ultimately increase the quality, transparency, and reproducibility of neuroimaging research.

Fig. 1. Addiction Cue-Reactivity Initiative (ACRI) fMRI Drug Cue Reactivity (FDCR) checklist development process and outcomes.

a Procedure flowchart The process has been roughly divided into distinct stages: the selection of the steering committee (in black) using the results of an earlier mentioned systematic review to choose the initial checklist items and expert panel candidates (in pink), checklist development phase (in red), expert panel selection (in purple), checklist commenting and revision phase (in green), checklist rating phase (in yellow) and data analysis and Delphi process finalization (in blue). The number of contributors to each section is displayed by ‘n =’. Within the graph, an overview of the structure of the checklist at each stage is presented in terms of number of categories, essential items to be reported and further recommendations and their categories. recom: recommendations. b Checklist Rating by Experts: Each item was rated from 1 to 5 (not important to extremely important). All the items met threshold 1 and were rated as moderately, highly, or extremely important by >70% of the raters. In addition, 24 items reached the more stringent threshold 2 of being rated as either highly or extremely important by 80% of raters (the ones that did not reach this threshold are marked with ‘†’) (the figure is adopted and modified from [15]).

Fig. 2. Development cycle of reporting checklists.

The inner circle depicts major steps with an example of relevant activities/tasks shown in the middle circle followed by an example of tools/mechanisms shown in the outer circle.

Fig. 3. Checklist development process and research process coverage in sample neuroimaging checklists.

The top panel assesses 6 sample checklists using the developmental cycle model. Dark blue means that the checklist fully reported that item in their initial publication, Light blue means partially reported, and Light yellow means not reported. The bottom panel shows the main steps in a research process and how various checklists covered those steps in their items. Light yellow indicates no coverage, Light blue indicates indirect coverage, and Dark blue indicates full coverage. ACRI-FDCR Addiction Cue-Reactivity Initiative fMRI drug cue-reactivity checklist, COBDIAS Committee on Best Practices in Data Analysis and Sharing, ContES Concurrent tES fMRI Checklist, MRI Magnetic Resonance Imaging, MEEG Magneticencephalography/Eelectroencephalography, MRSinMRS Minimum Reporting Standards for in vivo Magnetic Resonance Spectroscopy.

Fig. 4. Reporting status of the Addiction Cue-Reactivity Initiative (ACRI) fMRI Drug Cue Reactivity (FDCR) checklist.

This figure provides an overview of the reporting status of studies that employed the ACRI-FDCR checklist. The assessments are based on the analysis of 108 sample FDCR articles before the publication of the ACRI-FDCR checklist. a displays the percentage of articles that reported each specific checklist item. Each bar in this panel represents a checklist item, showing how frequently it was included in the published studies. b summarizes the overall reporting status of the articles as a whole. This panel aggregates the data to give a broader view of how well the articles adhere to the checklist criteria.