A methodological checklist for fMRI drug cue reactivity studies: development and expert consensus

Abstract

Cue reactivity is one of the most frequently used paradigms in functional magnetic resonance imaging (fMRI) studies of substance use disorders (SUDs). Although there have been promising results elucidating the neurocognitive mechanisms of SUDs and SUD treatments, the interpretability and reproducibility of these studies is limited by incomplete reporting of participants' characteristics, task design, craving assessment, scanning preparation and analysis decisions in fMRI drug cue reactivity (FDCR) experiments. This hampers clinical translation, not least because systematic review and meta-analysis of published work are difficult. This consensus paper and Delphi study aims to outline the important methodological aspects of FDCR research, present structured recommendations for more comprehensive methods reporting and review the FDCR literature to assess the reporting of items that are deemed important. Forty-five FDCR scientists from around the world participated in this study. First, an initial checklist of items deemed important in FDCR studies was developed by several members of the Enhanced NeuroImaging Genetics through Meta-Analyses (ENIGMA) Addiction working group on the basis of a systematic review. Using a modified Delphi consensus method, all experts were asked to comment on, revise or add items to the initial checklist, and then to rate the importance of each item in subsequent rounds. The reporting status of the items in the final checklist was investigated in 108 recently published FDCR studies identified through a systematic review. By the final round, 38 items reached the consensus threshold and were classified under seven major categories: 'Participants' Characteristics', 'General fMRI Information', 'General Task Information', 'Cue Information', 'Craving Assessment Inside Scanner', 'Craving Assessment Outside Scanner' and 'Pre- and Post-Scanning Considerations'. The review of the 108 FDCR papers revealed significant gaps in the reporting of the items considered important by the experts. For instance, whereas items in the 'General fMRI Information' category were reported in 90.5% of the reviewed papers, items in the 'Pre- and Post-Scanning Considerations' category were reported by only 44.7% of reviewed FDCR studies. Considering the notable and sometimes unexpected gaps in the reporting of items deemed to be important by experts in any FDCR study, the protocols could benefit from the adoption of reporting standards. This checklist, a living document to be updated as the field and its methods advance, can help improve experimental design, reporting and the widespread understanding of the FDCR protocols. This checklist can also provide a sample for developing consensus statements for protocols in other areas of task-based fMRI.

Extended Data Fig. 1 |. Inter-rater reliability for individual checklist items.

Inter-rater reliability assessed by Fleiss’ Kappa for each ENIGMA ACRI checklist item, calculated on the basis of the assessment of reporting status of the checklist items among 108 papers by three independent raters.

Extended Data Fig. 2 |. Relationships between reporting score and publication context.

a, Relation between the reporting score of each article and its word count. (Note that article word count is not exactly accurate, because it is measured by counting the words from the beginning of the introduction to the end of the discussion; thus, it might include the running title of each page, footnotes and the captions of figures and tables.) b, Relation between the reporting score of each article and its journal word limit. (Note that the word limitation for journals with no word limitation is counted as 15,000.) c, Relation between the reporting score of each article with journal impact factor. d, Article reporting scores across the years. The relations in panels a, b and c were assessed using linear regressions, whereas a Kruskal-Wallis test was performed for panel d.

Fig. 1 |. Schematic representation of key reportable aspects of an fMRI drug cue reactivity study.

1. Participants are recruited on the basis of explicit criteria, and baseline data are collected on participant demographics, handedness, psychiatric history and substance use history. 2. Participants undergo fMRI scanning with carefully selected hardware and software parameters, and data are analyzed through specified preprocessing and analysis pipelines for statistical inference. 3. Participants engage with drug and neutral cues during fMRI scanning, with cues of specified durations presented in events and/or blocks with a chosen temporal architecture. 4. These cues stimulate one or more sensory modalities and are typically matched in terms of psychological characteristics, such as induced arousal or valence, and/or physical characteristics, such as saturation and hue for pictorial cues. 5. and 6. Participants provide craving self-reports outside and/or inside the scanner, using various short and long-form instruments and hardware such as response boxes or joysticks. 7. In addition to pre-scanning sources of between-study variance such as task instructions and scanner familiarization, there are important post-scanning safety procedures such as craving-management interventions and additional assessments before participants leave the imaging center.

Fig. 2 |. A schematic of the entire Delphi study methodology.

The process has been roughly divided into distinct stages: the selection of the SC (in black) using the results of an earlier mentioned systematic review to choose the initial checklist items and expert committee candidates (in pink), checklist development phase (in red), expert panel selection (in purple), checklist commenting and revision phase (in green), checklist rating phase (in yellow) and data analysis and Delphi process finalization (in blue). The number of contributors to each section is displayed by ‘n =‘. To the left of the main graph, an overview of the structure of the checklist at each stage is presented. recom, recommendations.

Fig. 3 |. Ratings for 38 items in seven categories.

This figure depicts the rating of 49 raters (11 from the steering committee and 38 from the expert panel) for the checklist items. Each item was rated from 1 to 5 (not important to extremely important). All the items met threshold 1 and were rated as moderately, highly or extremely important by >70% of the raters. In addition, 24 items reached the more-stringent threshold 2 of being rated as either highly or extremely important by 80% of raters (the ones that did not reach this threshold are marked with ‘†’). Items are represented by their summary in the figure. Full text of the items is provided in Tables 1–6.

Fig. 4 |. Ratings for 75 additional recommendations in seven categories.

This figure depicts the rating of 49 raters (11 from the steering committee and 38 from the expert panel) for the checklist additional recommendations. Each additional recommendation was rated either ‘Yes’ or ‘No’ on the question of whether it should be included as a recommendation. Recommendations are represented by their summary in the figure. Full text of the recommendations is provided in Tables 1–6.

Fig. 5 |. State of reproducibility/transparency in fMRI drug cue reactivity research in the context of the ENIGMA-ACRI checklist.

Assessments by three independent raters on the basis of 108 FDCR articles. a, Percentage of articles that reported each checklist item. Note that the percentages are calculated out of applicable items for each article. For example, craving-rating technology was not applicable for an article without craving rating. b, Percentage of overall reporting status of articles.