Early Stopping in Experimentation With Real-Time Functional Magnetic Resonance Imaging Using a Modified Sequential Probability Ratio Test

Abstract

Introduction: Functional magnetic resonance imaging (fMRI) often involves long scanning durations to ensure the associated brain activity can be detected. However, excessive experimentation can lead to many undesirable effects, such as from learning and/or fatigue effects, discomfort for the subject, excessive motion artifacts and loss of sustained attention on task. Overly long experimentation can thus have a detrimental effect on signal quality and accurate voxel activation detection. Here, we propose dynamic experimentation with real-time fMRI using a novel statistically driven approach that invokes early stopping when sufficient statistical evidence for assessing the task-related activation is observed. Methods: Voxel-level sequential probability ratio test (SPRT) statistics based on general linear models (GLMs) were implemented on fMRI scans of a mathematical 1-back task from 12 healthy teenage subjects and 11 teenage subjects born extremely preterm (EPT). This approach is based on likelihood ratios and allows for systematic early stopping based on target statistical error thresholds. We adopt a two-stage estimation approach that allows for accurate estimates of GLM parameters before stopping is considered. Early stopping performance is reported for different first stage lengths, and activation results are compared with full durations. Finally, group comparisons are conducted with both early stopped and full duration scan data. Numerical parallelization was employed to facilitate completion of computations involving a new scan within every repetition time (TR). Results: Use of SPRT demonstrates the feasibility and efficiency gains of automated early stopping, with comparable activation detection as with full protocols. Dynamic stopping of stimulus administration was achieved in around half of subjects, with typical time savings of up to 33% (4 min on a 12 min scan). A group analysis produced similar patterns of activity for control subjects between early stopping and full duration scans. The EPT group, individually, demonstrated more variability in location and extent of the activations compared to the normal term control group. This was apparent in the EPT group results, reflected by fewer and smaller clusters. Conclusion: A systematic statistical approach for early stopping with real-time fMRI experimentation has been implemented. This dynamic approach has promise for reducing subject burden and fatigue effects.

Keywords: SPRT (sequential probability ratio test); adaptive fMRI; dynamic experimentation; early stopping fMRI; real-time fMRI.

FIGURE 1

(A) Sample 1-back protocols demonstrating the two difficulty levels. (B) Block design and timings of each difficulty level.

FIGURE 2

Schematic of the experimental setup of the dynamic real-time fMRI process. The equations were presented to the subject while the scans were acquired using a dedicated computer. FMRI scans were exported in real-time from the scanner computer to the Linux workstation using the Philips XTC program and CORBA interface. Scans were preprocessed on the Linux workstation and SPRT statistics were calculated. The results were relayed back to the stimulus presentation program with an instruction to either continue or terminate the stimulus. The program allowed the flexibility to automatically terminate the presentation of the difficulty level or it could be overridden by the experimenter to continue presenting the stimulus. Note: where hospital firewalls are not an issue, the setup can be simplified.

FIGURE 3

The active voxel counts shown are for the voxels-in-common between the full duration scan and early stop scan and counts are for active voxels unique to either the final scan or early stop scan. Top row: 2-block easy and hard levels, bottom row: 4-block easy and hard levels. Maximum number of possible scans is 238, minimum is 78 scans for 2 blocks first stage of easy and hard stimulus administration or 154 scans for 4 blocks first stage of easy and hard. Information given for the point where 80% of voxels have been classified as either active or non-active. Results reported uses α_E = 0.001, β_E = 0.1, and z = 3.10 (p < 0.001) at first scan after the first stage. The thick black line in each chart indicates the subject groups. Left of the black line (1–12) = control subjects, right of the black line (13–23) = EPT subjects.

FIGURE 4

Estimated standard deviations for the easy task parameter. Plots for 3 example active (left) and non-active (right) voxels from a control subject (subject 3) showing how the estimates decrease over time (scan number). (A) Based on 2-block first stage estimation, (B) 4-block first stage estimation.

FIGURE 5

Comparison of early stopping times using α_E = 0.001 and α_E = 0.0001. Based on 80% of voxels being classified. Both 2-block (top row) and 4-block (bottom row) first stage conditions are presented. β_E = 0.1 throughout. The thick black line in each chart indicates the subject groups. Left of the black line (1–12) = control subjects, right of the black line (13–23) = EPT subjects.

FIGURE 6

A comparison of the early stopping scans at 70, 80, and 90% of voxels classified as either active or non-active. Conducted using α_E = 0.001, β_E = 0.1. Both 2-block (top row) and 4-block (bottom row) first stage conditions are presented. The thick black line in each chart indicates the subject groups. Left of the black line (1–12) = control subjects, right of the black line (13–23) = EPT subjects.

FIGURE 7

Full brain activation maps showing the overlapping voxels between the different stopping points (using 2-blocks first stage, 4-blocks first stage and final scan). Top row shows the easy level and bottom row shows the hard level for 1 subject (number 9). The voxels that are active only at full duration are shown in blue. Those only active after 2-blocks or 4-blocks of stimulus administration are in red. Yellow shows the overlap between full duration and 2-block first stage early stopping scans. Green shows the overlap between full duration and 4-block first stage stopping scans. Thresholded at z = 3.10 at first scan after the first stage. Results overlaid on MNI template, slice z = 56 shown. R = right, A = anterior.

FIGURE 8

Plots showing the number of subjects that pass the framewise displacement threshold of 0.9 mm for each scan. Top: EPT subjects, bottom: control subjects.

FIGURE 9

Group results for the 1-back task. Analysis performed for control and EPT subjects using FSL. Early stopping with 2- and 4-blocks being initially administered is compared to full duration. Activations are overlaid on the MNI template brain. Red (A) = easy level results, Blue (B) hard level results. Group results thresholded at z = 3.10 (p < 0.001). Slice z = 59 is shown. R = right, L = left, A = anterior, P = posterior.