Bridging consciousness and cognition in memory and perception: evidence for both state and strength processes

Abstract

Subjective experience indicates that mental states are discrete, in the sense that memories and perceptions readily come to mind in some cases, but are entirely unavailable to awareness in others. However, a long history of psychophysical research has indicated that the discrete nature of mental states is largely epiphenomenal and that mental processes vary continuously in strength. We used a novel combination of behavioral methodologies to examine the processes underlying perception of complex images: (1) analysis of receiver operating characteristics (ROCs), (2) a modification of the change-detection flicker paradigm, and (3) subjective reports of conscious experience. These methods yielded converging results showing that perceptual judgments reflect the combined, yet functionally independent, contributions of two processes available to conscious experience: a state process of conscious perception and a strength process of knowing; processes that correspond to recollection and familiarity in long-term memory. In addition, insights from the perception experiments led to the discovery of a new recollection phenomenon in a long-term memory change detection paradigm. The apparent incompatibility between subjective experience and theories of cognition can be understood within a unified state-strength framework that links consciousness to cognition across the domains of perception and memory.

Figure 1. Illustration of state-strength theory.

Probability density functions (top) for evidence that stimuli come from class A or B (e.g., old or new items in a test of memory; pairs of same or different images in a test of perception). Classic strength theory postulates continuously-varying evidence distributions for items in different classes (normal distributions for A and B). In contrast, state-strength theory proposes that in addition to continuously-varying distributions, items may elicit discrete mental states (uniform distributions for A and B). Predicted receiver operating characteristics (ROCs) are shown on the bottom. ROCs plot the proportion of correct and incorrect responses across different levels of evidence strength. Strength theory predicts curvilinear ROCs with intercepts at (0,0) and (1,1). However, if some items are associated with a discrete mental state, the ROC intercepts will be shifted so that the left y-intercept occurs at a point corresponding to State A and the upper x-intercept occurs at a point corresponding to State B. The resulting ROC reflects a combination of state and strength processes. Parameter estimates in the inset show estimates of state and strength for these hypothetical ROCs. State and strength processes are, respectively, recollection and familiarity in memory, and perceiving and knowing in perception.

Figure 2. Perception of simple and complex visual images.

(A–B) Trial procedures and materials for the first series of experiments. (C–H) ROCs and estimates of Perceiving Same (Ps, the y-intercept), Perceiving Different (Pd, one minus the upper x-intercept) and Knowing (K, the degree of curvilinearity, measured as d′). Average parameter estimates from individuals' ROCs are in the insets; error bars show the standard error of the mean. Perception of simple stimuli (C; trial procedure shown in A) could be accounted for by the strength process of knowing, while perception of complex stimuli such as buildings, faces and fractals (D; trial procedure shown in B) was based on knowing as well as a state of perceiving differences. Subsequent experiments indicated that the results with complex stimuli generalized to different presentation conditions and stimuli, including (E) sequentially presented images without an intervening mask, (F) simultaneously presented images, (G) simultaneously presented images at a duration too brief for eye movements to occur, and (H) arrays of six objects that were either trial-unique or repeated in different combinations over trials.

Figure 3. Dissociating perceiving and knowing.

(A) Examples of global (left) and discrete (right) changes. Buildings were expanded or contracted slightly in the global change condition. A feature was added or removed in the discrete change condition (arrows were not presented in the experiment). The trial procedure was the same as in Fig. 2B. (B) ROCs and parameter estimates revealed a crossover dissociation; Pd was significantly greater for discrete compared to global changes [t(36) = 3.15, p = .003], while K was significantly greater for global compared to discrete changes [t(36) = 2.68, p = .01]. Ps did not differ, t<1. (C) Average quadratic coefficients of ROCs plotted in z-space; error bars show the standard error of the mean. Global change zROCs did not differ significantly from linearity [left global bar, M _quadratic = −0.12, SE = 0.06, t(18) = 1.84, p = .08; right global bar, M _quadratic = −0.04, SE = 0.05, t<1, ns], whereas discrete change zROCs were U-shaped, i.e. had significant positive quadratic components [from left to right, M _quadratic = 0.30, SE = 0.09, t(17) = 3.19, p = .005; M _quadratic = 0.39, SE = 0.16, t(5) = 2.43, p = .05; M _quadratic = 0.15, SE = 0.06, t(21) = 2.48, p = .022; M _quadratic = 0.21, SE = 0.08, t(18) = 2.57, p = .019;], ruling against a UVSD strength theory.

Figure 4. Tracking perceiving and knowing over time.

Same/different confidence responses across repetitions for each trial in the flicker paradigm. Data are shown for the ten individuals (numbers on top of each column of blocks) tested in the discrete change condition (A) and the ten individuals tested in the global change condition (B). ‘Different’ trials are the top row of blocks for each of the discrete and global change conditions; ‘same’ trials are the bottom row of blocks for each condition. Trials are sorted so that the fastest learning trials appear on the bottom of each block. In each block, every row is a trial, and the x-axis represents responses 1 through 10. Unsure responses are green, hotter colors indicate more confident ‘different’ responses; cooler colors indicate more confident ‘same’ responses. In the discrete change condition, the correct identification of differences showed an abrupt, step function transition to high confidence responses. In contrast, in the global change condition, the correct identification of differences showed step function transitions on some trials and more gradual transitions on other trials. The identification of sameness gradually transitioned from low to intermediate to high confidence for both conditions.

Figure 5. Relating step functions and gradual transitions to perceiving and knowing.

(A) ROC estimates of perceiving were highly correlated with the proportion of step function transitions, r = .898, p<.001. (B) ROC estimates of knowing were highly correlated with the proportion of gradual transitions (right), r = .814, p<.001.

Figure 6. ROCs and zROCs over time.

ROCs (left) and zROCs (right) were plotted for the discrete (top row) and global (bottom row) change conditions for each of the ten responses in the flicker paradigm. Average estimates of perceiving and knowing are in the insets of the ROCs; error bars show the standard error of the mean. The discrete change ROCs (A) were consistently linear, and the resulting zROCs (B) were U-shaped. The global change ROCs (C) were curvilinear with an upper x-intercept, and the zROCs (D) did not deviate from linearity.

Figure 7. Subjective availability of perceiving and knowing.

Correlations between ROC estimates and subjective reports of perceiving (A) and knowing (B). There was a positive correlation between ROC estimates and subjective reports of perceiving differences [Pd, r = .87, p<.001; r = .82, p<.001 for the first and second experiments, respectively] and of knowing [r = .67, p<.001; r = .79, p<.001 for the first and second experiments, respectively].

Figure 8. Generalizing from perception to long-term memory.

(A) Example of a scene from the long-term memory change detection paradigm. In the example shown here, the bottom image is missing a few items that are present in the top image, including the keyboard, the mouse, and the wall outlets. (B) ROCs in long-term memory change detection and in a performance-matched standard recognition memory procedure. Change detection compared to standard recognition led to an increase in recollection of new items [Rn; t(52) = 5.37, p<.001] and a decrease in the recollection of old items [Ro; t(52) = 4.67, p<.001] whereas estimates of familiarity did not differ [F; t<1]. Average parameter estimates are in the inset in (B); error bars show the standard error of the mean.