Does interaction matter? Testing whether a confidence heuristic can replace interaction in collective decision-making

Abstract

In a range of contexts, individuals arrive at collective decisions by sharing confidence in their judgements. This tendency to evaluate the reliability of information by the confidence with which it is expressed has been termed the 'confidence heuristic'. We tested two ways of implementing the confidence heuristic in the context of a collective perceptual decision-making task: either directly, by opting for the judgement made with higher confidence, or indirectly, by opting for the faster judgement, exploiting an inverse correlation between confidence and reaction time. We found that the success of these heuristics depends on how similar individuals are in terms of the reliability of their judgements and, more importantly, that for dissimilar individuals such heuristics are dramatically inferior to interaction. Interaction allows individuals to alleviate, but not fully resolve, differences in the reliability of their judgements. We discuss the implications of these findings for models of confidence and collective decision-making.

Keywords: Collective decision-making; Computational; Confidence; Heuristic; Interaction; Metacognition; Perception; Reaction time; Signal detection theory.

Fig. 1

Schematic of one experimental trial. On each trial, dyad members briefly viewed two intervals, each containing six contrast gratings. In one of the two intervals, one of the six gratings had a slightly higher level of contrast (see encircled grating). After viewing the two intervals, dyad members privately used a vertical bar to indicate which interval they thought contained the target and their confidence in this decision. Their responses were then shared, with keyboard response shown in blue (dotted line) and mouse response shown in yellow (dashed line). If they independently selected the same interval, they received feedback (colour-coded) and continued to the next trial. If they privately selected different intervals, they were first asked to make a joint decision, using an additional white vertical bar.

Fig. 2

Example of a psychometric function. The x-axis shows Δc, the contrast level in the second interval minus the contrast level in the first interval at the target location; negative values correspond to targets in the first interval and positive values correspond to targets in the second interval. The y-axis shows the proportion of trials in which the target was reported to be in the second interval. A highly sensitive observer would produce a steeply rising psychometric function with a large slope.

Fig. 3

Example of an ROC curve. The x-axis and the y-axis show the cumulative probabilities P(confidence | incorrect) and P(confidence | correct), respectively. The sum of the shaded areas provides an estimate of metacognitive accuracy. The more bowed the curve, the higher the metacognitive accuracy (i.e. the probability of high confidence given correct rises more rapidly than the probability of high confidence given incorrect).

Fig. 4

The collective benefit obtained from the MCS and MRTS algorithms depended on the similarity of dyad members’ sensitivities. The x-axis shows the ratio of the sensitivity of the worse dyad member relative to that of the better dyad member (S_min/S_max), with values near one corresponding to dyad members of nearly equal sensitivity. The y-axis shows (A) the ratio of the sensitivity of the MCS algorithm relative to that of the more sensitive dyad member (S_MCS/S_max) and (B) the ratio of the sensitivity of the MRTS algorithm relative to that of the more sensitive dyad member (S_MRTS/S_max), with values above one indicating a collective benefit over the more sensitive dyad member.

Fig. 5

Interacting dyad members took each other’s metacognitive ability into account when making joint decisions. The x-axis shows the ratio of the A_ROC of dyad member A relative to that of dyad member B. The y-axis shows the fraction of disagreement trials in which the dyad followed the decision of dyad member A instead of that of dyad member B.

Fig. 6

The relative benefit for interaction over the MCS and MRTS algorithms depended on the similarity of dyad members’ sensitivities. The x-axis shows the ratio of the sensitivity of the worse dyad member relative to that of the better dyad member (S_min/S_max), with values near one corresponding to dyad members of nearly equal sensitivity. The y-axis shows (A) the ratio of the sensitivity of the empirical dyad relative to that of the MCS algorithm (S_emp/S_MCS) and (B) the ratio of the sensitivity of the empirical dyad relative to that of the MRTS algorithm (S_emp/S_MRTS), with values above one indicating a relative benefit for interaction.

Fig. 7

The effect of normalising confidence estimates depended on the similarity of dyad members’ sensitivities. The x-axis shows the ratio of dyad members’ sensitivities (S_min/S_max), with values near one corresponding to dyad members of nearly equal sensitivity. The y-axis shows the ratio of the sensitivity of the MCS algorithm using normalised confidence estimates relative to that of the MCS algorithm using raw confidence estimates (S_MCS/S_rawMCS), with values above the horizontal line indicating a relative benefit for normalising confidence estimates.