Quantifying collective intelligence in human groups

Abstract

Collective intelligence (CI) is critical to solving many scientific, business, and other problems, but groups often fail to achieve it. Here, we analyze data on group performance from 22 studies, including 5,279 individuals in 1,356 groups. Our results support the conclusion that a robust CI factor characterizes a group's ability to work together across a diverse set of tasks. We further show that CI is predicted by the proportion of women in the group, mediated by average social perceptiveness of group members, and that it predicts performance on various out-of-sample criterion tasks. We also find that, overall, group collaboration process is more important in predicting CI than the skill of individual members.

Keywords: collective intelligence; human groups; team performance.

Fig. 1.

Study process. Step 1: Using the POGS, we conducted 22 studies involving 1,356 groups, 5,279 individuals, and four to eight tasks per study. Step 2: We perform meta-analytic factor analysis (across the 22 studies) and leave-one-out analysis to test the robustness of evidence of a general CI factor that explains a group’s performance. Step 3: We use a variety of predictors, including demographics, skill, social perceptiveness, and process measures to predict CI and to assess the relative predictive power of each set of predictors.

Fig. 2.

CI factor analysis and prediction. (A) Raw Pearson correlations between tasks and CI of the pooled data (all correlations are significant with at least P < 0.027). (B) Standardized factor loadings of the meta-analysis of each task on the first factor (CI). (C) Treating each of the eight tasks as a criterion task, we repeat the meta factor analysis (using the remaining seven tasks) to compute a restricted CI factor and predict the excluded criterion task (Pearson correlation with 95% confidence interval). (D) Regression coefficients for four different linear models predicting CI. Proportion of female group members is a significant predictor in models that do not control for Social Perceptiveness (showing coefficients from SI Appendix, Table S5).

Fig. 3.

Variable importance in predicting CI. Variable importance based on a random forest prediction model computed as the decrease of accuracy in prediction when a given variable is excluded from the model and expressed relative to the maximum.