How should social mixing be measured: comparing web-based survey and sensor-based methods

Abstract

Background: Contact surveys and diaries have conventionally been used to measure contact networks in different settings for elucidating infectious disease transmission dynamics of respiratory infections. More recently, technological advances have permitted the use of wireless sensor devices, which can be worn by individuals interacting in a particular social context to record high resolution mixing patterns. To date, a direct comparison of these two different methods for collecting contact data has not been performed.

Methods: We studied the contact network at a United States high school in the spring of 2012. All school members (i.e., students, teachers, and other staff) were invited to wear wireless sensor devices for a single school day, and asked to remember and report the name and duration of all of their close proximity conversational contacts for that day in an online contact survey. We compared the two methods in terms of the resulting network densities, nodal degrees, and degree distributions. We also assessed the correspondence between the methods at the dyadic and individual levels.

Results: We found limited congruence in recorded contact data between the online contact survey and wireless sensors. In particular, there was only negligible correlation between the two methods for nodal degree, and the degree distribution differed substantially between both methods. We found that survey underreporting was a significant source of the difference between the two methods, and that this difference could be improved by excluding individuals who reported only a few contact partners. Additionally, survey reporting was more accurate for contacts of longer duration, and very inaccurate for contacts of shorter duration. Finally, female participants tended to report more accurately than male participants.

Conclusions: Online contact surveys and wireless sensor devices collected incongruent network data from an identical setting. This finding suggests that these two methods cannot be used interchangeably for informing models of infectious disease dynamics.

Figure 1

Models of contact reporting probabilities. (a) Contact reporting probabilities based on pairs of survey reports (^ss), where a circle stands for a contact reported (grey) or not reported (white) by a survey participant. There are four possible combinations: (i) ${\mathit{N}}_1^{\mathit{ss}}$ where both contact partners report the contact, (ii) ${\mathit{N}}_2^{\mathit{ss}}$ and (iii) ${\mathit{N}}_3^{\mathit{ss}}$ where only one of the contact partners report the contact (since ${\mathit{N}}_2^{\mathit{ss}}$ and ${\mathit{N}}_3^{\mathit{ss}}$ are indistinguishable, we use their sum ${\mathit{N}}_{2+3}^{\mathit{ss}}$), and (iv) ${\mathit{N}}_4^{\mathit{ss}}$ where none of the two contact partners report the contact. (b) All possible combinations of survey reporting statuses, where P stands for the probability of reporting a contact and Q is the complement. (c) Survey reports and mote detections of contacts combined (^sm), where a rectangle stands for a contact detected (grey) or not detected (white) by a pair of motes. There are four possible combinations: (i) ${\mathit{N}}_1^{\mathit{sm}}$ where a contact pair is reported in the survey and recorded by motes, (ii) ${\mathit{N}}_2^{\mathit{sm}}$ and (iii) ${\mathit{N}}_3^{\mathit{sm}}$ where only the survey or motes recorded the contact, and (iv) ${\mathit{N}}_4^{\mathit{sm}}$ where a contact that actually took place was neither reported nor detected by a mote. (d)P_s and P_m stand for average survey and mote reporting probabilities, and Q_s and Q_m are the average probability of a survey or mote not recording a contact respectively. (e) The estimated proportion of the difference between survey and mote data that can be attributed to survey underreporting, based on the models (b) and (d). Here, ${\mathit{N}}_{1\text{'}}^{\mathit{sm}}$ is the estimated amount of contacts detected by the motes and also reported by the study participants if there was no underreporting (estimate is based on P), ${\mathit{N}}_{1\text{'}}^{\mathit{sm}}-{\mathit{N}}_1^{\mathit{sm}}$ is the estimated amount of mote-detected but not survey-reported contacts due to underreporting, and ${\mathit{N}}_{3\text{'}}^{\mathit{sm}}$, is the estimated amount of non-reporting due to differences in contact definitions between the survey and mote studies.

Figure 2

Survey degree by day of data submission. Box-and-whisker plots showing the reported degree distribution by day of data submission. Day 1 refers to the first day of data collection, March 14.

Figure 3

Mote versus survey degree. Left column (a, c, e): kernel density estimates for survey (red) and mote (blue) degree distributions. Right column (b, d, f): jittered scatterplots of the nodal degrees as measured by the mote study versus the nodal degrees as measured by the survey study. 1st row (a and b): all contacts included; 2nd row (c and d): only contacts longer than 5 minutes included; 3rd row (e and f): only contacts longer than 15 minutes included.

Figure 4

Individual reporting probabilities. Distributions of P_s (calculated for all individuals separately) rendered as box-and-whisker plots. P_s values are provided separately for the four duration categories < 5 minutes, 5 to 15 minutes, 15 to 60 minutes, and 1 to 4 hours. The red box-and-whisker plots show the distributions for all individuals, the olive ones for individuals who reported at least two contacts, the green ones for individuals who reported at least three contacts, the blue ones for individuals with at least four contacts, and the pink ones for individuals with at least five contacts. The whiskers extend from the median line to the highest/lowest value that is within 1.5 times the interquartile range.