Crowdsourcing: It Matters Who the Crowd Are. The Impacts of between Group Variations in Recording Land Cover

Abstract

Volunteered geographical information (VGI) and citizen science have become important sources data for much scientific research. In the domain of land cover, crowdsourcing can provide a high temporal resolution data to support different analyses of landscape processes. However, the scientists may have little control over what gets recorded by the crowd, providing a potential source of error and uncertainty. This study compared analyses of crowdsourced land cover data that were contributed by different groups, based on nationality (labelled Gondor and Non-Gondor) and on domain experience (labelled Expert and Non-Expert). The analyses used a geographically weighted model to generate maps of land cover and compared the maps generated by the different groups. The results highlight the differences between the maps how specific land cover classes were under- and over-estimated. As crowdsourced data and citizen science are increasingly used to replace data collected under the designed experiment, this paper highlights the importance of considering between group variations and their impacts on the results of analyses. Critically, differences in the way that landscape features are conceptualised by different groups of contributors need to be considered when using crowdsourced data in formal scientific analyses. The discussion considers the potential for variation in crowdsourced data, the relativist nature of land cover and suggests a number of areas for future research. The key finding is that the veracity of citizen science data is not the critical issue per se. Rather, it is important to consider the impacts of differences in the semantics, affordances and functions associated with landscape features held by different groups of crowdsourced data contributors.

Fig 1. Clusters countries with similar definitions of ‘forest’ based on minimum area, tree height and canopy cover.

Fig 2

The distribution of the data a) in the case study area and b) local detail showing the density of the data points shaded with a transparency term and the 50km analysis grid.

Fig 3

Kernel Density Estimation surfaces of the distributions of a) Shrub cover comparing Gondor and Non-Gondor groups and b) Forest cover comparing Expert and Non-Expert groups. Darker areas indicate a greater density of data points and in both cases the Expert and Gondor groups were randomly split and mapped.

Fig 4

a) An example of Geo-Wiki crowdsourced data points within a 50km buffer under a kernel centred on coordinates (0,0) and b) the weighting function used in the geographically weighted average approach.

Fig 5

The land cover data over a 50km grid a) generated from all contributors in the study area and b) with some local detail.

Fig 6

The land cover maps generated by data from a) all contributors, b) contributors from Gondor, c) Non-Gondor and d) a map of difference, with differences in red.

Fig 7. The correspondence matrix of the land cover maps generated from data contributed by Gondor and Non-Gondor subsets.

Diagonal agreement and low levels of off-diagonal disagreement are indicated in white, with increasing levels of off-diagonal disagreement shaded from light to dark orange.

Fig 8

The land cover maps generated by data from a) All Contributors, b) Expert contributors, c) Non-Experts and d) a map of difference, with differences in red.

Fig 9. The correspondence matrix of the land cover maps generated from data contributed by Expert and Non-Expert subsets.

Diagonal agreement and low levels of off-diagonal disagreement are indicated in white, with increasing levels of off-diagonal disagreement shaded from light to dark orange.