Testing inter-observer error under a collaborative research framework for studying lithic shape variability

Abstract

Evaluating error that arises through the aggregation of data recorded by multiple observers is a key consideration in many metric and geometric morphometric analyses of stone tool shape. One of the most common approaches involves the convergence of observers for repeat trails on the same set of artefacts: however, this is logistically and financially challenging when collaborating internationally and/or at a large scale. We present and evaluate a unique alternative for testing inter-observer error, involving the development of 3D printed copies of a lithic reference collection for distribution among observers. With the aim of reducing error, clear protocols were developed for photographing and measuring the replicas, and inter-observer variability was assessed on the replicas in comparison with a corresponding data set recorded by a single observer. Our results demonstrate that, when the photography procedure is standardized and dimensions are clearly defined, the resulting metric and geometric morphometric data are minimally affected by inter-observer error, supporting this method as an effective solution for assessing error under collaborative research frameworks. Collaboration is becoming increasingly important within archaeological and anthropological sciences in order to increase the accessibility of samples, encourage dual-project development between foreign and local researchers and reduce the carbon footprint of collection-based research. This study offers a promising validation of a collaborative research design whereby researchers remotely work together to produce comparable data capturing lithic shape variability.

Supplementary information: The online version contains supplementary material available at 10.1007/s12520-022-01676-2.

Keywords: 3D printing; Geometric morphometrics; Inter-observer reliability; Metric measurements; Stone tools.

Fig. 1

The six 3D printed replica tools. Original lithics were knapped and scanned by CS in preparation for 3D printing. Example photos were taken by SH. Scale = 3 cm

Fig. 2

Photographs from the 3D printing process. A The 3D model of the tool is sent to the machine for printing. B The resulting 3D prints once removed from the supports are cleaned using ethanol. 3D printing was carried out by LT and CS

Fig. 3

A schematic of the Elliptic Fourier fitting process that generates the raw shape data for geometric morphometrics. Coefficients of sine and cosine terms (harmonics) are computed to reconstruct the x (blue) and y (red) coordinates from an arbitrary starting point moving along the outline

Fig. 4

Boxplots demonstrating the distribution of length, width and thickness (mm) collected by multiple observers for each tool (1–6)

Fig. 5

Principal component (PC) contributions along the first 3 axes of variance within the multiple observer outline data

Fig. 6

Scatterplots (top row) and boxplots (bottom row) of repeat capture scores along principal components (PC) 1–3, demonstrating the clustering within tools (1–6). PC1 represents 59.7% of the total variance, whilst PC2 and PC3 account for 33.4% and 3%, respectively

Fig. 7

Scatterplots (top row) and boxplots (bottom row) of repeat capture scores along principal components (PC) 1–3, demonstrating the clustering within tools (symbols) and between data sets (colors). PC1 represents 60.4% of the total variance, whilst PC2 and PC3 account for 33.5% and 3.3%, respectively