Examining the validity and utility of two secondary sources of food environment data against street audits in England

Abstract

Background: Secondary data containing the locations of food outlets is increasingly used in nutrition and obesity research and policy. However, evidence evaluating these data is limited. This study validates two sources of secondary food environment data: Ordnance Survey Points of Interest data (POI) and food hygiene data from the Food Standards Agency (FSA), against street audits in England and appraises the utility of these data.

Methods: Audits were conducted across 52 Lower Super Output Areas in England. All streets within each Lower Super Output Area were covered to identify the name and street address of all food outlets therein. Audit-identified outlets were matched to outlets in the POI and FSA data to identify true positives (TP: outlets in both the audits and the POI/FSA data), false positives (FP: outlets in the POI/FSA data only) and false negatives (FN: outlets in the audits only). Agreement was assessed using positive predictive values (PPV: TP/(TP + FP)) and sensitivities (TP/(TP + FN)). Variations in sensitivities and PPVs across environment and outlet types were assessed using multi-level logistic regression. Proprietary classifications within the POI data were additionally used to classify outlets, and agreement between audit-derived and POI-derived classifications was assessed.

Results: Street audits identified 1172 outlets, compared to 1100 and 1082 for POI and FSA respectively. PPVs were statistically significantly higher for FSA (0.91, CI: 0.89-0.93) than for POI (0.86, CI: 0.84-0.88). However, sensitivity values were not different between the two datasets. Sensitivity and PPVs varied across outlet types for both datasets. Without accounting for this, POI had statistically significantly better PPVs in rural and affluent areas. After accounting for variability across outlet types, FSA had statistically significantly better sensitivity in rural areas and worse sensitivity in rural middle affluence areas (relative to deprived). Audit-derived and POI-derived classifications exhibited substantial agreement (p < 0.001; Kappa = 0.66, CI: 0.63-0.70).

Conclusions: POI and FSA data have good agreement with street audits; although both datasets had geographic biases which may need to be accounted for in analyses. Use of POI proprietary classifications is an accurate method for classifying outlets, providing time savings compared to manual classification of outlets.

Keywords: Foodscape; Sensitivity; Validity; ‘Administrative data’; ‘Commercial business list’; ‘Obesogenic environments’; ‘Positive predictive value’; ‘Retail food environment’; ‘Secondary data’; ‘Street audit’.

Fig. 1

Flow chart detailing data processing procedure. POI: Points of Interest data; FSA: Food Standards Agency Data

Fig. 2

Venn diagram illustrating the classification of outlets as true positives (TP), false positives (FP) and false negatives (FN). The left-hand oval represents all outlets identified in the audits, and the right-hand oval represents all outlets identified by the secondary data (POI or FSA). The region of overlap depicts outlets that were identified in both the audits and the dataset. The figure also shows the equations used to calculate sensitivity statistics and positive predictive values (PPV) and their respective odds, where P(X) represents the probability of event X

Fig. 3

Positive Predictive Values (PPV) and sensitivities for FSA and POI data. * statistically significant difference between datasets (p < 0.05). FSA: Food Standards Agency data. POI: Points of Interest data. PPV: positive predictive values