Comparison of nutrient profiling models for assessing the nutritional quality of foods: a validation study

Abstract

Nutrient profiling (NP) is a method for evaluating the healthfulness of foods. Although many NP models exist, most have not been validated. This study aimed to examine the content and construct/convergent validity of five models from different regions: Australia/New Zealand (FSANZ), France (Nutri-Score), Canada (HCST), Europe (EURO) and Americas (PAHO). Using data from the 2013 UofT Food Label Information Program (n15342 foods/beverages), construct/convergent validity was assessed by comparing the classifications of foods determined by each model to a previously validated model, which served as the reference (Ofcom). The parameters assessed included associations (Cochran-Armitage trend test), agreement (κ statistic) and discordant classifications (McNemar's test). Analyses were conducted across all foods and by food category. On the basis of the nutrients/components considered by each model, all models exhibited moderate content validity. Although positive associations were observed between each model and Ofcom (all P trend<0·001), agreement with Ofcom was 'near perfect' for FSANZ (κ=0·89) and Nutri-Score (κ=0·83), 'moderate' for EURO (κ=0·54) and 'fair' for PAHO (κ=0·28) and HCST (κ=0·26). There were discordant classifications with Ofcom for 5·3 % (FSANZ), 8·3 % (Nutri-Score), 22·0 % (EURO), 33·4 % (PAHO) and 37·0 % (HCST) of foods (all P<0·001). Construct/convergent validity was confirmed between FSANZ and Nutri-Score v. Ofcom, and to a lesser extent between EURO v. Ofcom. Numerous incongruencies with Ofcom were identified for HCST and PAHO, which highlights the importance of examining classifications across food categories, the level at which differences between models become apparent. These results may be informative for regulators seeking to adapt and validate existing models for use in country-specific applications.

Keywords: CFG Canada’s Food Guide; EURO WHO Regional Office for Europe; FSANZ Food Standards Australia New Zealand; FVNL fruits; HCST Health Canada Surveillance Tool; NP nutrient profiling; PAHO WHO Regional Office for the Americas/Pan American Health Organization; and legumes; nuts; vegetables; Construct validity; Content validity; Convergent validity; Healthfulness; Nutrient profiling; Nutritional quality; Validation.

Fig. 1

Overall associations between the proportions (%, 95 % CI) of foods classified as ‘less healthy’ by the models and quartiles of Ofcom scores (n 15 227; all P<0·001 for trend using the Cochran–Armitage trend test). Data were missing for 0·29–0·41 % (n 44 to 62) of foods across the comparisons with the Ofcom model. , Food Standards Australia New Zealand; , Nutri-Score; , Health Canada Surveillance Tool; , WHO Regional Office for Europe; , WHO Regional Office for the Americas/Pan American Health Organization.

Fig. 2

Overall proportions (%, 95 % CI) of ‘healthier ()’ and ‘less healthy ()’ foods and agreement (κ) between each model and the Ofcom model (n 15 227). Data were missing for 0·01–0·34 % (n 1 to 52) of foods within a model. Agreement was assessed using the κ statistic as follows: 0·01–0·20 ‘slight’; 0·21–0·40 ‘fair’; 0·41–0·60 ‘moderate’; 0·61–0·80 ‘substantial’; 0·81–0·99 ‘near perfect’^{( 39 )}. FSANZ, Food Standards Australia New Zealand; HCST, Health Canada Surveillance Tool; EURO, WHO Regional Office for Europe; PAHO, WHO Regional Office for the Americas/Pan American Health Organization.

Fig. 3

Agreement (κ, 95 % CI) and discordance (%, indicated above each line) between the Food Standards Australia New Zealand (FSANZ) and Ofcom model for all foods (n 15 183; data missing for n 44) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Agreement was assessed using the κ statistic as follows: 0·01–0·20 ‘slight’; 0·21–0·40 ‘fair’; 0·41–0·60 ‘moderate’; 0·61–0·80 ‘substantial’; 0·81–0·99 ‘near perfect’^{( 39 )}. Significant discordance in classifications between models using McNemar’s test (* P<0·05, ** P<0·001).

Fig. 4

Agreement (κ, 95 % CI) and discordance (%, indicated above each line) between the Nutri-Score and Ofcom model for all foods (n 15 183; data missing for n 44) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Agreement was assessed using the κ statistic as follows: 0·01–0·20 ‘slight’; 0·21–0·40 ‘fair’; 0·41–0·60 ‘moderate’; 0·61–0·80 ‘substantial’; 0·81–0·99 ‘near perfect’^{( 39 )}. Significant discordance in classifications between models using McNemar’s test (* P<0·05, ** P<0·001).

Fig. 5

Cross-classification analyses between five Nutri-Score classes v. quintiles of Ofcom scores for all foods (n 15 183; data missing for n 44) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Exact agreement occurs when a food is classified in the same classes/quintiles (e.g. Nutri-Score A and Ofcom quintile 1). Agreement within an adjacent (±1) class/quintile (e.g. Nutri-Score A and Ofcom quintile 2) and disagreement (e.g. Nutri-Score A and Ofcom quintile 3) also were assessed. Gross misclassification occurs when a food is classified in opposing classes/quintiles (e.g. Nutri-Score A and Ofcom quintile 5). , Gross misclassification; , disagreement; , agreement ±1 class/quintile; , exact agreement.

Fig. 6

Agreement (κ, 95 % CI) and discordance (%, indicated above each line) between the Health Canada Surveillance Tool (HCST) and Ofcom model for all foods (n 15 165; data missing for n 62) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Agreement was assessed using the κ statistic as follows: 0·01–0·20 ‘slight’; 0·21–0·40 ‘fair’; 0·41–0·60 ‘moderate’; 0·61–0·80 ‘substantial’; 0·81–0·99 ‘near perfect’^{( 39 )}. Significant discordance in classifications between models using McNemar’s test (* P<0·05, ** P<0·001).

Fig. 7

Cross-classification analyses between four Health Canada Surveillance Tool (HCST) tiers v. quartiles of Ofcom scores for all foods (n 15 165; data missing for n 62) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Exact agreement occurs when a food is classified in the same tiers/quartiles (e.g. HCST tier 1 and Ofcom quartile 1). Agreement within an adjacent (±1) tier/quartile (e.g. HCST tier 1 and Ofcom quartile 2) and disagreement (e.g. HCST tier 1 and Ofcom quartile 3) also were assessed. Gross misclassification occurs when a food is classified in opposing tiers/quartiles (e.g. HCST tier 1 and Ofcom quartile 4). , Gross misclassification; , disagreement; , agreement ±1 tier/quartile; , exact agreement.

Fig. 8

Agreement (κ, 95 % CI) and discordance (%, indicated above each line) between the WHO Regional Office for Europe (EURO) and Ofcom model for all foods (n 15 182; data missing for n45) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Agreement was assessed using the κ statistic as follows: 0·01–0·20 ‘slight’; 0·21–0·40 ‘fair’; 0·41–0·60 ‘moderate’; 0·61–0·80 ‘substantial’; 0·81–0·99 ‘near perfect’^{( 39 )}. Significant discordance in classifications between models using McNemar’s test (* P<0·01, ** P<0·001). The ‘X’ symbol represents a food category for which the κ statistic and McNemar’s test could not be conducted because 2×2 tables could not be generated (i.e. none of the desserts or dessert toppings/fillings was classified as ‘healthier’, and none of the eggs was classified as ‘less healthy’ by the EURO model).

Fig. 9

Agreement (κ, 95 % CI) and discordance (%, indicated above each line) between the WHO Regional Office for the Americas/Pan American Health Organization (PAHO) and Ofcom model for all foods (n 15 182; data missing for n 45) and twenty-two food categories from Schedule M of the Food and Drug Regulations ^{( 38 )}. Agreement was assessed using the κ statistic as follows: 0·01–0·20 ‘slight’; 0·21–0·40 ‘fair’’; 0·41–0·60 ‘moderate’; 0·61–0·80 ‘substantial’; 0·81–0·99 ‘near perfect’^{( 39 )}. Significant discordance in classifications between models using McNemar’s test (* P<0·01, ** P<0·001). The ‘X’ symbol represents a food category for which the κ statistic and McNemar’s test could not be conducted because 2×2 tables could not be generated (i.e. none of the packaged salads was classified as ‘healthier’ by the PAHO model).