Food and beverage environment analysis and monitoring system: a reliability study in the school food and beverage environment

Abstract

States and school districts around the country are developing policies that set nutrition standards for competitive foods and beverages sold outside of the US Department of Agriculture's reimbursable school lunch program. However, few tools exist for monitoring the implementation of these new policies. The objective of this research was to develop a computerized assessment tool, the Food and Beverage Environment Analysis and Monitoring System (FoodBEAMS), to collect data on the competitive school food environment and to test the inter-rater reliability of the tool among research and nonresearch professionals. FoodBEAMS was used to collect data in spring 2007 on the competitive foods and beverages sold in 21 California high schools. Adherence of the foods and beverages to California's competitive food and beverage nutrition policies for schools (Senate Bills 12 and 965) was determined using the data collected by both research and nonresearch professionals. The inter-rater reliability between the data collectors was assessed using the intraclass correlation coefficient. Researcher vs researcher and researcher vs nonresearcher inter-rater reliability was high for both foods and beverages, with intraclass correlation coefficients ranging from .972 to .987. Results of this study provide evidence that FoodBEAMS is a promising tool for assessing and monitoring adherence to nutrition standards for competitive foods sold on school campuses and can be used reliably by both research and nonresearch professionals.

Figure 1

Convergent Validity: Scatterplot of the EAT* versus FoodBEAMS* measurements of percent adherence by venue to California state standards for Beverages (SB 965) and Foods (SB 12). Note: The diagonal line represents where the dots would fall for theoretical perfect concordance, and a single dot may represent multiple data points. *The paper and pencil Environmental Assessment Tool (EAT) is considered to be the gold standard for collecting such data. *The web-based Food and Beverage Environmental Assessment and Monitoring System (FoodBEAMS™) is under experiment as a valid method for collecting such data.

Figure 2

Inter-rater Reliability: Scatterplot of Researcher versus Researcher FoodBEAMS measurements of percent adherence by venue to California state standards for Beverages (SB 965) and Foods (SB 12). Note: The diagonal line represents where the dots would fall for theoretical perfect concordance, and a single dot may represent multiple data points.