Theoretical limitations of the AOFAS scoring systems: an analysis using Monte Carlo modeling

Abstract

The AOFAS foot scores are four related outcome instruments based on the use of quantitative interval data and have seen increasing use in the literature. The mathematical construction of the scales is particularly notable for a very small number of intervals available to answer each component item and for quantitatively unequal intervals for some items. Monte Carlo computer modeling techniques were used to simulate the responses to each item for a variety of idealized patient populations with several different means, standard deviations, and levels of interaction between items. The continuous data describing each patient's responses were categorized into the finite number of available intervals in the AOFAS Hindfoot score. The resultant distributions of net scores often demonstrated bizarre, skewed behavior that bore little resemblance to the original distributions of continuous data. The effects were magnified as the ends of the scale were approached and when strong correlations between the items measuring pain and function were present. The distributions did not become distorted when the original continuous data were not rounded off into intervals but were simply weighted by their relative contribution to the AOFAS score and summed. The AOFAS scores, therefore, have inherently limited precision which is entirely due to the small number of response intervals available to answer each component item of the scale. Minor changes in a patient's response to a series of correlated questions can potentially make a drastic difference in their total score. Because the population distributions may be badly skewed, the use of parametric statistics with the AOFAS scores should be regarded with genuine suspicion, and appropriate refinements of the scales should be sought.