Assessing the informational value of parameter estimates in cognitive models

Abstract

Mathematical models of cognition often contain unknown parameters whose values are estimated from the data. A question that generally receives little attention is how informative such estimates are. In a maximum likelihood framework, standard errors provide a measure of informativeness. Here, a standard error is interpreted as the standard deviation of the distribution of parameter estimates over multiple samples. A drawback to this interpretation is that the assumptions that are required for the maximum likelihood framework are very difficult to test and are not always met. However, at least in the cognitive science community, it appears to be not well known that standard error calculation also yields interpretable intervals outside the typical maximum likelihood framework. We describe and motivate this procedure and, in combination with graphical methods, apply it to two recent models of categorization: ALCOVE (Kruschke, 1992) and the exemplar-based random walk model (Nosofsky & Palmeri, 1997). The applications reveal aspects of these models that were not hitherto known and bring a mix of bad and good news concerning estimation of these models.