Evaluation logic in practice: Findings from two empirical investigations of American Evaluation Association members

Abstract

Surveys of two independent random samples of American Evaluation Association (AEA) members were conducted to investigate application of the logic of evaluation in their evaluation practice. This logic consists of four parts: (1) establish criteria, (2) set standards, (3) measure performance on criteria and compare to standards, and (4) synthesize into a value judgment. Nearly three-fourths (71.84% ± 5.98%) of AEA members are unfamiliar with this logic, yet a majority also indicate its importance and utility for evaluation practice. Moreover, and despite unfamiliarity with the four steps of the logic of evaluation, many AEA members identify evaluative criteria (82.41% ± 3.34%), set performance standards (60.55% ± 7.39%), compare performance to standards (62.14% ± 5.98%), and synthesize into an evaluative conclusion (75.00% ± 5.80%) in their evaluation practice. Much like the working logic of evaluation, however, application of the general logic varies widely.

Keywords: American Evaluation Association; Evaluation logic; Evaluation practice; Logic of evaluation; Research on evaluation.