Clinically significant and practical! Enhancing precision does make a difference. Reply to McGlinchey and Jacobson, Hsu, and Speer

Abstract

Based on a secondary analysis of the Jacobson and Truax [Jacobson, N.S. & Truax, P. (1991).

Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.] data using both their own traditional approach and the refined method advanced by Hageman and Arrindell [Hageman, W.J.J.M., & Arrindell, W.A. (1999). Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behaviour Research and Therapy, 37, 1169-1193], McGlinchey and Jacobson [McGlinchey, J. B., & Jacobson, N. S. (1999). Clinically significant but impractical? A response to Hageman and Arrindell. Behaviour Research and Therapy, 37, 1211-1217.] reported practically identical findings on reliable and clinically significant change across the two approaches. This led McGlinchey and Jacobson to conclude that there is little practical gain in utilizing the refined method over the traditional approach. Close inspection of the data used by McGlinchey and Jacobson however revealed a serious mistake with respect to the value of the standard error of measurement that was employed in their calculations. When the proper index value was utilised, further re-analysis by the present authors disclosed clear differences (i.e. different classifications of S's) across the two approaches. Importantly, these differences followed exactly the same pattern as depicted in Table 2 in Hageman and Arrindell (1999). The theoretical advantages of the refined method, i.e. enhanced precision, appropriate distinction between analysis at the individual and group levels, and maximal comparability of findings across studies, exceed those of the traditional method. Application of the refined method may be carried out within approximately half an hour, which not only supports its practical manageability, but also challenges the suggestion of McGlinchey and Jacobson (1999) that the relevant method would be too complex (impractical) for the average scientist. The reader is offered the opportunity of obtaining an SPSS setup in the form of an ASCII text file by means of which the relevant calculations can be carried out. The ways in which the valuable commentaries by Hsu [Hsu, L. M. (1999). A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell. Behaviour Research and Therapy, 37, 1195-1202.] and Speer [Speer, D. C. (1999). What is the role of two-wave designs in clinical research? Comment on Hageman and Arrindell. Behaviour Research and Therapy, 37, 1203-1210.) contribute to a better understanding of the technical/statistical backgrounds of the traditional and refined methods were also discussed.