Classification and Regression Tree and Computer Adaptive Testing in Cardiac Rehabilitation: Instrument Validation Study

Abstract

Background: There is a need for shorter-length assessments that capture patient questionnaire data while attaining high data quality without an undue response burden on patients. Computerized adaptive testing (CAT) and classification and regression tree (CART) methods have the potential to meet these needs and can offer attractive options to shorten questionnaire lengths.

Objective: The objective of this study was to test whether CAT or CART was best suited to reduce the number of questionnaire items in multiple domains (eg, anxiety, depression, quality of life, and social support) used for a needs assessment procedure (NAP) within the field of cardiac rehabilitation (CR) without the loss of data quality.

Methods: NAP data of 2837 CR patients from a multicenter Cardiac Rehabilitation Decision Support System (CARDSS) Web-based program was used. Patients used a Web-based portal, MyCARDSS, to provide their data. CAT and CART were assessed based on their performances in shortening the NAP procedure and in terms of sensitivity and specificity.

Results: With CAT and CART, an overall reduction of 36% and 72% of NAP questionnaire length, respectively, was achieved, with a mean sensitivity and specificity of 0.765 and 0.817 for CAT, 0.777 and 0.877 for classification trees, and 0.743 and 0.40 for regression trees, respectively.

Conclusions: Both CAT and CART can be used to shorten the questionnaires of the NAP used within the field of CR. CART, however, showed the best performance, with a twice as large overall decrease in the number of questionnaire items of the NAP compared to CAT and the highest sensitivity and specificity. To our knowledge, our study is the first to assess the differences in performance between CAT and CART for shortening questionnaire lengths. Future research should consider administering varied assessments of patients over time to monitor their progress in multiple domains. For CR professionals, CART integrated with MyCARDSS would provide a feedback loop that informs the rehabilitation progress of their patients by providing real-time patient measurements.

Keywords: cardiac rehabilitation; computing methodologies; internet; mHealth; needs assessment; psychometrics.

Figure 1

Maximum, minimum, and mean number of items administered in each CAT (computerized adaptive testing) and classification and regression tree. GAD-7: Generalized Anxiety Disorder - 7; HADS-A: Hospital Anxiety and Depression Scale - Anxiety; HADS-D: Hospital Anxiety and Depression Scale - Depression; MPSSS: Multidimensional Perceived Social Support Scale; PHQ-9: Patient Health Questionnaire - 9; QLMI-P: Quality of Life after Myocardial Infarction - Physical dimension; and QLMI-S: Quality of Life after Myocardial Infarction - Social dimension.

Figure 2

Percentage decrease per questionnaire for computerized adaptive testing and classification and regression tree. QLMI-P: Quality of Life after Myocardial Infarction - Physical dimension; QLMI-S: Quality of Life after Myocardial Infarction - Social dimension; PHQ-9: Patient Health Questionnaire - 9; GAD-7: Generalized Anxiety Disorder - 7; HADS-A: Hospital Anxiety and Depression Scale - Anxiety; HADS-D: Hospital Anxiety and Depression Scale -Depression; MPSSS: Multidimensional Perceived Social Support Scale.