Development, validation and application of computer-linked knowledge questionnaires in diabetes education

Abstract

Multiple choice questionnaires (MCQs) capable of being marked manually or by a newly developed optical mark reader, or by use of an inexpensive inter-active microcomputer system have been developed for the separate assessment of insulin-dependent and non-insulin-dependent patient knowledge. Forty-six insulin-related and non-insulin-related multiple choice questions covering six main areas of knowledge were constructed for inclusion into draft questionnaires. From the responses of a total of 180 completed questionnaires, piloted in 18 randomly selected clinics in 14 Regional Health Authorities in England, psychometric analysis was performed to determine reliability, discrimination coefficients, and facility indices. Seventy-three per cent of insulin-dependent diabetic patients (IDDM) and 92% of non-insulin-dependent diabetic patients (NIDDM) MCQ correct options had facility indices within the acceptable range of 30 to 90%. 82% IDDM and 93% NIDDM correct options had discrimination coefficients exceeding 0.2. Questionnaire reliability (internal consistency) using the Kudor-Richardson (KR20) formula was IDDM 0.87 and NIDDM 0.82. Evidence in support of the IDDM questionnaire's criterion validity was based on significant differences (p less than 0.05) identified between a number of knowledge area scores stratified according to HbA1 levels. Prescriptive correction for screen display and automatic hard copy feedback was designed for both incorrect and omitted question options, providing both educational (patient) and analytical (clinic) documentation. Both technical and psychometric properties of these knowledge assessment instruments should be acceptable for diabetic knowledge evaluation and instruction.