Sensitivity and Specificity of Real-World Social Factor Screening Approaches

Abstract

Health care organizations are increasingly documenting patients for social risk factors in structured data. Two main approaches to documentation, ICD-10 Z codes and screening questions, face limited adoption and conceptual challenges. This study compared estimates of social risk factors obtained via screening questions and ICD-10 Z diagnoses coding, as used in clinical practice, to estiamtes from validated survey instruments in a sample of adult primary care and emergency department patients at an urban safety-net health system. Financial strain, transportation barriers, food insecurity, and housing instability were independently assessed using instruments with published reliability and validity. These four social factors were also being collected by the health system in screening questions or could be mapped to ICD-10 Z code diagnosis code concepts. Neither the screening questions nor ICD-10 Z codes performed particularly well in terms of accuracy. For the screening questions, the Area Under the Curve (AUC) scores were 0.609 for financial strain, 0.703 for transportation, 0.698 for food insecurity, and 0.714 for housing instability. For the ICD-10 Z codes, AUC scores tended to be lower in the range of 0.523 to 0.535. For both screening questions and ICD-10 Z codes, the measures were much more specific than sensitive. Under real world conditions, ICD-10 Z codes and screening questions are at the minimal, or below, threshold for being diagnostically useful approaches to identifying patients' social risk factors. Data collection support through information technology or novel approaches combining data sources may be necessary to improve the usefulness of these data.

Keywords: Safety-net; Social determinants; Survey; Validity.

Fig. 1

Combination of validated survey sample and existing electronic health record sources

Fig. 2

Percent of patients with social factors by screening approach. Note: EHR-based screening estimates were obtained from patients administered the screening questionnaire

Fig. 3

Differences in area under the curve values for ICD-10 Z codes alone or in combination with available electronic health record screening questions