Accuracy of Electronic Health Record Data for Identifying Stroke Cases in Large-Scale Epidemiological Studies: A Systematic Review from the UK Biobank Stroke Outcomes Group

Abstract

Objective: Long-term follow-up of population-based prospective studies is often achieved through linkages to coded regional or national health care data. Our knowledge of the accuracy of such data is incomplete. To inform methods for identifying stroke cases in UK Biobank (a prospective study of 503,000 UK adults recruited in middle-age), we systematically evaluated the accuracy of these data for stroke and its main pathological types (ischaemic stroke, intracerebral haemorrhage, subarachnoid haemorrhage), determining the optimum codes for case identification.

Methods: We sought studies published from 1990-November 2013, which compared coded data from death certificates, hospital admissions or primary care with a reference standard for stroke or its pathological types. We extracted information on a range of study characteristics and assessed study quality with the Quality Assessment of Diagnostic Studies tool (QUADAS-2). To assess accuracy, we extracted data on positive predictive values (PPV) and-where available-on sensitivity, specificity, and negative predictive values (NPV).

Results: 37 of 39 eligible studies assessed accuracy of International Classification of Diseases (ICD)-coded hospital or death certificate data. They varied widely in their settings, methods, reporting, quality, and in the choice and accuracy of codes. Although PPVs for stroke and its pathological types ranged from 6-97%, appropriately selected, stroke-specific codes (rather than broad cerebrovascular codes) consistently produced PPVs >70%, and in several studies >90%. The few studies with data on sensitivity, specificity and NPV showed higher sensitivity of hospital versus death certificate data for stroke, with specificity and NPV consistently >96%. Few studies assessed either primary care data or combinations of data sources.

Conclusions: Particular stroke-specific codes can yield high PPVs (>90%) for stroke/stroke types. Inclusion of primary care data and combining data sources should improve accuracy in large epidemiological studies, but there is limited published information about these strategies.

Fig 1. International Classification of Diseases (ICD) codes for cerebrovascular disease.

* 433: occlusion/stenosis of pre-cerebral arteries with or without infarction. † 434: thrombosis/embolism of cerebral arteries with or without infarction. Codes in blue text denote ICD-9 codes which most closely represent stroke when subdivided using additional coding available in the clinically modified version of ICD-9 (ICD-9-CM) used in North America. In ICD-9-CM, ‘with infarction’ (433.x1, 434.x1) is distinguished from ‘without infarction’ (433.x0, 434.x0). ‡ 436: acute, ill-defined cerebrovascular disease ¶ a pathological term for ischaemic stroke § G46: not a diagnostic code; may be used for the presenting symptoms of either stroke or TIA.

Fig 2. Selection of studies.

*Additional studies identified from bibliography screening. †Additional studies identified from review articles and bibliography screening.

Fig 3. Positive predictive values of codes for stroke.

H: hospital data, D: death certificates, H+D: hospital data and death certificates; x = number of coded events confirmed as ‘true cases’ by the reference standard; y = total number of coded events; x/y = PPV. Circles represent PPVs, and horizontal lines denote 95% confidence intervals (CIs). Circle size is proportional to the inverse variance of the PPV. Where more than one result was available for a particular study, the result for the largest number of coded events validated is shown. * Cerebrovascular codes: I60-I69+/-G45 (ICD-10) or 430–438 (ICD-9), unless otherwise specified † Mean PPV (taken from the range published in the study) ‡ Excluding codes 435 (TIA) and 438 (sequelae of cebrovascular disease) § Excluding code 435 (TIA) and including code 342 (hemiplegia and hemiparesis) ¶ Excluding code 435 (TIA) # Stroke-specific codes: 160, 161, 163, 164 (ICD-10), 430, 431, 434, 436 (ICD-9), 430, 431, 433.x1, 434.x1 (ICD-9-CM). ¥ Ischaemic stoke and unspecified stroke codes: I63, I64 (ICD-10), 434, 436 (ICD-9), 433.x1, 434.x1, 436 (ICD-9-CM) **Ischaemic stroke codes:163 (ICD-10), 434 (ICD-9), 433.x1, 434.x1 (ICD-9-CM) †† Haemorrhagic stroke codes:I60, I61 (ICD-10), 430, 431 (ICD-9) ‡‡ Subarachnoid haemorrhage stroke codes:I60 (ICD-10), 430 (ICD-9) ¶¶ Intracerebral haemorrhage stroke codes:I61 (ICD-10), 431 (ICD-9)

Fig 4. Positive predictive values of codes for ischaemic stroke.

H: hospital data, D: death certificates, H+D: hospital data and death certificates; x = number of coded events confirmed as ‘true cases’ by the reference standard; y = total number of coded events; x/y = PPV. Circles represent PPVs, and horizontal lines denote 95% confidence intervals (CIs). Circle size is proportional to the inverse variance of the PPV. Where more than one result was available for a particular study, the result for the largest number of coded events validated is shown. † Mean PPV (taken from the range published in the study) ¥ Ischaemic stoke and unspecified stroke codes: I63, I64 (ICD-10), 434, 436 (ICD-9), 433.x1, 434.x1, 436 (ICD-9-CM) **Ischaemic stroke codes:163 (ICD-10), 434 (ICD-9), 433.x1, 434.x1 (ICD-9-CM)

Fig 5. Positive predictive values of codes for haemorrhagic stroke.

H: hospital data, D: death certificates, H+D: hospital data and death certificates; x = number of coded events confirmed as ‘true cases’ by the reference standard; y = total number of coded events; x/y = PPV. Circles represent PPVs, and horizontal lines denote 95% confidence intervals (CIs). Circle size is proportional to the inverse variance of the PPV. Where more than one result was available for a particular study, the result for the largest number of coded events validated is shown. † Mean PPV (taken from the range published in the study) †† Haemorrhagic stroke codes:I60, I61 (ICD-10), 430, 431 (ICD-9) ‡‡ Subarachnoid haemorrhage stroke codes:I60 (ICD-10), 430 (ICD-9) ¶¶ Intracerebral haemorrhage stroke codes:I61 (ICD-10), 431 (ICD-9)