Inter-reader variability of SPECT MPI readings in low- and middle-income countries: Results from the IAEA-MPI Audit Project (I-MAP)

Abstract

Background: Consistency of results between different readers is an important issue in medical imaging, as it affects portability of results between institutions and may affect patient care. The International Atomic Energy Agency (IAEA) in pursuing its mission of fostering peaceful applications of nuclear technologies has supported several training activities in the field of nuclear cardiology (NC) and SPECT myocardial perfusion imaging (MPI) in particular. The aim of this study was to verify the outcome of those activities through an international clinical audit on MPI where participants were requested to report on studies distributed from a core lab.

Methods: The study was run in two phases: in phase 1, SPECT MPI studies were distributed as raw data and full processing was requested as per local practice. In phase 2, images from studies pre-processed at the core lab were distributed. Data to be reported included summed stress score (SSS); summed rest score (SRS); summed difference score (SDS); left ventricular (LV) ejection fraction (EF) and end- diastolic volume (EDV). Qualitative appraisals included the assessment of perfusion and presence of ischemia, scar or mixed patterns, presence of transient ischemic dilation (TID), and risk for cardiac events (CE). Twenty-four previous trainees from low- and middle-income countries participated (core participants group) and their results were assessed for inter-observer variability in each of the two phases, and for changes between phases. The same evaluations were performed for a group of eleven international experts (experts group). Results were also compared between the groups.

Results: Expert readers showed an excellent level of agreement for all parameters in both phase 1 and 2. For core participants, the concordance of all parameters in phase 1 was rated as good to excellent. Two parameters which were re-evaluated in phase 2, namely SSS and SRS, showed an increased level of concordance, up to excellent in both cases. Reporting of categorical variables by expert readers remained almost unchanged between the two phases, while core participants showed an increase in phase 2. Finally, pooled LVEF values did not show a significant difference between core participants and experts. However, significant differences were found between LVEF values obtained using different software packages for cardiac analysis.

Conclusions: In this study, inter-observer agreement was moderate-to-good for core group readers and good-to-excellent for expert readers. The quality of reporting is affected by the quality of processing. These results confirm the important role of the IAEA training activities in improving imaging in low- and middle-income countries.

Keywords: CAD; Gated SPECT; Image interpretation; Myocardial ischemia and infarction.

Figure 1

Worldwide distribution of both participants and experts. Red dot identifies the Core Lab (Nuclear Medicine Dept, University of Brescia). The size of dots reflects more than one participant from the same country

Figure 2

Example of a case study as distributed in phase 2: Male; 76 y-o; Family history of CAD; Hypertensive; Inferior AMI in 1970; relapse 1 year later; 1991 Coro: 75% stenosis RCA; distal occlusion RCA; 75% stenosis distal LAD; occlusion D2; patent LCX; OMT until 2009; Referred for MPI in 2009; Bicycle exercise; max workload 60 W for 3’; typical chest pain; ECG: inferior-lateral ST downslope and runs of NSVT. CAD, Coronary artery disease; AMI, acute myocardial infarction; RCA, right coronary artery; LCX, left circumflex; OMT, optimized medical therapy; MPI, myocardial perfusion imaging; NSVT, non-sustained ventricular tachycardia

Figure 3

ICC values for continuous variables (EDV, LVEF, SSS, SRS). Calculation of EDV and LVEF was not requested for phase 2. EDV-S/R, end-diastolic volume post-stress/at rest; LVEF-S/R, left Ventricle Ejection Fraction post stress/at rest; SSS, summed stress score; SRS, summed rest score

Figure 4

Calculated Fleiss’ kappa values for categorical variables. TID, Transient Ischemic Dilation; SDS, Summed Differential Score stratified; PHR, patient high risk)

Figure 5

SSS value as function of the number of hypoperfusion clusters. Green boxes represent patients whose perfusion has been judged as abnormal. Blue boxes represent normal perfusion judgments. (A) Experts phase 1, (B) Experts phase 2, (C) Core participants phase 1, (D) Core participants phase 2. The figure shows box-and-whiskers plot, showing the median, quartiles, and outlier and extreme values for a scale variable. The interquartile range (IQR) is the difference between the 75th and 25th percentiles and corresponds to the length of the box. Circles outside the boxes represent OUTLIERS. Outliers are values between 1.5 IQR’s and 3 IQR’s from the end of a box. Stars represent EXTREME whose values are more than 3 IQR’s from the end of a box

Figure 6

Case #11 of phase 1: Female, 71 y-o; Type 2 diabetes on medication with metformin; Hypertension treated by vasodilators (Enalapril); 5 yrs before MPI cardioversion for atrial fibrillation. On chronic therapy with warfarin and propafenone; H 162 cm W 79 kg; Referred for chest pain not related to efforts; Dipyridamol stress test (0.84 mg/kg/5 minutes); Rest BP 170/95, 67 BPM; At the end of Dipyridamol infusion BP = 150/100, 95 BPM