Inter-rater agreement and checklist validation for postoperative wound assessment using smartphone images in vascular surgery

Abstract

Objective: Surgical site infection (SSI) is the most common nosocomial infection, in vascular surgery patients, who experience a high rate of readmission. Facilitating transition from hospital to outpatient care with digital image-based wound monitoring has the potential to detect and to enable treatment of SSI at an early stage. In this study, we evaluated whether smartphone digital images can supplant in-person evaluation of postoperative vascular surgery wounds.

Methods: We developed a wound assessment checklist using previously validated criteria. We recruited adults who underwent a vascular surgical procedure between 2014 and 2015, involving an incision of at least 3 cm, from a high-volume academic vascular surgery service. Vascular surgery care providers evaluated wounds in person using the assessment checklist; a different group of providers evaluated wounds by a smartphone digital image. Inter-rater agreement coefficients for wound characteristics and treatment plan were calculated within and between the in-person group and the digital image group; the sensitivity and specificity of digital images relative to in-person evaluation were determined.

Results: We assessed a total of 80 wounds. Regardless of modality, inter-rater agreement was poor when wounds were evaluated for the presence of ecchymosis and redness; moderate for cellulitis; and high for the presence of a drain, necrosis, or dehiscence. As expected, the presence of drainage was more readily observed in person. Inter-rater agreement was high for both in-person and image-based assessment with respect to course of treatment, with near-perfect agreement for treatments ranging from antibiotics to surgical débridement to hospital readmission. No difference in agreement emerged when raters evaluated poor-quality compared with high-quality images. For most parameters, specificity was higher than sensitivity for image-based compared with "gold standard" in-person assessment.

Conclusions: Using smartphone digital images is a valid method for evaluating postoperative vascular surgery wounds and is comparable to in-person evaluation with regard to most wound characteristics. The inter-rater reliability for determining treatment recommendations was universally high. Remote wound monitoring and assessment may play an integral role in future transitional care models to decrease readmission for SSI in vascular or other surgical patients. These findings will inform smartphone implementation in the clinical care setting as wound images transition from informal clinical communication to becoming part of the care standard.

Figure 1

The shape represents the value for Gwet's agreement coefficient (AC); the error bar represents the bootstrapped 95% confidence interval for the AC.