Systems engineering analysis of diagnostic referral closed-loop processes

Abstract

Background: Closing loops to complete diagnostic referrals remains a significant patient safety problem in most health systems, with 65%-73% failure rates and significant delays common despite years of improvement efforts, suggesting new approaches may be useful. Systems engineering (SE) methods increasingly are advocated in healthcare for their value in studying and redesigning complex processes.

Objective: Conduct a formative SE analysis of process logic, variation, reliability and failures for completing diagnostic referrals originating in two primary care practices serving different demographics, using dermatology as an illustrating use case.

Methods: An interdisciplinary team of clinicians, systems engineers, quality improvement specialists, and patient representatives collaborated to understand processes of initiating and completing diagnostic referrals. Cross-functional process maps were developed through iterative group interviews with an urban community-based health centre and a teaching practice within a large academic medical centre. Results were used to conduct an engineering process analysis, assess variation within and between practices, and identify common failure modes and potential solutions.

Results: Processes to complete diagnostic referrals involve many sub-standard design constructs, with significant workflow variation between and within practices, statistical instability and special cause variation in completion rates and timeliness, and only 21% of all process activities estimated as value-add. Failure modes were similar between the two practices, with most process activities relying on low-reliability concepts (eg, reminders, workarounds, education and verification/inspection). Several opportunities were identified to incorporate higher reliability process constructs (eg, simplification, consolidation, standardisation, forcing functions, automation and opt-outs).

Conclusion: From a systems science perspective, diagnostic referral processes perform poorly in part because their fundamental designs are fraught with low-reliability characteristics and mental models, including formalised workaround and rework activities, suggesting a need for different approaches versus incremental improvement of existing processes. SE perspectives and methods offer new ways of thinking about patient safety problems, failures and potential solutions.

Keywords: diagnostic errors; failure modes and effects analysis (FMEA); human factors; process mapping; statistical process control.

Figure 1

Process stability and variability in diagnostic dermatology referral closed-loop failures and times until closure, stratified by skin lesions concerning for cancer and skin lesions non-concerning for cancer. (A) Skin lesions concerning for cancer monthly per cent of closed dermatology referrals (p statistical control chart). (B) Skin lesions concerning for cancer monthly average time until closing referrals (x-bar statistical control chart). (C) Non-concerning skin lesions monthly per cent of closed dermatology referrals (p statistical control chart). (D) Non-concerning skin lesions monthly average time until closing referrals (x-bar statistical control chart).

Figure 2

Process maps for diagnostic referral processes showing the ordering, scheduling and follow-up processes for in-network and out-of-network referrals. (A) General referral process and (B) dermatology referral process. Labels on each process map indicate activity categorisations (see online supplemental material for larger versions of figure). Q, data querying; E, data entry/storage; W, delay/waiting; I, inspection/verification; R, reminder; S, rework; T, travel/transportation; V, value add.

Figure 3

Example of FMEA results for diagnostic referral process and process improvement ideas produced from FMEA. (A) Identified failure modes, causes and RPNs are summarised in each column, along with the RPN score difference between the two study practices. Grey indicates minimum RPN score; orange denotes additional RPN score for site 1 over site 2; blue denotes additional RPN score for site 2 over site 1. Failure modes with the highest RPNs (to the right of the red vertical line) were prioritised for potential process redesign solutions. (B) Evaluated highest scoring process improvement ideas based on feasibility versus potential impact. Grey colour scale represents the degree of impact to patients (least amount of patients impacted to broader impact). Improvement ideas are categorised as workflow, IT/EHR-based or resource/staffing changes. Green outlines represent improvement ideas that in combination would be implemented through a referral tracking tab in the EHR (see online supplemental material for larger versions). EHR, electronic health record; FMEA, failure modes and effects analysis; RPN, risk priority number.