Healthcare Process Modeling to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals): Development and evaluation of a conceptual framework

Abstract

Objective: There are signals of clinicians' expert and knowledge-driven behaviors within clinical information systems (CIS) that can be exploited to support clinical prediction. Describe development of the Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals).

Materials and methods: We employed an iterative framework development approach that combined data-driven modeling and simulation testing to define and refine a process for phenotyping clinician behaviors. Our framework was developed and evaluated based on the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) predictive model to detect and leverage signals of clinician expertise for prediction of patient trajectories.

Results: Seven themes-identified during development and simulation testing of the CONCERN model-informed framework development. The HPM-ExpertSignals conceptual framework includes a 3-step modeling technique: (1) identify patterns of clinical behaviors from user interaction with CIS; (2) interpret patterns as proxies of an individual's decisions, knowledge, and expertise; and (3) use patterns in predictive models for associations with outcomes. The CONCERN model differentiated at risk patients earlier than other early warning scores, lending confidence to the HPM-ExpertSignals framework.

Discussion: The HPM-ExpertSignals framework moves beyond transactional data analytics to model clinical knowledge, decision making, and CIS interactions, which can support predictive modeling with a focus on the rapid and frequent patient surveillance cycle.

Conclusions: We propose this framework as an approach to embed clinicians' knowledge-driven behaviors in predictions and inferences to facilitate capture of healthcare processes that are activated independently, and sometimes well before, physiological changes are apparent.

Keywords: clinical informatics; conceptual framework; electronic health records; predictive modeling.

Figure 1.

Approach to iterative conceptual framework development leveraging thematic analyses of processes and findings from data driven modeling and simulation testing for triangulation of themes. Thematic analysis of the iterative processes and contextual information that informed development of the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) model were triangulated with thematic analysis of clinical subject matter expert perceptions of the CONCERN model during simulation testing. These triangulated findings were used to define a conceptual framework for phenotyping clinician behaviors to detect and leverage signals of clinician expertise for prediction of patient trajectories.

Figure 2.

Time-varying survival regression. Forest plot of the covariates used in Cox time-varying proportional hazards model and associated statistics. HR: hazard ratio; MEWS: Modified Early Warning Score; NEWS: National Early Warning Score.

Figure 3.

Comparison of log likelihood ratios at various hours before event. The likelihood ratio, defined as L(x,h) = P(x | patient has an event h hours in the future) / P(x | patient does not have an event h hours in the future). For example, L(‘CONCERN score = yellow’, 6) quantifies how well the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) algorithm separate the probability measures induced by whether the patient has an event 6 hours in the future after observing a “yellow” score. Larger values represent more weight given to the numerator vs the denominator, while smaller values represent more weight given to the denominator. MEWS: Modified Early Warning Score; NEWS: National Early Warning Score.

Figure 4.

Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals). The framework is focused on information that can be mined from clinical data structures, is generated by clinician processes, and is driven by knowledge-based behaviors in order to identify features from user interaction with clinical systems, which are patterns of clinical behaviors and can be interpreted and used in predictions.