Identifying patterns in administrative tasks through structural topic modeling: A study of task definitions, prevalence, and shifts in a mental health practice's operations during the COVID-19 pandemic

Abstract

Objective: This case study illustrates the use of natural language processing for identifying administrative task categories, prevalence, and shifts necessitated by a major event (the COVID-19 [coronavirus disease 2019] pandemic) from user-generated data stored as free text in a task management system for a multisite mental health practice with 40 clinicians and 13 administrative staff members.

Materials and methods: Structural topic modeling was applied on 7079 task sequences from 13 administrative users of a Health Insurance Portability and Accountability Act-compliant task management platform. Context was obtained through interviews with an expert panel.

Results: Ten task definitions spanning 3 major categories were identified, and their prevalence estimated. Significant shifts in task prevalence due to the pandemic were detected for tasks like billing inquiries to insurers, appointment cancellations, patient balances, and new patient follow-up.

Conclusions: Structural topic modeling effectively detects task categories, prevalence, and shifts, providing opportunities for healthcare providers to reconsider staff roles and to optimize workflows and resource allocation.

Keywords: COVID19; mental health; natural language processing; task management; topic modeling.

Figure 1.

A structural topic modeling approach for identifying and categorizing tasks performed by administrators in a medical practice.

Figure 2.

Converting raw task description data to task sequence. This task (Task_ID 5ec49fc3) has 4 user-created events numbered 13 962 through 13 965. The task description is free text entered by the first user (e025dc59). The task was assigned to user 13a81065. User 13a81065 completed 3 events as part of the task and entered comments for each event, after which the task was assigned to user ed682e9a, who completed the fourth event, entered the last comment, and marked the task COMPLETED. A task sequence is a composite of the task description and all comments associated with the task, as shown in the box at the bottom.

Figure 3.

Spectrum of task categories discovered: from primarily client facing (left) to internal (middle) to primarily insurance facing (right). Some tasks (topics 3, 10) are in between the internal and insurance-facing categories.

Figure 4.

Estimated topic prevalence (expected topic proportions) in the corpus.

Figure 5.

Analysis of changes in the prevalence of topics in the pre–COVID-19 (coronavirus disease 2019) period and COVID-19 period datasets. The scale on the vertical axis is irrelevant; it simply lists the topics in order. The horizontal axis shows the size of the incremental effect (positive for topics on the right-hand side of the graph and negative on the left-hand side of the graph) of an indicator variable (1 = COVID-19, 0 = pre–COVID-19) on the prevalence of the corresponding topic.

Figure 6.

An example of a task sequence with high prevalence of topic 10 (claim resolution).