Identifying Topics in Patient and Doctor Conversations Using Natural Language Processing Methods [Internet]

Excerpt

Background: Amid electronic health records and laboratory tests, office-based patient and provider communication is still the heart of primary care. Patients often present multiple complaints and, consequently, may be expressing intense emotions, requiring physicians to decide how to balance competing demands. How physicians navigate conversations has implications for patient satisfaction, payments, and quality of care. However, traditional observational measures of emotions and what is discussed during clinical visits are slow and costly and do not scale for use in clinical practice. We evaluated machine-learning methods for automated annotation of medical topics and emotional valence in patient-provider dialogue transcripts.

Objectives: The specific aims of the current study were to develop and evaluate natural language processing (NLP) models that predict (1) topics of conversations and (2) emotional valence of patient-provider interactions.

Methods: Using transcripts from primary care visits, we developed machine-learning models that predicted talk-turn (eg, speaker statements in a conversation that are bounded at the beginning and end by statements of the other speaker) topics and emotion labels of patient-provider interactions. First, we developed nonsequential NLP models that predicted topic labels on the basis of single or multiple local talk-turns (eg, logistic classifiers, support vector machines, word-level gated recurrent units), as well as fully sequential models that integrated information across talk-turn sequences (eg, conditional random fields, hidden Markov models, hierarchical gated recurrent units). Second, we compared 2 machine-learning models, a recurrent neural network with a hierarchical structure and a logistic regression classifier, to recognize the emotional valence of each utterance in the transcripts.

Results: For medical topics, sequential models had greater topic classification accuracy at the talk-turn level and greater precision at the visit level. Nonsequential models had higher topic recall scores at the visit level compared with sequential models. The agreement of emotion ratings from the recurrent neural-network model with human ratings was comparable to that of human-human interrater agreement. The weighted average of the correlation coefficients for the recurrent neural-network model with human raters was 0.60, as was the human rater agreement.

Conclusions: Incorporating sequential information across talk-turns improved the accuracy of topic prediction in patient-provider dialogue by smoothing out noisy information from talk-turns. The recurrent neural-network model predicted the emotional valence of patients and physicians during primary care visits with similar reliability as human raters. As an initial machine learning-based evaluation of topic and emotion recognition in primary care visit conversations, our work provides valuable baselines for applications that might be helpful in monitoring physician navigation of patient concerns, supporting physicians in empathic communication, or examining the role of emotion in patient-centered care.

Limitations: Although our results are promising, more advanced prediction techniques and larger labeled data sets will likely be required to improve prediction performance.