The doctor will polygraph you now

Abstract

Artificial intelligence (AI) methods have been proposed for the prediction of social behaviors that could be reasonably understood from patient-reported information. This raises novel ethical concerns about respect, privacy, and control over patient data. Ethical concerns surrounding clinical AI systems for social behavior verification can be divided into two main categories: (1) the potential for inaccuracies/biases within such systems, and (2) the impact on trust in patient-provider relationships with the introduction of automated AI systems for "fact-checking", particularly in cases where the data/models may contradict the patient. Additionally, this report simulated the misuse of a verification system using patient voice samples and identified a potential LLM bias against patient-reported information in favor of multi-dimensional data and the outputs of other AI methods (i.e., "AI self-trust"). Finally, recommendations were presented for mitigating the risk that AI verification methods will cause harm to patients or undermine the purpose of the healthcare system.

Keywords: Ethics; Machine learning; Science, technology and society.

Fig. 1. A possible workflow of a clinical AI system for social behavior verification.

This workflow has the following components: (1) patient data is input into the verification model, (2) the AI model verifies/rejects the claim of the patient, and (3) the assessment is factored into downstream recommendations.

Fig. 2. Demographic statistics for patient samples used to generate synthetic audio data.

These statistics include the distribution of gender and age within the dataset as well as the prevalence of voice/respiratory conditions which may be confounding factors in the prediction of smoking status.

Fig. 3. Data generation pipeline for privacy-aware experiments with voice data and LLM APIs.

The pipeline included the following steps: (1) acoustic features extracted from real-world voice recordings were structured into a data generation prompt for Llama 3.1, (2) Llama 3.1 was run locally to generate synthetic acoustic features which met specific constraints related to similarity and data privacy, and (3) resultant synthetic acoustic data was input into the APIs of LLMs for experimentation purposes.