Healthcare's New Horizon With ChatGPT's Voice and Vision Capabilities: A Leap Beyond Text

Abstract

The integration of artificial intelligence (AI) in healthcare is responsible for a paradigm shift in medicine. OpenAI's recent augmentation of their Generative Pre-trained Transformer (ChatGPT) large language model (LLM) with voice and image recognition capabilities (OpenAI, Delaware) presents another potential transformative tool for healthcare. Envision a healthcare setting where professionals engage in dynamic interactions with ChatGPT to navigate the complexities of atypical medical scenarios. In this innovative landscape, practitioners could solicit ChatGPT's expertise for concise summarizations and insightful extrapolations from a myriad of web-based resources pertaining to similar medical conditions. Furthermore, imagine patients using ChatGPT to identify abnormalities in medical images or skin lesions. While the prospects are diverse, challenges such as suboptimal audio quality and ensuring data security necessitate cautious integration in medical practice. Drawing insights from previous ChatGPT iterations could provide a prudent roadmap for navigating possible challenges. This editorial explores some possible horizons and potential hurdles of ChatGPT's enhanced functionalities in healthcare, emphasizing the importance of continued refinements and vigilance to maximize the benefits while minimizing risks. Through collaborative efforts between AI developers and healthcare professionals, another fusion of AI and healthcare can evolve into enriched patient care and enhanced medical experience.

Keywords: artificial intelligence chatgpt-4; dalle-3; image recognition; user-centric interface; voice recognition technology.

Figure 1. Imaginary example of two healthcare professionals talking about the new features of ChatGPT of voice and image interactions

The picture was crafted by the authors on 11 Oct 2023 on ChatGPT-4, using the AI-enabled OpenAI’s DALL·E 3, using the prompt: “Use DALL·E 3 to draw 3D image of 2 healthcare professionals talking together, one saying “ChatGPT can now see, hear and speak with us!” and the other replying: How could that impact healthcare?, digital art”. ChatGPT-4 added to the image the following text: “Prompt: Digital art of a diverse pair of healthcare professionals standing in a well-lit hospital corridor. A professional of Middle Eastern descent in a blue uniform enthusiastically mentions, 'ChatGPT can now see, hear and speak with us!'. The other professional, of Indian descent and wearing a surgical mask hanging from the neck, contemplates and responds, 'How could that impact healthcare?'. Nearby, a hospital bed and vital signs monitor can be seen.” The "DALL·E 3"-generated image was then edited by the authors, to correct the spelling "typo" that DALL·E 3 made. The final image is shown here.