A streaming brain-to-voice neuroprosthesis to restore naturalistic communication

Kaylo T Littlejohn^#^{1

2

3}, Cheol Jun Cho^#^{1

2

3}, Jessie R Liu^{2

3}, Alexander B Silva^{2

3

4}, Bohan Yu¹, Vanessa R Anderson², Cady M Kurtz-Miott², Samantha Brosler^{2

3

4}, Anshul P Kashyap¹, Irina P Hallinan², Adit Shah¹, Adelyn Tu-Chan⁵, Karunesh Ganguly^{3

5}, David A Moses^{2

3}, Edward F Chang^{6

7

8}, Gopala K Anumanchipalli^{9

10

11}

Affiliations

¹ Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA.
² Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA.
³ Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA.
⁴ Graduate Program in Bioengineering, University of California, Berkeley-University of California, San Francisco, Berkeley, CA, USA.
⁵ Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
⁶ Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA. edward.chang@ucsf.edu.
⁷ Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA. edward.chang@ucsf.edu.
⁸ Graduate Program in Bioengineering, University of California, Berkeley-University of California, San Francisco, Berkeley, CA, USA. edward.chang@ucsf.edu.
⁹ Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA. gopala@berkeley.edu.
¹⁰ Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA. gopala@berkeley.edu.
¹¹ Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA. gopala@berkeley.edu.

^# Contributed equally.

PMID: 40164740
DOI: 10.1038/s41593-025-01905-6

A streaming brain-to-voice neuroprosthesis to restore naturalistic communication

Kaylo T Littlejohn et al. Nat Neurosci. 2025 Apr.

. 2025 Apr;28(4):902-912.

doi: 10.1038/s41593-025-01905-6. Epub 2025 Mar 31.

Authors

Affiliations

¹ Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA.
² Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA.
³ Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA.
⁴ Graduate Program in Bioengineering, University of California, Berkeley-University of California, San Francisco, Berkeley, CA, USA.
⁵ Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
⁶ Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA. edward.chang@ucsf.edu.
⁷ Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA. edward.chang@ucsf.edu.
⁸ Graduate Program in Bioengineering, University of California, Berkeley-University of California, San Francisco, Berkeley, CA, USA. edward.chang@ucsf.edu.
⁹ Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, USA. gopala@berkeley.edu.
¹⁰ Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA. gopala@berkeley.edu.
¹¹ Weill Institute for Neuroscience, University of California, San Francisco, San Francisco, CA, USA. gopala@berkeley.edu.

^# Contributed equally.

PMID: 40164740
DOI: 10.1038/s41593-025-01905-6

Abstract

Natural spoken communication happens instantaneously. Speech delays longer than a few seconds can disrupt the natural flow of conversation. This makes it difficult for individuals with paralysis to participate in meaningful dialogue, potentially leading to feelings of isolation and frustration. Here we used high-density surface recordings of the speech sensorimotor cortex in a clinical trial participant with severe paralysis and anarthria to drive a continuously streaming naturalistic speech synthesizer. We designed and used deep learning recurrent neural network transducer models to achieve online large-vocabulary intelligible fluent speech synthesis personalized to the participant's preinjury voice with neural decoding in 80-ms increments. Offline, the models demonstrated implicit speech detection capabilities and could continuously decode speech indefinitely, enabling uninterrupted use of the decoder and further increasing speed. Our framework also successfully generalized to other silent-speech interfaces, including single-unit recordings and electromyography. Our findings introduce a speech-neuroprosthetic paradigm to restore naturalistic spoken communication to people with paralysis.

PubMed Disclaimer

Conflict of interest statement

Competing interests: K.T.L., J.R.L., D.A.M., E.F.C. and G.K.A. are inventors of pending provisional University of California, San Francisco, patent applications relevant to the neural decoding approaches used in this work. G.K.A. and E.F.C. are inventors on patent application PCT/US2020/028926, D.A.M. and E.F.C. are inventors on patent application PCT/US2020/043706, and E.F.C. is an inventor on patent US9905239B2, which are broadly relevant to the neural decoding approaches in this work. E.F.C. is a cofounder of, and D.A.M. is a director at, Echo Neurotechnologies. The other authors declare no competing interests.

References

1. Felgoise, S. H., Zaccheo, V., Duff, J. & Simmons, Z. Verbal communication impacts quality of life in patients with amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. Frontotemporal Degener. 17, 179–183 (2016). - PubMed
1. Huggins, J. E., Wren, P. A. & Gruis, K. L. What would brain–computer interface users want? Opinions and priorities of potential users with amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. 12, 318–324 (2011). - PubMed - PMC
1. Branco, M. P. et al. Brain–computer interfaces for communication: preferences of individuals with locked-in syndrome. Neurorehabil. Neural Repair 35, 267–279 (2021). - PubMed - PMC
1. Peters, B., O’Brien, K. & Fried-Oken, M. A recent survey of augmentative and alternative communication use and service delivery experiences of people with amyotrophic lateral sclerosis in the United States. Disabil. Rehabil. Assist. Technol. 1121–1134 (2024).
1. Peters, B. et al. Brain–computer interface users speak up: the Virtual Users’ Forum at the 2013 International Brain–Computer Interface Meeting. Arch. Phys. Med. Rehabil. 96, S33–S37 (2015). - PubMed - PMC

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
- Nature Publishing Group

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

A streaming brain-to-voice neuroprosthesis to restore naturalistic communication

Affiliations

A streaming brain-to-voice neuroprosthesis to restore naturalistic communication

Authors

Affiliations

Abstract

Conflict of interest statement

References

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources