Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Clinical Trial
. 2018 Nov 21;13(11):e0204566.
doi: 10.1371/journal.pone.0204566. eCollection 2018.

Cortical control of a tablet computer by people with paralysis

Paul Nuyujukian  1   2   3   4   5   6 , Jose Albites Sanabria  7   8 , Jad Saab  7   8   9 , Chethan Pandarinath  1   2   10   11 , Beata Jarosiewicz  1   2   8   12 , Christine H Blabe  1 , Brian Franco  13 , Stephen T Mernoff  9   14 , Emad N Eskandar  15   16 , John D Simeral  7   8   9   13 , Leigh R Hochberg  7   8   9   13   17 , Krishna V Shenoy  2   3   4   5   6   18   19 , Jaimie M Henderson  2   4   5
Affiliations
Clinical Trial

Cortical control of a tablet computer by people with paralysis

Paul Nuyujukian et al. PLoS One. .

Abstract

General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.

PubMed Disclaimer

Conflict of interest statement

I have read the journal’s policy and the authors of this manuscript have the following competing interests: KVS is a consultant for Neuralink Inc. and on the Scientific Advisory Boards of Cognescent Inc. and Heal Inc. JMH is a consultant for Circuit Therapeutics and Enspire DBS and is on the Surgical Advisory board of Neuropace Inc. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Figures

Fig 1
Fig 1. Research setup.
a Schematic of research setup with T6. We recorded from 96-channel electrode arrays implanted in motor cortex. The neural signals extracted from the arrays were passed into a decoding algorithm which output a two dimensional cursor velocity and a click signal. The output of the decoder was presented as a wireless Bluetooth mouse interface and paired with a computer tablet. The participants used this interface to control the tablet and perform common tasks like email and web browsing. b Example task timeline with T5 from trial day 124. Shortest vertical black lines represent general user interface clicks, shorter gray lines represent single character text entry, and taller gray lines represent autocompletion of text.
Fig 2
Fig 2. Thresholded spiking actvity of participants’ arrays.
Each panel, corresponding to a specified 96-channel array, shows the threshold crossing waveforms recorded over 60 seconds on the specified trial day. a is T6’s array. b and c are T5’s lateral and medial arrays, respectively. d and e are T9’s lateral and medial arrays, respectively. Scale bars represent 150 uV (vertical) and 500 us (horizontal). Data are from the following trial days: 1013 (T6), 124 (T5), and 218 (T9). Plot construction identical to that of Fig 5 of [33].
Fig 3
Fig 3
a T6 browsing the web. Overlay in center of the image is a screen capture of the tablet. b T6 composing an email (trial day 1001). Both images are taken from S1 Video.
Fig 4
Fig 4
a T9 performing a video search. b T9 searching for artists from a music streaming program. Both images are taken from S2 Video.
Fig 5
Fig 5
a T5 chatting in real time with research staff. b T5 checking the forecast. Both images are taken from S3 Video.
Fig 6
Fig 6. Tasks of interest.
a T6 playing music with a keyboard application. b T9 using a calculator application. Both images are taken from S4 Video.
See this image and copyright information in PMC

References

    1. Crepeau EB, Cohn ES. Narrative as a Key to Understanding In: Boyt BA, Gillen G, editors. Willard and Spackman’s Occupational Therapy. 12th ed Wolters Kluwer/Lippincott Williams and Wilkins Health; 2013. p. 96–102.
    1. Ball LJ, Beukelman DR, Pattee GL. Communication effectiveness of individuals with amyotrophic lateral sclerosis. J Commun Disord. 2004;37(3):197–215. 10.1016/j.jcomdis.2003.09.002 - DOI - PubMed
    1. Bodine C. Assistive Technology In: Frontera W, editor. DeLisa’s Physical Medicine and Rehabilitation. 5th ed Wolters Kluwer/Lippincott Williams and Wilkins Health; 2010. p. 1997–2016.
    1. Fried-Oken M, Mooney A, Peters B. Supporting communication for patients with neurodegenerative disease. NeuroRehabilitation. 2015;37(1):69–87. 10.3233/NRE-151241 - DOI - PMC - PubMed
    1. Wolpaw JR, Ramoser H, McFarland DJ, Pfurtscheller G. EEG-Based communication: improved accuracy by response verification. IEEE Transactions on Rehabilitation Engineering. 1998;6:326–33. 10.1109/86.712231 - DOI - PubMed

Publication types