Sleep-Deep-Learner is taught sleep-wake scoring by the end-user to complete each record in their style

Abstract

Sleep-wake scoring is a time-consuming, tedious but essential component of clinical and preclinical sleep research. Sleep scoring is even more laborious and challenging in rodents due to the smaller EEG amplitude differences between states and the rapid state transitions which necessitate scoring in shorter epochs. Although many automated rodent sleep scoring methods exist, they do not perform as well when scoring new datasets, especially those which involve changes in the EEG/EMG profile. Thus, manual scoring by expert scorers remains the gold standard. Here we take a different approach to this problem by using a neural network to accelerate the scoring of expert scorers. Sleep-Deep-Learner creates a bespoke deep convolution neural network model for individual electroencephalographic or local-field-potential (LFP) records via transfer learning of GoogLeNet, by learning from a small subset of manual scores of each EEG/LFP record as provided by the end-user. Sleep-Deep-Learner then automates scoring of the remainder of the EEG/LFP record. A novel REM sleep scoring correction procedure further enhanced accuracy. Sleep-Deep-Learner reliably scores EEG and LFP data and retains sleep-wake architecture in wild-type mice, in sleep induced by the hypnotic zolpidem, in a mouse model of Alzheimer's disease and in a genetic knock-down study, when compared to manual scoring. Sleep-Deep-Learner reduced manual scoring time to 1/12. Since Sleep-Deep-Learner uses transfer learning on each independent recording, it is not biased by previously scored existing datasets. Thus, we find Sleep-Deep-Learner performs well when used on signals altered by a drug, disease model, or genetic modification.

Keywords: Alzheimer’s disease model; automated sleep–wake scoring; basic sleep research; deep learning; hypnotics; in vivo pharmacology; transfer learning; translational sleep research.

Figure 1.

Illustration of the workflow for the end-user and subsequent automated scoring by Sleep-Deep-Learner. (A) The end-user first manually scores one-twelfth of each file in their preferred sleep-scoring software package. We used Sirenia Sleep. EEG/LFP/EMG signals are then exported as European data format files. Scores are exported/converted to.txt,.tsv, or.xls files. The manual scores for each file will be used to train a new model for each record. Next a spreadsheet should be created with all the edf file names in the first column and the chosen EEG or LFP channel in the second column. Sleep-Deep-Learner reads this spreadsheet to work through all the files without supervision by the end user. Finally, the end-user instructs Sleep-Deep-Learner to complete scoring of the files via its native graphical user interface. (B) Sleep-Deep-Learner iterates through each file. For each iteration, GoogLeNet is loaded. The final learnable layers are replaced with new layers specific to each sleep–wake state. The model created for each file is then trained on a subset of wavelet transform images labeled as wake, NREM sleep, or REM sleep by the end-user’s manual scoring for that specific file. The novel trained model created for each file is then used to classify (score) all epochs from that file.

Figure 2.

Sleep-Deep-Learner scores mouse EEG/EMG signal with high reliability compared with human expert scores and retain the proportion of sleep–wake states. (A) F1 scores for wakefulness, NREM sleep, and REM sleep reveal reliable scoring respective to manual scores by the expert scorer on which Sleep-Deep-Learner was trained. (B) The proportion of wake, NREM sleep, and REM sleep from 24-hour records when determined by manual scoring. (C) The proportion of wake, NREM sleep, and REM sleep from 24-hour records when determined by Sleep-Deep-Learner’s automated scoring, which is highly comparable to the one with manual scoring (B). On a 24-hour record-by-record basis, the proportion of wakefulness (D), NREM sleep (E), and REM sleep (F) correlate highly when determined by manual scoring versus Sleep-Deep-Learner’s automated scoring. Data were from eleven 24-hour recordings from six mice. Error bars represent SEM.

Figure 3.

Sleep-Deep-Learner retains sleep architecture of manual scoring. (A) A representative multitaper time-frequency plot of EEG from a 24-hour record with aligned hypnograms drawn from manual scores and Sleep-Deep-Learner’s automated scores revealing highly matching sleep–wake architecture captured by manual versus automated scoring. (B) Time-weighted bout analysis reveals the proportion of time (y-axis) the mice spent in bouts of various binned durations (x-axis) of wakefulness, NREM sleep, and REM sleep as determined from manual scores. (C) The same time-weighted bout analysis is shown of wakefulness, NREM sleep, and REM sleep as determined from automated scores. (D) Scatter plots with fitted least squares lines reveal the close relationship between time-weighted bouts for all three sleep–wake states on a record-by-record basis. Pearson’s correlation coefficients reveal statistically highly significant correlations between all three sleep–wake states comparing manual versus Sleep-Deep-Learner’s automated scores. Data were from eleven 24-hour recordings from six mice. Error bars represent SEM.

Figure 4.

Sleep-Deep-Learner retains sleep architecture of hypnotic-drug (zolpidem) induced sleep compared with manual scoring. (A) A representative multitaper time-frequency plot of EEG from a 3-hour record in which 5 mg/kg zolpidem was delivered by intraperitoneal injection with aligned hypnograms drawn from manual scores and Sleep-Deep-Learner’s automated scores demonstrating the highly matched scoring between manual and automated scoring. Of note, because the length of the recording was 3 hours, only 20 minutes of manual scoring was used for training the network for each file in this case. (B) F1 scores between manual and Sleep-Deep-Learner’s automated sleep scores reveal high reliability of drug-induced NREM sleep and overall reliability, with reliability of REM sleep in a similar range to WT records. The relative time spent in each sleep–wake state as determined by manual scores (C) versus Sleep-Deep-Learner’s automated scores (D) show similar values. N = 4. Error bars represent SEM.

Figure 5.

Sleep-Deep-Learner retains sleep architecture in local-field potential (LFP) data from a mouse model of Alzheimer’s disease (APP/PS1) compared with manual scoring. (A) F1 scores between manual and Sleep-Deep-Learner’s automated sleep scores were computed with 24-hour LFP recordings in frontal cortex performed on 5 APP/PS1 mice. The F1 scores reveal that Sleep-Deep-Learner scored NREM sleep, wake, and all states combined with high accuracy. The F1 score of REM sleep scoring was slightly lower than the other states, but in a similar range to the WT case shown in Figure 1. The relative time spent in each sleep–wake state as determined by manual scores (B) was highly similar to Sleep-Deep-Learner automated scores (C). N = 5. Error bars represent SEM.

Figure 6.

Sleep-Deep-Learner retains sleep architecture in local-field potential (LFP) data from hippocampus of mice compared with manual scoring. (A) F1 scores between manual and Sleep-Deep-Learner’s automated scores were computed with 24-hour LFP recordings performed in the hippocampus of 4 PV-Cre mice. The F1 scores reveal high reliability of Sleep-Deep-Learner in scoring NREM sleep, wake, and all states combined. The F1 score of REM sleep scoring was slightly lower than other states, but in a similar range to the WT case shown in Figure 1. The relative time spent in each sleep–wake state as determined by manual scores (B) was highly similar to Sleep-Deep-Learner’s automated scores (C) N = 4. Error bars represent SEM.

Figure 7.

The significant increase in NREM sleep delta power due to knocking down the expression of GABA_A alpha3 subunits in the thalamic reticular nucleus (Uygun et al., 2022) is reproduced when Sleep-Deep-Learner scored the data. Sleep–wake states were determined by manual scoring in panels A–C, from Uygun et al., 2022. (A) Baseline time-frequency power dynamics reveal elevated delta (1.5 – 4 Hz) during NREM sleep prior to rapid eye movement sleep (REM) transitions, during the whole 12 h light period. (B) Following the knock-down of alpha3 containing GABA_A receptors in the thalamic reticular nucleus (α3KD), high delta in NREM sleep prior to REM sleep transitions was further increased. (C) In contrast to their own BL levels , α3KD mice produced more delta power during NREM sleep prior to REM sleep transitions [t (5) = 2.14, p = .04]. Sleep–wake states determined by Sleep-Deep-Learner’s automated scores in panels (D)–(F) reproduced the effects found by manual scoring [t (5) = 6.6, p = .0006]. Significance was tested using one-tailed paired t-tests. Thick lines indicate mean; envelopes indicate SEM. N = 6.