Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2013 Jan 15;9(1):81-7.
doi: 10.5664/jcsm.2350.

The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring

Richard S Rosenberg  1 , Steven Van Hout
Affiliations

The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring

Richard S Rosenberg et al. J Clin Sleep Med. .

Abstract

Study objectives: The program provides a unique opportunity to compare a large number of scorers with varied levels of experience to determine sleep stage scoring agreement. The objective is to examine areas of disagreement to inform future revisions of the AASM Manual for the Scoring of Sleep and Associated Events.

Methods: The sample included 9 record fragments, 1,800 epochs and more than 3,200,000 scoring decisions. More than 2,500 scorers, most with 3 or more years of experience, participated. The analysis determined agreement with the score chosen by the majority of scorers.

Results: Sleep stage agreement averaged 82.6%. Agreement was highest for stage R sleep with stages N2 and W approaching the same level. Scoring agreement for stage N3 sleep was 67.4% and was lowest for stage N1 at 63.0%. Scorers had particular difficulty with the last epoch of stage W before sleep onset, the first epoch of stage N2 after stage N1 and the first epoch of stage R after stage N2. Discrimination between stages N2 and N3 was particularly difficult for scorers.

Conclusions: These findings suggest that with current rules, inter-scorer agreement in a large group is approximately 83%, a level similar to that reported for agreement between expert scorers. Agreement in the scoring of stages N1 and N3 sleep was low. Modifications to the scoring rules to improve scoring during sleep stage transitions may result in improvement.

PubMed Disclaimer

Figures

Figure 1
Figure 1. Transition from stage W to stage N1 sleep (n = 7)
Figure 2
Figure 2. Transition from stage N1 to stage N2 sleep
Figure 3
Figure 3. Transition from stage N2 to stage R sleep
Figure 4
Figure 4. Transitions involving stage N3 sleep
Figure 5
Figure 5. Stage scores during a CPAP titration study
See this image and copyright information in PMC

Comment in

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Rechtschaffen A, Kales A. Washington, DC: US Department of Health, Education and Welfare Public Health Service -- NIH/NIND; 1968. A manual of standardized terminology, techniques and scoring system fo sleep stages of human subjects.
    1. Iber C, Ancoli-Israel S, Chesson A, Quan S for the American Academy of Sleep Medicine. Westchester, IL: American Academy of Sleep Medicine; 2007. The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications.
    1. Grigg-Damberger MM. The AASM scoring manual four years later. J Clin Sleep Med. 2012;8:323–32. - PMC - PubMed
    1. Moser D, Anderer P, Gruber G, et al. Sleep classification according to AASM and Rechtschaffen – Kales: effects on sleep scoring parameters. Sleep. 2009;32:139–49. - PMC - PubMed
    1. Danker-Hopfe H, Anderer P, Zeitlhofer J, et al. Interrater reliability for sleep scoring according to the Rechtschaffen – Kales and the new AASM standard. J Sleep Res. 2009;18:74–84. - PubMed