Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter June 6, 2022

Automatic sleep scoring with LSTM networks: impact of time granularity and input signals

  • Alexandra-Maria Tăuțan ORCID logo EMAIL logo , Alessandro C. Rossi and Bogdan Ionescu


Supervised automatic sleep scoring algorithms are usually trained using sleep stage labels manually annotated on 30 s epochs of PSG data. In this study, we investigate the impact of using shorter epochs with various PSG input signals for training and testing a Long Short Term Memory (LSTM) neural network. An LSTM model is evaluated on the provided 30 s epoch sleep stage labels from a publicly available dataset, as well as on 10 s subdivisions. Additionally, three independent scorers re-labeled a subset of the dataset on shorter time windows. The automatic sleep scoring experiments were repeated on the re-annotated subset.The highest performance is achieved on features extracted from 30 s epochs of a single channel frontal EEG. The resulting accuracy, precision and recall were of 92.22%, 67.58% and 66.00% respectively. When using a shorter epoch as input, the performance decreased by approximately 20%. Re-annotating a subset of the dataset on shorter time epochs did not improve the results and further altered the sleep stage detection performance. Our results show that our feature-based LSTM classification algorithm performs better on 30 s PSG epochs when compared to 10 s epochs used as input. Future work could be oriented to determining whether varying the epoch size improves classification outcomes for different types of classification algorithms.

Corresponding author: Alexandra-Maria Tăuțan, University Politehnica of Bucharest, Splaiul Independenței 313, 060042, Bucharest, Romania; and Onera Health, Torenallee 42-54, 5617BD Eindhoven, The Netherlands, E-mail:


The authors would like to thank Alexandra Jahaleanu, Marjolein Vliexs and Makrina Sekeri from Onera Health for their support in the re-annotation process.

  1. Research funding: None declared.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Not applicable.

  5. Ethical approval: Not applicable.


1. Perry, G, Patil, S, Presley-Cantrell, L. Raising awareness of sleep as a healthy behavior how much sleep is needed and are we there ? why is sleep a public health issue? Call to action. Prev Chronic Dis 2013;10:130081. in Google Scholar PubMed PubMed Central

2. Killgore, WDS, Balkin, TJ, Yarnell, AM, Capaldi, VF. Sleep deprivation impairs recognition of specific emotions. Neurobiol Sleep Circadian Rhythms 2017;3:10–6. in Google Scholar PubMed PubMed Central

3. Schwarz, J, Axelsson, J, Gerhardsson, A, Tamm, S, Fischer, H, Kecklund, G, et al.. Mood impairment is stronger in young than in older adults after sleep deprivation. J Sleep Res 2019;28. in Google Scholar PubMed PubMed Central

4. Nollet, M, Wisden, W, Franks, NP. Sleep deprivation and stress: a reciprocal relationship. Interface Focus 2020;10:3. in Google Scholar PubMed PubMed Central

5. Hudson, AN, Van Dongen, HPA, Honn, KA. Sleep deprivation, vigilant attention, and brain function: a review. Neuropsychopharmacology 2020;45:21–30. in Google Scholar PubMed PubMed Central

6. Wu, H, Dunnett, S, Ho, YS, Chang, RCC. The role of sleep deprivation and circadian rhythm disruption as risk factors of Alzheimer’s disease. Front Neuroendocrinol 2019;54:100764. in Google Scholar PubMed

7. St Louis, EK, Boeve, AR, Boeve, BF. REM sleep behavior disorder in Parkinson’s disease and other synucleinopathies. Mov Disord 2017;32:645–58. in Google Scholar PubMed

8. Romanella, SM, Roe, D, Tatti, E, Cappon, D, Paciorek, R, Testani, E, et al.. The sleep side of aging and alzheimer’s disease. Sleep Med 2021;77:209–25. in Google Scholar PubMed PubMed Central

9. Acquavella, J, Mehra, J, Bron, M, Suomi, JMH, Hess, GP. Prevalence of narcolepsy and other sleep disorders and frequency of diagnostic tests from 2013-2016 in insured patients actively seeking care. J Clin Sleep Med 2020;16:1255–63. in Google Scholar PubMed PubMed Central

10. Berry, RB, Brooks, R, Gamaldo, CE, Harding, SM, Lloyd, RM, Quan, SF, et al.. The AASM manual for the scoring of sleep and associated events. Version 2.4.Am. Acad. Sleep Med 2017.Search in Google Scholar

11. Loh, HW, Ooi, CP, Vicnesh, J, Oh, SL, Fuast, O, Gertych, A, et al.. Automated detection of sleep stages using deep learning techniques: a systematic review of the last decade (2010–2020). Appl Sci Switz 2020;10:1–25. in Google Scholar

12. Chriskos, P, Frantzidis, CA, Nday, CM, Gkivogkli, PT, Bamidis, PD, Kourtidou-Papadeli, C. A review on current trends in automatic sleep staging through bio-signal recordings and future challenges. Sleep Med Rev 2021;55:101377. in Google Scholar PubMed

13. Fernandez-Varela, I, Hernandez-Pereira, E, Alvarez-Estevez, D, Moret-Bonillo, V. A convolutional network for sleep stages classification; 2019. arXiv:1902.05748.10.3390/proceedings2181174Search in Google Scholar

14. Tautan, AM, Rossi, AC, De Franciso, R, Ionescu, B. Automatic sleep stage detection using a single channel frontal EEG. Presented at the 7th IEEE international conference on e-health and bioengineering – EHB, Iasi Romania; 2019.10.1109/EHB47216.2019.8969973Search in Google Scholar

15. Biswal, S, Kulas, J, Sun, H, Goparaju, B, Westover, MB, Bianchi, MT, et al.. SLEEPNET: automated sleep staging system via deep learning; 2017. arXiv, 1–17.Search in Google Scholar

16. Hassan, AR, Bashar, SK, Bhuiyan, MIH. On the classification of sleep states by means of statistical and spectral features from single channel Electroencephalogram. In: 2015 International conference on advances in computing, communications and informatics (ICACCI), Kochi, India; 2015:2238–43 pp. in Google Scholar

17. Alickovic, E, Subasi, A. Ensemble SVM method for automatic sleep stage classification. IEEE Trans Instrum Meas 2018;67:1258–65. in Google Scholar

18. Hassan, AR, Bhuiyan, MIH. Automatic sleep scoring using statistical features in the EMD domain and ensemble methods. Biocybern Biomed Eng 2016;36:248–55. in Google Scholar

19. Sundararajan, K, Georgievska, G, te Lindert, BHW, Gehrman, PR, Ramauter, J, Mazzotti, DR, et al.. Sleep classification from wrist-worn accelerometer data using random forests. Sci Rep 2021;11:1–10. in Google Scholar PubMed PubMed Central

20. Cen, L, Yu, ZL, Tang, Y, Shi, W, Kluge, T, Ser, W. Deep learning method for sleep stage classification. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2017;796–802. in Google Scholar

21. Phan, H, Ch, OY, Koch, P, Mertins, A, Vos, MD. Fusion of end-to-end deep learning models for sequence-to-sequence sleep staging 2019:1829–33. in Google Scholar

22. Muller, B, Gabelein, WD, Schulz, H. A taxonomic analysis of sleep stages. Sleep 2006;29:967–74. in Google Scholar PubMed

23. Cesari, M, Stefani, A, Penzel, T, Ibrahim, A, Hackner, H, Heidbreder, A, et al.. Inter-rater sleep stage scoring reliability between manual scoring from two European sleep centers and automatic scoring performed by the artificial intelligence-based Stanford-STAGES algorithm. J Clin Sleep Med 2021;17:1237–47. in Google Scholar PubMed PubMed Central

24. Koch, H, Jennum, P, Christensen, JAE. Automatic sleep classification using adaptive segmentation reveals an increased number of rapid eye movement sleep transitions. J Sleep Res 2019;28. in Google Scholar PubMed

25. Álvarez-Estévez, D, Fernández-Pastoriza, JM, Hernández-Pereira, E, Moret-Bonillo, V. A method for the automatic analysis of the sleep macrostructure in continuum. Expert Syst Appl 2013;40:1796–803. in Google Scholar

26. Supratak, A, Dong, H, Wu, C, Guo, Y, DeepSleepNet. A model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Trans Neural Syst Rehabil Eng 2017;25:1998–2008. in Google Scholar

27. Phan, H, Andreotti, F, Cooray, N, Chen, OY, de Vos, M. SeqSleepNet : end-to-end hierarchical recurrent neural network for sleep staging. IEEE Trans Neural Syst Rehabil Eng 2019;27:400–10. in Google Scholar

28. Rosenburg, R, van Hout, S. The American academy of sleep medicine interscorer reliability program: sleep stage scoring. J Clin Sleep Med 2013;9.10.5664/jcsm.2350Search in Google Scholar PubMed PubMed Central

29. Goldberger, AA, Glass, L, Hausdorff, JM, Ivanov, P, Ivanov, PC, Mark, RG, et al.. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 200AD;101:215–20. in Google Scholar PubMed

30. Ghassemi, MM, Moody, BE, Lehman, LW, Song, C, Li, Q, Sun, H, et al.. The PhysioNet/computing in cardiology challenge 2018. Computing in Cardiology Conference (CinC) 2018;20–3. in Google Scholar PubMed PubMed Central

31. Iber, C, Ancoli-Israel, S, Chesson, A, Stuart, Q. The AASM manual for the scoring of sleep and associated events 2007.Search in Google Scholar

32. Gwet, K. Handbook of inter-rater reliability, 4th ed. Gaithersburg: Advanced Analytics LLC; 2014.Search in Google Scholar

33. Grigg-Damberger, MM. The AASM scoring manual four years later. J Clin Sleep Med 2012;8:323–32. in Google Scholar PubMed PubMed Central

34. Cohen, JA. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960:37–46.10.1177/001316446002000104Search in Google Scholar

35. Fleiss, JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971;76:378–82. in Google Scholar

36. Tautan, A, Rossi, AC, De Francisco, R, Ionescu, B. Automatic sleep stage detection: a study on the influence of various PSG input signals. Presented at the 42nd annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC) in conjunction with the 43rd annual conference of the Canadian Medical and Biological Engineering Society 2020.10.1109/EMBC44109.2020.9175628Search in Google Scholar PubMed

37. Tautan, AM, Rossi, AC, De Francisco, R, Ionescu, B. Dimensionality reduction for EEG-based sleep stage detection: comparison of autoencoders, principal component analysis and factor analysis. Biomed Tech 2020;66:125–36. in Google Scholar PubMed

38. Laza, R, Pavón, R, Reboiro-Jato, M, Fdez-Riverola, F. Evaluating the effect of unbalanced data in biomedical document classification. J Integr Bioinformat 2011;8:177. in Google Scholar

39. More, A. Survey of resampling techniques for improving classification performance in unbalanced datasets 2016;10000:1–7.Search in Google Scholar

40. Pozzolo, AD, Caelen, O, Bontempi, G. When is undersampling effective in unbalanced classification tasks? Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2015;9284:200–15.10.1007/978-3-319-23528-8_13Search in Google Scholar

41. Goodfellow, I, Bengio, Y, Courville, A. Deep learning. MIT Press; 2016. [Online]. Available from: in Google Scholar

42. Yu, Y, Si, X, Hu, C, Zhang, J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 2019;31:1235–70. in Google Scholar PubMed

43. Hochreiter, S, Schmidhuber, J. Long short-term memory. Neural Comput 1997;9:1735–80. in Google Scholar PubMed

44. Kingma, DP, Ba, JL. Adam: a method for stochastic optimization. In: 3rd int conf learn represent ICLR 2015 - conf track proc. San Diego, USA: ICLR; 2015:1–15 pp.Search in Google Scholar

45. Pedregosa, F, Varoquaux, G, Gramfort, A, Michel, V, Thirion, B, Grisel, O, et al.. Scikit-learn: machine learning in Python. J Mach Learn Res 2011;12:2825–30.Search in Google Scholar

46. Barnas, G, Mackenzie, CF, Skacel, M, Hempleman, SC, Wicke, KM, Skacel, CM, et al.. Amplitude dependency of regional chest wall resistance and elastance at normal breathing frequencies. Am Rev Respir Dis 1988;140. in Google Scholar PubMed

47. Meng, Q, Catchpoole, D, Skillicorn, D, Kennedy, PJ, Relational autoencoder for feature extraction, arXiv, 364–371, 2018.10.1109/IJCNN.2017.7965877Search in Google Scholar

48. Chen, Z, Yang, F. Intelligent terminal based on single channel EEG signal LSTM sleep staging. Comput Sci Appl 2019;9:1156–68. in Google Scholar

49. Sun, Y, Wang, B, Jin, J, Wang, X. Deep convolutional network method for automatic sleep stage classification based on neurophysiological signals 2018:1–5.10.1109/CISP-BMEI.2018.8633058Search in Google Scholar

50. Hao, J, Luo, S, Pan, L A novel sleep staging algorithm based on hybrid neural network 2019;2019:1–4.10.1109/ICEIEC.2019.8784612Search in Google Scholar

51. Yang, Y, Zheng, X, Yuan, F. A study on automatic sleep stage classification based on CNN-LSTM. In: Int conf comput sci educ. Colombo, Sri Lanka: Int conf comput sci educ; 2018:1–5 pp.10.1145/3265689.3265693Search in Google Scholar

52. Michielli, N, Acharya, UR, Molinari, F. Cascaded LSTM recurrent neural network for automated sleep stage classification using single-channel EEG signals. Comput Biol Med 2019;106:71–81. in Google Scholar PubMed

53. Kuo, C, Chen, G, Lin, N. Automatic sleep staging using deep long short-term memory: validation in large-scale datasets 2019;58–64.10.1145/3365966.3365980Search in Google Scholar

54. Huang, Y, Zhang, Y. A two-phase model for sleep staging using single channel EEG. 2018 Int. Conf. Big Data Artif. Intell. BDAI 2018;1:123–7.10.1109/BDAI.2018.8546683Search in Google Scholar

55. Zhang, L, Fabbri, D, Upender, R, Kent, D. Automated sleep stage scoring of the sleep heart health study using deep neural networks. Sleep 2019;42:zsz159. in Google Scholar PubMed PubMed Central

56. Rahman, MM, Bhuiyan, MIH, Hassan, AR. Sleep stage classification using single-channel EOG. Comput Biol Med 2018;102:211–20. in Google Scholar PubMed

57. Hassan, AR, Bhuiyan, MIH. A decision support system for automatic sleep staging from EEG signals using tunable Q-factor wavelet transform and spectral features. J Neurosci Methods 2016;271:107–18. in Google Scholar PubMed

58. Hassan, AR, Subasi, A. A decision support system for automated identification of sleep stages from single-channel EEG signals. Knowl-Based Syst 2017;115;128–124. in Google Scholar

59. Zaremba, W, Sutskever, I, Vinyals, O. Recurrent neural network regularization 2014;2013:1–8.Search in Google Scholar

Received: 2021-12-08
Accepted: 2022-05-17
Published Online: 2022-06-06
Published in Print: 2022-08-26

© 2022 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 26.9.2023 from
Scroll to top button