Towards Safer Flights: A Multi-modality Fusion Technology-based Cognitive Load Recognition Framework

Yuhan Li, Ke Li, Shaofan Wang, Yuangan Li, Jia'Ao Chen, Dongsheng Wen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

The cognitive overload experienced by pilots in arduous circumstances is related to the psychological state and sympathetic response of the human subject. The physiological signals of subjects are predictive and reliable in detecting their mental states, thus preventing cognitive overload. This study proposes a Multi-modality Fusion Technology (MFT) based model for recognizing pilot cognitive load through pilots' physiological signals, including electrocardiosignal (ECG), photoplethysmography (PPG), electrodermal response (EDA), electromyography signal (EMG), respiration signal (RESP) and skin temperature signal (SKT). From these signals features are extracted and then fused at the feature layer into a united vector. This vector is then sent into the model for learning. In the decision level, individual decisions from several models are combined and a decision is finalized. Various subjects were involved in experimental data collection on a flight simulator, and the collected data were used to train and test the model. The model is evaluated through both 10-fold cross-validation and Leave-One-person-Out (LOO) cross-validation.

Original languageEnglish
Title of host publicationProceedings of 2022 IEEE 4th International Conference on Civil Aviation Safety and Information Technology, ICCASIT 2022
EditorsHuabo Sun
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages525-530
Number of pages6
ISBN (Electronic)9781665467667
DOIs
StatePublished - 2022
Externally publishedYes
Event4th IEEE International Conference on Civil Aviation Safety and Information Technology, ICCASIT 2022 - Dali, China
Duration: 12 Oct 202214 Oct 2022

Publication series

NameProceedings of 2022 IEEE 4th International Conference on Civil Aviation Safety and Information Technology, ICCASIT 2022

Conference

Conference4th IEEE International Conference on Civil Aviation Safety and Information Technology, ICCASIT 2022
Country/TerritoryChina
CityDali
Period12/10/2214/10/22

Keywords

  • cognitive load
  • ensemble learning
  • machine learning
  • multi-modality
  • physiological signals
  • pilot states
  • workload recognition

Fingerprint

Dive into the research topics of 'Towards Safer Flights: A Multi-modality Fusion Technology-based Cognitive Load Recognition Framework'. Together they form a unique fingerprint.

Cite this