A System for Drivers' Cognitive Load Estimation Based on Deep Convolutional Neural Networks and Facial Feature Analysis

Shyngyskhan Abilkassov, Merey Kairgaliyev, Bauyrzhan Zhakanov, Berdakh Abibullaev

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The driver's cognitive load estimation on the road is a critical factor for safety measurement that can reduce the overall number of road accidents. Factors such as distraction from driving, being "lost in thought, "conversation, cell phone use, and drowsiness, can be used to compute a driver's attention level on driving. To minimize the number of fatal and critical circumstances on the road, this work investigates a decision tree-based deep-learning algorithm focusing on the estimation of the driver's cognitive load. We construct and evaluate the performance of multimodal deep learning approaches, which combine two separately trained ResNet50 convolutional neural networks on data acquired from the driver's side and front face images. Further, we compare two different approaches for driver drowsiness level estimation using different computer vision algorithms. The first approach is implemented via the blinking ratio method using 68-landmarks of the face by calculating the blinking and yawning ratio. In contrast, the second one uses the contour area approach for drowsiness identification using morphological operations. As differences between driver behavioral classes are more distinct from side camera images than from the frontal image, the former has been chosen to be used for deep learning classification, and eventually, the accuracy of 92% has been achieved. On the other hand, the frontal image was successfully used to robustly detect the driver's drowsiness level by using computer vision techniques on facial landmarks. As a result, the decision tree was constructed to estimate the driver's driving safety level based on deep learning side posture classification and blinking ratio methods.

Original languageEnglish
Title of host publicationProceedings - 2021 22nd IEEE International Conference on Industrial Technology, ICIT 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages994-1000
Number of pages7
ISBN (Electronic)9781728157306
DOIs
Publication statusPublished - Mar 10 2021
Event22nd IEEE International Conference on Industrial Technology, ICIT 2021 - Valencia, Spain
Duration: Mar 10 2021Mar 12 2021

Publication series

NameProceedings of the IEEE International Conference on Industrial Technology
Volume2021-March

Conference

Conference22nd IEEE International Conference on Industrial Technology, ICIT 2021
Country/TerritorySpain
CityValencia
Period3/10/213/12/21

ASJC Scopus subject areas

  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'A System for Drivers' Cognitive Load Estimation Based on Deep Convolutional Neural Networks and Facial Feature Analysis'. Together they form a unique fingerprint.

Cite this