Evaluation of manual and non-manual components for sign language recognition

Medet Mukushev; Arman Sabyrov; Alfarabi Imashev; Kenessary Koishybay; Vadim Kimmelman; Anara Sandygulova

Evaluation of manual and non-manual components for sign language recognition

Medet Mukushev, Arman Sabyrov, Alfarabi Imashev, Kenessary Koishybay, Vadim Kimmelman, Anara Sandygulova

Nazarbayev University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

21 Citations (Scopus)

Abstract

The motivation behind this work lies in the need to differentiate between similar signs that differ in non-manual components present in any sign. To this end, we recorded full sentences signed by five native signers and extracted 5200 isolated sign samples of twenty frequently used signs in Kazakh-Russian Sign Language (K-RSL), which have similar manual components but differ in non-manual components (i.e. facial expressions, eyebrow height, mouth, and head orientation). We conducted a series of evaluations in order to investigate whether non-manual components would improve sign's recognition accuracy. Among standard machine learning approaches, Logistic Regression produced the best results, 78.2% of accuracy for dataset with 20 signs and 77.9% of accuracy for dataset with 2 classes (statement vs question).

Original language	English
Title of host publication	LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings
Editors	Nicoletta Calzolari, Frederic Bechet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Publisher	European Language Resources Association (ELRA)
Pages	6073-6078
Number of pages	6
ISBN (Electronic)	9791095546344
Publication status	Published - 2020
Event	12th International Conference on Language Resources and Evaluation, LREC 2020 - Marseille, France Duration: May 11 2020 → May 16 2020

Publication series

Name	LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings

Conference

Conference	12th International Conference on Language Resources and Evaluation, LREC 2020
Country/Territory	France
City	Marseille
Period	5/11/20 → 5/16/20

Funding

This work was supported by the Nazarbayev University Faculty Development Competitive Research Grant Program 2019-2021 “Kazakh Sign Language Automatic Recognition System (K-SLARS)”. Award number is 110119FD4545.

Keywords

Information extraction
Machine learning methods
Sign language Recognition
Statistical

ASJC Scopus subject areas

Language and Linguistics
Education
Library and Information Sciences
Linguistics and Language

Cite this

Mukushev, M., Sabyrov, A., Imashev, A., Koishybay, K., Kimmelman, V., & Sandygulova, A. (2020). Evaluation of manual and non-manual components for sign language recognition. In N. Calzolari, F. Bechet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, & S. Piperidis (Eds.), LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings (pp. 6073-6078). (LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings). European Language Resources Association (ELRA).

Evaluation of manual and non-manual components for sign language recognition. / Mukushev, Medet; Sabyrov, Arman; Imashev, Alfarabi et al.
LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings. ed. / Nicoletta Calzolari; Frederic Bechet; Philippe Blache; Khalid Choukri; Christopher Cieri; Thierry Declerck; Sara Goggi; Hitoshi Isahara; Bente Maegaard; Joseph Mariani; Helene Mazo; Asuncion Moreno; Jan Odijk; Stelios Piperidis. European Language Resources Association (ELRA), 2020. p. 6073-6078 (LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Mukushev, M, Sabyrov, A, Imashev, A, Koishybay, K, Kimmelman, V & Sandygulova, A 2020, Evaluation of manual and non-manual components for sign language recognition. in N Calzolari, F Bechet, P Blache, K Choukri, C Cieri, T Declerck, S Goggi, H Isahara, B Maegaard, J Mariani, H Mazo, A Moreno, J Odijk & S Piperidis (eds), LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings. LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings, European Language Resources Association (ELRA), pp. 6073-6078, 12th International Conference on Language Resources and Evaluation, LREC 2020, Marseille, France, 5/11/20.

Mukushev M, Sabyrov A, Imashev A, Koishybay K, Kimmelman V, Sandygulova A. Evaluation of manual and non-manual components for sign language recognition. In Calzolari N, Bechet F, Blache P, Choukri K, Cieri C, Declerck T, Goggi S, Isahara H, Maegaard B, Mariani J, Mazo H, Moreno A, Odijk J, Piperidis S, editors, LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings. European Language Resources Association (ELRA). 2020. p. 6073-6078. (LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings).

Mukushev, Medet ; Sabyrov, Arman ; Imashev, Alfarabi et al. / Evaluation of manual and non-manual components for sign language recognition. LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings. editor / Nicoletta Calzolari ; Frederic Bechet ; Philippe Blache ; Khalid Choukri ; Christopher Cieri ; Thierry Declerck ; Sara Goggi ; Hitoshi Isahara ; Bente Maegaard ; Joseph Mariani ; Helene Mazo ; Asuncion Moreno ; Jan Odijk ; Stelios Piperidis. European Language Resources Association (ELRA), 2020. pp. 6073-6078 (LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings).

@inproceedings{47d4cde3620b48af85417173605b7761,

title = "Evaluation of manual and non-manual components for sign language recognition",

abstract = "The motivation behind this work lies in the need to differentiate between similar signs that differ in non-manual components present in any sign. To this end, we recorded full sentences signed by five native signers and extracted 5200 isolated sign samples of twenty frequently used signs in Kazakh-Russian Sign Language (K-RSL), which have similar manual components but differ in non-manual components (i.e. facial expressions, eyebrow height, mouth, and head orientation). We conducted a series of evaluations in order to investigate whether non-manual components would improve sign's recognition accuracy. Among standard machine learning approaches, Logistic Regression produced the best results, 78.2\% of accuracy for dataset with 20 signs and 77.9\% of accuracy for dataset with 2 classes (statement vs question).",

keywords = "Information extraction, Machine learning methods, Sign language Recognition, Statistical",

author = "Medet Mukushev and Arman Sabyrov and Alfarabi Imashev and Kenessary Koishybay and Vadim Kimmelman and Anara Sandygulova",

note = "Funding Information: This work was supported by the Nazarbayev University Faculty Development Competitive Research Grant Program 2019-2021 “Kazakh Sign Language Automatic Recognition System (K-SLARS)”. Award number is 110119FD4545. Publisher Copyright: {\textcopyright} European Language Resources Association (ELRA), licensed under CC-BY-NC; 12th International Conference on Language Resources and Evaluation, LREC 2020 ; Conference date: 11-05-2020 Through 16-05-2020",

year = "2020",

language = "English",

series = "LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings",

publisher = "European Language Resources Association (ELRA)",

pages = "6073--6078",

editor = "Nicoletta Calzolari and Frederic Bechet and Philippe Blache and Khalid Choukri and Christopher Cieri and Thierry Declerck and Sara Goggi and Hitoshi Isahara and Bente Maegaard and Joseph Mariani and Helene Mazo and Asuncion Moreno and Jan Odijk and Stelios Piperidis",

booktitle = "LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings",

}

TY - GEN

T1 - Evaluation of manual and non-manual components for sign language recognition

AU - Mukushev, Medet

AU - Sabyrov, Arman

AU - Imashev, Alfarabi

AU - Koishybay, Kenessary

AU - Kimmelman, Vadim

AU - Sandygulova, Anara

N1 - Funding Information: This work was supported by the Nazarbayev University Faculty Development Competitive Research Grant Program 2019-2021 “Kazakh Sign Language Automatic Recognition System (K-SLARS)”. Award number is 110119FD4545. Publisher Copyright: © European Language Resources Association (ELRA), licensed under CC-BY-NC

PY - 2020

Y1 - 2020

N2 - The motivation behind this work lies in the need to differentiate between similar signs that differ in non-manual components present in any sign. To this end, we recorded full sentences signed by five native signers and extracted 5200 isolated sign samples of twenty frequently used signs in Kazakh-Russian Sign Language (K-RSL), which have similar manual components but differ in non-manual components (i.e. facial expressions, eyebrow height, mouth, and head orientation). We conducted a series of evaluations in order to investigate whether non-manual components would improve sign's recognition accuracy. Among standard machine learning approaches, Logistic Regression produced the best results, 78.2% of accuracy for dataset with 20 signs and 77.9% of accuracy for dataset with 2 classes (statement vs question).

AB - The motivation behind this work lies in the need to differentiate between similar signs that differ in non-manual components present in any sign. To this end, we recorded full sentences signed by five native signers and extracted 5200 isolated sign samples of twenty frequently used signs in Kazakh-Russian Sign Language (K-RSL), which have similar manual components but differ in non-manual components (i.e. facial expressions, eyebrow height, mouth, and head orientation). We conducted a series of evaluations in order to investigate whether non-manual components would improve sign's recognition accuracy. Among standard machine learning approaches, Logistic Regression produced the best results, 78.2% of accuracy for dataset with 20 signs and 77.9% of accuracy for dataset with 2 classes (statement vs question).

KW - Information extraction

KW - Machine learning methods

KW - Sign language Recognition

KW - Statistical

UR - https://www.scopus.com/pages/publications/85095539455

UR - https://www.scopus.com/inward/citedby.url?scp=85095539455&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85095539455

T3 - LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings

SP - 6073

EP - 6078

BT - LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings

A2 - Calzolari, Nicoletta

A2 - Bechet, Frederic

A2 - Blache, Philippe

A2 - Choukri, Khalid

A2 - Cieri, Christopher

A2 - Declerck, Thierry

A2 - Goggi, Sara

A2 - Isahara, Hitoshi

A2 - Maegaard, Bente

A2 - Mariani, Joseph

A2 - Mazo, Helene

A2 - Moreno, Asuncion

A2 - Odijk, Jan

A2 - Piperidis, Stelios

PB - European Language Resources Association (ELRA)

T2 - 12th International Conference on Language Resources and Evaluation, LREC 2020

Y2 - 11 May 2020 through 16 May 2020

ER -

Evaluation of manual and non-manual components for sign language recognition

Abstract

Publication series

Conference

Funding

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this