Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf

Alikhan Abutalipov; Aigerim Janaliyeva; Medet Mukushev; Antonio Cerone; Anara Sandygulova

doi:10.1007/978-3-030-70650-0_14

Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf

Alikhan Abutalipov
, Aigerim Janaliyeva
, Medet Mukushev
, Antonio Cerone
, Anara Sandygulova

Nazarbayev University

Nazarbayev University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

2 Downloads (Pure)

Abstract

This paper showcases the work that aims at building a user-friendly mobile application of a reverse dictionary to translate sign languages to spoken languages. The concept behind the reverse dictionary is the ability to perform a video-based search by demonstrating a handshape in front of a mobile phone’s camera. The user would be able to use this feature in two ways. Firstly, the user would be able to search for a word by showing a handshape for the application to provide a list of signs that contain that handshape. Secondly, the user could fingerspell the word letter by letter in front of the camera for the application to return the sign that corresponds to that word. The user can then look through the suggested videos and see their written translations. To offer other functionalities, the application also has Search by Category and Search by Word options. Currently, the reverse dictionary supports translations from Russian Sign Language (RSL) to Russian language.

Original language	English
Title of host publication	From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers
Editors	Juliana Bowles, Giovanna Broccia, Mirco Nanni
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	217-226
Number of pages	10
ISBN (Print)	9783030706494
DOIs	https://doi.org/10.1007/978-3-030-70650-0_14
Publication status	Published - 2021
Event	9th International Symposium on From Data Models and Back, DataMod 2020 - Virtual, Online Duration: Oct 20 2020 → Oct 20 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12611 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Symposium on From Data Models and Back, DataMod 2020
City	Virtual, Online
Period	10/20/20 → 10/20/20

Funding

Acknowledgment. This work was supported by the Nazarbayev University Faculty Development Competitive Research Grant Program 2019–2021 “Kazakh Sign Language Automatic Recognition System (K-SLARS)”. Award number is 110119FD4545.

Keywords

Fingerspelling recognition
Human-computer interaction
iOS application
Reverse dictionary
Russian Sign Language (RSL)
Sign language dictionary
Video-based search interface

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-70650-0_14

Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf 47bca526Final published version, 4.43 MB

Cite this

Abutalipov, A., Janaliyeva, A., Mukushev, M., Cerone, A., & Sandygulova, A. (2021). Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf. In J. Bowles, G. Broccia, & M. Nanni (Eds.), From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers (pp. 217-226). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12611 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-70650-0_14

Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf. / Abutalipov, Alikhan; Janaliyeva, Aigerim; Mukushev, Medet et al.
From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers. ed. / Juliana Bowles; Giovanna Broccia; Mirco Nanni. Springer Science and Business Media Deutschland GmbH, 2021. p. 217-226 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12611 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abutalipov, A, Janaliyeva, A, Mukushev, M, Cerone, A & Sandygulova, A 2021, Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf. in J Bowles, G Broccia & M Nanni (eds), From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12611 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 217-226, 9th International Symposium on From Data Models and Back, DataMod 2020, Virtual, Online, 10/20/20. https://doi.org/10.1007/978-3-030-70650-0_14

Abutalipov A, Janaliyeva A, Mukushev M, Cerone A , Sandygulova A. Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf. In Bowles J, Broccia G, Nanni M, editors, From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2021. p. 217-226. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-70650-0_14

Abutalipov, Alikhan ; Janaliyeva, Aigerim ; Mukushev, Medet et al. / Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf. From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers. editor / Juliana Bowles ; Giovanna Broccia ; Mirco Nanni. Springer Science and Business Media Deutschland GmbH, 2021. pp. 217-226 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{6a03cc4dd978479d97eb960847aa0393,

title = "Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf",

abstract = "This paper showcases the work that aims at building a user-friendly mobile application of a reverse dictionary to translate sign languages to spoken languages. The concept behind the reverse dictionary is the ability to perform a video-based search by demonstrating a handshape in front of a mobile phone{\textquoteright}s camera. The user would be able to use this feature in two ways. Firstly, the user would be able to search for a word by showing a handshape for the application to provide a list of signs that contain that handshape. Secondly, the user could fingerspell the word letter by letter in front of the camera for the application to return the sign that corresponds to that word. The user can then look through the suggested videos and see their written translations. To offer other functionalities, the application also has Search by Category and Search by Word options. Currently, the reverse dictionary supports translations from Russian Sign Language (RSL) to Russian language.",

keywords = "Fingerspelling recognition, Human-computer interaction, iOS application, Reverse dictionary, Russian Sign Language (RSL), Sign language dictionary, Video-based search interface",

author = "Alikhan Abutalipov and Aigerim Janaliyeva and Medet Mukushev and Antonio Cerone and Anara Sandygulova",

note = "Funding Information: Acknowledgment. This work was supported by the Nazarbayev University Faculty Development Competitive Research Grant Program 2019–2021 “Kazakh Sign Language Automatic Recognition System (K-SLARS)”. Award number is 110119FD4545. Publisher Copyright: {\textcopyright} 2021, The Author(s).; 9th International Symposium on From Data Models and Back, DataMod 2020 ; Conference date: 20-10-2020 Through 20-10-2020",

year = "2021",

doi = "10.1007/978-3-030-70650-0\_14",

language = "English",

isbn = "9783030706494",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "217--226",

editor = "Juliana Bowles and Giovanna Broccia and Mirco Nanni",

booktitle = "From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers",

address = "Germany",

}

TY - GEN

T1 - Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf

AU - Abutalipov, Alikhan

AU - Janaliyeva, Aigerim

AU - Mukushev, Medet

AU - Cerone, Antonio

AU - Sandygulova, Anara

N1 - Funding Information: Acknowledgment. This work was supported by the Nazarbayev University Faculty Development Competitive Research Grant Program 2019–2021 “Kazakh Sign Language Automatic Recognition System (K-SLARS)”. Award number is 110119FD4545. Publisher Copyright: © 2021, The Author(s).

PY - 2021

Y1 - 2021

N2 - This paper showcases the work that aims at building a user-friendly mobile application of a reverse dictionary to translate sign languages to spoken languages. The concept behind the reverse dictionary is the ability to perform a video-based search by demonstrating a handshape in front of a mobile phone’s camera. The user would be able to use this feature in two ways. Firstly, the user would be able to search for a word by showing a handshape for the application to provide a list of signs that contain that handshape. Secondly, the user could fingerspell the word letter by letter in front of the camera for the application to return the sign that corresponds to that word. The user can then look through the suggested videos and see their written translations. To offer other functionalities, the application also has Search by Category and Search by Word options. Currently, the reverse dictionary supports translations from Russian Sign Language (RSL) to Russian language.

AB - This paper showcases the work that aims at building a user-friendly mobile application of a reverse dictionary to translate sign languages to spoken languages. The concept behind the reverse dictionary is the ability to perform a video-based search by demonstrating a handshape in front of a mobile phone’s camera. The user would be able to use this feature in two ways. Firstly, the user would be able to search for a word by showing a handshape for the application to provide a list of signs that contain that handshape. Secondly, the user could fingerspell the word letter by letter in front of the camera for the application to return the sign that corresponds to that word. The user can then look through the suggested videos and see their written translations. To offer other functionalities, the application also has Search by Category and Search by Word options. Currently, the reverse dictionary supports translations from Russian Sign Language (RSL) to Russian language.

KW - Fingerspelling recognition

KW - Human-computer interaction

KW - iOS application

KW - Reverse dictionary

KW - Russian Sign Language (RSL)

KW - Sign language dictionary

KW - Video-based search interface

UR - https://www.scopus.com/pages/publications/85103590191

UR - https://www.scopus.com/pages/publications/85103590191#tab=citedBy

U2 - 10.1007/978-3-030-70650-0_14

DO - 10.1007/978-3-030-70650-0_14

M3 - Conference contribution

AN - SCOPUS:85103590191

SN - 9783030706494

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 217

EP - 226

BT - From Data to Models and Back - 9th International Symposium, DataMod 2020, Revised Selected Papers

A2 - Bowles, Juliana

A2 - Broccia, Giovanna

A2 - Nanni, Mirco

PB - Springer Science and Business Media Deutschland GmbH

T2 - 9th International Symposium on From Data Models and Back, DataMod 2020

Y2 - 20 October 2020 through 20 October 2020

ER -

Handshape Classification in a Reverse Dictionary of Sign Languages for the Deaf

Abstract

Publication series

Conference

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this