Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL

Anna Kuznetsova; Alfarabi Imashev; Medet Mukushev; Anara Sandygulova; Vadim Kimmelman

Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL

Anna Kuznetsova, Alfarabi Imashev, Medet Mukushev, Anara Sandygulova, Vadim Kimmelman

Department of Robotics and Mechatronics

Research output: Contribution to conference › Paper › peer-review

8 Citations (Scopus)

Abstract

This paper presents a study that compares non-manual markers of polar and wh-questions to statements in Kazakh-Russian Sign Language (KRSL) in a dataset collected for NLP tasks. The primary focus of the study is to demonstrate the utility of computer vision solutions for the linguistic analysis of non-manuals in sign languages, although additional corrections are required to account for biases in the output. To this end, we analyzed recordings of 10 triplets of sentences produced by 9 native signers using both manual annotation and computer vision solutions (such as OpenFace). We utilize and improve the computer vision solution, and briefly describe the results of the linguistic analysis.

Original language	English
Pages	49-59
Number of pages	11
Publication status	Published - 2021
Event	1st International Workshop on Automatic Translation for Signed and Spoken Languages, AT4SSL 2021 - Virtual, Online, United States Duration: Aug 16 2021 → Aug 20 2021

Conference

Conference	1st International Workshop on Automatic Translation for Signed and Spoken Languages, AT4SSL 2021
Country/Territory	United States
City	Virtual, Online
Period	8/16/21 → 8/20/21

ASJC Scopus subject areas

Language and Linguistics
Artificial Intelligence
Software
Linguistics and Language

Cite this

@conference{6d0d6d71c9ca43b09720429e3c7320b7,

title = "Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL",

abstract = "This paper presents a study that compares non-manual markers of polar and wh-questions to statements in Kazakh-Russian Sign Language (KRSL) in a dataset collected for NLP tasks. The primary focus of the study is to demonstrate the utility of computer vision solutions for the linguistic analysis of non-manuals in sign languages, although additional corrections are required to account for biases in the output. To this end, we analyzed recordings of 10 triplets of sentences produced by 9 native signers using both manual annotation and computer vision solutions (such as OpenFace). We utilize and improve the computer vision solution, and briefly describe the results of the linguistic analysis.",

author = "Anna Kuznetsova and Alfarabi Imashev and Medet Mukushev and Anara Sandygulova and Vadim Kimmelman",

note = "Publisher Copyright: {\textcopyright} 2021 Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages, AT4SSL 2021. All rights reserved.; 1st International Workshop on Automatic Translation for Signed and Spoken Languages, AT4SSL 2021 ; Conference date: 16-08-2021 Through 20-08-2021",

year = "2021",

language = "English",

pages = "49--59",

}

TY - CONF

T1 - Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL

AU - Kuznetsova, Anna

AU - Imashev, Alfarabi

AU - Mukushev, Medet

AU - Sandygulova, Anara

AU - Kimmelman, Vadim

PY - 2021

Y1 - 2021

N2 - This paper presents a study that compares non-manual markers of polar and wh-questions to statements in Kazakh-Russian Sign Language (KRSL) in a dataset collected for NLP tasks. The primary focus of the study is to demonstrate the utility of computer vision solutions for the linguistic analysis of non-manuals in sign languages, although additional corrections are required to account for biases in the output. To this end, we analyzed recordings of 10 triplets of sentences produced by 9 native signers using both manual annotation and computer vision solutions (such as OpenFace). We utilize and improve the computer vision solution, and briefly describe the results of the linguistic analysis.

AB - This paper presents a study that compares non-manual markers of polar and wh-questions to statements in Kazakh-Russian Sign Language (KRSL) in a dataset collected for NLP tasks. The primary focus of the study is to demonstrate the utility of computer vision solutions for the linguistic analysis of non-manuals in sign languages, although additional corrections are required to account for biases in the output. To this end, we analyzed recordings of 10 triplets of sentences produced by 9 native signers using both manual annotation and computer vision solutions (such as OpenFace). We utilize and improve the computer vision solution, and briefly describe the results of the linguistic analysis.

UR - http://www.scopus.com/inward/record.url?scp=85129402144&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85129402144&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85129402144

SP - 49

EP - 59

T2 - 1st International Workshop on Automatic Translation for Signed and Spoken Languages, AT4SSL 2021

Y2 - 16 August 2021 through 20 August 2021

ER -

Using Computer Vision to Analyze Non-manual Marking of Questions in KRSL

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this