Character-based feature extraction with LSTM networks for POS-tagging task

Aibek Makazhanov; Zhandos Yessenbayev

doi:10.1109/ICAICT.2016.7991654

Character-based feature extraction with LSTM networks for POS-tagging task

Aibek Makazhanov
, Zhandos Yessenbayev

Private Institution National Laboratory Astana

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

9 Citations (Scopus)

Abstract

In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

Original language	English
Title of host publication	Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781509018406
DOIs	https://doi.org/10.1109/ICAICT.2016.7991654
Publication status	Published - Jul 25 2017
Event	10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016 - Baku, Azerbaijan Duration: Oct 12 2016 → Oct 14 2016

Publication series

Name	Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings

Conference

Conference	10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016
Country/Territory	Azerbaijan
City	Baku
Period	10/12/16 → 10/14/16

Funding

ACKNOWLEDGMENT This work was supported by the Nazarbayev University under the grant number 064-2016/013-2016 and the Ministry of Education of Science of the Republic of Kazakhstan under the research program number 349//049-2016.

Keywords

character-based features
continuous vector space word representations
LSTM networks
out-of-vocabulary words
POS-tagging

ASJC Scopus subject areas

Computer Vision and Pattern Recognition
Computer Science Applications
Computer Networks and Communications
Information Systems
Modelling and Simulation

Access to Document

10.1109/ICAICT.2016.7991654

Cite this

Makazhanov, A., & Yessenbayev, Z. (2017). Character-based feature extraction with LSTM networks for POS-tagging task. In Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings Article 7991654 (Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICAICT.2016.7991654

Character-based feature extraction with LSTM networks for POS-tagging task. / Makazhanov, Aibek; Yessenbayev, Zhandos.
Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. 7991654 (Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Makazhanov, A & Yessenbayev, Z 2017, Character-based feature extraction with LSTM networks for POS-tagging task. in Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings., 7991654, Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings, Institute of Electrical and Electronics Engineers Inc., 10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016, Baku, Azerbaijan, 10/12/16. https://doi.org/10.1109/ICAICT.2016.7991654

Makazhanov A, Yessenbayev Z. Character-based feature extraction with LSTM networks for POS-tagging task. In Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. 7991654. (Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings). doi: 10.1109/ICAICT.2016.7991654

Makazhanov, Aibek ; Yessenbayev, Zhandos. / Character-based feature extraction with LSTM networks for POS-tagging task. Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. (Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings).

@inproceedings{3598d49522ff4a448fd7f4e97d3ab2b2,

title = "Character-based feature extraction with LSTM networks for POS-tagging task",

abstract = "In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.",

keywords = "character-based features, continuous vector space word representations, LSTM networks, out-of-vocabulary words, POS-tagging",

author = "Aibek Makazhanov and Zhandos Yessenbayev",

note = "Funding Information: ACKNOWLEDGMENT This work was supported by the Nazarbayev University under the grant number 064-2016/013-2016 and the Ministry of Education of Science of the Republic of Kazakhstan under the research program number 349//049-2016. Publisher Copyright: {\textcopyright} 2016 IEEE.; 10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016 ; Conference date: 12-10-2016 Through 14-10-2016",

year = "2017",

month = jul,

day = "25",

doi = "10.1109/ICAICT.2016.7991654",

language = "English",

series = "Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings",

address = "United States",

}

TY - GEN

T1 - Character-based feature extraction with LSTM networks for POS-tagging task

AU - Makazhanov, Aibek

AU - Yessenbayev, Zhandos

N1 - Funding Information: ACKNOWLEDGMENT This work was supported by the Nazarbayev University under the grant number 064-2016/013-2016 and the Ministry of Education of Science of the Republic of Kazakhstan under the research program number 349//049-2016. Publisher Copyright: © 2016 IEEE.

PY - 2017/7/25

Y1 - 2017/7/25

N2 - In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

AB - In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

KW - character-based features

KW - continuous vector space word representations

KW - LSTM networks

KW - out-of-vocabulary words

KW - POS-tagging

UR - https://www.scopus.com/pages/publications/85034254537

UR - https://www.scopus.com/pages/publications/85034254537#tab=citedBy

U2 - 10.1109/ICAICT.2016.7991654

DO - 10.1109/ICAICT.2016.7991654

M3 - Conference contribution

AN - SCOPUS:85034254537

T3 - Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings

BT - Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016

Y2 - 12 October 2016 through 14 October 2016

ER -

Character-based feature extraction with LSTM networks for POS-tagging task

Abstract

Publication series

Conference

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this