Character-based feature extraction with LSTM networks for POS-tagging task

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

Original languageEnglish
Title of host publicationApplication of Information and Communication Technologies, AICT 2016 - Conference Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509018406
DOIs
Publication statusPublished - Jul 25 2017
Event10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016 - Baku, Azerbaijan
Duration: Oct 12 2016Oct 14 2016

Conference

Conference10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016
CountryAzerbaijan
CityBaku
Period10/12/1610/14/16

Fingerprint

Tagging
Feature Extraction
Feature extraction
Word problem
Vector spaces
Vector space
Output
Model
Experiment
Experiments
Character
Language

Keywords

  • character-based features
  • continuous vector space word representations
  • LSTM networks
  • out-of-vocabulary words
  • POS-tagging

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Computer Networks and Communications
  • Information Systems
  • Modelling and Simulation

Cite this

Makazhanov, A., & Yessenbayev, Z. (2017). Character-based feature extraction with LSTM networks for POS-tagging task. In Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings [7991654] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICAICT.2016.7991654

Character-based feature extraction with LSTM networks for POS-tagging task. / Makazhanov, Aibek; Yessenbayev, Zhandos.

Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. 7991654.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Makazhanov, A & Yessenbayev, Z 2017, Character-based feature extraction with LSTM networks for POS-tagging task. in Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings., 7991654, Institute of Electrical and Electronics Engineers Inc., 10th IEEE International Conference on Application of Information and Communication Technologies, AICT 2016, Baku, Azerbaijan, 10/12/16. https://doi.org/10.1109/ICAICT.2016.7991654
Makazhanov A, Yessenbayev Z. Character-based feature extraction with LSTM networks for POS-tagging task. In Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. 7991654 https://doi.org/10.1109/ICAICT.2016.7991654
Makazhanov, Aibek ; Yessenbayev, Zhandos. / Character-based feature extraction with LSTM networks for POS-tagging task. Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017.
@inproceedings{3598d49522ff4a448fd7f4e97d3ab2b2,
title = "Character-based feature extraction with LSTM networks for POS-tagging task",
abstract = "In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.",
keywords = "character-based features, continuous vector space word representations, LSTM networks, out-of-vocabulary words, POS-tagging",
author = "Aibek Makazhanov and Zhandos Yessenbayev",
year = "2017",
month = "7",
day = "25",
doi = "10.1109/ICAICT.2016.7991654",
language = "English",
booktitle = "Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Character-based feature extraction with LSTM networks for POS-tagging task

AU - Makazhanov, Aibek

AU - Yessenbayev, Zhandos

PY - 2017/7/25

Y1 - 2017/7/25

N2 - In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

AB - In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

KW - character-based features

KW - continuous vector space word representations

KW - LSTM networks

KW - out-of-vocabulary words

KW - POS-tagging

UR - http://www.scopus.com/inward/record.url?scp=85034254537&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034254537&partnerID=8YFLogxK

U2 - 10.1109/ICAICT.2016.7991654

DO - 10.1109/ICAICT.2016.7991654

M3 - Conference contribution

AN - SCOPUS:85034254537

BT - Application of Information and Communication Technologies, AICT 2016 - Conference Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -