RELIEF-MM

Effective modality weighting for multimedia information retrieval

Turgay Yilmaz, Adnan Yazici, Masaru Kitsuregawa

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algorithm with classifier predictions. RELIEF-MM employs an improved weight estimation function, which exploits the representation and reliability capabilities of modalities, as well as the discrimination capability, without any increase in the computational complexity. The comprehensive experiments conducted on TRECVID 2007, TRECVID 2008 and CCV datasets validate RELIEF-MM as an efficient, accurate and robust way of modality weighting for multimedia data.

Original languageEnglish
Pages (from-to)389-413
Number of pages25
JournalMultimedia Systems
Volume20
Issue number4
DOIs
Publication statusPublished - Jan 1 2014

Fingerprint

Information retrieval
Feature extraction
Computational complexity
Classifiers
Fusion reactions
Experiments

Keywords

  • Feature weighting
  • Multimedia information retrieval
  • Multimodal fusion
  • RELIEF

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

RELIEF-MM : Effective modality weighting for multimedia information retrieval. / Yilmaz, Turgay; Yazici, Adnan; Kitsuregawa, Masaru.

In: Multimedia Systems, Vol. 20, No. 4, 01.01.2014, p. 389-413.

Research output: Contribution to journalArticle

Yilmaz, Turgay ; Yazici, Adnan ; Kitsuregawa, Masaru. / RELIEF-MM : Effective modality weighting for multimedia information retrieval. In: Multimedia Systems. 2014 ; Vol. 20, No. 4. pp. 389-413.
@article{ba2d395f164a4cb1a4c64539eddef78b,
title = "RELIEF-MM: Effective modality weighting for multimedia information retrieval",
abstract = "Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algorithm with classifier predictions. RELIEF-MM employs an improved weight estimation function, which exploits the representation and reliability capabilities of modalities, as well as the discrimination capability, without any increase in the computational complexity. The comprehensive experiments conducted on TRECVID 2007, TRECVID 2008 and CCV datasets validate RELIEF-MM as an efficient, accurate and robust way of modality weighting for multimedia data.",
keywords = "Feature weighting, Multimedia information retrieval, Multimodal fusion, RELIEF",
author = "Turgay Yilmaz and Adnan Yazici and Masaru Kitsuregawa",
year = "2014",
month = "1",
day = "1",
doi = "10.1007/s00530-014-0360-6",
language = "English",
volume = "20",
pages = "389--413",
journal = "Multimedia Systems",
issn = "0942-4962",
publisher = "Springer Verlag",
number = "4",

}

TY - JOUR

T1 - RELIEF-MM

T2 - Effective modality weighting for multimedia information retrieval

AU - Yilmaz, Turgay

AU - Yazici, Adnan

AU - Kitsuregawa, Masaru

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algorithm with classifier predictions. RELIEF-MM employs an improved weight estimation function, which exploits the representation and reliability capabilities of modalities, as well as the discrimination capability, without any increase in the computational complexity. The comprehensive experiments conducted on TRECVID 2007, TRECVID 2008 and CCV datasets validate RELIEF-MM as an efficient, accurate and robust way of modality weighting for multimedia data.

AB - Fusing multimodal information in multimedia data usually improves the retrieval performance. One of the major issues in multimodal fusion is how to determine the best modalities. To combine the modalities more effectively, we propose a RELIEF-based modality weighting approach, named as RELIEF-MM. The original RELIEF algorithm is extended for weaknesses in several major issues: class-specific feature selection, complexities with multi-labeled data and noise, handling unbalanced datasets, and using the algorithm with classifier predictions. RELIEF-MM employs an improved weight estimation function, which exploits the representation and reliability capabilities of modalities, as well as the discrimination capability, without any increase in the computational complexity. The comprehensive experiments conducted on TRECVID 2007, TRECVID 2008 and CCV datasets validate RELIEF-MM as an efficient, accurate and robust way of modality weighting for multimedia data.

KW - Feature weighting

KW - Multimedia information retrieval

KW - Multimodal fusion

KW - RELIEF

UR - http://www.scopus.com/inward/record.url?scp=84903278200&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84903278200&partnerID=8YFLogxK

U2 - 10.1007/s00530-014-0360-6

DO - 10.1007/s00530-014-0360-6

M3 - Article

VL - 20

SP - 389

EP - 413

JO - Multimedia Systems

JF - Multimedia Systems

SN - 0942-4962

IS - 4

ER -