Analytical study of performance of linear discriminant analysis in stochastic settings

Amin Zollanvari, Jianping Hua, Edward R. Dougherty

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

This paper provides exact analytical expressions for the first and second moments of the true error for linear discriminant analysis (LDA) when the data are univariate and taken from two stochastic Gaussian processes. The key point is that we assume a general setting in which the sample data from each class do not need to be identically distributed or independent within or between classes. We compare the true errors of designed classifiers under the typical i.i.d. model and when the data are correlated, providing exact expressions and demonstrating that, depending on the covariance structure, correlated data can result in classifiers with either greater error or less error than when training with uncorrelated data. The general theory is applied to autoregressive and moving-average models of the first order, and it is demonstrated using real genomic data.

Original languageEnglish
Pages (from-to)3017-3029
Number of pages13
JournalPattern Recognition
Volume46
Issue number11
DOIs
Publication statusPublished - Nov 2013
Externally publishedYes

Fingerprint

Discriminant analysis
Classifiers

Keywords

  • Auto-regressive models
  • Correlated data
  • Expected error
  • Gaussian processes
  • Linear discriminant analysis
  • Moving-average models
  • Non-i.i.d data
  • Stochastic settings

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Analytical study of performance of linear discriminant analysis in stochastic settings. / Zollanvari, Amin; Hua, Jianping; Dougherty, Edward R.

In: Pattern Recognition, Vol. 46, No. 11, 11.2013, p. 3017-3029.

Research output: Contribution to journalArticle

Zollanvari, Amin ; Hua, Jianping ; Dougherty, Edward R. / Analytical study of performance of linear discriminant analysis in stochastic settings. In: Pattern Recognition. 2013 ; Vol. 46, No. 11. pp. 3017-3029.
@article{9da7e95bb42a48e2bc1d86aeb581ca7f,
title = "Analytical study of performance of linear discriminant analysis in stochastic settings",
abstract = "This paper provides exact analytical expressions for the first and second moments of the true error for linear discriminant analysis (LDA) when the data are univariate and taken from two stochastic Gaussian processes. The key point is that we assume a general setting in which the sample data from each class do not need to be identically distributed or independent within or between classes. We compare the true errors of designed classifiers under the typical i.i.d. model and when the data are correlated, providing exact expressions and demonstrating that, depending on the covariance structure, correlated data can result in classifiers with either greater error or less error than when training with uncorrelated data. The general theory is applied to autoregressive and moving-average models of the first order, and it is demonstrated using real genomic data.",
keywords = "Auto-regressive models, Correlated data, Expected error, Gaussian processes, Linear discriminant analysis, Moving-average models, Non-i.i.d data, Stochastic settings",
author = "Amin Zollanvari and Jianping Hua and Dougherty, {Edward R.}",
year = "2013",
month = "11",
doi = "10.1016/j.patcog.2013.04.002",
language = "English",
volume = "46",
pages = "3017--3029",
journal = "Pattern Recognition",
issn = "0031-3203",
publisher = "Elsevier",
number = "11",

}

TY - JOUR

T1 - Analytical study of performance of linear discriminant analysis in stochastic settings

AU - Zollanvari, Amin

AU - Hua, Jianping

AU - Dougherty, Edward R.

PY - 2013/11

Y1 - 2013/11

N2 - This paper provides exact analytical expressions for the first and second moments of the true error for linear discriminant analysis (LDA) when the data are univariate and taken from two stochastic Gaussian processes. The key point is that we assume a general setting in which the sample data from each class do not need to be identically distributed or independent within or between classes. We compare the true errors of designed classifiers under the typical i.i.d. model and when the data are correlated, providing exact expressions and demonstrating that, depending on the covariance structure, correlated data can result in classifiers with either greater error or less error than when training with uncorrelated data. The general theory is applied to autoregressive and moving-average models of the first order, and it is demonstrated using real genomic data.

AB - This paper provides exact analytical expressions for the first and second moments of the true error for linear discriminant analysis (LDA) when the data are univariate and taken from two stochastic Gaussian processes. The key point is that we assume a general setting in which the sample data from each class do not need to be identically distributed or independent within or between classes. We compare the true errors of designed classifiers under the typical i.i.d. model and when the data are correlated, providing exact expressions and demonstrating that, depending on the covariance structure, correlated data can result in classifiers with either greater error or less error than when training with uncorrelated data. The general theory is applied to autoregressive and moving-average models of the first order, and it is demonstrated using real genomic data.

KW - Auto-regressive models

KW - Correlated data

KW - Expected error

KW - Gaussian processes

KW - Linear discriminant analysis

KW - Moving-average models

KW - Non-i.i.d data

KW - Stochastic settings

UR - http://www.scopus.com/inward/record.url?scp=84878876237&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878876237&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2013.04.002

DO - 10.1016/j.patcog.2013.04.002

M3 - Article

VL - 46

SP - 3017

EP - 3029

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

IS - 11

ER -