Analytic study of performance of error estimators for linear discriminant analysis

Amin Zollanvari, Ulisses M. Braga-Neto, Edward R. Dougherty

Research output: Contribution to journalArticle

34 Citations (Scopus)

Abstract

We derive double asymptotic analytical expressions for the first moments, second moments, and cross-moments with the actual error for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis in the multivariate Gaussian model under the assumption of a common known covariance matrix and a fixed Mahalanobis distance as dimensionality approaches infinity. Sample sizes for the two classes need not be the same; they are only assumed to reach a fixed, but arbitrary, asymptotic ratio with the dimensionality. From the asymptotic moment representations, we directly obtain double asymptotic expressions for the bias, variance, and RMS of the error estimators. The asymptotic expressions presented here generally provide good small sample approximations, as demonstrated via numerical experiments. The applicability of the theoretical results is illustrated by finding the minimum sample size to bound the RMS in gene-expression classification.

Original languageEnglish
Article number5872073
Pages (from-to)4238-4255
Number of pages18
JournalIEEE Transactions on Signal Processing
Volume59
Issue number9
DOIs
Publication statusPublished - Sep 2011
Externally publishedYes

Fingerprint

Discriminant analysis
Covariance matrix
Gene expression
Experiments

Keywords

  • Double asymptotics
  • error estimation
  • genomic signal processing
  • leave-one-out
  • linear discriminant analysis
  • resubstitution
  • root-mean square (RMS)

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing

Cite this

Analytic study of performance of error estimators for linear discriminant analysis. / Zollanvari, Amin; Braga-Neto, Ulisses M.; Dougherty, Edward R.

In: IEEE Transactions on Signal Processing, Vol. 59, No. 9, 5872073, 09.2011, p. 4238-4255.

Research output: Contribution to journalArticle

Zollanvari, Amin ; Braga-Neto, Ulisses M. ; Dougherty, Edward R. / Analytic study of performance of error estimators for linear discriminant analysis. In: IEEE Transactions on Signal Processing. 2011 ; Vol. 59, No. 9. pp. 4238-4255.
@article{228d22e6fe234d878e62a0c5a1363427,
title = "Analytic study of performance of error estimators for linear discriminant analysis",
abstract = "We derive double asymptotic analytical expressions for the first moments, second moments, and cross-moments with the actual error for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis in the multivariate Gaussian model under the assumption of a common known covariance matrix and a fixed Mahalanobis distance as dimensionality approaches infinity. Sample sizes for the two classes need not be the same; they are only assumed to reach a fixed, but arbitrary, asymptotic ratio with the dimensionality. From the asymptotic moment representations, we directly obtain double asymptotic expressions for the bias, variance, and RMS of the error estimators. The asymptotic expressions presented here generally provide good small sample approximations, as demonstrated via numerical experiments. The applicability of the theoretical results is illustrated by finding the minimum sample size to bound the RMS in gene-expression classification.",
keywords = "Double asymptotics, error estimation, genomic signal processing, leave-one-out, linear discriminant analysis, resubstitution, root-mean square (RMS)",
author = "Amin Zollanvari and Braga-Neto, {Ulisses M.} and Dougherty, {Edward R.}",
year = "2011",
month = "9",
doi = "10.1109/TSP.2011.2159210",
language = "English",
volume = "59",
pages = "4238--4255",
journal = "IEEE Transactions on Signal Processing",
issn = "1053-587X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "9",

}

TY - JOUR

T1 - Analytic study of performance of error estimators for linear discriminant analysis

AU - Zollanvari, Amin

AU - Braga-Neto, Ulisses M.

AU - Dougherty, Edward R.

PY - 2011/9

Y1 - 2011/9

N2 - We derive double asymptotic analytical expressions for the first moments, second moments, and cross-moments with the actual error for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis in the multivariate Gaussian model under the assumption of a common known covariance matrix and a fixed Mahalanobis distance as dimensionality approaches infinity. Sample sizes for the two classes need not be the same; they are only assumed to reach a fixed, but arbitrary, asymptotic ratio with the dimensionality. From the asymptotic moment representations, we directly obtain double asymptotic expressions for the bias, variance, and RMS of the error estimators. The asymptotic expressions presented here generally provide good small sample approximations, as demonstrated via numerical experiments. The applicability of the theoretical results is illustrated by finding the minimum sample size to bound the RMS in gene-expression classification.

AB - We derive double asymptotic analytical expressions for the first moments, second moments, and cross-moments with the actual error for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis in the multivariate Gaussian model under the assumption of a common known covariance matrix and a fixed Mahalanobis distance as dimensionality approaches infinity. Sample sizes for the two classes need not be the same; they are only assumed to reach a fixed, but arbitrary, asymptotic ratio with the dimensionality. From the asymptotic moment representations, we directly obtain double asymptotic expressions for the bias, variance, and RMS of the error estimators. The asymptotic expressions presented here generally provide good small sample approximations, as demonstrated via numerical experiments. The applicability of the theoretical results is illustrated by finding the minimum sample size to bound the RMS in gene-expression classification.

KW - Double asymptotics

KW - error estimation

KW - genomic signal processing

KW - leave-one-out

KW - linear discriminant analysis

KW - resubstitution

KW - root-mean square (RMS)

UR - http://www.scopus.com/inward/record.url?scp=79960987402&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79960987402&partnerID=8YFLogxK

U2 - 10.1109/TSP.2011.2159210

DO - 10.1109/TSP.2011.2159210

M3 - Article

AN - SCOPUS:79960987402

VL - 59

SP - 4238

EP - 4255

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

SN - 1053-587X

IS - 9

M1 - 5872073

ER -