Exact representation of the second-order moments for resubstitution and leave-one-out error estimation for linear discriminant analysis in the univariate heteroskedastic Gaussian model

Amin Zollanvari, Ulisses Braga-Neto, Edward R. Dougherty

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

This paper provides exact analytical expressions for the bias, variance, and RMS for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis (LDA) in the univariate heteroskedastic Gaussian model. Neither the variances nor the sample sizes for the two classes need be the same. The generality of heteroskedasticity (unequal variances) is a fundamental feature of the work presented in this paper, which distinguishes it from past work. The expected resubstitution and leave-one-out errors are represented by probabilities involving bivariate Gaussian distributions. Their second moments and cross-moments with the actual error are represented by 3- and 4-variate Gaussian distributions. From these, the bias, deviation variance, and RMS for resubstitution and leave-one-out as estimators of the actual error can be computed. The RMS expressions are applied to the determination of sample size and illustrated in biomarker classification.

Original languageEnglish
Pages (from-to)908-917
Number of pages10
JournalPattern Recognition
Volume45
Issue number2
DOIs
Publication statusPublished - Feb 2012
Externally publishedYes

Fingerprint

Discriminant analysis
Error analysis
Gaussian distribution
Biomarkers

Keywords

  • Error estimation
  • Genomics
  • Heteroskedasticity
  • Leave-one-out
  • Linear discriminant analysis
  • Resubstitution
  • RMS

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

@article{6545aeb66efa42c29217fe5b096f61fc,
title = "Exact representation of the second-order moments for resubstitution and leave-one-out error estimation for linear discriminant analysis in the univariate heteroskedastic Gaussian model",
abstract = "This paper provides exact analytical expressions for the bias, variance, and RMS for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis (LDA) in the univariate heteroskedastic Gaussian model. Neither the variances nor the sample sizes for the two classes need be the same. The generality of heteroskedasticity (unequal variances) is a fundamental feature of the work presented in this paper, which distinguishes it from past work. The expected resubstitution and leave-one-out errors are represented by probabilities involving bivariate Gaussian distributions. Their second moments and cross-moments with the actual error are represented by 3- and 4-variate Gaussian distributions. From these, the bias, deviation variance, and RMS for resubstitution and leave-one-out as estimators of the actual error can be computed. The RMS expressions are applied to the determination of sample size and illustrated in biomarker classification.",
keywords = "Error estimation, Genomics, Heteroskedasticity, Leave-one-out, Linear discriminant analysis, Resubstitution, RMS",
author = "Amin Zollanvari and Ulisses Braga-Neto and Dougherty, {Edward R.}",
year = "2012",
month = "2",
doi = "10.1016/j.patcog.2011.08.006",
language = "English",
volume = "45",
pages = "908--917",
journal = "Pattern Recognition",
issn = "0031-3203",
publisher = "Elsevier",
number = "2",

}

TY - JOUR

T1 - Exact representation of the second-order moments for resubstitution and leave-one-out error estimation for linear discriminant analysis in the univariate heteroskedastic Gaussian model

AU - Zollanvari, Amin

AU - Braga-Neto, Ulisses

AU - Dougherty, Edward R.

PY - 2012/2

Y1 - 2012/2

N2 - This paper provides exact analytical expressions for the bias, variance, and RMS for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis (LDA) in the univariate heteroskedastic Gaussian model. Neither the variances nor the sample sizes for the two classes need be the same. The generality of heteroskedasticity (unequal variances) is a fundamental feature of the work presented in this paper, which distinguishes it from past work. The expected resubstitution and leave-one-out errors are represented by probabilities involving bivariate Gaussian distributions. Their second moments and cross-moments with the actual error are represented by 3- and 4-variate Gaussian distributions. From these, the bias, deviation variance, and RMS for resubstitution and leave-one-out as estimators of the actual error can be computed. The RMS expressions are applied to the determination of sample size and illustrated in biomarker classification.

AB - This paper provides exact analytical expressions for the bias, variance, and RMS for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis (LDA) in the univariate heteroskedastic Gaussian model. Neither the variances nor the sample sizes for the two classes need be the same. The generality of heteroskedasticity (unequal variances) is a fundamental feature of the work presented in this paper, which distinguishes it from past work. The expected resubstitution and leave-one-out errors are represented by probabilities involving bivariate Gaussian distributions. Their second moments and cross-moments with the actual error are represented by 3- and 4-variate Gaussian distributions. From these, the bias, deviation variance, and RMS for resubstitution and leave-one-out as estimators of the actual error can be computed. The RMS expressions are applied to the determination of sample size and illustrated in biomarker classification.

KW - Error estimation

KW - Genomics

KW - Heteroskedasticity

KW - Leave-one-out

KW - Linear discriminant analysis

KW - Resubstitution

KW - RMS

UR - http://www.scopus.com/inward/record.url?scp=80052964854&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052964854&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2011.08.006

DO - 10.1016/j.patcog.2011.08.006

M3 - Article

AN - SCOPUS:80052964854

VL - 45

SP - 908

EP - 917

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

IS - 2

ER -