On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers

Amin Zollanvari, Ulisses M. Braga-Neto, Edward R. Dougherty

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

Error estimation is a problem of high current interest in many areas of application. This paper concerns the classical problem of determining the performance of error estimators in small-sample settings under a Gaussianity parametric assumption. We provide here for the first time the exact sampling distribution of the resubstitution and leave-one-out error estimators for linear discriminant analysis (LDA) in the univariate case, which is valid for any sample size and combination of parameters (including unequal variances and sample sizes for each class). In the multivariate case, we provide a quasi-binomial approximation to the distribution of both the resubstitution and leave-one-out error estimators for LDA, under a common but otherwise arbitrary class covariance matrix, which is assumed to be known in the design of the LDA discriminant. We provide numerical examples, using both synthetic and real data, that indicate that these approximations are accurate, provided that LDA classification error is not too large.

Original languageEnglish
Pages (from-to)2705-2723
Number of pages19
JournalPattern Recognition
Volume42
Issue number11
DOIs
Publication statusPublished - Nov 2009
Externally publishedYes

Fingerprint

Discriminant analysis
Classifiers
Sampling
Covariance matrix
Error analysis

Keywords

  • Error estimation
  • Leave-one-out
  • Linear discriminant analysis
  • Parametric classification
  • Resubstitution
  • Sampling distribution

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers. / Zollanvari, Amin; Braga-Neto, Ulisses M.; Dougherty, Edward R.

In: Pattern Recognition, Vol. 42, No. 11, 11.2009, p. 2705-2723.

Research output: Contribution to journalArticle

Zollanvari, Amin ; Braga-Neto, Ulisses M. ; Dougherty, Edward R. / On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers. In: Pattern Recognition. 2009 ; Vol. 42, No. 11. pp. 2705-2723.
@article{53a27c6e11f441eb837cb3c50f5f14c8,
title = "On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers",
abstract = "Error estimation is a problem of high current interest in many areas of application. This paper concerns the classical problem of determining the performance of error estimators in small-sample settings under a Gaussianity parametric assumption. We provide here for the first time the exact sampling distribution of the resubstitution and leave-one-out error estimators for linear discriminant analysis (LDA) in the univariate case, which is valid for any sample size and combination of parameters (including unequal variances and sample sizes for each class). In the multivariate case, we provide a quasi-binomial approximation to the distribution of both the resubstitution and leave-one-out error estimators for LDA, under a common but otherwise arbitrary class covariance matrix, which is assumed to be known in the design of the LDA discriminant. We provide numerical examples, using both synthetic and real data, that indicate that these approximations are accurate, provided that LDA classification error is not too large.",
keywords = "Error estimation, Leave-one-out, Linear discriminant analysis, Parametric classification, Resubstitution, Sampling distribution",
author = "Amin Zollanvari and Braga-Neto, {Ulisses M.} and Dougherty, {Edward R.}",
year = "2009",
month = "11",
doi = "10.1016/j.patcog.2009.05.003",
language = "English",
volume = "42",
pages = "2705--2723",
journal = "Pattern Recognition",
issn = "0031-3203",
publisher = "Elsevier",
number = "11",

}

TY - JOUR

T1 - On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers

AU - Zollanvari, Amin

AU - Braga-Neto, Ulisses M.

AU - Dougherty, Edward R.

PY - 2009/11

Y1 - 2009/11

N2 - Error estimation is a problem of high current interest in many areas of application. This paper concerns the classical problem of determining the performance of error estimators in small-sample settings under a Gaussianity parametric assumption. We provide here for the first time the exact sampling distribution of the resubstitution and leave-one-out error estimators for linear discriminant analysis (LDA) in the univariate case, which is valid for any sample size and combination of parameters (including unequal variances and sample sizes for each class). In the multivariate case, we provide a quasi-binomial approximation to the distribution of both the resubstitution and leave-one-out error estimators for LDA, under a common but otherwise arbitrary class covariance matrix, which is assumed to be known in the design of the LDA discriminant. We provide numerical examples, using both synthetic and real data, that indicate that these approximations are accurate, provided that LDA classification error is not too large.

AB - Error estimation is a problem of high current interest in many areas of application. This paper concerns the classical problem of determining the performance of error estimators in small-sample settings under a Gaussianity parametric assumption. We provide here for the first time the exact sampling distribution of the resubstitution and leave-one-out error estimators for linear discriminant analysis (LDA) in the univariate case, which is valid for any sample size and combination of parameters (including unequal variances and sample sizes for each class). In the multivariate case, we provide a quasi-binomial approximation to the distribution of both the resubstitution and leave-one-out error estimators for LDA, under a common but otherwise arbitrary class covariance matrix, which is assumed to be known in the design of the LDA discriminant. We provide numerical examples, using both synthetic and real data, that indicate that these approximations are accurate, provided that LDA classification error is not too large.

KW - Error estimation

KW - Leave-one-out

KW - Linear discriminant analysis

KW - Parametric classification

KW - Resubstitution

KW - Sampling distribution

UR - http://www.scopus.com/inward/record.url?scp=67649390743&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67649390743&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2009.05.003

DO - 10.1016/j.patcog.2009.05.003

M3 - Article

AN - SCOPUS:67649390743

VL - 42

SP - 2705

EP - 2723

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

IS - 11

ER -