On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers

Amin Zollanvari, Ulisses M. Braga-Neto, Edward R. Dougherty

Research output: Contribution to journalArticlepeer-review

32 Citations (Scopus)

Abstract

Error estimation is a problem of high current interest in many areas of application. This paper concerns the classical problem of determining the performance of error estimators in small-sample settings under a Gaussianity parametric assumption. We provide here for the first time the exact sampling distribution of the resubstitution and leave-one-out error estimators for linear discriminant analysis (LDA) in the univariate case, which is valid for any sample size and combination of parameters (including unequal variances and sample sizes for each class). In the multivariate case, we provide a quasi-binomial approximation to the distribution of both the resubstitution and leave-one-out error estimators for LDA, under a common but otherwise arbitrary class covariance matrix, which is assumed to be known in the design of the LDA discriminant. We provide numerical examples, using both synthetic and real data, that indicate that these approximations are accurate, provided that LDA classification error is not too large.

Original languageEnglish
Pages (from-to)2705-2723
Number of pages19
JournalPattern Recognition
Volume42
Issue number11
DOIs
Publication statusPublished - Nov 1 2009

Keywords

  • Error estimation
  • Leave-one-out
  • Linear discriminant analysis
  • Parametric classification
  • Resubstitution
  • Sampling distribution

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers'. Together they form a unique fingerprint.

Cite this