### Abstract

Cross-validation is commonly used to estimate the overall error rate of a designed classifier in a small-sample expression study. The true error of the classifier is a function of the prior probabilities of the classes. With random sampling these can be estimated consistently in terms of the class sample sizes, but when sampling is separate, meaning these sample sizes are determined prior to sampling, there are no reasonable estimates from the data and the prior probabilities must be 'estimated' outside the experiment. We have conducted a set of simulations to study the bias of cross-validation as a function of these 'estimates'. The results show that a poor choice for estimating these probabilities can significantly increase the bias of cross-validation as an estimator of the true error.

Original language | English |
---|---|

Title of host publication | Proceedings - IEEE International Workshop on Genomic Signal Processing and Statistics |

Pages | 98-99 |

Number of pages | 2 |

DOIs | |

Publication status | Published - 2013 |

Externally published | Yes |

Event | 2013 IEEE International Workshop on Genomic Signal Processing and Statistics, GENSIPS 2013 - Houston, TX, United States Duration: Nov 17 2013 → Nov 19 2013 |

### Other

Other | 2013 IEEE International Workshop on Genomic Signal Processing and Statistics, GENSIPS 2013 |
---|---|

Country | United States |

City | Houston, TX |

Period | 11/17/13 → 11/19/13 |

### Fingerprint

### ASJC Scopus subject areas

- Biochemistry, Genetics and Molecular Biology (miscellaneous)
- Computational Theory and Mathematics
- Signal Processing
- Biomedical Engineering

### Cite this

*Proceedings - IEEE International Workshop on Genomic Signal Processing and Statistics*(pp. 98-99). [6735947] https://doi.org/10.1109/GENSIPS.2013.6735947

**Effect of mixing probabilities on the bias of cross-validation under separate sampling.** / Zollanvari, Amin; Braga-Neto, Ulisses M.; Dougherty, Edward R.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

*Proceedings - IEEE International Workshop on Genomic Signal Processing and Statistics.*, 6735947, pp. 98-99, 2013 IEEE International Workshop on Genomic Signal Processing and Statistics, GENSIPS 2013, Houston, TX, United States, 11/17/13. https://doi.org/10.1109/GENSIPS.2013.6735947

}

TY - GEN

T1 - Effect of mixing probabilities on the bias of cross-validation under separate sampling

AU - Zollanvari, Amin

AU - Braga-Neto, Ulisses M.

AU - Dougherty, Edward R.

PY - 2013

Y1 - 2013

N2 - Cross-validation is commonly used to estimate the overall error rate of a designed classifier in a small-sample expression study. The true error of the classifier is a function of the prior probabilities of the classes. With random sampling these can be estimated consistently in terms of the class sample sizes, but when sampling is separate, meaning these sample sizes are determined prior to sampling, there are no reasonable estimates from the data and the prior probabilities must be 'estimated' outside the experiment. We have conducted a set of simulations to study the bias of cross-validation as a function of these 'estimates'. The results show that a poor choice for estimating these probabilities can significantly increase the bias of cross-validation as an estimator of the true error.

AB - Cross-validation is commonly used to estimate the overall error rate of a designed classifier in a small-sample expression study. The true error of the classifier is a function of the prior probabilities of the classes. With random sampling these can be estimated consistently in terms of the class sample sizes, but when sampling is separate, meaning these sample sizes are determined prior to sampling, there are no reasonable estimates from the data and the prior probabilities must be 'estimated' outside the experiment. We have conducted a set of simulations to study the bias of cross-validation as a function of these 'estimates'. The results show that a poor choice for estimating these probabilities can significantly increase the bias of cross-validation as an estimator of the true error.

UR - http://www.scopus.com/inward/record.url?scp=84897694038&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84897694038&partnerID=8YFLogxK

U2 - 10.1109/GENSIPS.2013.6735947

DO - 10.1109/GENSIPS.2013.6735947

M3 - Conference contribution

SN - 9781479934621

SP - 98

EP - 99

BT - Proceedings - IEEE International Workshop on Genomic Signal Processing and Statistics

ER -