A probabilistic scoring method in multiple-choice testing incorporating partial knowledge

Desmond Adair, Martin Jaeger

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

The traditional method for scoring multiple-choice tests is when a student picks one option as the answer. This method however, has been heavily criticized, as it can lead to students guessing and also takes no account of a student's partial knowledge. The scheme developed here, based on simple probability theory, is tested against another method which takes partial knowledge into account namely, the elimination method for multiplechoice tests. The scheme is also tested against the conventional dichotomous method of scoring. It was found that the relatively easy-to-use probabilistic scoring method gave similar results to the elimination testing, whereas lower results were found when compared with the conventional dichotomous method. The results due to the probabilistic scoring method were consistently more reliable than those due to the conventional dichotomous method.

Original languageEnglish
Title of host publicationCurriculum Development, Innovation and Reform
PublisherNova Science Publishers, Inc.
Pages109-124
Number of pages16
ISBN (Print)9781626184282
Publication statusPublished - May 2013

Fingerprint

student

Keywords

  • Computer-aided assessment
  • Multiple-choice assessment
  • Scoring method

ASJC Scopus subject areas

  • Social Sciences(all)

Cite this

Adair, D., & Jaeger, M. (2013). A probabilistic scoring method in multiple-choice testing incorporating partial knowledge. In Curriculum Development, Innovation and Reform (pp. 109-124). Nova Science Publishers, Inc..

A probabilistic scoring method in multiple-choice testing incorporating partial knowledge. / Adair, Desmond; Jaeger, Martin.

Curriculum Development, Innovation and Reform. Nova Science Publishers, Inc., 2013. p. 109-124.

Research output: Chapter in Book/Report/Conference proceedingChapter

Adair, D & Jaeger, M 2013, A probabilistic scoring method in multiple-choice testing incorporating partial knowledge. in Curriculum Development, Innovation and Reform. Nova Science Publishers, Inc., pp. 109-124.
Adair D, Jaeger M. A probabilistic scoring method in multiple-choice testing incorporating partial knowledge. In Curriculum Development, Innovation and Reform. Nova Science Publishers, Inc. 2013. p. 109-124
Adair, Desmond ; Jaeger, Martin. / A probabilistic scoring method in multiple-choice testing incorporating partial knowledge. Curriculum Development, Innovation and Reform. Nova Science Publishers, Inc., 2013. pp. 109-124
@inbook{88c2169f572746ab8992a2a648cfe626,
title = "A probabilistic scoring method in multiple-choice testing incorporating partial knowledge",
abstract = "The traditional method for scoring multiple-choice tests is when a student picks one option as the answer. This method however, has been heavily criticized, as it can lead to students guessing and also takes no account of a student's partial knowledge. The scheme developed here, based on simple probability theory, is tested against another method which takes partial knowledge into account namely, the elimination method for multiplechoice tests. The scheme is also tested against the conventional dichotomous method of scoring. It was found that the relatively easy-to-use probabilistic scoring method gave similar results to the elimination testing, whereas lower results were found when compared with the conventional dichotomous method. The results due to the probabilistic scoring method were consistently more reliable than those due to the conventional dichotomous method.",
keywords = "Computer-aided assessment, Multiple-choice assessment, Scoring method",
author = "Desmond Adair and Martin Jaeger",
year = "2013",
month = "5",
language = "English",
isbn = "9781626184282",
pages = "109--124",
booktitle = "Curriculum Development, Innovation and Reform",
publisher = "Nova Science Publishers, Inc.",
address = "United States",

}

TY - CHAP

T1 - A probabilistic scoring method in multiple-choice testing incorporating partial knowledge

AU - Adair, Desmond

AU - Jaeger, Martin

PY - 2013/5

Y1 - 2013/5

N2 - The traditional method for scoring multiple-choice tests is when a student picks one option as the answer. This method however, has been heavily criticized, as it can lead to students guessing and also takes no account of a student's partial knowledge. The scheme developed here, based on simple probability theory, is tested against another method which takes partial knowledge into account namely, the elimination method for multiplechoice tests. The scheme is also tested against the conventional dichotomous method of scoring. It was found that the relatively easy-to-use probabilistic scoring method gave similar results to the elimination testing, whereas lower results were found when compared with the conventional dichotomous method. The results due to the probabilistic scoring method were consistently more reliable than those due to the conventional dichotomous method.

AB - The traditional method for scoring multiple-choice tests is when a student picks one option as the answer. This method however, has been heavily criticized, as it can lead to students guessing and also takes no account of a student's partial knowledge. The scheme developed here, based on simple probability theory, is tested against another method which takes partial knowledge into account namely, the elimination method for multiplechoice tests. The scheme is also tested against the conventional dichotomous method of scoring. It was found that the relatively easy-to-use probabilistic scoring method gave similar results to the elimination testing, whereas lower results were found when compared with the conventional dichotomous method. The results due to the probabilistic scoring method were consistently more reliable than those due to the conventional dichotomous method.

KW - Computer-aided assessment

KW - Multiple-choice assessment

KW - Scoring method

UR - http://www.scopus.com/inward/record.url?scp=84892810758&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84892810758&partnerID=8YFLogxK

M3 - Chapter

SN - 9781626184282

SP - 109

EP - 124

BT - Curriculum Development, Innovation and Reform

PB - Nova Science Publishers, Inc.

ER -