Aggregating student peer assessment during capstone projects

Desmond Adair, Martin Jaeger

Research output: Contribution to journalArticle

Abstract

Student assessment of other student's work has many potential benefits to learning for both the assessor and the assessed. However, sources of peer assessment provide, quite often, subjective evidence, which can be conflicting, uncertain and even ignorant. One of the key elements to providing an overall quality assessment of a student's work derived from the assessment by his/her peers is the use of an appropriate method of combining or fusing these heterogeneous evidence sources. Since the development of the belief theory introduced by Shafer in the 1970s, many combination rules have been proposed in the literature with two main methods selected here. The first is an evidential reasoning (ER) approach, the kernel of which is an ER algorithm developed on the basis of the framework and the evidence combination rule of the Dempster-Shafer (DS) theory. It has been claimed also in the literature that Dempster's rule generates counter-intuitive and unacceptable results in practical situations, so an approach based on the Dezert-Smarandache (DSmT) theory of fusion will also be explored, in particular the PCR6 rule of proportional conflict redistribution. Results for peer assessment marksallocatedby astudent cohort,consistingof20students,duringtheircapstoneprojects,and,aggregatedusingeachof thesetwoapproachesarecompared witheachother andwith resultsobtainedby themoretraditionalAveragingRule(AR) approach. It is clear from the findings that when subjective evidence is aggregated then the simple AR approach as the accepted combination method is in doubt. It also seems that the DS method of aggregation seems the best alternative to traditional averaging.

Original languageEnglish
Pages (from-to)216-224
Number of pages9
JournalInternational Journal of Engineering Education
Volume33
Issue number1
Publication statusPublished - 2017

Fingerprint

Students
student
evidence
Fusion reactions
Agglomeration
redistribution
aggregation
learning
literature

Keywords

  • Aggregating
  • Capstone
  • Evidence
  • Fusion

ASJC Scopus subject areas

  • Education
  • Engineering(all)

Cite this

Aggregating student peer assessment during capstone projects. / Adair, Desmond; Jaeger, Martin.

In: International Journal of Engineering Education, Vol. 33, No. 1, 2017, p. 216-224.

Research output: Contribution to journalArticle

@article{eb8a27c5669345cda44139cb30fd2133,
title = "Aggregating student peer assessment during capstone projects",
abstract = "Student assessment of other student's work has many potential benefits to learning for both the assessor and the assessed. However, sources of peer assessment provide, quite often, subjective evidence, which can be conflicting, uncertain and even ignorant. One of the key elements to providing an overall quality assessment of a student's work derived from the assessment by his/her peers is the use of an appropriate method of combining or fusing these heterogeneous evidence sources. Since the development of the belief theory introduced by Shafer in the 1970s, many combination rules have been proposed in the literature with two main methods selected here. The first is an evidential reasoning (ER) approach, the kernel of which is an ER algorithm developed on the basis of the framework and the evidence combination rule of the Dempster-Shafer (DS) theory. It has been claimed also in the literature that Dempster's rule generates counter-intuitive and unacceptable results in practical situations, so an approach based on the Dezert-Smarandache (DSmT) theory of fusion will also be explored, in particular the PCR6 rule of proportional conflict redistribution. Results for peer assessment marksallocatedby astudent cohort,consistingof20students,duringtheircapstoneprojects,and,aggregatedusingeachof thesetwoapproachesarecompared witheachother andwith resultsobtainedby themoretraditionalAveragingRule(AR) approach. It is clear from the findings that when subjective evidence is aggregated then the simple AR approach as the accepted combination method is in doubt. It also seems that the DS method of aggregation seems the best alternative to traditional averaging.",
keywords = "Aggregating, Capstone, Evidence, Fusion",
author = "Desmond Adair and Martin Jaeger",
year = "2017",
language = "English",
volume = "33",
pages = "216--224",
journal = "International Journal of Engineering Education",
issn = "0949-149X",
publisher = "Tempus Publications",
number = "1",

}

TY - JOUR

T1 - Aggregating student peer assessment during capstone projects

AU - Adair, Desmond

AU - Jaeger, Martin

PY - 2017

Y1 - 2017

N2 - Student assessment of other student's work has many potential benefits to learning for both the assessor and the assessed. However, sources of peer assessment provide, quite often, subjective evidence, which can be conflicting, uncertain and even ignorant. One of the key elements to providing an overall quality assessment of a student's work derived from the assessment by his/her peers is the use of an appropriate method of combining or fusing these heterogeneous evidence sources. Since the development of the belief theory introduced by Shafer in the 1970s, many combination rules have been proposed in the literature with two main methods selected here. The first is an evidential reasoning (ER) approach, the kernel of which is an ER algorithm developed on the basis of the framework and the evidence combination rule of the Dempster-Shafer (DS) theory. It has been claimed also in the literature that Dempster's rule generates counter-intuitive and unacceptable results in practical situations, so an approach based on the Dezert-Smarandache (DSmT) theory of fusion will also be explored, in particular the PCR6 rule of proportional conflict redistribution. Results for peer assessment marksallocatedby astudent cohort,consistingof20students,duringtheircapstoneprojects,and,aggregatedusingeachof thesetwoapproachesarecompared witheachother andwith resultsobtainedby themoretraditionalAveragingRule(AR) approach. It is clear from the findings that when subjective evidence is aggregated then the simple AR approach as the accepted combination method is in doubt. It also seems that the DS method of aggregation seems the best alternative to traditional averaging.

AB - Student assessment of other student's work has many potential benefits to learning for both the assessor and the assessed. However, sources of peer assessment provide, quite often, subjective evidence, which can be conflicting, uncertain and even ignorant. One of the key elements to providing an overall quality assessment of a student's work derived from the assessment by his/her peers is the use of an appropriate method of combining or fusing these heterogeneous evidence sources. Since the development of the belief theory introduced by Shafer in the 1970s, many combination rules have been proposed in the literature with two main methods selected here. The first is an evidential reasoning (ER) approach, the kernel of which is an ER algorithm developed on the basis of the framework and the evidence combination rule of the Dempster-Shafer (DS) theory. It has been claimed also in the literature that Dempster's rule generates counter-intuitive and unacceptable results in practical situations, so an approach based on the Dezert-Smarandache (DSmT) theory of fusion will also be explored, in particular the PCR6 rule of proportional conflict redistribution. Results for peer assessment marksallocatedby astudent cohort,consistingof20students,duringtheircapstoneprojects,and,aggregatedusingeachof thesetwoapproachesarecompared witheachother andwith resultsobtainedby themoretraditionalAveragingRule(AR) approach. It is clear from the findings that when subjective evidence is aggregated then the simple AR approach as the accepted combination method is in doubt. It also seems that the DS method of aggregation seems the best alternative to traditional averaging.

KW - Aggregating

KW - Capstone

KW - Evidence

KW - Fusion

UR - http://www.scopus.com/inward/record.url?scp=85015240482&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015240482&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:85015240482

VL - 33

SP - 216

EP - 224

JO - International Journal of Engineering Education

JF - International Journal of Engineering Education

SN - 0949-149X

IS - 1

ER -