Examining the Reliability of Adaptive Comparative Judgement (ACJ) as an Assessment Tool in Educational Settings

Autor/in	Kimbell, Richard
Titel	Examining the Reliability of Adaptive Comparative Judgement (ACJ) as an Assessment Tool in Educational Settings
Quelle	In: International Journal of Technology and Design Education, 32 (2022) 3, S.1515-1529 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0957 7572
DOI	10.1007/s10798-021-09654-w
Schlagwörter	Student Evaluation; Evaluation Methods; Alternative Assessment; Value Judgment; Computer Software; Mathematics; Reliability; Educational Innovation + Suchen Sie Ihr Suchwort? Schulnote; Studentische Bewertung; Werturteil; Mathematik; Reliabilität; Instructional innovation; Bildungsinnovation
Abstract	Conventional approaches to assessment involve teachers and examiners judging the quality of learners work by reference to lists of criteria or other 'outcome' statements. This paper explores a quite different method of assessment using 'Adaptive Comparative Judgement' (ACJ) that was developed within a research project at Goldsmiths University of London between 2004 and 2010. The method was developed into a tool that enabled judges to distinguish better/worse performances not by allocating numbers through mark schemes, but rather by direct, holistic, judgement. The tool was successfully deployed through a series of national and international research and development exercises. But game-changing innovations are never flaw-less first time out (Golley, Jet: Frank Whittle and the Invention of the Jet Engine, Datum Publishing, Liphook Hampshire, 2009; Dyson, Against the odds: an autobiography, Texere Publishing, Knutsford Cheshire, 2001) and a series of careful investigations resulted in a problem being identified within the workings of ACJ (Bramley, Investigating the reliability of Adaptive Comparative Judgment, Cambridge Assessment Research Report, UK, Cambridge, 2015). The issue was with the 'adaptive' component of the algorithm that, under certain conditions, appeared to exaggerate the reliability statistic. The problem was 'worked' by the software company running ACJ and a solution found. This paper reports the whole sequence of events--from the original innovation, through deployment, the emergent problem, and the resulting solution that was published at an international conference (Rangel Smith and Lynch in: PATT36 International Conference. Research & Practice in Technology Education: Perspectives on Human Capacity and Development, 2018) and subsequently deployed within a modified ACJ algorithm. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01