Comparison of Passing Scores Determined by the Angoff Method in Different Item Samples

Autor/inn/en	Kara, Hakan; Cetin, Sevda
Titel	Comparison of Passing Scores Determined by the Angoff Method in Different Item Samples
Quelle	In: International Journal of Assessment Tools in Education, 7 (2020) 1, S.80-97 (18 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	ORCID (Kara, Hakan) ORCID (Cetin, Sevda)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2148-7456
Schlagwörter	Cutting Scores; Standard Setting (Scoring); Sampling; Error of Measurement; Test Items; Difficulty Level; Middle School Students; Student Placement; Mathematics Tests; New Jersey + Suchen Sie Ihr Suchwort? Bewertungsskala; Messfehler; Test content; Testaufgabe; Schwierigkeitsgrad; Middle school; Middle schools; Student; Students; Mittelschule; Mittelstufenschule; Schüler; Schülerin; Schülerpraktikum
Abstract	In this study, the efficiency of various random sampling methods to reduce the number of items rated by judges in an Angoff standard-setting study was examined and the methods were compared with each other. Firstly, the full-length test was formed by combining Placement Test 2012 and 2013 mathematics subsets. After then, simple random sampling (SRS), content stratified (C-SRS), item-difficulty stratified (D-SRS) and content-by-difficulty random sampling (CD-SRS) methods were used to constitute different length of subsets (30%, 40%, 50%, 70%) from the full-test. In total, 16 different study conditions (4 methods x 4 subsets) were investigated. In data analysis part, ANOVA analysis was conducted to examine whether minimum passing scores (MPSs) for the subsets were significantly different from the MPSs of the full-length test. As a follow-up analysis, RMSE and SEE (Standard Error of Estimation) values were calculated for each study condition. Results indicated that the estimated Angoff MPSs were significantly different from the full-test Angoff MPS (45.12) only in the study conditions of 30%-C-SRS, 40% C-SRS, 30% D-SRS and 30%-CD-SRS. According to RMSE values, the C-SRS method had the smallest error while the SRS method had the biggest one. Moreover, SEE examinations revealed that to achieve estimations similar to the full-test Angoff MPS (within one SEE), it is sufficient to get 50% of items with the C-SRS method. C-SRS method was the more effective one compared to the others in reducing the number of items rated by judges in MPS setting studies conducted with the Angoff method. (As Provided).
Anmerkungen	International Journal of Assessment Tools in Education. Pamukkale University, Faculty of Education, Kinikli Campus, Denizli 20070, Turkey. e-mail: ijate.editor@gmail.com; Web site: https://ijate.net/index.php/ijate
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01