Comparison of Classical Test Theory vs. Multi-Facet Rasch Theory

Autor/inn/en	Polat, Murat; Turhan, Nihan S.; Toraman, Cetin
Titel	Comparison of Classical Test Theory vs. Multi-Facet Rasch Theory
Quelle	In: Pegem Journal of Education and Instruction, 12 (2022) 2, S.213-225 (13 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Zusatzinformation	ORCID (Turhan, Nihan S.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2146-0655
Schlagwörter	Comparative Analysis; Test Theory; Item Response Theory; Student Evaluation; Language Tests; Writing Evaluation; Scores; Scoring Rubrics; Correlation; Validity; Undergraduate Students + Suchen Sie Ihr Suchwort? Testtheorie; Item-Response-Theorie; Schulnote; Studentische Bewertung; Language test; Sprachtest; Scoring formulas; Auswertungsbogen; Korrelation; Gültigkeit
Abstract	Testing English writing skills could be multi-dimensional; thus, the study aimed to compare students' writing scores calculated according to Classical Test Theory (CTT) and Multi-Facet Rasch Model (MFRM). The research was carried out in 2019 with 100 university students studying at a foreign language preparatory class and four experienced instructors who participated in the study as raters. Data of the study were collected by using a writing rubric consisting of four components (content, organization, grammar and vocabulary). Participants' writing scores were analysed thoroughly both by CTT and MFRM. At the first step, the participants' writing scores were calculated by taking the means of the writing points given by the graders in the CTT model. Then, the MFRM was applied to the data through a three-facet design considering the rater, student and rubric components as MFRM facets respectively. Finally, ability estimates obtained and reported in the logit scale via Rasch Analysis were converted into the analytic rubric's component scores used throughout the scoring procedure. Finally, two sets of writing scores were calculated and compared according to both measurement models. Considering the findings, it was summoned that there was a positive and high correlation between the ability estimates found according to the CTT and the MFRM. However, the mean score difference calculated according to both theories was still significant. Moreover, the analyses showed that criterion validity of the writing scores obtained via the MFRM was higher than the scores obtained via the CTT. (As Provided).
Anmerkungen	Pegem Academy Publishing and Educational Guidance Services TLC. Mesrutiyet Caddesi, No: 45, Ankara, Kizilay 06420, Turkey. e-mail: editor@pegegog.net; Web site: http://www.pegegog.net/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01