Exploring Difficult-to-Score Essays with a Hyperbolic Cosine Accuracy Model and Coh-Metrix Indices

Autor/inn/en	Wang, Jue; Engelhard, George; Combs, Trenton
Titel	Exploring Difficult-to-Score Essays with a Hyperbolic Cosine Accuracy Model and Coh-Metrix Indices
Quelle	In: Journal of Experimental Education, 91 (2023) 1, S.125-144 (20 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Wang, Jue) ORCID (Engelhard, George)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0973
DOI	10.1080/00220973.2021.1993774
Schlagwörter	Writing Evaluation; Scoring; Accuracy; Computational Linguistics; Essays; Difficulty Level; Readability; Language Usage; Semantics; Syntax; Models; Evaluators + Suchen Sie Ihr Suchwort? Bewertung; Linguistics; Computerlinguistik; Essay; Aufsatzunterricht; Schwierigkeitsgrad; Lesbarkeit; Sprachgebrauch; Semantik; Analogiemodell
Abstract	Unfolding models are frequently used to develop scales for measuring attitudes. Recently, unfolding models have been applied to examine rater severity and accuracy within the context of rater-mediated assessments. One of the problems in applying unfolding models to rater-mediated assessments is that the substantive interpretations of the latent variable are challenging. The purpose of this study is to evaluate the use of Coh-Metrix indices as an approach for defining the characteristics of essays related to their scoring difficulty. Essays from a rater-mediated writing assessment were selected (N = 100), and rater accuracy was defined using an unfolding model. The scoring difficulty of essays along the unfolding scale (i.e., difficult-to-score below, easy-to-score, and difficult-to-score above) was related to essay feature indices defined with the Coh-Metrix text analyzer. Six areas of essay features were considered: descriptive, text readability, lexical diversity, latent semantic analysis, syntactic complexity, and word information. The results suggest that the substantive interpretation of the underlying unfolding scale can be enhanced by examining essay characteristics with Coh-Metrix indices. This methodology offers a promising approach for understanding the unfolding continuum and gaining information about the characteristics of difficult-to-score essays. Suggestions and implications for evaluating rater-mediated assessment systems using unfolding models are discussed. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01