A Rubric for the Detection of Students in Crisis

Autor/inn/en	Burkhardt, Amy; Lottridge, Susan; Woolf, Sherri
Titel	A Rubric for the Detection of Students in Crisis
Quelle	In: Educational Measurement: Issues and Practice, 40 (2021) 2, S.72-80 (9 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
DOI	10.1111/emip.12410
Schlagwörter	Scoring Rubrics; Identification; At Risk Students; Standardized Tests; Interrater Reliability + Suchen Sie Ihr Suchwort? Scoring formulas; Auswertungsbogen; Identifikation; Identifizierung; Standadised tests; Standardisierter Test; Interrater-Reliabilität
Abstract	For some students, standardized tests serve as a conduit to disclose sensitive issues of harm or distress that may otherwise go unreported. By detecting this writing, known as "crisis papers," testing programs have a unique opportunity to assist in mitigating the risk of harm to these students. The use of machine learning to automatically detect such writing is necessary in the context of online tests and automated scoring. To achieve a detection system that is accurate, humans must first consistently label the data that are used to train the model. This paper argues that the existing guidelines are not sufficient for this task and proposes a three-level rubric to guide the collection of the training data. In showcasing the fundamental machine learning procedures for creating an automatic text classification system, the following evidence emerges as support of the operational use of this rubric. First, hand-scorers largely agree with one another in assigning labels to text according to the rubric. Additionally, when this labeled data are used to train a baseline classifier, the model exhibits promising performance. Recommendations are made for improving the hand-scoring training process, with the ultimate goal of quickly and accurately assisting students in crisis. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01