Cost Analysis and Cost-Effectiveness of Hand-Scored and Automated Approaches to Writing Screening

Autor/inn/en	Matta, Michael; Keller-Margulis, Milena A.; Mercer, Sterett H.
Titel	Cost Analysis and Cost-Effectiveness of Hand-Scored and Automated Approaches to Writing Screening
Quelle	(2022), (55 Seiten) PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Matta, Michael) ORCID (Keller-Margulis, Milena A.) ORCID (Mercer, Sterett H.) Weitere Informationen
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Cost Effectiveness; Scoring; Automation; Writing Tests; Resource Allocation; Predictive Validity; Accuracy; Curriculum Based Assessment; Elementary School Students; Writing Skills + Suchen Sie Ihr Suchwort? Kosten-Nutzen-Analyse; Kosten-Nutzen-Denken; Bewertung; Writing test; Schreibtest; Ressourcenallokation; Writing skill; Schreibfertigkeit
Abstract	Although researchers have investigated technical adequacy and usability of written-expression curriculum-based measures (WE-CBM), the economic implications of different scoring approaches have largely been ignored. The absence of such knowledge can undermine the effective allocation of resources and lead to the adoption of suboptimal measures for the identification of students at risk for poor writing outcomes. Therefore, we used the Ingredients method to compare implementation costs and cost-effectiveness of hand-calculated and automated scoring approaches. Data analyses were conducted on secondary data from a study that evaluated predictive validity and diagnostic accuracy of quantitative approaches for scoring WE-CBM samples. Findings showed that automated approaches offered more economic solutions than hand-calculated methods; for automated scores, the effects were stronger when the free writeAlizer R package was employed, whereas for hand-calculated scores, simpler WE-CBM metrics were less costly than more complex metrics. Sensitivity analyses confirmed the relative advantage of automated scores when the number of classrooms, students, and assessment occasions per school year increased; again, writeAlizer was less sensitive to the changes in the ingredients than the other approaches. Finally, the visualization of the cost-effectiveness ratio illustrated that writeAlizer offered the optimal balance between implementation costs and diagnostic accuracy, followed by complex hand-calculated metrics and a proprietary automated program. Implications for the use of hand-calculated and automated scores for the universal screening of written expression with elementary students are discussed. [This paper was published in "Journal of School Psychology" v92 p80-95 Jun 2022.] (As Provided).
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01