Literaturnachweis - Detailanzeige
| Autor/inn/en | Mozer, Reagan; Miratrixy, Luke; Relyea, Jackie Eunjung; Kim, James S. |
|---|---|
| Institution | Annenberg Institute for School Reform at Brown University |
| Titel | Bridging Human and Machine Scoring in Experimental Assessments of Writing: Tools, Tips, and Lessons Learned from a Field Trial in Education. EdWorkingPaper No. 21-493 |
| Quelle | (2021), (53 Seiten)
PDF als Volltext kostenfreie Datei |
| Sprache | englisch |
| Dokumenttyp | gedruckt; online; Monographie |
| Schlagwörter | Scoring; Automation; Data Analysis; Natural Language Processing; Artificial Intelligence; Intervention; Elementary School Students; Grade 1; Grade 2; Persuasive Discourse; Writing Evaluation; Instructional Effectiveness; Content Area Writing |
| Abstract | In a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the hand-coded scores as a measured outcome. This process is both time and labor-intensive, which creates a persistent barrier for large-scale assessments of text. Furthermore, enriching ones understanding of a found impact on text outcomes via secondary analyses can be difficult without additional scoring efforts. Machine-based text analytic and data mining tools offer one potential avenue to help facilitate research in this domain. For instance, we could augment a traditional impact analysis that examines a single human-coded outcome with a suite of automatically generated secondary outcomes. By analyzing impacts across a wide array of text-based features, we can then explore what an overall change signifies, in terms of how the text has evolved due to treatment. In this paper, we propose several different methods for supplementary analysis in this spirit. We then present a case study of using these methods to enrich an evaluation of a classroom intervention on young children's writing. We argue that our rich array of findings move us from "it worked" to "it worked because" by revealing how observed improvements in writing were likely due, in part, to the students having learned to marshal evidence and speak with more authority. Relying exclusively on human scoring, by contrast, is a lost opportunity. (As Provided). |
| Anmerkungen | Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: AISR_Info@brown.edu; Web site: http://www.annenberginstitute.org |
| Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
| Update | 2024/1/01 |
Literaturbeschaffung und Bestandsnachweise in Bibliotheken prüfen
Permalink als QR-Code
Inhalt auf sozialen Plattformen teilen (nur vorhanden, wenn Javascript eingeschaltet ist)
Teile diese Seite:
