Literaturnachweis - Detailanzeige
| Autor/inn/en | Donoghue, John R.; McClellan, Catherine A.; Hess, Melinda R. |
|---|---|
| Titel | Investigating Constructed-Response Scoring over Time: The Effects of Study Design on Trend Rescore Statistics. Research Report. ETS RR-22-15 |
| Quelle | In: ETS Research Report Series, (2022), (16 Seiten)Infoseite zur Zeitschrift
PDF als Volltext kostenfreie Datei |
| Sprache | englisch |
| Dokumenttyp | gedruckt; online; Zeitschriftenaufsatz |
| Schlagwörter | Item Response Theory; Test Construction; Scoring; Testing; Scores; Sampling; Statistical Distributions; Interrater Reliability; Probability; Computation |
| Abstract | When constructed-response items are administered for a second time, it is necessary to evaluate whether the current Time B administration's raters have drifted from the scoring of the original administration at Time A. To study this, Time A papers are sampled and rescored by Time B scorers. Commonly the scores are compared using the proportion of exact agreement across times and/or t-statistics comparing Time A means to Time B means. It is common to treat these rescores with procedures that assume a multinomial sampling model, which is incorrect. The correct, product-multinomial model reflects the stratification of Time A scores. Using direct computation, the research report demonstrates that both proportion of exact agreement and the t-statistic can deviate substantially from expected behavior, providing misleading results. Reweighting the rescore table gives each statistic the correct expected value but does not guarantee that the usual sampling distributions hold. It is also noted that the results apply to a wider class of situations in which a set of papers is scored by one group of raters or scoring engine and then a sample is selected to be evaluated by a different group of raters or scoring engine. (As Provided). |
| Anmerkungen | Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/ |
| Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
| Update | 2024/1/01 |
Literaturbeschaffung und Bestandsnachweise in Bibliotheken prüfen
Permalink als QR-Code
Inhalt auf sozialen Plattformen teilen (nur vorhanden, wenn Javascript eingeschaltet ist)
Teile diese Seite:
