Literaturnachweis - Detailanzeige
| Autor/inn/en | Lambert, Richard G.; Holcomb, T. Scott; Bottoms, Bryndle |
|---|---|
| Institution | University of North Carolina at Charlotte, Center for Educational Measurement and Evaluation (CEME) |
| Titel | Examining Inter-Rater Reliability of Evaluators Judging Teacher Performance: Proposing an Alternative to Cohen's Kappa. CEME Technical Report. CEMETR-2022-02 |
| Quelle | (2022), (40 Seiten)
PDF als Volltext kostenfreie Datei |
| Sprache | englisch |
| Dokumenttyp | gedruckt; online; Monographie |
| Schlagwörter | Interrater Reliability; Evaluators; Rating Scales; Teacher Evaluation; Preschool Teachers; Private Education; Scoring |
| Abstract | The validity of the Kappa coefficient of chance-corrected agreement has been questioned when the prevalence of specific rating scale categories is low and agreement between raters is high. The researchers proposed the Lambda Coefficient of Rater-Mediated Agreement as an alternative to Kappa to address these concerns. Lambda corrects for chance agreement based on specific assumptions about raters and the rater-mediated assessment process including rater-specific tendencies for strict or lenient ratings. Actual ratings of teacher profiles from an interrater reliability exercise confirmed the shortcomings of Kappa. The rater data also demonstrated the robustness of Lambda-1, Lambda-2, Gwet's AC1, and Gwet's AC2 to the data conditions that are problematic for Kappa. All four alternative chance-corrected agreement coefficients showed less variability across the 65 raters than Kappa. However, AC-2 was undetermined for 57 of the 65 raters. Simulation data demonstrated the robustness of the Lambda Coefficient of Rater-Mediated Agreement to the data conditions that are problematic for Kappa. (As Provided). |
| Anmerkungen | Center for Educational Measurement and Evaluation. Department of Educational Leadership, UNC Charlotte, 9201 University City Blvd., Charlotte, NC 28223. Tel: 704-687-8867; Fax: 704-687-1629; e-mail: ceme-coed@uncc.edu; Web site: https://ceme.charlotte.edu/ |
| Erfasst von | ERIC (Education Resources Information Center), Washington, DC |
| Update | 2024/1/01 |
Literaturbeschaffung und Bestandsnachweise in Bibliotheken prüfen
Permalink als QR-Code
Inhalt auf sozialen Plattformen teilen (nur vorhanden, wenn Javascript eingeschaltet ist)
Teile diese Seite:
