Related Critical Psychometric Issues and Their Resolutions during Development of PE Metrics

Autor/inn/en	Fox, Connie; Zhu, Weimo; Park, Youngsik; Fisette, Jennifer L.; Graber, Kim C.; Dyson, Ben; Avery, Marybell; Franck, Marian; Placek, Judith H.; Rink, Judy; Raynes, De
Titel	Related Critical Psychometric Issues and Their Resolutions during Development of PE Metrics
Quelle	In: Measurement in Physical Education and Exercise Science, 15 (2011) 2, S.138-154 (17 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1091-367X
Schlagwörter	Measures (Individuals); Psychometrics; Interrater Reliability; Training; Scaling; High Stakes Tests; Standards; Observation; Coding; Physical Education; Scoring Rubrics; Kindergarten; Grade 5; Grade 2; Item Response Theory; Item Banks; Test Items; Student Evaluation; Measurement; Test Construction; Performance Tests; Psychomotor Skills; Item Analysis; Test Bias + Suchen Sie Ihr Suchwort? Messdaten; Psychometry; Psychometrie; Interrater-Reliabilität; Ausbildung; Scale construction; Skalenkonstruktion; Standard; Beobachtung; Codierung; Programmierung; Körpererziehung; Sportunterricht; Scoring formulas; Auswertungsbogen; School year 05; 5. Schuljahr; Schuljahr 05; School year 02; 2. Schuljahr; Schuljahr 02; Item-Response-Theorie; Test content; Testaufgabe; Schulnote; Studentische Bewertung; Messverfahren; Testaufbau; Leistungsbeurteilung; Leistungsermittlung; Leistungsmessung; Leistungsüberprüfung; Psychomotorische Aktivität; Itemanalyse; Testkritik
Abstract	In addition to validity and reliability evidence, other psychometric qualities of the PE Metrics assessments needed to be examined. This article describes how those critical psychometric issues were addressed during the PE Metrics assessment bank construction. Specifically, issues included (a) number of items or assessments needed, (b) training protocol for required intra- and inter-rater objectivity, and (c) the development of a score scale. First, using a subsample of data from the PE Metrics study, in which students were assessed using four assessments, the impact of the number of assessments was examined. It was found that at least two assessments are needed when applying PE Metrics for the purpose of high stakes testing. Single individual assessments can still be used in teaching practice, but the results must be interpreted with caution. Second, with the training protocol developed for PE Metrics, satisfactory intra-rater objectivity can be achieved. When two or more raters are involved in rating, however, an additional monitoring protocol should be employed so that inter-rater objectivity can be monitored and controlled. Third, to help allow for a consistent interpretation and reporting of PE Metrics results, a score scale was developed. Other related issues, such as test fairness and setting performance standards, were discussed, and future directions concerning PE Metrics maintenance and continuing development were outlined. (Contains 11 tables.) (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10