An Examination of Rater Drift within a Generalizability Theory Framework

Autor/inn/en	Harik, Polina; Clauser, Brian E.; Grabovsky, Irina; Nungester, Ronald J.; Swanson, Dave; Nandakumar, Ratna
Titel	An Examination of Rater Drift within a Generalizability Theory Framework
Quelle	In: Journal of Educational Measurement, 46 (2009) 1, S.43-58 (16 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0655
DOI	10.1111/j.1745-3984.2009.01068.x
Schlagwörter	Generalizability Theory; Performance Based Assessment; Performance Tests; Clinical Experience; Mathematical Models; Scores + Suchen Sie Ihr Suchwort? Leistungsermittlung; Leistungsbeurteilung; Leistungsmessung; Leistungsüberprüfung; Mathematical model; Mathematisches Modell
Abstract	The present study examined the long-term usefulness of estimated parameters used to adjust the scores from a performance assessment to account for differences in rater stringency. Ratings from four components of the USMLE[R] Step 2 Clinical Skills Examination data were analyzed. A generalizability-theory framework was used to examine the extent to which rater-related sources of error could be eliminated through statistical adjustment. Particular attention was given to the stability of these estimated parameters over time. The results suggest that rater stringency estimates obtained at a point in time and then used to adjust ratings over a period of months may substantially decrease in usefulness. In some cases, over several months, the use of these adjustments may become counterproductive. Additionally, it is hypothesized that the rate of deterioration in the usefulness of estimated parameters may be a function of the characteristics of the scale. (As Provided).
Anmerkungen	Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail: customerservices@blackwellpublishing.com; Web site: http://www.blackwellpublishing.com/jnl_default.asp
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10