The Impact of Process Instructions on Judges' Use of Examinee Performance Data in Angoff Standard Setting Exercises

Autor/inn/en	Mee, Janet; Clauser, Brian E.; Margolis, Melissa J.
Titel	The Impact of Process Instructions on Judges' Use of Examinee Performance Data in Angoff Standard Setting Exercises
Quelle	In: Educational Measurement: Issues and Practice, 32 (2013) 3, S.27-35 (9 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0731-1745
Schlagwörter	Standard Setting (Scoring); Judges; Data; Decision Making; Performance; Accuracy; Expertise + Suchen Sie Ihr Suchwort? Bewertungsskala; Judge; Richter; Daten; Decision-making; Entscheidungsfindung; Achievement; Leistung; Expert appraisal
Abstract	Despite being widely used and frequently studied, the Angoff standard setting procedure has received little attention with respect to an integral part of the process: how judges incorporate examinee performance data in the decision-making process. Without performance data, subject matter experts have considerable difficulty accurately making the required judgments. Providing data introduces the very real possibility that judges will turn their content-based judgments into norm-referenced judgments. This article reports on three Angoff standard setting panels for which some items were randomly assigned to have incorrect performance data. Judges were informed that some of the items were accompanied by inaccurate data, but were not told which items they were. The purpose of the manipulation was to assess the extent to which changing the instructions given to the judges would impact the extent to which they relied on the performance data. The modified instructions resulted in the judges making less use of the performance data than judges participating in recent parallel studies. The relative extent of the change judges made did not appear to be substantially influenced by the accuracy of the data. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10