Assessing Raters: What Factors Predict Discernment in Novice Creativity Raters?

Autor/inn/en	Ceh, Simon Majed; Edelmann, Carina; Hofer, Gabriela; Benedek, Mathias
Titel	Assessing Raters: What Factors Predict Discernment in Novice Creativity Raters?
Quelle	In: Journal of Creative Behavior, 56 (2022) 1, S.41-54 (14 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Ceh, Simon Majed) ORCID (Hofer, Gabriela) ORCID (Benedek, Mathias)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0022-0175
DOI	10.1002/jocb.515
Schlagwörter	Novices; Evaluators; Creativity; Personality Traits; Prediction; Task Analysis; Time; Motivation; Rating Scales; Interrater Reliability + Suchen Sie Ihr Suchwort? Kreativität; Individual characteristics; Personality characteristic; Persönlichkeitsmerkmal; Vorhersage; Aufgabenanalyse; Zeit; psychologische; Motivation (psychologisch); Rating-Skala; Interrater-Reliabilität
Abstract	Creativity research crucially relies on creativity evaluations by external raters, but it is not clear what properties characterize good raters. In the present study, we investigated whether rater personality and rater creativity are related to discernment (i.e., the ability to distinguish creative from uncreative responses) when evaluating others' responses. To this end, 166 novice raters evaluated a pre-selected set of responses from the alternate uses task for creativity. We further assessed the rater's creativity and personality and explored the role of task factors such as rating motivation and rating time. We found that creativity in terms of creative potential and creative achievement independently predicted rater discernment. At the level of personality, openness to experience was related to higher discernment, but not any other Big-Five factors or dark triad traits. Rater discernment also increased with rating time, rating motivation, and rater leniency (i.e., the tendency to appreciate creative responses). Finally, we seized the large rater sample to estimate to what extent inter-rater reliability depends on rater number and how dependable ratings are across the rating scale. Together, these findings highlight relevant factors at person, task, and method level predicting individual discernment, which should be considered to ensure good inter-rater reliability in creativity ratings. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01