A Multilevel Factor Analysis of Third-Party Evaluations of Noncognitive Constructs Used in Admissions Decision Making

Autor/inn/en	Oliveri, Maria; McCaffrey, Daniel; Ezzo, Chelsea; Holtzman, Steven
Titel	A Multilevel Factor Analysis of Third-Party Evaluations of Noncognitive Constructs Used in Admissions Decision Making
Quelle	In: Applied Measurement in Education, 30 (2017) 4, S.297-313 (17 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0895-7347
DOI	10.1080/08957347.2017.1353989
Schlagwörter	Factor Analysis; Decision Making; College Admission; Likert Scales; Graduate Study; College Applicants; Personality Measures; Hierarchical Linear Modeling; Scores; Inferences; Test Interpretation; Validity + Suchen Sie Ihr Suchwort? Faktorenanalyse; Decision-making; Entscheidungsfindung; Hochschulzugang; Hochschulzulassung; Zulassung; Likert-Skala; Aufbaustudium; Graduiertenstudium; Hauptstudium; College applications; Studienbewerber; Inference; Inferenz; Test analysis; Testauswertung; Gültigkeit
Abstract	The assessment of noncognitive traits is challenging due to possible response biases, "subjectivity" and "faking." Standardized third-party evaluations where an external evaluator rates an applicant on their strengths and weaknesses on various noncognitive traits are a promising alternative. However, accurate score-based inferences from third-party evaluations requires disentangling score variance due to raters versus applicants by utilizing a multilevel factor analysis (MFA). To date, MFA is highly underutilized in the measurement field. In this study, we apply the MFA to analyze third-party evaluations using data from the Personal Potential Index (PPI). The PPI is a third-party measure used to evaluate graduate school applicants noncognitive traits to help inform admissions decisions. We analyzed 12,693 ratings of 6,249 applicants divided into two randomly selected subgroups. We conducted multilevel exploratory factor analysis with one subgroup and tested the hypothesized structure with the other subgroup. This work illustrates the advantages and challenges of using MFA approach to support the meaningful and valid interpretation of scores from third-party evaluations. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01