A Study of the Inter-Rater Reliability of University Application Readers in a Holistic Admissions Review Process

Autor/in	Moody Rideout, Blaire Lauren
Titel	A Study of the Inter-Rater Reliability of University Application Readers in a Holistic Admissions Review Process
Quelle	(2017), (161 Seiten) PDF als Volltext Verfügbarkeit Ph.D. Dissertation, Bowling Green State University
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
ISBN	978-1-3697-7301-9
Schlagwörter	Hochschulschrift; Dissertation; Interrater Reliability; College Applicants; Holistic Approach; Evaluation Methods; College Admission; Undergraduate Students; Content Analysis; Evaluators + Suchen Sie Ihr Suchwort? Thesis; Dissertations; Academic thesis; Interrater-Reliabilität; College applications; Studienbewerber; Holistischer Ansatz; Hochschulzugang; Hochschulzulassung; Zulassung; Inhaltsanalyse
Abstract	In 2015, the American Council on Education surveyed undergraduate admission and enrollment management leaders at 338 four-year institutions to understand holistic admissions review (Espinosa, Gaertner, and Orfield, 2015). In the report titled, Race, Class and College Access: Achieving Diversity in a Shifting Legal Landscape, 92% of selective institutions reported utilizing a holistic admissions review process and 72% cited the Gratz v. Bollinger (2003) and Grutter v. Bollinger (2003) court cases as the impetus for increasing their use (Espinosa et al., 2015). Holistic admissions review has led to admission directors relying heavily on applicant qualities beyond GPA and test scores, such as leadership and extracurricular experience (Hoover, 2013; Sandlin, 2008; Spencer, 2008). Yet, little research has been conducted on the reliability of holistic admissions as an assessment or review tool, which is inherently subjective. This study sought to examine a holistic admissions review process at one institution to simply show if variance occurred and possible explanations for it. The study analyzed reader reviews for approximately 15,000 individual undergraduate admission reviews over three years. The primary method was focused on the inter-rater reliability of the application reviews. If there was significant variance for particular reviews, there was an analysis of the content behind such reviews. Findings revealed that subjective measures of performance review, and in this case holistic admissions review, may not be susceptible to high levels of variability with 97% of reader reviews showing agreement for this study. Overall, just 3% of the subjective application reviews showed variability. The content analysis of the 208 application reviews with variance revealed 20 categories within six themes of Concern, Diversity, Extracurricular, Lacking Substance, Missing Information, and Writing Style. Additionally, female and international applicants were more represented in the variability group when comparing to applicants overall. Implications for future research and professional practice should focus on reader characteristics and experiences, quantity of readers utilized, reader review systematization, and reader hiring, training, and supervision. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.] (As Provided).
Anmerkungen	ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01