Automated Summary Evaluation with Inbuilt Rubric Method: An Alternative to Constructed Responses and Multiple-Choice Tests Assessments

Autor/inn/en	Martínez-Huertas, José Á.; Jastrzebska, Olga; Olmos, Ricardo; León, José A.
Titel	Automated Summary Evaluation with Inbuilt Rubric Method: An Alternative to Constructed Responses and Multiple-Choice Tests Assessments
Quelle	In: Assessment & Evaluation in Higher Education, 44 (2019) 7, S.1029-1041 (13 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0260-2938
DOI	10.1080/02602938.2019.1570079
Schlagwörter	Automation; Scoring Rubrics; Alternative Assessment; Test Reliability; Test Validity; Student Evaluation; Reading Tests; Multiple Choice Tests; Reading Comprehension; High School Students; Undergraduate Students; Foreign Countries; Spain + Suchen Sie Ihr Suchwort? Scoring formulas; Auswertungsbogen; Testreliabilität; Testvalidität; Schulnote; Studentische Bewertung; Lesetest; Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Leseverstehen; High school; High schools; Student; Students; Oberschule; Schüler; Schülerin; Studentin; Ausland; Spanien
Abstract	Automated summary evaluation is proposed as an alternative to rubrics and multiple-choice tests in knowledge assessment. Inbuilt rubric is a recent Latent Semantic Analysis (LSA) method that implements rubrics in an artificially-generated semantic space. It was compared with classical LSA's cosine-based methods assessing knowledge in a within-subjects design regarding two validation sources: a comparison with the results of rubric scores and multiple-choice tests, and the sensitivity of predicting the academic level of the test-taker. Results showed a higher reliability for inbuilt rubric (from Pearson correlation coefficient 0.81 to 0.49) over the classical LSA method (from 0.61 to 0.34), and a higher sensitivity using binary logistic regressions and effect sizes to predict academic level. It is concluded that inbuilt rubric has a qualitatively higher reliability and validity than classical LSA methods in a way that is complementary to models based on semantic networks. Thus, it is concluded that new automated summary evaluation approaches such as the inbuilt rubric method can be practical in terms of reliability and efficiency, and, thus, they can offer an affordable and valuable form of knowledge assessment in different educational levels. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01