Development and Validation of a Rating Scale for Summarization as an Integrated Task

Autor/inn/en	Li, Jiuliang; Wang, Qian
Titel	Development and Validation of a Rating Scale for Summarization as an Integrated Task
Quelle	In: Asian-Pacific Journal of Second and Foreign Language Education, 6 (2021), Artikel 11 (21 Seiten) PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Li, Jiuliang)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2363-5169
DOI	10.1186/s40862-021-00113-6
Schlagwörter	Validity; Rating Scales; Writing Skills; Writing Evaluation; Language Tests; Scoring Rubrics; Evaluators; Scores; Correlation; Test Construction; Student Attitudes + Suchen Sie Ihr Suchwort? Gültigkeit; Rating-Skala; Writing skill; Schreibfertigkeit; Language test; Sprachtest; Scoring formulas; Auswertungsbogen; Korrelation; Testaufbau; Schülerverhalten
Abstract	Summary writing is essential for academic success, and has attracted renewed interest in academic research and large-scale language test. However, less attention has been paid to the development and evaluation of the scoring scales of summary writing. This study reports on the validation of a summary rubric that represented an approach to scale development with limited resources out of consideration for practicality. Participants were 83 students and three raters. Diagnostic evaluation of the scale components and categories was based on raters' perception of their use and the scores of students' summaries which were analyzed using multifaceted Rasch measurement (MFRM). Correlation analysis revealed significant relationships among the scoring components, but the coefficients among some of the components were over high. MFRM analysis provided evidence in support of the usefulness of the scoring rubric, but also suggested the need of a refinement of the components and categories. According to the raters, the rubric was ambiguous in addressing some crucial text features. This study has implications for summarization task design, scoring scale development and validation in particular. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2022/1/01