Calibrated Parsing Items Evaluation: A Step towards Objectifying the Translation Assessment

Autor/inn/en	Akbari, Alireza; Shahnazari, Mohammadtaghi
Titel	Calibrated Parsing Items Evaluation: A Step towards Objectifying the Translation Assessment
Quelle	In: Language Testing in Asia, 9 (2019), Artikel 8 (27 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2229-0443
DOI	10.1186/s40468-019-0083-x
Schlagwörter	Test Items; Translation; Computer Software; Evaluators; Probability; Validity; Reliability; Comparative Analysis; Language Tests; Evaluation Methods; Second Languages; Language Processing + Suchen Sie Ihr Suchwort? Test content; Testaufgabe; Wahrscheinlichkeitsrechnung; Wahrscheinlichkeitstheorie; Gültigkeit; Reliabilität; Language test; Sprachtest; Second language; Zweitsprache; Sprachverarbeitung
Abstract	The present research paper introduces a translation evaluation method called Calibrated Parsing Items Evaluation (CPIE hereafter). This evaluation method maximizes translators' performance through identifying the parsing items with an optimal p-docimology and d-index (item discrimination). This method checks all the possible parses (annotations) in a source text by means of the Brat Visualization Stanford CoreNLP software. CPIE takes a step towards the objectification of translation assessment by allowing evaluators to assess values (impacts) of the items in source texts via docimologically justified parsing items. For this paper, 16 evaluators were recruited to score translation drafts by means of the holistic, analytic, Preselected Items Evaluation (PIE) methods and CPIE method. For the present research paper, "F-Statistics," "Probability Plot," "Spearman rho," and "Regression Variable Plot" were applied to the evaluators' translation assessments to ensure the degree of validity and reliability of CPIE compared to the holistic, analytic, and PIE methods, respectively. The results indicated that the CPIE method was more consistent and valid in terms of docimologically justified parsing items. The limitations and the possibilities of the CPIE method in web-based platforms were also discussed. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01