Comparing Different Trend Estimation Approaches in Country Means and Standard Deviations in International Large-Scale Assessment Studies

Autor/inn/en	Robitzsch, Alexander; Lüdtke, Oliver
Titel	Comparing Different Trend Estimation Approaches in Country Means and Standard Deviations in International Large-Scale Assessment Studies
Quelle	In: Large-scale Assessments in Education, 11 (2023), Artikel 26 (37 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Robitzsch, Alexander) ORCID (Lüdtke, Oliver)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
DOI	10.1186/s40536-023-00176-6
Schlagwörter	Educational Trends; Trend Analysis; International Assessment; Achievement Tests; Foreign Countries; Secondary School Students; Item Response Theory; Test Items; Scaling; Models; Simulation; Alternative Assessment; Reading; Mathematics; Sciences; Error Patterns; Testing Problems; Program for International Student Assessment + Suchen Sie Ihr Suchwort? Bildungsentwicklung; Trendanalyse; Achievement test; Achievement; Testing; Test; Tests; Leistungsbeurteilung; Leistungsüberprüfung; Leistung; Testdurchführung; Testen; Ausland; Sekundarschüler; Item-Response-Theorie; Test content; Testaufgabe; Scale construction; Skalenkonstruktion; Analogiemodell; Simulation program; Simulationsprogramm; Leseprozess; Lesen; Mathematik; Science; Wissenschaft; Fehlertyp
Abstract	One major aim of international large-scale assessments (ILSA) like PISA is to monitor changes in student performance over time. To accomplish this task, a set of common items (i.e., link items) is repeatedly administered in each assessment. Linking methods based on item response theory (IRT) models are used to align the results from the different assessments on a common scale. This work employs the one-parameter logistic (1PL) and the two-parameter logistic (2PL) IRT models as scaling models for dichotomous item response data. The present article discusses different types of trend estimates in country means and standard deviations for countries in ILSA. These types differ in three aspects. First, the trend can be assessed by an indirect or direct linking approach for linking a country's performance at an international metric. Second, the linking for the trend estimation can rely on either all items or only the link items. Third, item parameters can be assumed to be invariant or noninvariant across countries. It is shown that the most often employed trend estimation methods of original trends and marginal trends can be conceived as particular cases of indirect and direct linking approaches, respectively. Through a simulation study and analytical derivations, it is demonstrated that trend estimates using a direct linking approach and those that rely on only link items outperformed alternatives for the 1PL model with uniform country differential item functioning (DIF) and the 2PL model with uniform and nonuniform country DIF. We also illustrated the performance of the different scaling models for assessing the PISA trend from PISA 2006 to PISA 2009 in the cognitive domains of reading, mathematics, and science. In this empirical application, linking errors based on jackknifing testlets were utilized that adequately quantify DIF effects in the uncertainty of trend estimates. (As Provided).
Anmerkungen	Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01