An Exploration of Comparability Issues in Educational Research: Scale Linking, Equating, and Propensity Score Weighting

Autor/in	Wu, Tong
Titel	An Exploration of Comparability Issues in Educational Research: Scale Linking, Equating, and Propensity Score Weighting
Quelle	(2023), (152 Seiten) PDF als Volltext Verfügbarkeit Ph.D. Dissertation, The University of North Carolina at Charlotte
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
ISBN	979-8-3794-3753-4
Schlagwörter	Hochschulschrift; Dissertation; Educational Research; Comparative Analysis; Equated Scores; Weighted Scores; Item Response Theory; Item Analysis; Error of Measurement; Test Bias; Validity + Suchen Sie Ihr Suchwort? Thesis; Dissertations; Academic thesis; Bildungsforschung; Pädagogische Forschung; Item-Response-Theorie; Itemanalyse; Messfehler; Testkritik; Gültigkeit
Abstract	This three-article dissertation aims to address three methodological challenges to ensure comparability in educational research, including scale linking, test equating, and propensity score (PS) weighting. The first study intends to improve test scale comparability by evaluating the effect of six missing data handling approaches, including listwise deletion (LWD), treating missing data as incorrect responses (IN), corrected item mean imputation (CM), imputing with a response function (RF), multiple imputation (MI), and full information likelihood information (FIML), on item response theory (IRT) scale linking accuracy when missing data occur within common items. The relative performance of these six missing data treatment methods under two missing mechanisms is explored with simulated data. Results show that RF, MI, and FIML produce fewer errors for conducting scale linking, whereas LWD is associated with the most errors regardless of testing conditions. The second study aims to ensure test score comparability by proposing a new equating method to account for rater errors in rater-mediated assessments. Specifically, the performance of using an IRT observed-score equating method with a hierarchical rater model (HRM) is investigated under various conditions. The newly proposed equating method leads to comparable bias, SE, and RMSE to that of a traditional IRT observed-score equating method with the use of generalized partial credit model (GPCM) as normal raters scoring the new test forms. However, when aberrant raters are involved in the scoring process, the HRM IRT observed-score equating method generally produces more accurate results in bias and RMSE, though generates comparable SEs to the traditional method. The third study examines the performance of six covariate balance diagnostics when using PS weighting method with multilevel data. Specifically, a set of simulated conditions is used to examine the ability of within-cluster and pooled absolute standardized bias (ASB), variance ratio (VR), and percent bias reduction (PBR) methods to identify a correct PS model. In addition, the association between the balance statistics and the bias in treatment effect is explored. Within-cluster ASB and PBR are observed to be associated with the most accurate results in the choice of PS model as compared to other diagnostics. Pooled ASB is found to have the highest association with the treatment effect bias. By advancing the methodology for addressing comparability issues, the dissertation intends to enhance the validity and improve the quality of educational research. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.] (As Provided).
Anmerkungen	ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01