Empirical vs. Expected IRT-Based Reliability Estimation in Computerized Multistage Testing (MST)

Autor/inn/en	Zhang, Yanwei; Breithaupt, Krista; Tessema, Aster; Chuah, David
Titel	Empirical vs. Expected IRT-Based Reliability Estimation in Computerized Multistage Testing (MST)
Quelle	(2006), (21 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Individual Testing; Test Reliability; Programming; Error of Measurement; Programming Languages; Adaptive Testing; Scores + Suchen Sie Ihr Suchwort? Persönlichkeitstest; Testreliabilität; Programmierung; Messfehler
Abstract	Two IRT-based procedures to estimate test reliability for a certification exam that used both adaptive (via a MST model) and non-adaptive design were considered in this study. Both procedures rely on calibrated item parameters to estimate error variance. In terms of score variance, one procedure (Method 1) uses the empirical ability distribution from a particular sample of examinees, and the other procedure (Method 2) assumes a normal distribution of ability and is sample-free. Due to the problem of sampling restriction in adaptive tests, Method 1 was modified (Method 1 extension) to "beef up" the sample and estimate reliability for each testlet in a MST panel before aggregating the estimates into an overall estimate for a test form or route. Overall, results imply that Method 1 and Method 2 tend to produce similar results for both adaptive and non-adaptive tests on the panel level and the test section level. Method 1 should not be applied to individual test forms in adaptive tests by a MST design. In the latter case, the modified or extended procedure can be used to alleviate the problem of restricted sample. The algorithms of the discussed procedures can be implemented in common statistical programming language such as SAS and SPSS as flexible alternatives to the theoretical and empirical reliability estimates computed, for example, by the BILOG-MG software. (Contains 6 tables and 1 figure.) (Author).
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10