Extended Multivariate Generalizability Theory with Complex Design Structures

Autor/inn/en	Brennan, Robert L.; Kim, Stella Y.; Lee, Won-Chan
Titel	Extended Multivariate Generalizability Theory with Complex Design Structures
Quelle	In: Educational and Psychological Measurement, 82 (2022) 4, S.617-642 (26 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Kim, Stella Y.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0013-1644
DOI	10.1177/00131644211049746
Schlagwörter	Multivariate Analysis; Generalizability Theory; Multiple Choice Tests; Test Construction; Test Items; Error Patterns; Test Bias; Test Reliability; Scores + Suchen Sie Ihr Suchwort? Multivariate Analyse; Multiple choice examinations; Multiple-choice tests, Multiple-choice examinations; Multiple-Choice-Verfahren; Testaufbau; Test content; Testaufgabe; Fehlertyp; Testkritik; Testreliabilität
Abstract	This article extends multivariate generalizability theory (MGT) to tests with different random-effects designs for each level of a fixed facet. There are numerous situations in which the design of a test and the resulting data structure are not definable by a single design. One example is mixed-format tests that are composed of multiple-choice and free-response items, with the latter involving variability attributable to both items and raters. In this case, two distinct designs are needed to fully characterize the design and capture potential sources of error associated with each item format. Another example involves tests containing both testlets and one or more stand-alone sets of items. Testlet effects need to be taken into account for the testlet-based items, but not the stand-alone sets of items. This article presents an extension of MGT that faithfully models such complex test designs, along with two real-data examples. Among other things, these examples illustrate that estimates of error variance, error-tolerance ratios, and reliability-like coefficients can be biased if there is a mismatch between the user-specified universe of generalization and the complex nature of the test. (As Provided).
Anmerkungen	SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01