Confidence Bounds and Power for the Reliability of Observational Measures on the Quality of a Social Setting

Autor/inn/en	Shin, Yongyun; Raudenbush, Stephen W.
Titel	Confidence Bounds and Power for the Reliability of Observational Measures on the Quality of a Social Setting
Quelle	In: Psychometrika, 77 (2012) 3, S.543-560 (18 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0033-3123
DOI	10.1007/s11336-012-9266-4
Schlagwörter	Generalizability Theory; Neighborhoods; Intervals; Child Care Centers; Social Scientists; Performance Based Assessment; Observation; Educational Environment; Interaction; Reliability; Tests; Measurement; Evaluation Methods; Sample Size; Psychometrics + Suchen Sie Ihr Suchwort? Neighbourhoods; Nachbarschaft; Child care facilities; Child care services; Kinderzentrum; Kinderbetreuung; Social scientist; Sozialwissenschaftler; Leistungsermittlung; Beobachtung; Lernumgebung; Pädagogische Umwelt; Schulumwelt; Interaktion; Reliabilität; Examination; Prüfung; Examen; Messverfahren; Psychometry; Psychometrie
Abstract	Social scientists are frequently interested in assessing the qualities of social settings such as classrooms, schools, neighborhoods, or day care centers. The most common procedure requires observers to rate social interactions within these settings on multiple items and then to combine the item responses to obtain a summary measure of setting quality. A key aspect of the quality of such a summary measure is its reliability. In this paper we derive a confidence interval for reliability, a test for the hypothesis that the reliability meets a minimum standard, and the power of this test against alternative hypotheses. Next, we consider the problem of using data from a preliminary field study of the measurement procedure to inform the design of a later study that will test substantive hypotheses about the correlates of setting quality. The preliminary study is typically called the "generalizability study" or "G study" while the later, substantive study is called the "decision study" or "D study." We show how to use data from the G study to estimate reliability, a confidence interval for the reliability, and the power of tests for the reliability of measurement produced under alternative designs for the D study. We conclude with a discussion of sample size requirements for G studies. (As Provided).
Anmerkungen	Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: service-ny@springer.com; Web site: http://www.springerlink.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10