Effect of Observation Mode on Measures of Teaching

Literaturnachweis - Detailanzeige

Trefferliste

Autor/inn/en	McCaffrey, Daniel F.; Casabianca, Jodi M.
Institution	Society for Research on Educational Effectiveness (SREE)
Titel	Effect of Observation Mode on Measures of Teaching
Quelle	(2013), (9 Seiten) PDF als Volltext kostenfreie Datei Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Monographie
Schlagwörter	Urban Schools; Classroom Observation Techniques; Video Technology; Teacher Evaluation; Quality Control; Affordances; Middle School Teachers; Secondary School Teachers; Reliability; Scores; Minority Group Students; Economically Disadvantaged; Eligibility; Lunch Programs; Breakfast Programs; Error of Measurement; Mathematics Teachers; Intervention; Models + Suchen Sie Ihr Suchwort? Urban area; Urban areas; School; Schools; Stadtregion; Stadt; Schule; Teacher appraisal; Lehrerbeurteilung; Qualitätskontrolle; Middle school; Middle schools; Teacher; Teachers; Mittelschule; Mittelstufenschule; Lehrer; Lehrerin; Lehrende; Reliabilität; Eignung; Mittagessen; Schulfrühstück; Messfehler; Mathematics; Mathematik; Analogiemodell
Abstract	As the education reform movement increasingly focuses on teachers and teaching, educators, policy-makers, and researchers need valid and reliable measures that can be used to evaluate individual teachers, provide guidance for improving teaching performance, and support research in ways that advance instruction and classroom dialog and practice. A new generation of classroom evaluation tools has recently been developed to support evaluation of teaching. Live observations tend to be the standard for studies of teaching and teacher evaluations in practice. They have the benefit of the observer being in the teacher's physical classroom. This is valuable for teacher evaluations because it gives observation scores credibility among teachers. Using video provides particular affordances because they create a permanent record and teachers can review them to evaluate their own instruction as professional development (Miller, 2007; Sherin & Han, 2004; van Es & Sherin, 2010). Videos can be scored by multiple raters, which can reduce error by averaging scores. The use of video also allows for scores to be audited as a part of quality control and videos can be evaluated using multiple scoring protocols to assess the robustness of inferences to a protocol. For most of these reasons, many recent studies of classrooms have made use of videos (Bill and Melinda Gates Foundation, 2012). Given these affordances, an important issue is to understand the comparability of the nature and quality of information created through these two observation modes. Nearly 20 years ago Jaeger (1993) identified mode of observation as potentially contributing to the psychometric properties of measuring teaching, but little research on mode effects has occurred since. This study is a first step toward rectifying the dearth in knowledge on the effects of observation mode on the psychometric properties of classroom teaching evaluations. It tests for observation mode effects on inferences about teaching, classrooms, and teachers. Research questions include the following: (1) Do raters systematically give higher scores using one observation mode or the other? (2) Does the observation mode affect the rank ordering of scores? (3) Does the observation mode affect the size of the standard errors of measurement or the reliability of scores? (4) What are the implications of errors for inferences about the teaching in a lesson or for a classroom for a year? The authors use data collected for the study Toward an Understanding of Classroom Context (TUCC) to test for mode effects in the scores and inferences about the teaching in lessons and in classrooms. TUCC took place in middle and high schools in an urban fringe mid-Atlantic school district that serves roughly 90 percent students of color and 55 percent students who are eligible for free or reduced price meals. The study concluded that there are small mode effects on score means but they are relatively small and most likely inconsequential. Modes do not rank order teachers differently; it is the measurement error that results in differences between live observations and video scoring in the ordering of classrooms or lessons. Differences in the decomposition of variance across modes are a result of the differences in scoring dates. Scoring trends and the differences in timing across the modes are the only significant difference between modes. They have important implications for studies using classroom observations. Live scoring will confound rater learning with lessons and video scoring can avoid this confound. Figures are appended. (ERIC).
Anmerkungen	Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01