Estimating within-group agreement in small groups. A proposed adjustment for the average deviation index.

Autor/inn/en	Lohse-Bossenz, Hendrik; Kunina-Habenicht, Olga; Kunter, Mareike
Titel	Estimating within-group agreement in small groups. A proposed adjustment for the average deviation index.
Quelle	In: European journal of work and organizational psychology, 23 (2014) 3, S. 456-468 PDF als Volltext Link als defekt meldenVerfügbarkeit
Beigaben	Literaturangaben
Zusatzinformation	Forschungsdaten, Studiendetails und Erhebungsinstrumente
Sprache	englisch
Dokumenttyp	online; gedruckt; Zeitschriftenaufsatz
ISSN	1359-432X; 1464-0643
DOI	10.1080/1359432X.2012.748189
Schlagwörter	Methode; Schätzung; Aggregation; Datenstruktur; Übereinstimmung
Abstract	In many research contexts where a multilevel data structure is present, researchers have to aggregate individual responses to the group level (e.g., team climate) to test specific hypotheses. An established way of justifying this aggregation is to show sufficient within-group agreement. An increasingly common measure of within-group agreement is the Average Deviation Index (ADM). The study elaborates on the properties of the ADM within small groups. This is a crucial topic, as many multilevel studies incorporate teams of fewer than ten members. Comparing practical and critical values for interpreting ADM magnitudes shows that in small groups the ADM is more likely to overestimate the agreement by receiving a smaller ADM value by chance. We assume the calculation procedure of the ADM to be one reason for the overestimation and therefore propose an adjusted calculation. After exploring the properties of the adjustment with simulated data, the original and adjusted calculations are applied to an empirical example with multiple ratings from 48 experts. Compared to the original ADM, using the adjusted ADM leads to a less biased agreement estimate within smaller groups. However, researchers are encouraged to use practical as well as critical values to interpret the level of agreement when using the ADM. (Orig.).
Erfasst von	DIPF \| Leibniz-Institut für Bildungsforschung und Bildungsinformation, Frankfurt am Main
Update	2024/2