Psychophysics of the McGurk and Other Audiovisual Speech Integration Effects

Autor/inn/en	Jiang, Jintao; Bernstein, Lynne E.
Titel	Psychophysics of the McGurk and Other Audiovisual Speech Integration Effects
Quelle	In: Journal of Experimental Psychology: Human Perception and Performance, 37 (2011) 4, S.1193-1209 (17 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0096-1523
DOI	10.1037/a0023100
Schlagwörter	Video Technology; Acoustics; Syllables; Auditory Stimuli; Visual Stimuli; Responses; Visual Perception; Auditory Perception; Speech Communication; Young Adults; Statistical Analysis; Factor Analysis; Correlation; Prediction; Experimental Psychology + Suchen Sie Ihr Suchwort? Akustik; Silbe; Auditive Stimulation; Visuelle Wahrnehmung; Auditive Wahrnehmung; Akustische Wahrnehmung; Young adult; Junger Erwachsener; Statistische Analyse; Faktorenanalyse; Korrelation; Vorhersage; Experimentelle Psychologie
Abstract	When the auditory and visual components of spoken audiovisual nonsense syllables are mismatched, perceivers produce four different types of perceptual responses, auditory correct, visual correct, fusion (the so-called "McGurk effect"), and combination (i.e., two consonants are reported). Here, quantitative measures were developed to account for the distribution of the four types of perceptual responses to 384 different stimuli from four talkers. The measures included mutual information, correlations, and acoustic measures, all representing audiovisual stimulus relationships. In Experiment 1, open-set perceptual responses were obtained for acoustic /b[script a]/ or /l[script a]/ dubbed to video /b[script a], d[script a], g[script a], v[script a], z[script a], l[script a], w[script a], [eth][script a]/. The talker, the video syllable, and the acoustic syllable significantly influenced the type of response. In Experiment 2, the best predictors of response category proportions were a subset of the physical stimulus measures, with the variance accounted for in the perceptual response category proportions between 17% and 52%. That audiovisual stimulus relationships can account for perceptual response distributions supports the possibility that internal representations are based on modality-specific stimulus relationships. (Contains 1 footnote, 6 tables, and 6 figures.) (As Provided).
Anmerkungen	American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002-4242. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: order@apa.org; Web site: http://www.apa.org/publications
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10