Cross-Modal Interactions during Perception of Audiovisual Speech and Nonspeech Signals: An fMRI Study

Autor/inn/en	Hertrich, Ingo; Dietrich, Susanne; Ackermann, Hermann
Titel	Cross-Modal Interactions during Perception of Audiovisual Speech and Nonspeech Signals: An fMRI Study
Quelle	In: Journal of Cognitive Neuroscience, 23 (2011) 1, S.221-237 (17 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0898-929X
DOI	10.1162/jocn.2010.21421
Schlagwörter	Brain Hemisphere Functions; Diagnostic Tests; Speech Communication; Phonetics; Auditory Perception; Short Term Memory; Followup Studies; Cognitive Processes; Acoustics; Phonology; Auditory Stimuli; Visual Stimuli + Suchen Sie Ihr Suchwort? Diagnostic test; Diagnostischer Test; Phonetik; Fonetik; Auditive Wahrnehmung; Akustische Wahrnehmung; Akustik; Kurzzeitgedächtnis; Follow-up studies; Kontaktstudium; Cognitive process; Kognitiver Prozess; Fonologie; Auditive Stimulation
Abstract	During speech communication, visual information may interact with the auditory system at various processing stages. Most noteworthy, recent magnetoencephalography (MEG) data provided first evidence for early and preattentive phonetic/phonological encoding of the visual data stream--prior to its fusion with auditory phonological features [Hertrich, I., Mathiak, K., Lutzenberger, W., & Ackermann, H. Time course of early audiovisual interactions during speech and non-speech central-auditory processing: An MEG study. "Journal of Cognitive Neuroscience, 21," 259-274, 2009]. Using functional magnetic resonance imaging, the present follow-up study aims to further elucidate the topographic distribution of visual-phonological operations and audiovisual (AV) interactions during speech perception. Ambiguous acoustic syllables--disambiguated to /pa/ or /ta/ by the visual channel (speaking face)--served as test materials, concomitant with various control conditions (nonspeech AV signals, visual-only and acoustic-only speech, and nonspeech stimuli). (i) Visual speech yielded an AV-subadditive activation of primary auditory cortex and the anterior superior temporal gyrus (STG), whereas the posterior STG responded both to speech and nonspeech motion. (ii) The inferior frontal and the fusiform gyrus of the right hemisphere showed a strong phonetic/phonological impact (differential effects of visual /pa/ vs. /ta/) upon hemodynamic activation during presentation of speaking faces. Taken together with the previous MEG data, these results point at a dual-pathway model of visual speech information processing: On the one hand, access to the auditory system via the anterior supratemporal "what" path may give rise to direct activation of "auditory objects." On the other hand, visual speech information seems to be represented in a right-hemisphere visual working memory, providing a potential basis for later interactions with auditory information such as the McGurk effect. (As Provided).
Anmerkungen	MIT Press. 55 Hayward Street, Cambridge, MA 02142. Tel: 617-253-2889; Fax: 617-253-1709; e-mail: journals-orders@mit.edu; Web site: http://www.mitpressjournals.org/loi/jocn
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10