Exploring What Is Encoded in Distributional Word Vectors: A Neurobiologically Motivated Analysis

Autor/in	Utsumi, Akira
Titel	Exploring What Is Encoded in Distributional Word Vectors: A Neurobiologically Motivated Analysis
Quelle	In: Cognitive Science, 44 (2020) 6, (33 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1551-6709
DOI	10.1111/cogs.12844
Schlagwörter	Cognitive Processes; Biology; Semantics; Neurological Organization; Artificial Intelligence; Prediction; Emotional Response; Social Influences; Language + Suchen Sie Ihr Suchwort? Cognitive process; Kognitiver Prozess; Biologie; Semantik; Künstliche Intelligenz; Vorhersage; Emotionales Verhalten; Sozialer Einfluss; Languages; Sprache
Abstract	The pervasive use of distributional semantic models or word embeddings for both cognitive modeling and practical application is because of their remarkable ability to represent the meanings of words. However, relatively little effort has been made to explore what types of information are encoded in distributional word vectors. Knowing the internal knowledge embedded in word vectors is important for cognitive modeling using distributional semantic models. Therefore, in this paper, we attempt to identify the knowledge encoded in word vectors by conducting a computational experiment using Binder et al.'s (2016) featural conceptual representations based on neurobiologically motivated attributes. In an experiment, these conceptual vectors are predicted from text-based word vectors using a neural network and linear transformation, and prediction performance is compared among various types of information. The analysis demonstrates that abstract information is generally predicted more accurately by word vectors than perceptual and spatiotemporal information, and specifically, the prediction accuracy of cognitive and social information is higher. Emotional information is also found to be successfully predicted for abstract words. These results indicate that language can be a major source of knowledge about abstract attributes, and they support the recent view that emphasizes the importance of language for abstract concepts. Furthermore, we show that word vectors can capture some types of perceptual and spatiotemporal information about concrete concepts and some relevant word categories. This suggests that language statistics can encode more perceptual knowledge than often expected. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01