Do Large Language Models Know What Humans Know?

Autor/inn/en	Trott, Sean; Jones, Cameron; Chang, Tyler; Michaelov, James; Bergen, Benjamin
Titel	Do Large Language Models Know What Humans Know?
Quelle	In: Cognitive Science, 47 (2023) 7, (21 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0364-0213
DOI	10.1111/cogs.13309
Schlagwörter	Models; Language Processing; Beliefs; Child Development; Language Acquisition; Linguistic Theory; Task Analysis; Artificial Intelligence; Comparative Analysis; Computational Linguistics; Learning Processes; Interpersonal Relationship + Suchen Sie Ihr Suchwort? Analogiemodell; Sprachverarbeitung; Belief; Glaube; Kindesentwicklung; Sprachaneignung; Spracherwerb; Linguistische Theorie; Aufgabenanalyse; Künstliche Intelligenz; Linguistics; Computerlinguistik; Learning process; Lernprozess; Interpersonal relation; Interpersonal relations; Interpersonelle Beziehung; Zwischenmenschliche Beziehung
Abstract	Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre-registered analyses, we present a linguistic version of the False Belief Task to both human participants and a large language model, GPT-3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans nor does it explain the full extent of their behavior--despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01