Active Inference, Epistemic Value, and Vicarious Trial and Error

Autor/inn/en	Pezzulo, Giovanni; Cartoni, Emilio; Rigoli, Francesco; io-Lopez, Léo; Friston, Karl
Titel	Active Inference, Epistemic Value, and Vicarious Trial and Error
Quelle	In: Learning & Memory, 23 (2016) 7, S.322-338 (17 Seiten) PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1072-0502
DOI	10.1101/lm.041780.116
Schlagwörter	Interference (Learning); Epistemology; Physiology; Neurology; Brain Hemisphere Functions; Simulation; Correlation; Decision Making; Habituation; Behavior Patterns; Role; Memory; Bayesian Statistics; Animals; Rewards; Task Analysis; Models + Suchen Sie Ihr Suchwort? Erkenntnistheorie; Physiologie; Neurologie; Simulation program; Simulationsprogramm; Korrelation; Decision-making; Entscheidungsfindung; Rollen; Gedächtnis; Animal; Tier; Tiere; Reward; Belohnung; Aufgabenanalyse; Analogiemodell
Abstract	Balancing habitual and deliberate forms of choice entails a comparison of their respective merits--the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are "forward sweeps" of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)--the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice. (As Provided).
Anmerkungen	Cold Spring Harbor Laboratory Press. 500 Sunnyside Boulevard, Woodbury, NY 11797-2924. Tel: 800-843-4388; Tel: 516-367-8800; Fax: 516-422-4097; e-mail: cshpres@cshl.edu; Web site: http://learnmem.cshlp.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01