Tools
Tools
Schwartenbeck, Philipp, FitzGerald, Thomas 
ORCID: https://orcid.org/0000-0002-3855-1591, Dolan, Raymond J. and Friston, Karl J.
(2013)
Exploration, novelty, surprise, and free energy minimization.
Frontiers in Psychology, 4.
ISSN 1664-1078
Abstract
This paper reviews recent developments under the free energy principle that introduce a normative perspective on classical economic (utilitarian) decision-making based on (active) Bayesian inference. It has been suggested that the free energy principle precludes novelty and complexity, because it assumes that biological systems-like ourselves-try to minimize the long-term average of surprise to maintain their homeostasis. However, recent formulations show that minimizing surprise leads naturally to concepts such as exploration and novelty bonuses. In this approach, agents infer a policy that minimizes surprise by minimizing the difference (or relative entropy) between likely and desired outcomes, which involves both pursuing the goal-state that has the highest expected utility (often termed exploitation) and visiting a number of different goal-states (exploration). Crucially, the opportunity to visit new states increases the value of the current state. Casting decision-making problems within a variational framework, therefore, predicts that our behavior is governed by both the entropy and expected utility of future states. This dissolves any dialectic between minimizing surprise and exploration or novelty seeking.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | active inference,exploration,exploitation,novelty,reinforcement learning,free energy,active inference,brain |
| Faculty \ School: | Faculty of Social Sciences > School of Psychology |
| Depositing User: | Pure Connector |
| Date Deposited: | 14 Apr 2016 15:01 |
| Last Modified: | 07 Mar 2024 01:57 |
| URI: | https://ueaeprints.uea.ac.uk/id/eprint/58249 |
| DOI: | 10.3389/fpsyg.2013.00710 |
Actions (login required)
 |
View Item |