Searching for Worth: the impact of effort throughout reward processing

Abstract

What drives behavior? This fundamental inquiry has been the cornerstone of disciplines spanning from philosophy to biology. In neuroscience and economics, however, the prevailing viewpoint posits that reward plays a central role in motivating an organism’s behavior, such that one is willing to endure larger costs for more valuable rewards. Within this framework, effort naturally becomes an integral component of reward valuation since the subjective value of a reward can be observed through the level of effort one is willing to expend to obtain it.

Despite the inherent connection between effort and reward, the impact that effort costs have on reward value – and the neural architecture that underlies that valuation process – remains unclear. Extant research indicates that the processing of effort sometimes occurs in the same brain regions as other cost-based rewards, while at other times, it engages distinct prefrontal regions, suggesting that effort demands are treated separately from other reward-related costs. Additionally, substantial evidence suggests that effort requirements diminish the value of prospective rewards, as demonstrated by the consistent preference of animals and humans for pursuing less effortful rewards. Interestingly, this preference reverses once a reward is obtained such that more value is attributed to rewards earned through greater effort. This dissociation presents a unique opportunity to explore how effort and reward are integrated during reward receipt and incorporated into representations of future predicted rewards. Nevertheless, previous studies tend to focus on select phases of effort-based reward processing in isolation, obscuring a fuller understanding of this valuation process. This factor, coupled with variability in experimental paradigms and insufficient control for confounders like success probability, potentially contribute to inconsistencies in previous findings, necessitating new research focusing on how pure effort demand impacts reward processing.

In this thesis, we employed a diverse range of techniques to examine the influence of effort costs at different stages of reward processing. Firstly, we conducted a comprehensive meta-analysis, utilizing both coordinate-based and image-based approaches to identify brain regions consistently involved in signaling prospective effort demands and the net value of effort-based rewards. Our findings indicate that during the evaluation and selection of cues, the net value of effort-based rewards is represented by a system comprised of the ventromedial prefrontal cortex (vmPFC) and the ventral striatum, regions known to play a role in processing the subjective value of rewards across various domains, costs, and stages of processing. The meta-analysis also revealed a specific role for the pre-sensory motor area (pre-SMA), which tracked both effort costs and net value, albeit in a different manner than the vmPFC. These results provide robust evidence for the distinct contributions of the vmPFC and pre-SMA in valuing incentivized effort costs and highlight their central involvement in the network that drives motivated behavior.

While preparing the meta-analysis, we observed that many experimental designs included features that generated confounds that could have potentially contributed to conflicting findings from previous studies. Motivated by this observation, we aimed to design a simple experimental paradigm that would allow us to isolate pure effort and reward value signals by controlling critical confounding factors, measuring responses across different phases of reward processing, and assessing effort-based reward signals in the absence of option comparison. Surprisingly, across two experiments, we failed to observe any net value signal during cue evaluation at the pre-SMA/dACC (as putatively measured by frontal midline theta). Instead, effects in component P3 suggest that higher magnitude and less probable rewards were more salient than lower magnitude and more probable rewards, and that paid effort costs amplified this effect. Additionally, we observed enhanced theta power for successful high effort outcomes, although theta did not track reward magnitude. These findings suggest that the pre-SMA/dACC does not track prospective effort when success rates are equivalent between effort conditions and in the absence of option comparison. Finally, effort-related effects at feedback indicate that rewards earned through high effort are more salient, suggesting a general attentional bias toward high effort outcomes.

We next aimed to test whether this attentional bias could potentially modulate individuals learning speed. Specifically, we were interested in exploring how cognitive effort modulates the reinforcing value of received rewards. Our results indicated that participants successfully acquired cue-reward contingencies but that learning rates were unaffected by effort demands. Instead, we observed an effect of performance feedback, such that learning rates were higher when participants successfully completed the effort task than when efforts were unsuccessful.

Collectively, the findings presented in this thesis provide several crucial insights into the field of effort-based reward processing. Firstly, we provide compelling evidence that the net value of prospective effort-based rewards engages a system involving the vmPFC and pre-SMA, with supporting roles played by the ventral striatum, anterior cingulate cortex, and anterior insula. This suggests that effort is treated similarly to other costs within the framework of value-based decision-making. Furthermore, we demonstrate that when reward probabilities are fully dissociated from effort requirements, frontal midline theta primarily reflects expended effort during reward delivery and that effort enhances the salience of rewards during feedback, which suggests that more attention is allocated to rewards obtained through greater effort. Lastly, we investigated the impact of cognitive effort demands on reward learning and found that positive performance feedback, but not effort demands, facilitated the updating of cue-reward contingencies.

In conclusion, this thesis significantly advances our understanding of effort-based reward valuation by shedding light on the neural mechanisms involved at different stages of reward processing. This collection of work provides conclusive evidence for the roles of the vmPFC and pre-SMA in valuing prospective incentivized effort costs. The current findings also provide additional evidence in support of the idea that effort is weighed differently in early stages of reward processing (i.e., choice, cue evaluation, etc.) than in later stages (i.e. feedback, learning, etc.) and that performance feedback may be more relevant in certain contexts than sheer effort expenditure. Critically, these findings also underscore the need to reevaluate current methodologies in the field of neuroeconomics, as the failure to replicate certain established effects calls for a comprehensive reevaluation of existing practices.

To enter the videoconference click here (MS Teams, Dec 11, 11:00h)