# 16.6 Summary

Decisions are prepared and made in the brain so that numerous physiological correlates of decision making can be found in the human and monkey cortex. The fields of cognitive neuroscience associated with these questions are called ‘neuroeconomics’ and ‘neuroscience of volition’.

An influential computational model describes decision making as the competition of several populations of excitatory neurons which share a common pool of inhibitory neurons. Under suitable conditions, the explicit model of inhibitory neurons can be replaced by an effective inhibitory coupling between excitatory populations. In a rate model, the competitive interactions between two excitatory populations can be understood using phase plane analysis. Equivalently, the decision process can be described as downward motion in an energy landscape which plays the role of a Liapunov function. The energy picture is valid for any rate model where all units of the network are coupled by symmetric interactions.

The drift-diffusion model which has been used in the past as a black-box model for reaction time distributions and choice preferences can, under appropriate assumptions, be related to a rate model of competitively interacting populations of neurons.

# Literature

There are several accessible introductions to the problem of decision making in neuroeconomics (402; 418; 189). The neurophysiological correlates of decision making are reviewed in Gold and Shadlen (191); Romo and Salinas (444); Deco et al. (120, 121).

The competitive model of decision making that we presented in this chapter is discussed in Wang et al. (543) and Wong and Wang (558), but competitive interaction through inhibition is a much older topic in the field of computational neuroscience and artificial neural networks (198; 271; 215; 209). Competitive models of spiking neurons with shared inhibition have also been applied to other tasks of perceptual decision making, e.g., (317).

The energy as a Liapunov function for rate-models of neurons has been introduced by Cohen and Grossberg (100). In the context of associative memories (to be discussed in the next chapter) energy functions have been used for binary neuron models by Hopfield (226) and for rate models by Hopfield (227).

Drift-diffusion models have been reviewed by Ratcliff and Rouder (422); Ratcliff and McKoon (421). The relation of drift-diffusion models to neuronal decision models has been discussed by Bogacz et al. (61) and Wong and Wang (558) and has been worked out in the general case by Roxin and Ledberg (447).

A highly recommendable overview of the neuroscience around the questions of volition is Haggard (203) who reviews both the original Libet-experiment (297) and its modern variants. The fMRI study of Soon et al. (490) is also accessible to the non-specialized reader.

# Exercises

1. 1.

Phase plane analysis of a binary decision process . Consider the following system (in unit-free variables)

 $\displaystyle{{\text{d}}h_{E,1}\over{\text{d}}t}$ $\displaystyle=$ $\displaystyle-h_{E,1}+(w_{EE}-\alpha)\,g_{E}(h_{E,1})-\alpha\,g_{E}(h_{E,2})+h% ^{\rm ext}_{1}$ (16.18) $\displaystyle{{\text{d}}h_{E,2}\over{\text{d}}t}$ $\displaystyle=$ $\displaystyle-h_{E,2}+(w_{EE}-\alpha)\,g_{E}(h_{E,2})-\alpha\,g_{E}(h_{E,1})+h% ^{\rm ext}_{2}$ (16.19)

where $\alpha=1$ and $w_{EE}=1.5$ . The function $g(h)$ is piecewise linear: $g(h)=0$ for $h<-0.2$ ; $g(h)=0.1+0.5h$ for $-0.2\leq h\leq 0.2$ ; $g(h)=h$ for $0.2 $g(h)=0.4+0.5h$ for $0.8\leq h\leq 1.2$ ; and $g(h)=1$ for $h>1.2$ .

(i) Draw the two nullclines ( $dh_{1}/dt=0$ and $dh_{2}/dt=0$ ) in the phase plane with horizontal axis $h_{1}$ and vertical axis $h_{2}$ for the case $h^{\rm ext}_{1}=h^{\rm ext}_{2}=0.8$ .

(ii) Add flow arrows on the nullclines.

(iii) Set $h^{\rm ext}_{1}=h^{\rm ext}_{2}=b$ and study the fixed point on the diagonal $h_{1}=h_{2}=h^{*}$ . Find an expression for $h^{*}(b)$ under the assumption that the fixed point is in the region where $g(h)=h$ . Analyze the stability of this fixed point.

(iv) We now drop the assumption that the fixed point is in the region where $g(h)=h$ . Consider an arbitrary sufficiently smooth function $g(h)$ as well as arbitrary couplings $\alpha$ and $w_{EE}$ . and give a formula for the fixed point on the diagonal

(v) Assume now that $\alpha=0.75$ and $w_{EE}=1.5$ . Linearize about the fixed point in (iv) and calculate the two eigenvalues.

Hint: Introduce a parameter $\beta=0.75g^{\prime}(h^{*})$ .

(vi) Show that the fixed point is stable for $g^{\prime}(h^{*})=0$ and unstable for $g^{\prime}(h^{*})=1$ . At which value of $\beta$ does it change stability?

(vii) Describe in words your findings. What happens with a weak or a strong unbiased input to the decision model?

2. 2.

Winner-take-all in artificial neural networks .

Consider a network of formal neurons described by activities $A_{k}=(h_{k}-1)$ for $1\leq h_{k}\leq 2$ , $A_{k}=0$ for $h_{k}<1$ and $A_{k}=1$ for $h_{k}>2$ . We write $A_{k}=g_{E}(h_{k})$ .

The update happens in discrete time according to

 $h_{k}(t+\Delta t)=w_{0}\,g(h_{k}(t))-\alpha\,\sum_{j\neq k}g_{E}(h_{j}(t))+h_{% k}^{\rm ext}(t)\,.$ (16.20)

The external input vanishes for $t\leq 0$ . For $t>0$ the input to unit $k$ is $h_{k}^{\rm ext}=(0.5)^{k}+1.0$ .

(i) Set $w_{0}=2$ and $\alpha=1$ . Follow the evolution of the activities for three time steps.

(ii) What happens if you change $\alpha$ ? What happens if you keep $\alpha=1$ but decrease $w_{0}$ ?

(iii) Derive sufficient conditions so that the only fixed point is $A_{k}=\delta_{k,1}$ , i.e. only the unit with the strongest input is active. Assume that the maximal external input to the maximally excited neuron is $h_{k}^{\rm ext}\leq 2$ .

3. 3.

Energy picture . Consider the energy function

 $E(x)=[1-(I_{A}+I_{B})]\,x^{2}+{1\over 4}x^{4}+(I_{A}-I_{B})\,x$ (16.21)

where $I_{A}$ and $I_{B}$ are inputs in support of option A and B, respectively.

(i) Draw qualitatively the energy landscape in the absence of input, $I_{A}=I_{B}=0$ .

(ii) Draw qualitatively the energy landscape for $I_{B}=0$ while $I_{A}$ takes one of the three values $\{0.5,1.0,1.5\}$ .

(iii) Draw the energy landscape for $I_{A}=I_{B}=c$ while $c$ varies in the range $[0.5,1.5]$ .

(iv) Determine the flow $\Delta x=-\Delta t\,\eta\,dE/dx$ for a small positive parameter $\eta$ for all the relevant cases from (i) - (iii).

(v) Compare your results with Fig. 16.9 .