16 Competing Populations and Decision Making

16.4 Alternative Decision Models

In the previous two sections, we discussed decision models as the competitive interaction between two or more populations of excitatory neurons. In this section we present two different models of decision making, i.e., the energy picture in subsection 16.4.1 and the drift-diffusion model in subsection 16.4.2. Both models are phenomenological concepts to describe decision making. However, both models are also related to the phase diagram of the model of two neuronal populations, encountered in Section 16.3.

A B
Fig. 16.9: Energy picture of decision making. A. Decisions correspond to a ball rolling down an energy landscape, plotted as a function of a formal decision variable xx. A value of x=0x=0 indicates that no decision is taken (top, no input), whereas a value of x=±1x=\pm 1 reflects a decision for options A or B, respectively. An input favoring option A deforms and tilts the energy landscape so that the minimum of the energy is in the neighborhood of x=1x=1 (middle). A strong but unbiased input, creates two energy minima corresponding to options A and B. Only one of the two options can be taken by the rolling ball (bottom). B. The corresponding phase plane diagram of the input potentials hAh_{A} (horizontal axis) and hBh_{B} (vertical axis). Fixed points are indicated as filled (stable) or empty (saddle) circles. The decision variable xx moves along the axis (short-dashed gray line) perpendicular to the diagonal (long-dashed black line) and is replotted again as a one-dimensional flow on the right-hand side of the figure.

16.4.1 The energy picture

Binary decisions can be visualized as a ball in a hilly energy landscape. Once the ball rolls in a certain direction, a decision starts to form. The decision is finalized when the ball approaches the energy minimum; cf. Fig. 16.9A. The decision variable xx reflects a decision for option A, if it approaches a fixed point in the neighborhood of x1x\approx 1 and a decision for option B for x-1x\approx-1. If the variable xx is trapped in a minimum close to x=0x=0, no decision is taken.

The dynamics of the decision process can be formulated as gradient descent

dxdt=-ηdEdx{dx\over dt}=-\eta{dE\over dx} (16.11)

with a positive constant η\eta. In other words, in a short time step Δt\Delta t, the decision variable moves by an amount -ηΔtdE/dx-\eta\Delta t\,dE/dx. Thus, if the slope is positive, the movement is toward the left. As a result, the movement is always downhill, so that the energy decreases along the trajectory x(t)x(t). We can calculate the change of the energy along the trajectory

dE(x(t))dt=dEdxdxdt=-η(dEdx)20 .{dE(x(t))\over dt}={dE\over dx}\,{dx\over dt}=-\eta\left({dE\over dx}\right)^{% 2}\leq 0\,. (16.12)

Therefore the energy plays the role of a Liapunov function of the system, i.e., a quantity that cannot increase along the trajectory of a dynamical system .

Interestingly, the energy picture can be related to the phase plane analysis of the two-dimensional model that we encountered earlier in Figs. 16.6B and 16.7. The diagonal of the phase plane plays the role of the boundary between the options A and B while the variable xx indicates the projection onto an axis orthogonal to the diagonal. x=0x=0 is the unbiased, undecided position on the diagonal; cf. Fig. 16.9. In the case of strong unbiased input, the one-dimensional flow diagram of the variable xx presents a reasonable summary of the flow pattern in the two-dimensional system, because the saddle point in the phase plane is attractive along the diagonal and is reached rapidly while the flow in the perpendicular direction is much slower (543; 61; 558).

The above arguments regarding the Liapunov function of the network can be made more precise and formulated as general theorem (100; 227). We consider an arbitrary network of KK neuronal population 1jK1\leq j\leq K with population rate Aj=g(hj)0A_{j}=g(h_{j})\geq 0 where gg is a gain function with derivative g>0g^{\prime}>0 and hh follows the dynamics

τdhjdt=-hj+RIj+kwjkg(hk)\tau\,{{\text{d}}h_{j}\over{\text{d}}t}=-h_{j}+R\,I_{j}\,+\sum_{k}w_{jk}g(h_{k% })\, (16.13)

with fixed inputs IjI_{j}. If the coupling is symmetric, i.e., wij=wjiw_{ij}=w_{ji} then the energy

E=-12⁢ijwijAiAj-iAiRIi+i0Aig-1(a)daE=-\sum_{i}\sum_{j}w_{ij}A_{i}A_{j}-\sum_{i}A_{i}\,R\,I_{i}+\sum_{i}\int_{0}^{% A_{i}}g^{-1}(a)\,{\text{d}}a\, (16.14)

is a Liapunov function of the dynamics.

The proof follows by taking the derivative. We exploit the fact that wij=wjiw_{ij}=w_{ji} and apply the chain rule dAi/dt=g(hi)dhi/dtdA_{i}/dt=g^{\prime}(h_{i})dh_{i}/dt so as to find

dEdt\displaystyle{dE\over dt} =\displaystyle= -i[jwijAj]g(hi)dhidt-iRIig(hi)dhidt+ig-1(Ai)g(hi)dhidt\displaystyle-\sum_{i}\left[\sum_{j}w_{ij}A_{j}\right]\,g^{\prime}(h_{i}){dh_{% i}\over dt}-\sum_{i}R\,I_{i}\,g^{\prime}(h_{i}){dh_{i}\over dt}+\sum_{i}g^{-1}% (A_{i})\,g^{\prime}(h_{i}){dh_{i}\over dt} (16.15)
=\displaystyle= -τig(hi)[dhidt]20 .\displaystyle-\tau\sum_{i}g^{\prime}(h_{i})\left[{dh_{i}\over dt}\right]^{2}% \leq 0\,.

In the second line we have used Eq. (16.13). Furthermore, since the neuronal gain function stays below a biologically sustainable firing rate g(x)Amaxg(x)\leq A^{\rm max}, the energy is bounded from below. Therefore the flow of a symmetric network of interacting populations will always converge to one of the stable fixed points corresponding to an energy minimum, – unless the initial condition is chosen to lie on an unstable fixed point in which case the dynamics stays there until it is perturbed by some input.

Example: Binary decision network revisited

The binary decision network of Eqs. (16.7) and (16.8) with effective inhibition α\alpha and recurrent interactions w0w_{0} consists of two populations. Interactions are symmetric since w12=w21=-αw_{12}=w_{21}=-\alpha. Therefore the energy function

E=-w012⁢[A12+A22]+αA1A2-R[I1A1+I2A2]+0A1g-1(x)dx+0A2g-1(y)dyE=-w_{0}[A_{1}^{2}+A_{2}^{2}]+2\alpha A_{1}A_{2}-R\,[I_{1}A_{1}+I_{2}A_{2}]+% \int_{0}^{A_{1}}g^{-1}(x)dx+\int_{0}^{A_{2}}g^{-1}(y)dy (16.16)

is a Liapunov function of the dynamics defined in Eqs. (16.7) and (16.8). Since the dynamics is bounded, there must be stable fixed points.

16.4.2 Drift-Diffusion Model

Fig. 16.10: Drift-diffusion model. The decision variable x(t)x(t) starts from x0x_{0} and undergoes a biased random walk toward one of the two thresholds, marked by horizontal dashed lines. In the first trial (thin solid line), the choice is option AA and the reaction time indicated as tAt_{A}. In another trial (thick solid line) the trajectory hits, after a time tBt_{B} the threshold for option B.

The drift-diffusion model is a phenomenological model to describe choice preferences and distributions of reaction times in binary decision making tasks (422). At each trial of a decision experiment, a decision variable xx is initialized at time t0t_{0} at a value x(t0)=x0x(t_{0})=x_{0}. Thereafter, the decision variable evolves according to

dxdt=(IA-IB)+σξ(t){dx\over dt}=(I_{A}-I_{B})+\sigma\xi(t) (16.17)

where ξ(t)\xi(t) is Gaussian white noise of unit mean and variance σ2\sigma^{2}. An input IA>IBI_{A}>I_{B} causes a ‘drift’ of the variable xx toward positive values while the noise ξ\xi leads to a ‘diffusion’-like motion of the trajectory; hence the name ‘drift-diffusion model’.

The reaction time is the time at which the variable xx reaches one of two thresholds, ΘA\Theta_{A} or ΘB\Theta_{B}, respectively (Fig. 16.10). For example tBt_{B} defined by tB=min{t|x(t)=ΘB}t_{B}={\rm min}\left\{t|x(t)=\Theta_{B}\right\} is the reaction time in a trial where the choice falls on option BB.

Parameters of the phenomenological drift-diffusion model are the values of thresholds ΘA\Theta_{A} and ΘB\Theta_{B}, and the strength of the input IA-IBI_{A}-I_{B} compared to that of the noise. The initial condition x0x_{0} can be identical in all trials or chosen in each trial independently from a small interval that reflects uncontrolled variations in the bias of the subject. The time t0t_{0} is typically the moment when the subject receives the choice stimulus, but it is also possible to start the drift-diffusion process a few milliseconds later so as to account for the propagation delay from the sensors to the brain (422; 421).

Example: Drift-diffusion model versus neuronal models

In the original formulation, the drift-diffusion model was used as a ‘black box’, i.e., a phenomenological model with parameters that can be fitted to match the distribution of reaction times and choice preferences to behavioral experiments. Interestingly, however, variants of one-dimensional drift-diffusion models can be derived from the models of neural populations with competitive interaction that we have discussed in earlier sections of this chapter (61; 558; 447). The essential idea can be best explained in the energy picture; cf. Fig. 16.9. We assume a small amount of noise. In the absence of input, the decision variable jitters around the stable fixed point x=0x=0. Its momentary value x0x\approx 0 serves as an initial condition, once the input is switched on. Suppose the input is strong but unbiased. Two new valleys form around x±1x\approx\pm 1. However, in the neighborhood of x=0x=0 the landscape is flat, so that noise leads to a diffusive motion of the trajectory. A biased input IA>IBI_{A}>I_{B} tilts the energy landscape to the right which causes a corresponding drift term in the diffusion process. The location where the slope of the valley becomes steep can be associated with the threshold ΘA\Theta_{A} in the diffusion model.