The simplest coupling scheme is all-to-all connectivity within a population. All connections have the same strength. If we want to change the number $N$ of neurons in the simulation of a population, an appropriate scaling law is

This scaling law is a mathematical abstraction that enables us to formally take the limit of $N\to\infty$ while keeping the expected input that a neuron receives from its partners in the population fixed. In the limit of $N\to\infty$ limit, the fluctuations disappear and the expected input can be considered as the actual input to any of the $N$ neurons. Of course, real populations are of finite size, so that some fluctuations always remain. But as $N$ increases the fluctuations decrease.

A slightly more intricate all-to-all coupling scheme is the following: weights $w_{ij}$ are drawn from a Gaussian distribution with mean $J_{0}/N$ and standard deviation $\sigma_{0}/\sqrt{N}$. In this case, each neuron in the population sees a somewhat different input, so that fluctuations of the membrane potential are of the order $\sigma_{0}$ even in the limit of large $N$ (146).

Experimentally the probability $p$ that a neuron inside a cortical column makes a functional connection to another neuron in the same column is in the range of 10 percent, but varies across layers; cf. Fig. 12.5.

In simulations, we can fix a connection probability $p$ and choose connections randomly with probability $p$ among all the possible $N^{2}$ connections. In this case, the number of presynaptic input links $C_{j}$ to a postsynaptic neuron $j$ has a mean value of $\langle C_{j}\rangle=pN$, but fluctuates between one neuron and the next with variance $p(1-p)N$.

Alternatively, we can take one model neuron $j=1,2,3,\dots N$ after the other and choose randomly $C=pN$ presynaptic partners for it (each neuron can be picked only once as a presynaptic partner for a given postsynaptic neuron $j$). In this case all neurons have, by construction, the same number of input links $C_{j}=C$. By an analogous selection scheme, we could also fix the number of output links to exactly $p\,N$ as opposed of simply imposing $p\,N$ as the average value.

Whatever the exact scheme to construct random connectivity, the number of input connections per neuron increases linearly with the size $N$ of the population; see Fig. 12.6B. It is therefore useful to scale the strength of the connections as

so that the mean input to typical neuron does not change if the number of model neurons in the simulated population is increased. Since in a bigger network individual inputs have less effects with the scaling according to Eq. (12.7), the amount of fluctuations in the input decreases with population size; compare Fig. 12.7.

The number of synapses onto the dendrites of a single pyramidal neuron is estimated to lie in the range of a few thousand (249). Thus, when one simulates networks of a hundred thousand neurons or millions of neurons, a modeling approach based on a fixed connection probability in the range of 10 percent cannot be correct. Moreover, in an animal participating in an experiment, not all neurons will be active at the same time. Rather only a few subgroups will be active, the composition of which depends on the stimulation conditions and the task. In other words, the number of inputs converging onto a single neuron may be much smaller than a thousand.

We can construct a random network with a fixed number $C$ of inputs by the following procedure. We pick one model neuron $j=1,2,3,\dots N$ after the other and choose randomly its $C$ presynaptic partners; see Fig. 12.6C. Whenever the network size $N$ is much bigger than $C$, the inputs to a given neuron can be thought of as random samples from the current network activity. No scaling of the connections with the population size $N$ is necessary; see Fig. 12.8.

In the simulations of Fig. 12.2, we have assumed a network of excitatory and inhibitory neurons. In other words, our network consists of two interacting populations. The combination of one excitatory and one inhibitory population can be exploited for the scaling of synaptic weights, if the effects of excitation and inhibition are ’balanced’.

In the discussion of scaling in the previous paragraphs, it was mentioned that a fixed connection probability of $p$ and a scaling of weights $w_{ij}=J_{0}/(p\,N)$ leads to a mean neuronal input current that is insensitive to the size of the simulation; however, fluctuations decrease with increasing $N$. Is there a possibility to work with a fixed connection probability $p$ and yet control the size of the fluctuations while changing $N$?

In a network of two populations, one excitatory and one inhibitory, it is possible to adjust parameters such that the mean input current into a typical neuron vanishes. The condition is that the total amount of excitation and inhibition cancel each other, so that excitation and inhibition are ‘balanced’. The resulting network is called a balanced network or a population with balanced excitation and inhibition (532; 539).

If the network has balanced excitation and inhibition the mean input current to a typical neuron is automatically zero and we do not have to apply a weight rescaling scheme to control the mean. Instead, we can scale synaptic weights so as to control specifically the amount of fluctuations of the input current around zero. An appropriate choice is

With this choice, a change in the size of the network hardly affects the mean and variance of the input current into a typical neuron; cf. Fig. 12.9. Note that in simulations of networks of integrate-and-fire neurons, the mean input current to the model neurons is in practice often controlled, and adjusted, through a common constant input to all neurons. In Figs. 12.7 - 12.9 we simply report the main effects of network size on the population activity and synaptic currents; the analysis of the observed phenomena will be postponed to Section 12.4.

In the previous paragraph we already considered two populations, one of them excitatory and the other one inhibitory. Let us generalize the arguments to a network consisting of several populations; cf. Fig. 12.10. It is convenient to visualize the neurons as being arranged in spatially separate pools, but this is not necessary. All neurons could, for example, be physically localized in the same column of the visual cortex, but of three different types: excitatory, fastspiking inhibitory, and non-fastspiking interneurons.

We assume that neurons are homogeneous within each pool. The activity of neurons in pool $n$ is

where $N_{n}$ is the number of neurons in pool $n$ and $\Gamma_{n}$ denotes the set of neurons that belong to pool $n$. We assume that each neuron $i$ in pool $n$ receives input from all neurons $j$ in pool $m$ with strength $w_{ij}=J_{nm}/N_{m}$; cf. Fig. 12.10. The time course $\alpha_{ij}(s)$ caused by a spike of a presynaptic neuron $j$ may depend on the synapse type, i.e., a connection from a neuron in pool $m$ to a neuron in pool $n$, but not on the identity of the two neurons. The input current to a neuron $i$ in group $\Gamma_{n}$ is generated by the spikes of all neurons in the network,

where $\alpha_{nm}(t-t_{j}^{(f)})$ denotes the time course of a postsynaptic current caused by spike firing at time $t_{j}^{(f)}$ of the presynaptic neuron $j$ which is part of population $m$. We use Eq. (12.9) to replace the sum on the right-hand side of Eq. (12.10) and obtain

We have dropped the index $i$ since the input current is the same for all neurons in pool $n$.

Thus, we conclude that the interaction between neurons of different pools can be summarized by the population activity $A(t)$ of the respective pools. Note that Eq. (12.11) is a straightforward generalization of Eq. (12.4) and could have been ‘guessed’ immediately; external input $I^{\rm ext}$ could be added as in Eq. (12.4).

The cortex is a rather thin sheet of cells. Cortical columns extend vertically across the sheet. As we have seen before, the connection probability within a column depends on the layer where pre- and postsynaptic neurons are located. In addition to this vertical connectivity, neurons make many horizontal connections to neurons in other cortical columns in the same, but also in other areas of the brain. Within the same brain area the probability of making a connection is often modeled as distance dependent. Note that distance dependence is a rather coarse feature, because the actual connectivity depends also on the function of the pre- and postsynaptic cell. In the primary visual area, for example, it has been found that pyramidal neurons with a preferred orientation for horizontal bars are more likely to make connections to other columns with a similar preferred orientation (27).

For models of distance-dependent connectivity it is necessary to assign to each model neuron $i$ a location $x(i)$ on the two-dimensional cortical sheet. Two different algorithmic procedures can be used to assign distance-dependent connectivity (Fig. 12.11). The first one assumes full connectivity with a strength $w_{ij}$ which falls off with distance

where $x(i)$ and $x(j)$ denote the location of post- and presynaptic neurons, respectively, and $w$ is a function into the real numbers. For convenience, one may assume finite support so that $w$ vanishes for distances $|x(i)-x(j)|>d$.

The second alternative is to give all connections the same weight, but to assume that the probability $P$ of forming a connection depends on the distance

where $x(i)$ and $x(j)$ denote the location of post- and presynaptic neurons, respectively and $P$ is a function into the interval $[0,1]$.

The physical location of a neuron in a population often reflects the task of a neuron. In the auditory system, for example, neurons are organized along an axis that reflects the neurons’ preferred frequency. A neuron at one end of the axis will respond maximally to low-frequency tones; a neuron at the other end to high frequencies. As we move along the axis the preferred frequency changes gradually. In the visual cortex, the preferred orientation changes gradually as one moves along the cortical sheet. For neurons organized in a spatially extended multidimensional network, a description by discrete pools does not seem appropriate. We will indicate in this section that a transition from discrete pools to a continuous population is possible. Here we give a short heuristic motivation of the equations. A thorough derivation along a slightly different line of arguments will be performed in Chapter 18.

To keep the notation simple, we consider a population of neurons that extends along a one-dimensional axis; cf. Fig. 12.12. We assume that the interaction between a pair of neurons $i,j$ depends only on their location $x$ or $y$ on the line. If the location of the presynaptic neuron is $y$ and that of the postsynaptic neuron is $x$, then $w_{ij}=w(x,y)$. In other words we assume full, but spatially dependent connectivity and neglect potential random components in the connectivity pattern.

In order to use the notion of population activity as defined in Eq. (12.11), we start by discretizing space in segments of size $d$. The number of neurons in the interval $[n\,d,(n+1)\,d]$ is $N_{n}=\rho\,d$ where $\rho$ is the spatial density. Neurons in that interval form the group $\Gamma_{n}$.

We now change our notation with respect to the discrete population and replace the subscript $m$ in the population activity $A_{m}$ by the spatial position of the neurons in that group

Since the efficacy of a pair of neurons with $i\in\Gamma_{n}$ and $j\in\Gamma_{m}$ is by definition $w_{ij}=J_{nm}/N_{m}$ with $N_{m}=\rho\,d$, we have $J_{nm}=\rho\,d\,w(nd,md)$. We use this in Eq. (12.11) and find for the input current

where $\alpha(s)$ describes the time course of the postsynaptic current caused by spike firing in one of the presynaptic neurons. For $d\to 0$, the summation on the right-hand side can be replaced by an integral and we arrive at

which is the final result. To rephrase Eq. (12.16) in words, the input to neurons at location $x$ depends on the spatial distribution of the population activity convolved with the spatial coupling filter $w(x,y)$ and the temporal filter $\alpha(s)$. The population activity $A(y,t-s)\Delta s$ is the number of spikes in a short interval $\Delta s$ summed across neurons in the neighborhood around $y$ normalized by the number of neurons in that neighborhood.