The interval distribution which has already been introduced in Chapter 7 plays a central role in the formulation of the population equation. Before we formulate the evolution of the population activity in terms of this interval distribution , we specify some necessary assumptions. Two key assumptions are that the population is homogeneous and contains non-adaptive neurons so that we can work in the framework of a time-dependent renewal theory. These assumptions will eventually be relaxed in Section 14.1.4 where we extend the theory to the case of a population of adaptive neurons and in Section 14.6 where we treat populations of finite size.
In order to formulate a first version of an integral equation for a population of neurons, we start in the framework of time-dependent renewal theory (Chapter 7). To do so, we have to assume the state of a neuron at time to be completely described by (i) its last firing time ; (ii) the input it received for times ; (iii) the characteristics of potential noise sources, be it noise in the input, noise in the neuronal parameters, or noise in the output.
Are these assumptions overly restrictive or can we still place an interesting and rich set of neuron models in the class of models consistent with assumptions (i) - (iii) of time-dependent renewal theory?
The state of a single-variable integrate-and-fire neuron , for example one of the nonlinear integrate-and-fire models of Chapter 5, is completely characterized by its momentary membrane potential . After firing at a time , the membrane potential is reset, so that all information that arrived before the firing is ‘forgotten’. Integration continues with . Therefore knowledge of last firing time and the input current for is sufficient to predict the momentary state of neuron at time and therefore (for a deterministic model) its next firing time. If the integrate-and-fire model is furthermore subject to noise in the form of stochastic spike arrivals of known rate (but unknown spike arrival times), we will not be able to predict the neuron’s exact firing time. Instead we are interested in the probability density
that the next spike occurs around time given that the last spike was at time and the neuron was subject to an input for . In practice, it might be difficult to write down an exact analytical formula for for noise models corresponding to stochastic spike arrival or diffusive noise (cf. Chapter 8), but the quantity is still well defined. We call the generalized interval distribution (cf. Chapter 9).
For several other noise models, there exist explicit mathematical expressions for . Consider, for example, integrate-and-fire models with slow noise in the parameters, defined as follows. After each firing time, the value for the reset or the value of the membrane time constant is drawn from a predefined distribution. Between two resets, the membrane potential evolves deterministically. As a consequence, the distribution of the next firing time can be predicted from the distribution of parameters and the deterministic solution of the threshold crossing equations (183). Such a model of slow noise in the parameters can also be considered as an approximation to a heterogeneous population of neurons where different neurons in the population have slightly different, but fixed parameters.
Another tractable noise model is the ‘escape noise’ model which is also the basis for the Generalized Linear Models of spiking neurons already discussed in Chapter 9. The short-term memory approximation SRM of the Spike Response Model with escape noise (cf. Eq. (9.19) in Ch. 9) is a prime example of a model that fits into the framework of renewal theory specified by assumptions (i) - (iii).
The escape noise model can be used in combination with any linear or nonlinear integrate-and-fire model. For a suitable choice of the escape function it provides also an excellent approximation to diffusive noise in the input; cf. Section 9.4 in Ch. 9. An example of how to formulate a nonlinear integrate-and-fire model with escape noise is given below.
The major limitation of time-dependent renewal theory is that, for the moment, we need to exclude adaptation effects, because the momentary state of adaptive neurons not only on the last firing time (and the input in the past) but also on the firing times of earlier spikes. Section 14.1.4 shows that adaptation can be included by extending the renewal theory to a quasi-renewal theory. The treatment of adaptive neurons is important because most cortical neurons exhibit adaptation (Ch. 6).
The integral equation (182; 183; 552) for activity dynamics with time-dependent renewal theory states that the activity at time depends on the fraction of active neurons at earlier times times the probability of observing a spike at given a spike at :
Equation (14.5) is easy to understand. The kernel is the probability density that the next spike of a neuron, which is under the influence of an input , occurs at time given that its last spike was at . The number of neurons which have fired at is proportional to and the integral runs over all times in the past. The interval distribution depends on the total input (both the external input and the synaptic input from other neurons in the population). For an unconnected population, corresponds to the external drive. For connected neurons, is the sum of the external input and the recurrent input from other neurons in the population; the case of connected neurons will be further analyzed in Section 14.2.
We emphasize that on the left-hand side of Eq. (14.5) is the expected activity at time while on the right-hand side is the observed activity in the past. In the limit of the number of neurons in the population going to infinity, the fraction of neurons that actually fire in a short time is the same as its expected value . Therefore, Eq. (14.5) becomes exact in the limit of , so that we can use the same symbol for on both sides of the equation. Finite-size effects are discussed in Section 14.6.
We conclude this section with some final remarks on the form of Eq. (14.5). First, we observe that we can multiply the activity value on both sides of the equation with a constant and introduce a new variable . If solves the equation, then will solve it as well. Thus, Eq. (14.5) cannot predict the correct normalization of the activity . This reflects the fact that, instead of defining the activity by a spike count divided by the number of neurons, we could have chosen to work directly with the spike count per unit of time or any other normalization. The proper normalization consistent with our definition of the population activity is derived below in Section 14.1.3.
Second, even though Eq. (14.5) is linear in the variable , it is in fact a highly nonlinear equation in the drive, because the kernel depends nonlinearly on the input . However, numerical implementations of the integral-equations lead to rapid schemes that predict the population activity in response to changing input, even in the highly nonlinear regime when strong input transiently synchronizes neurons in the population (Fig. 14.1).
To derive Eq. (14.5), we recall that is the probability density that a neuron fires at time given its last spike at and an input for . Integration of the probability density over time gives the probability that a neuron which has fired at fires its next spike at some arbitrary time between and . Just as in Chapter 7, we can define a survival probability,
i.e., the probability that a neuron which has fired its last spike at “survives” without firing up to time .
We now return to the homogeneous population of neurons in the limit of and assume that the firing of different neurons at time is independent, given that we know the history (the input and last spike) of each neuron. The technical term for such a situation is ’conditional independence’. We consider the network state at time and label all neurons by their last firing time . The proportion of neurons at time which have fired their last spike between and (and have not fired since) is expected to be
For an interpretation of the integral on the right-hand side of Eq. (14.7), we recall that is the fraction of neurons that have fired in the interval . Of these a fraction are expected to survive from to without firing. Thus among the neurons that we observe at time the proportion of neurons that have fired their last spike between and is expected to be ; cf. Fig. 14.2.
Finally, we use the fact that the total number of neurons remains constant. All neurons have fired at some point in the past.88Neurons which have never fired before are assigned a formal firing time . Thus, if we extend the lower bound of the integral on the right-hand side of Eq. 14.7 to and the upper bound to , the left-hand side becomes equal to one,
because all neurons have fired their last spike in the interval . Since the number of neurons remains constant, the normalization of Eq. 14.8 must hold at arbitrary times . Eq. (14.8) is an implicit equation for the population activity and the starting point for the discussions in this and the following chapters.
We repeat an important remark concerning the normalization of the activity. Since Eq. (14.5) is defined as the derivative of Eq. (14.8), the integration constant on the left-hand side of Eq. (14.8) is lost. This is most easily seen for constant activity . In this case the variable can be eliminated on both sides of Eq. (14.5) so that it yields the trivial statement that the interval distribution is normalized to unity. Equation (14.5) is therefore invariant under a rescaling of the activity with any constant , as mentioned earlier. To get the correct normalization of the activity we have to return to Eq. (14.8).
The integral equations presented so far are restricted to non-adapting neurons, because we assumed that the knowledge of the input and of the last firing time was sufficient to characterize the state of a neuron. For adapting neurons, however, the whole spiking history can shape the interspike interval distribution. In this section we extend the integral equation (14.5) to the case of adapting neurons.
For an isolated adaptive neuron in the presence of noise, the probability density of firing around time will, in general, depend on its past firing times where denotes the most recent spike time. The central insight is that, in a population of neurons, we can approximate the past firing times by the population activity in the past. Indeed, the probability of one of the neurons having a past spike time in the interval is .
Just as in the time-dependent renewal theory, we will treat the most recent spike of each neuron explicitly. For all spikes with we approximate the actual spike train history of an individual by the average history as summarized in the population activity . Let be the probability of observing a spike at time given the last spike at time , the input-current and the activity history until time , then we can rewrite equation (14.5) in a format suitable for adaptive neurons
In Section 14.5 we explain the methods to describe populations of adaptive neurons in more detail. Appropriate numerical schemes (Section 14.1.5) make it possible to describe the response of a population of adaptive neurons to arbitrary time-dependent input; see Fig. 14.4.
Integral equations can be solved numerically. However, the type of equation we face in Eq. (14.5) or Eq. (14.12) can not be cast in the typical Volterra or Fredholm forms for which efficient numerical methods have been extensively studied (301; 31). In the following, we describe a method that can be used to solve Eq. (14.5), Eq. (14.12) or Eq. (14.8), but we take as an example the quasi-renewal equivalent of Eq. (14.8)
To derive a numerical algorithm, we proceed in three steps. First, truncation of the integral in Eq. (14.13) at some lower bound. Second, the discretization of this integral and, third, the discretization of the integral defining the survivor function. Here we will use the rectangle rule for the discretization of every integral. Note that adaptive quadratures or Monte-Carlo methods could present more efficient alternatives.
The first step is the truncation. The probability to survive a very long time is essentially zero. Let be a period of time such that the survivor is very small. Then, we can truncate the infinite integral in 14.13
Next, we proceed to discretize the integral on small bins of size . Let be the vector made of the fraction of neurons at with last spike within and . This definition means that the th element is . The element with is then the momentary population activity in the time step starting at time , , since . Therefore we arrive at
Finally, we discretize the integral defining the survival function to obtain as a function of . Because of , we find for sufficiently small time steps
Note that we work in a moving coordinate because the index always counts time steps backward starting from the present time . Therefore and refer to the same group of neurons, i.e., those that have fired their last spike around time . Eq. (14.16) indicates that the number of neurons who survive from up to decrease in each time step by an exponential factor. In Eq. (14.16), can be either the renewal conditional intensity of a non-adaptive neuron model (e.g., Eq. (14.3)) or the effective ’quasi-renewal’ intensity of adaptive neurons (see Section 14.5). Together, Eq. (14.15) and (14.16) can be used to solve iteratively.
© Cambridge University Press. This book is in copyright. No reproduction of any part of it may take place without the written permission of Cambridge University Press.