# 1.3 Integrate-And-Fire Models

We have seen in the previous section that, to a first and rough approximation, neuronal dynamics can be conceived as a summation process (sometimes also called ‘integration’ process) combined with a mechanism that triggers action potentials above some critical voltage. Indeed in experiments firing times are often defined as the moment when the membrane potential reaches some threshold value from below. In order to build a phenomenological model of neuronal dynamics, we describe the critical voltage for spike initiation by a formal threshold $\vartheta$. If the voltage $u_{i}(t)$ (that contains the summed effect of all inputs) reaches $\vartheta$ from below, we say that neuron $i$ fires a spike. The moment of threshold crossing defines the firing time $t_{i}^{(f)}$.

The model makes use of the fact that neuronal action potentials of a given neuron always have roughly the same form. If the shape of an action potential is always the same, then the shape cannot be used to transmit information: rather information is contained in the presence or absence of a spike. Therefore action potentials are reduced to ‘events’ that happen at a precise moment in time.

Neuron models where action potentials are described as events are called ’Integrate-and-Fire’ models. No attempt is made to describe the shape of an action potential. Integrate-and-fire models have two separate components that are both necessary to define their dynamics: first, an equation that describes the evolution of the membrane potential $u_{i}(t)$; and second, a mechanism to generate spikes.

In the following we introduce the simplest model in the class of integrate-and-fire models using the following two ingredients: (i) a linear differential equation to describe the evolution of the membrane potential; (ii) a threshold for spike firing. This model is called the ‘Leaky Integrate-and-Fire’ Model. Generalized integrate-and-fire models that will be discussed in Part II of the book can be seen as variations of this basic model.

# 1.3.1 Integration of Inputs

The variable $u_{i}$ describes the momentary value of the membrane potential of neuron $i$. In the absence of any input, the potential is at its resting value $u_{\rm rest}$. If an experimentalist injects a current $I(t)$ into the neuron, or if the neuron receives synaptic input from other neurons, the potential $u_{i}$ will be deflected from its resting value.

In order to arrive at an equation that links the momentary voltage $u_{i}(t)-u_{\rm rest}$ to the input current $I(t)$, we use elementary laws from the theory of electricity. A neuron is surrounded by a cell membrane, which is a rather good insulator. If a short current pulse $I(t)$ is injected into the neuron, the additional electrical charge $q=\int I(t^{\prime})dt^{\prime}$ has to go somewhere: it will charge the cell membrane (Fig. 1.6A). The cell membrane therefore acts like a capacitor of capacity $C$. Because the insulator is not perfect, the charge will, over time, slowly leak through the cell membrane. The cell membrane can therefore be characterized by a finite leak resistance $R$.

The basic electrical circuit representing a leaky integrate-and-fire model consists of a capacitor $C$ in parallel with a resistor $R$ driven by a current $I(t)$; see Fig. 1.6.

If the driving current $I(t)$ vanishes, the voltage across the capacitor is given by the battery voltage $u_{\rm rest}$. For a biological explanation of the battery we refer the reader to the next chapter. Here we have simply inserted the battery ‘by hand’ into the circuit so as to account for the resting potential of the cell (Fig. 1.6A).

In order to analyze the circuit, we use the law of current conservation and split the driving current into two components,

 $I(t)=I_{R}+I_{C}$ (1.3)

The first component is the resistive current $I_{R}$ which passes through the linear resistor $R$. It can be calculated from Ohm’s law as $I_{R}=u_{R}/R$ where $u_{R}=u-u_{\rm rest}$ is the voltage across the resistor. The second component $I_{C}$ charges the capacitor $C$. From the definition of the capacity as $C=q/u$ (where $q$ is the charge and $u$ the voltage), we find a capacitive current $I_{C}={\text{d}}q/{\text{d}}t=C\,{\text{d}}u/{\text{d}}t$. Thus

 $I(t)={u(t)-u_{\rm rest}\over R}+C\,{{\text{d}}u\over{\text{d}}t}\,.$ (1.4)

We multiply Eq. (1.4) by $R$ and introduce the time constant $\tau_{m}=R\,C$ of the ‘leaky integrator’. This yields the standard form

 $\tau_{m}\,{{\text{d}}u\over{\text{d}}t}=-[u(t)-u_{\rm rest}]+R\,I(t)\,.$ (1.5)

We refer to $u$ as the membrane potential and to $\tau_{m}$ as the membrane time constant of the neuron.

From the mathematical point of view, Eq. (1.5) is a linear differential equation. From the point of view of an electrical engineer, it is the equation of a leaky integrator or $RC$-circuit where resistor $R$ and capacitor $C$ are arranged in parallel. From the point of view of the neuroscientist, Eq. (1.5) is called the equation of a passive membrane.

What is the solution of Eq. (1.5)? We suppose that, for whatever reason, at time $t=0$ the membrane potential takes a value $u_{\rm rest}+\Delta u$. For $t>0$ the input vanishes $I(t)=0$. Intuitively we expect that, if we wait long enough, the membrane potential relaxes to its resting value $u_{\rm rest}$. Indeed, the solution of the differential equation with initial condition $u(t_{0})=u_{\rm rest}+\Delta u$ is

 $u(t)-u_{\rm rest}=\Delta u\,\exp\left(-{t-t_{0}\over\tau_{m}}\right)\quad{\rm for% ~{}}t>t_{0}\,.$ (1.6)

Thus, in the absence of input, the membrane potential decays exponentially to its resting value. The membrane time constant $\tau_{m}=RC$ is the characteristic time of the decay. For a typical neuron it is in the range of 10ms, and hence rather long compared to the duration of a spike which is on the order of 1ms.

The validity of the solution (1.6) can be checked by taking the derivative on both sides of the equation. Since it is the solution in the absence of input, it is sometimes called the ‘free’ solution.

# 1.3.2 Pulse Input

Before we continue with the definition of the integrate-and-fire model and its variants, let us study the dynamics of the passive membrane defined by Eq. (1.5) in a simple example. Suppose that the passive membrane is stimulated by a constant input current $I(t)=I_{0}$ which starts at $t=0$ and ends at time $t=\Delta$. For the sake of simplicity we assume that the membrane potential at time $t=0$ is at its resting value $u(0)=u_{\rm rest}$.

As a first step, let us calculate the time course of the membrane potential. The trajectory of the membrane potential can be found by integrating (1.5) with the initial condition $u(0)=u_{\rm rest}$. The solution for $0 is

 $u(t)=u_{\rm rest}+R\,I_{0}\left[1-\exp\left(-{t\over\tau_{m}}\right)\right]\,.$ (1.7)

If the input current never stopped, the membrane potential (1.7) would approach for $t\to\infty$ the asymptotic value $u(\infty)=u_{\rm rest}+R\,I_{0}$. We can understand this result by looking at the electrical diagram of the $RC$-circuit in Fig. 1.6. Once a steady state is reached, the charge on the capacitor no longer changes. All input current must then flow through the resistor. The steady-state voltage at the resistor is therefore $RI_{0}$ so that the total membrane voltage is $u_{\rm rest}+RI_{0}$.

Example: Short pulses and the Dirac $\delta$ function

For short pulses the steady state value is never reached. At the end of the pulse, the value of the membrane potential is given according to Eq. (1.7) by $u(\Delta)=u_{\rm rest}+R\,I_{0}\left[1-\exp\left(-{\Delta\over\tau_{m}}\right)% \right]\,.$ For pulse durations $\Delta\ll\tau_{m}$ (where $\ll$ means much smaller than) we can expand the exponential term into a Taylor series: $\exp(x)=1+x+x^{2}/2...\,$. To first order in $x=-{\Delta\over\tau_{m}}$ we find

 $u(\Delta)=u_{\rm rest}+R\,I_{0}{\Delta\over\tau_{m}}\quad{\rm for~{}}\Delta\ll% \tau_{m}.$ (1.8)

Thus, the voltage deflection depends linearly on the amplitude and the duration of the pulse (Fig. 1.7, thick line).

We now make the duration $\Delta$ of the pulse shorter and shorter while increasing the amplitude of the current pulse to a value $I_{0}=q/\Delta$, so that the integral $\int I(t){\text{d}}t=q$ remains constant. In other words, the total charge $q$ delivered by the current pulse is always the same. Interestingly, the voltage deflection at the end of the pulse calculated from Eq. (1.8) remains unaltered, however short we make the pulse. Indeed, from Eq. (1.8) we find $u(\Delta)-u_{\rm rest}=q\,R/\tau_{m}=q/C$ where we have used $\tau_{m}=RC$. Thus we can consider the limit of an infinitely short pulse

 $I(t)=q\,\delta(t)=\lim_{\Delta\to 0}{q\over\Delta}\quad{\rm for~{}}{0 (1.9)

$\delta(t)$ is called the Dirac $\delta$-function. It is defined by $\delta(x)=0$ for $x\neq 0$ and $\int_{-\infty}^{\infty}\delta(x){\text{d}}x=1$.

Obviously, the Dirac $\delta$-function is a mathematical abstraction since it is practically impossible to inject a current with an infinitely short and infinitely strong current pulse into a neuron. Whenever we encounter a $\delta$-function, we should remember that, as a stand-alone object, it looks strange, but becomes meaningful as soon as we integrate over it. Indeed the input current defined in Eq. (1.9) needs to be inserted into the differential equation (1.5) and integrated. The mathematical abstraction of the Dirac $\delta$ function suddenly makes a lot of sense, because the voltage change induced by a short current pulse is always the same, whenever the duration of the pulse $\Delta$ is much shorter than the time constant $\tau_{m}$. Thus, the exact duration of the pulse is irrelevant, as long as it is short enough.

With the help of the $\delta$-function, we no longer have to worry about the time course of the membrane potential during the application of the current pulse: the membrane potential simply jumps at time $t=0$ by an amount $q/C$. Thus, it is as if we added instantaneously a charge $q$ onto the capacitor of the $RC$ circuit.

What happens for times $t>\Delta$? The membrane potential evolves from its new initial value $u_{\rm rest}+q/C$ in the absence of any further input. Thus we can use the ‘free’ solution from Eq. (1.6) with $t_{0}=\Delta$ and $\Delta u=q/C$.

We can summarize the considerations of this subsection by the following statement. The solution of the linear differential equation with pulse input

 $\tau_{m}\,{{\text{d}}u\over{\text{d}}t}=-[u(t)-u_{\rm rest}]+R\,q\,\delta(t)\,.$ (1.10)

is $u(t)=u_{\rm rest}$ for $t\leq 0$ and given by

 $u(t)-u_{\rm rest}=q\,{R\over\tau_{m}}\,\exp\left(-{t\over\tau_{m}}\right)\quad% {\rm for~{}}t>0$ (1.11)

The right-hand side of the equation is called the impulse-response function or Green’s function of the linear differential equation.

# 1.3.3 The Threshold for Spike Firing

Throughout this book, the term ‘firing time’ refers to the moment when a given neuron emits an action potential $t^{(f)}$. The firing time $t^{(f)}$ in the leaky integrate-and-fire model is defined by a threshold criterion

 $t^{(f)}:\quad u(t^{(f)})=\vartheta\,.$ (1.12)

The form of the spike is not described explicitly. Rather, the firing time is noted and immediately after $t^{(f)}$ the potential is reset to a new value $u_{r}<\vartheta$,

 $\lim_{\delta\to 0;\delta>0}u(t^{(f)}+\delta)=u_{r}\,.$ (1.13)

For $t>t^{(f)}$ the dynamics is again given by (1.5) until the next threshold crossing occurs. The combination of leaky integration (1.5) and reset (1.13) defines the leaky integrate-and-fire model (495). The voltage trajectory of a leaky integrate-and-fire model driven by a constant current $I_{0}$ is shown in Fig. 1.9.

For the firing times of neuron $i$ we write $t_{i}^{(f)}$ where $f=1,2,\dots$ is the label of the spike. Formally, we may denote the spike train of a neuron $i$ as the sequence of firing times

 $S_{i}(t)=\sum_{f}\delta(t-t_{i}^{(f)})$ (1.14)

where $\delta(x)$ is the Dirac $\delta$ function introduced before, with $\delta(x)=0$ for $x\neq 0$ and $\int_{-\infty}^{\infty}\delta(x){\text{d}}x=1$. Spikes are thus reduced to points in time (Fig. 1.8). We remind the reader that the $\delta$-function is a mathematical object that needs to be inserted into an integral in order to give meaningful results.

# 1.3.4 Time-dependent Input (*)22Sections marked by an asterisk are mathematically more advanced and can be omitted during a first reading of the book.

We study a leaky integrate-and-fire model which is driven by an arbitrary time-dependent input current $I(t)$; cf. Fig. 1.9B. The firing threshold has a value $\vartheta$ and after firing the potential is reset to a value $u_{r}<\vartheta$.

In the absence of a threshold, the linear differential equation (1.5) has a solution

 $u(t)=u_{\rm rest}+{R\over\tau_{m}}\int_{0}^{\infty}\exp\left(-{s\over\tau_{m}}% \right)\,I(t-s)\,{\text{d}}s\,.$ (1.15)

where $I(t)$ is an arbitrary input current and $\tau_{m}=RC$ is the membrane time constant. We assume here that the input current is defined for a long time back into the past: $t\to-\infty$ so that we do not have to worry about the initial condition. A sinusoidal current $I(t)=I_{0}\sin(\omega\,t)$ or a step current pulse, $I(t)=I_{0}\Theta(t)$ where $\Theta$ denotes the Heaviside step function with $\Theta(t)=0$ for $t\leq 0$ and $\Theta(t)=1$ for $t>0$, are two examples of a time-dependent current, but the solution, Eq. (1.15), is also valid for every other time-dependent input current.

So far our leaky integrator does not have a threshold. What happens to the solution Eq. (1.15), if we add a threshold $\vartheta$? Each time the membrane potential hits the threshold, the variable $u$ is reset from $\vartheta$ to $u_{r}$. In the electrical circuit diagram, the reset of the potential corresponds to removing a charge $q_{r}=C\,(\vartheta-u_{r})$ from the capacitor (Fig. 1.6) or, equivalently, adding a negative charge $-q_{r}$ onto the capacitor. Therefore, the reset corresponds to a short current pulse $I_{r}=-q_{r}\,\delta(t-t^{(f)})$ at the moment of the firing $t^{(f)}$. Indeed, it is not unusual to say that a neuron ‘discharges’ instead of ‘fires’. Since the reset happens each time the neuron fires, the reset current is

 $I_{r}=-q_{r}\,\sum_{f}\delta(t-t^{(f)})=-C\,(\vartheta-u_{r})S(t)\,,$ (1.16)

where $S(t)$ denotes the spike train, defined in Eq. (1.14).

The short current pulse corresponding to the ‘discharging’ is treated mathematically just like any other time-dependent input current. The total current $I(t)+I_{r}(t)$, consisting of the stimulating current and the reset current, is inserted into the solution (1.15) to give the final result

 $u(t)=u_{\rm rest}+\sum_{f}(u_{r}-\vartheta)\,\exp\left(-{t-t^{(f)}\over\tau_{m% }}\right)+{R\over\tau_{m}}\int_{0}^{\infty}\exp\left(-{s\over\tau_{m}}\right)% \,I(t-s)\,{\text{d}}s\,,$ (1.17)

where the firing times $t^{(f)}$ are defined by the threshold condition

 $t^{(f)}=\left\{t|u(t)=\vartheta\right\}\,.$ (1.18)

Note that with our definition of the Dirac $\delta$-function in Eq. (1.9), the discharging reset follows immediately after the threshold crossing, so that the natural sequence of events – first firing, then reset – is respected.

Eq. (1.17) looks rather complicated. It has, however, a simple explanation. In Sect. 1.3.2 we have seen that a short input pulse at time $t^{\prime}$ causes at time $t$ a response of the membrane proportional to $\exp\left(-{t-t^{\prime}\over\tau_{m}}\right)$, sometimes called the impulse response function or Green’s function; cf. Eq. (1.11). The second term on the right-hand side of Eq. (1.17) is the effect of the discharging current pulses at the moment of the reset.

In order to interpret the last term on the right-hand side, we think of a stimulating current $I(t)$ as consisting of a rapid sequence of discrete and short current pulses. In discrete time, there would be a different current pulse in each time step. Because of the linearity of the differential equation, the effect of all these short current pulses can be added. When we return from discrete time to continuous time, the sum of the impulse response functions turns into the integral on the right-hand side of Eq. (1.17).

# 1.3.5 Linear Differential Equation vs. Linear Filter: Two Equivalent Pictures (*)

The leaky integrate-and-fire model is defined by the differential equation (1.5), i.e.,

 $\tau_{m}\,{{\text{d}}u\over{\text{d}}t}=-[u(t)-u_{\rm rest}]+R\,I(t)\,,$ (1.19)

combined with the reset condition

 $\lim_{\delta\to 0;\delta>0}u(t^{(f)}+\delta)=u_{r}\,,$ (1.20)

where $t^{(f)}$ are the firing times

 $t^{(f)}=\left\{t|u(t)=\vartheta\right\}\,.$ (1.21)

As we have seen in the previous subsection, the linear equation can be integrated and yields the solution (1.17). It is convenient to rewrite the solution in the form

 $u(t)=\int_{0}^{\infty}\eta(s)S(t-s)ds+\int_{0}^{\infty}\kappa(s)I(t-s)\,{\text% {d}}s\,.$ (1.22)

where we have introduced filters $\eta(s)=(u_{r}-\vartheta)\,\exp\left(-{s\over\tau_{m}}\right)$ and $\kappa(s)={1\over C}\exp\left(-{s\over\tau_{m}}\right)\,$. Interestingly, Eq. (1.22) is much more general than the leaky integrate-and-fire model, because the filters do not need to be exponentials but could have any arbitrary shape. The filter $\eta$ describes the reset of the membrane potential and, more generally, accounts for neuronal refractoriness. The filter $\kappa$ summarizes the linear electrical properties of the membrane. Eq. (1.22) in combination with the threshold condition (1.21) is the basis of the Spike Response Model and Generalized Linear Models, which will be discussed in Part II.

# 1.3.6 Periodic drive and Fourier transform (*)

Formally, the complex Fourier transform of a real-valued function $f(t)$ with argument $t$ on the real line is

 $\hat{f}(\omega)=\int_{-\infty}^{\infty}f(t)\,e^{-i\omega t}{\text{d}}t=|\hat{f% }(\omega)|\,e^{i\phi_{f}(\omega)}$ (1.23)

where $|\hat{f}(\omega)|$ and $\phi_{f}(\omega)$ are called amplitude and phase of the Fourier transform at frequency $\omega$. The mathematical condition for a well-defined Fourier transform is that the function $f$ be Lebesgue integrable with integral $\int_{-\infty}^{\infty}|f(t)|{\text{d}}t<\infty$. If $f$ is a function of time, then $\hat{f}(\omega)$ is a function of frequency. An inverse Fourier transform leads back from frequency-space to the original space, i.e., time.

For a linear system, the above definition gives rise to several convenient rules for Fourier-transformed equations. For example, let us consider the system

 $u(t)=\int_{-\infty}^{\infty}\kappa(s)\,I(t-s)\,{\text{d}}s\,,$ (1.24)

where $I(t)$ is a real-valued input (e.g., a current), $u(t)$ the real-valued system output (e.g., a voltage) and $\kappa$ a linear response filter, or kernel, with $\kappa(s)=0$ for $s<0$ because of causality. The convolution on the right-hand side of Eq. (1.24) turns after Fourier transformation into a simple multiplication, as shown by the following steps of calculation:

 $\displaystyle\hat{u}(\omega)$ $\displaystyle=$ $\displaystyle\int_{-\infty}^{\infty}\left[\int_{-\infty}^{\infty}\kappa(s)\,I(% t-s)\,{\text{d}}s\right]\,\,e^{-i\omega t}{\text{d}}t$ (1.25) $\displaystyle=$ $\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\kappa(s)e^{-i% \omega s}\,I(t-s)\,e^{-i\omega(t-s)}{\text{d}}s{\text{d}}t$ $\displaystyle=$ $\displaystyle\hat{\kappa}(\omega)\,\hat{I}(\omega)$

where we introduced in the last step the variable $t^{\prime}=t-s$ and used the definition (1.23) of the Fourier transform.

Similarly, the derivative $du/dt$ of a function $u(t)$ can be Fourier-transformed using the product rule of integration. The Fourier transform of the derivative of $u(t)$ is $i\omega\hat{u}(\omega)$.

While introduced here as a purely mathematical operation, it is often convenient to visualize the Fourier transform in the context of a physical system driven by a periodic input. Consider the linear system of Eq. (1.24) with an input

 $I(t)=I_{0}\,e^{i\omega t}\,.$ (1.26)

A short comment on the notation. If the input is a current, it should be real-valued, as opposed to a complex number. We therefore take $I_{0}$ as a real and positive number and focus on the real part of the complex equation (1.26) as our physical input. When we perform a calculation with complex numbers, we therefore implicitly assume that, at the very end, we only take the real part of solution. However, the calculation with complex numbers turns out to be convenient for the steps in between.

Inserting the periodic drive, Eq. (1.26), into Eq. (1.24) yields

 $\displaystyle u(t)$ $\displaystyle=$ $\displaystyle\int_{-\infty}^{\infty}\kappa(s)\,I_{0}e^{i\omega(t-s)}\,{\text{d% }}s=\left[\int_{-\infty}^{\infty}\kappa(s)\,e^{-i\omega s}\,{\text{d}}s\right]% \,I_{0}e^{i\omega t}\,.$ (1.27)

Hence, if the input is periodic at frequency $\omega$ the output is so, too. The term in square brackets is the Fourier transform of the linear filter. We write $u(t)=u_{0}\,e^{i\phi_{\kappa}(\omega)}\,e^{i\omega t}$. The ratio between the amplitude of the output and that of the input is

 ${u_{0}\over I_{0}}=|\hat{\kappa}(\omega)|\,.$ (1.28)

The phase $\phi_{\kappa}(\omega)$ of the Fourier-transformed linear filter $\kappa$ corresponds to phase shift between input and output or, to say it differently, a delay $\Delta=\phi_{\kappa}/\omega=\phi_{\kappa}\,T/2\pi$ where $T$ is the period of the oscillation. Fourier transforms will play a role in the discussion of signal processing properties of connected networks of neurons in Part III of the book.

Example: Periodic drive of a passive membrane

We consider the differential equation of the passive membrane defined in Eq. (1.5) and choose voltage units such that $u_{\rm rest}=0$, i.e.,

 $\tau_{m}\,{{\text{d}}u\over{\text{d}}t}=-u(t)+R\,I(t)\,.$ (1.29)

The solution, given by Eq. (1.15), corresponds to the convolution of the input $I(t)$ with a causal linear filter $\kappa(s)=(1/C)\,e^{(-s/\tau_{m})}$ for $s>0$. In order to determine the response amplitude $u_{0}$ to a periodic drive $I(t)=I_{0}\,e^{i\omega t}$ we need to calculate the Fourier transform of $\kappa$:

 $\displaystyle|\hat{\kappa}(\omega)|$ $\displaystyle=$ $\displaystyle\left|{1\over C}\int_{0}^{\infty}e^{-t\over\tau_{m}}\,e^{-i\omega t% }\,{\text{d}}t\right|={1\over C}\left|{\tau_{m}\over 1+i\omega\tau_{m}}\right|\,.$ (1.30)

For $\omega\tau_{m}\gg 1$ the right-hand side is proportional to $\omega^{-1}$. Therefore the amplitude of the response to a periodic input decreases at high frequencies.