9 Noisy Output: Escape Rate and Soft Threshold

9.2 Likelihood of a spike train

In the previous subsection, we calculated the probability of firing in a short time step Δt\Delta t from the continuous-time firing intensity ρ(t)\rho(t). Here we ask a similar question, but not on the level of a single spike, but on that of a full spike train.

Suppose we know that a spike train has been generated by an escape noise process

ρ(t)=f(u(t)-ϑ)\rho(t)=f(u(t)-\vartheta) (9.9)

as in Eq. (9.2), where the membrane potential u(t)u(t) arises from the dynamics of one of the generalized integrate-and-fire models such as the SRM.

The likelihood LnL^{n} that spikes occur at the times t(1),,t(f),,t(n)t^{(1)},\dots,t^{(f)},\dots,t^{(n)} is (69)

Ln({t(1),t(2),,t(n)})=ρ(t(1))ρ(t(2))ρ(t(n))exp[-0Tρ(s)ds],\displaystyle L^{n}(\{t^{(1)},t^{(2)},\dots,t^{(n)}\})=\rho(t^{(1)})\cdot\rho(% t^{(2)})\cdot\,\dots\rho(t^{(n)})\,\exp[-\int_{0}^{T}\rho(s){\text{d}}s]\,, (9.10)

where [0,T][0,T] is the observation interval. The product on the right-hand side contains the momentary firing intensity ρ(t(f))\rho(t^{(f)}) at the firing times t(1),t(2),,t(n)t^{(1)},t^{(2)},\dots,t^{(n)}. The exponential factor takes into account that the neuron needs to ‘survive’ without firing in the intervals between the spikes.

In order to highlight this interpretation it is convenient to take a look at Fig. 9.6A and rewrite Eq. (9.10) in the equivalent form

Ln({t(1),t(2),,t(n)})=\displaystyle L^{n}(\{t^{(1)},t^{(2)},\dots,t^{(n)}\})= exp[-0t(1)ρ(s)ds]\displaystyle\exp[-\int_{0}^{t^{(1)}}\rho(s){\text{d}}s]\cdot (9.11)
ρ(t(1))\displaystyle\rho(t^{(1)}) exp[-t(1)t(2)ρ(s)ds]\displaystyle\exp[-\int_{t^{(1)}}^{t^{(2)}}\rho(s){\text{d}}s]\cdot
ρ(t(2))\displaystyle\rho(t^{(2)}) exp[-t(2)t(3)ρ(s)ds]\displaystyle\exp[-\int_{t^{(2)}}^{t^{(3)}}\rho(s){\text{d}}s]\cdot\,\dots
ρ(t(n))\displaystyle\rho(t^{(n)}) exp[-t(n)Tρ(s)ds].\displaystyle\exp[-\int_{t^{(n)}}^{T}\rho(s){\text{d}}s]\,.

Intuitively speaking, the likelihood of finding nn spikes at times t(f)t^{(f)} depends on the instantaneous rate at the time of the spikes and the probability to survive the intervals in between without firing; cf. the survivor function introduced in Chapter 7, Eq. (7.26).

Instead of the likelihood, it is sometimes more convenient to work with the logarithm of the likelihood, called the log-likelihood

logLn({t(1),,ti(f),,t(n)})=-0Tρ(s)ds+f=1nlogρ(ti(f)).\displaystyle{\rm log}L^{n}(\{t^{(1)},\dots,t_{i}^{(f)},\dots,t^{(n)}\})=-\int% _{0}^{T}\rho(s){\text{d}}s+\sum_{f=1}^{n}{\rm log}\rho(t_{i}^{(f)}). (9.12)

The three formulations Eqs. (9.10) – (9.12) are equivalent and it is a matter of taste if one is preferred over the others.

Example: Discrete-time version of likelihood and generative model

In a discrete-time simulation of a SRM or generalized integrate-and-fire model with escape noise, we divide the simulation interval [0,T][0,T] into NN time steps Δt=T/N\Delta t=T/N; cf. Fig. 9.6B. In each time step, we first calculate the membrane potential u(t)u(t) and then generate a spike with probability Pt=PF(u(t))P_{t}=P_{F}(u(t)) given by Eq. (9.8). To do so, we call on the computer a random number rtr_{t} between zero and unity. If Pt>rtP_{t}>r_{t}, a spike is generated, i.e., the spike count number in this time bin is nt=1n_{t}=1. The probability of finding an empty time bin (spike count nt=0n_{t}=0) is

Prob{silentin[t,t+Δt]}=1-Pt=exp{-Δtρ(t)}{\rm Prob}\left\{{\rm silent~{}in~{}}[t,t+\Delta t]\right\}=1-P_{t}=\exp\left% \{-\Delta t\,\rho(t)\right\}\, (9.13)

where we have assumed that time bins are short enough so that the membrane potential does not change a lot from one time step to the next. The spike train can be summarized as sequence {0,0,1,0,0,0,1,0,}\{0,0,1,0,0,0,1,0,\dots\} of NN binary numbers nt{0,1}n_{t}\in\{0,1\}. We emphasize that, with the above discrete-time spike generation model it is impossible to have two spikes in a time bin Δt\Delta t. This reflects the fact that, because of neuronal refractoriness, neurons cannot emit two spikes in a time bin shorter than, say, half the duration of an action potential.

Since we call an independent random number in each time bin, a specific spike train occurs with a probability given by the product of the probabilities per bin:

Ptotal\displaystyle P_{\rm total} =\displaystyle= Πbinswithspike[Pt]Πemptybins[1-Pt]\displaystyle\Pi_{bins~{}with~{}spike}[P_{t}]\,\cdot\,\Pi_{empty~{}bins}[1-P_{% t}] (9.14)
=\displaystyle= Πt{[Pt]nt[1-Pt]1-nt},\displaystyle\Pi_{t}\left\{[P_{t}]^{n_{t}}\,\cdot\,[1-P_{t}]^{1-n_{t}}\right\}, (9.15)

where the product runs over all time bins and nt{0,1}n_{t}\in\{0,1\} is the spike count number in each bin.

We now switch our perspective and study an observed spike train in continuous time with spike firings at times t(f)t^{(f)} with 0<t(1),t(2),,t(n)<T0<t^{(1)},t^{(2)},\dots,t^{(n)}<T. The spike train was generated by a real neuron in a slice experiment under current injection with a known current I(t)I(t). We now ask the following question: What is the probability that the observed spike train could have been generated by our model? Thus, we consider our discrete-time model with escape noise as a generative model of the spike train. To do so, we calculate the voltage using a discrete-time version of Eq. (9.1). Given the (known) injected input I(t)I(t) and the (observed) spike times t(f)t^{(f)}, we can calculate the voltage u(t)u(t) of our model and therefore the probability of firing in each time step.

The observed spike trains has nn spikes at times 0<t(1),t(2),,t(n)<T0<t^{(1)},t^{(2)},\dots,t^{(n)}<T. Let us denote time bins containing a spike by tkt_{k} with index kk (i.e., t1<t(1)t1+Δtt_{1}<t^{(1)}\leq t_{1}+\Delta t; t2<t(2)t2+Δt;t_{2}<t^{(2)}\leq t_{2}+\Delta t;\dots) and empty bins by tkt_{k^{\prime}} with a dummy index kk^{\prime}. Using Eq. (9.14), the probability that the observed spike train could have been generated by our model is

Ptotal=Πk=1n[Ptk]Πk,tktk[1-Ptk].\displaystyle P_{\rm total}=\Pi_{k=1}^{n}[P_{t_{k}}]\,\cdot\,\Pi_{k^{\prime},t% _{k^{\prime}}\neq t_{k}}[1-P_{t_{k^{\prime}}}]\,. (9.16)

Eq. (9.16) is the starting point for finding optimal parameters of neuron models (Chapter 10).

We can gain some additional insights, if we rearrange the terms on the right-hand side of Eq. (9.16) differently. All time bins that fall into the interval between two spikes can be regrouped as follows

Π{k|tk<tktk+1}[1-Ptk]\displaystyle\Pi_{\{k^{\prime}|t_{k}<t_{k^{\prime}}\leq t_{k+1}\}}[1-P_{t_{k^{% \prime}}}] =\displaystyle= Π{k|tk<tktk+1}exp{-Δtρ(tk)}\displaystyle\Pi_{\{k^{\prime}|t_{k}<t_{k^{\prime}}\leq t_{k+1}\}}\exp\left\{-% \Delta t\,\rho(t_{k^{\prime}})\right\} (9.17)
=\displaystyle= exp{-tk<tk<tk+1Δtρ(tk)}.\displaystyle\exp\left\{-\sum_{t_{k}<t_{k^{\prime}}<t_{k+1}}\Delta t\,\rho(t_{% k^{\prime}})\right\}\,.

Ideally, a spike train in continuous time should be described by a model in continuous time. We therefore ask whether the discretization of time is critical or whether we can get rid of it. The transition to continuous time corresponds to the limit of NN\to\infty while TT is kept fixed. The Riemann-sum on the right-hand side of Eq. (9.17) then turns into an integral. In the same limit, the contribution of the bins containing a spike to Eq. (9.16) can be written as Ptk=ρ(tk)ΔtP_{t_{k}}=\rho(t_{k})\Delta t. We are led back from the discrete-time generative model to Eq. (9.11) in continuous time with the replacement

Ptotal=Ln({t(1),t(2),t(n)})(Δt)n,P_{\rm total}=L^{n}(\{t^{(1)},t^{(2)},\dots t^{(n)}\})(\Delta t)^{n}\,, (9.18)

i.e., the continuum limit exists. In other words, once the time steps Δt\Delta t are short enough, the exact discretization scheme does not play a role. Alternative, more efficient, sampling schemes exist that avoid discretization (73); see Section 10.3.3 in Chapter 10.

Fig. 9.6: Likelihood of a spike train. Three spikes (thick vertical lines) have been observed in the interval [0,T][0,T]. A. The instantaneous firing rates at the moment of the spikes are ρ(t(1)),ρ(t(2)),ρ(t(3))\rho(t^{(1)}),\rho(t^{(2)}),\rho(t^{(3)}), respectively. The intervals without spike firing are indicated by horizontal braces. The probability to stay quiescent during these intervals decays exponentially with the time-dependent rate ρ(t)\rho(t^{\prime}). B. Likelihood of a spike train in discrete time. Top: The probability that a time bin tkt_{k^{\prime}} contains no spike is 1-Ptk1-P_{t_{k^{\prime}}} whereas the probability that a spike occurs in bin tkt_{k} is PtkP_{t_{k}}. Middle: The spike count numbers nt{0,1}n_{t}\in\{0,1\} in each time bin. Bottom: The same spike train can also be described using a finer time discretization.