2024

Introduction

  • Suppose we are interested in understanding receptive field properties of a particular neuron.

  • We know from past research that – in the absence of all external inputs – this neuron has a baseline firing rate of 10 spikes per second on average.

  • However, the baseline firing rate is not constant, but fluctuates around this average value.

Baseline firing in a cool neuron

Cool stimulus-driven firing

Still cool?

How about now?

And now?

We need a formal method

  • Suppose we know the Expected Value of the baseline firing rate of this neuron is 10 spikes per second.

  • Let \(X\) to be the random variable that generates the observed number of spikes per second during the stimulus.

  • We want to know if the neuron is stimulus-driven.

  • That is, we want to know if the Expected Value of \(X\) is greater than 10 spikes per second.

  • \(\mathbb{E}(X) > 10\)???

  • \(\mu_X > 10\)???

Imagine two universes

  • Null Universe: The neuron is not stimulus-driven. The Expected Value of \(X\) is 10 spikes per second.

  • Alternative Universe: The neuron is stimulus-driven.

  • We want to know which universe we are in.

  • We write this as:

\[ \begin{align*} H_0 &: \mu_X = 10 \\ H_1 &: \mu_X > 10 \end{align*} \]

Start by building a model of the Null

  • The Null (\(H_0\)) universe is the universe where the neuron is not stimulus-driven.

  • The Expected Value of \(X_{H0}\) is 10 spikes per second.

  • Assume that the distribution of \(X_{H0}\) is Normal with mean 10 and standard deviation \(\sigma_X = 1\).

The Null Universe

  • \(X \sim \mathcal{N}(\mu_X=10, \sigma_X=1)\)

Is this a good model?

  • Firing rate - i.e., \(X_{H0}\) – is continuous and can take any value between 0 and \(\infty\).

  • \(X \sim \mathcal{N}(\mu_X=10, \sigma_X=1)\) is continuous but is defined for \((-\infty, \infty)\).

  • firing rate cannot be negative but our model can be negative. This is a weakness of the model.

  • However, \(X \sim \mathcal{N}(\mu_X=10, \sigma_X=1)\) assigns very near zero probability to negative values, so this model may not be too bad afterall.

Sample from \(X \rightarrow\) compare to Null

  • Stimulate 25 times and observe the spikes per second in each case.

We had to wrangle the data

  • Remember that our experiment generates the spike time raster plots from the beginning of this slide deck.

  • We had to wrangle the data to get the number of spikes per second from those plots.

  • This is where your data.table skills would come in handy.

Our raw data isn’t a good fit for our question

  • Recall that our question is expressed as follows:

\[ \begin{align*} H_0 &: \mu_X = 10 \\ H_1 &: \mu_X > 10 \end{align*} \]

  • Our raw data doesn’t immediately and unambiguously tell us about \(\mu_X\).

We need an estimate of \(\mu_X\)

  • We can estimate \(\mu_X\) by taking the sample mean of the observed firing rates.

  • We denote estimate of \(\mu_X\) as \(\widehat{\mu_X}\).

  • \(\widehat{\mu_X} = \bar{x}\)

  • \(\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i\)

The sample mean: visual

How likely is \(\bar{x}\) under the Null?

  • This is the same as asking: \(P(\bar{X}_{H0} > \bar{x}_{obs})\)

  • How can we compute this probability?

  • We know that we need area under the curve of the probability distribution given a continuous distribution.

  • This may give us the idea on the next slide…

It may be very tempting to do this

What’s wrong with this approach?

  • Can you see what’s wrong with this approach?

  • Suspense…

It’s the wrong probability distribution

  • The random variable that generates the raw data (\(X\)) is different from the random variable that generates the sample mean (\(\bar{X}\)).

  • Since we are asking about how likely an observed sample mean is under the Null, we should be using the distribution of the sample mean.

Raw data vs sample mean

How likely is the Null Universe?

  • The shaded area gives us \(P(\bar{X}_{H0} > \bar{x}_{obs})\)

  • This is called the p-value.

Why do we include all the probability to the right?

  • We are interested in the probability that the Expected Value – i.e., the population mean of the firing rate is greater than 10 spikes per second.

  • The inequality is baked into the our original question.

H0 vs H1: Making a decision?

  • If the p-value is very small then the Null is very unlikely and we reject it.

  • But how small is very small?

  • This is choice we have to make and we will discuss it more when we discuss inferential errors and statistical power.

  • For now, it’s enough to say that in pychology and neuroscience a p-value of 0.05 is often used.

  • This number is called the Type I error rate and is denoted by the symbol \(\alpha\).

Making the decision

  • For this example, the p-value turns out to be:
## [1] 0.08729804

Summary Recipe

  1. Specify the null and alternative hypotheses (\(H_0\) and \(H_1\)) in terms of a population parameter \(\theta\).

  2. Specify the type I error rate – denoted by the symbol \(\alpha\) – you are willing to tolerate.

  3. Specify the sample statistic \(\widehat{\theta}\) that you will use to estimate the population parameter \(\theta\) in step 1 and state how it is distributed under the assumption that \(H_0\) is true.

  4. Obtain a random sample and use it to compute the sample statistic from step 3. Call this value \(\widehat{\theta}_{\text{obs}}\).

  5. If \(\widehat{\theta}_{\text{obs}}\) or a more extreme outcome is very unlikely to occur under the assumption that \(H_0\) is true, then reject \(H_0\). Otherwise, do not reject \(H_0\).

Question

Consider the following figure. Can a Normal test be conducted on the basis of the information provided to you in this figure or is something essential missing?

Answer
  • The observed data can be obtained by reading from panel A. We see that there are 10 observations so \(n=10\)

  • Panel B shows that the random variable that generates the raw data is Normal with mean 5 and standard deviation 2.5.

  • Panel C shows that the distribution that generates the test statistic is Normal with mean 5, and standard deviation that looks to be about 2.5/sqrt(10).

  • Since the distribution of the test statistic is Normal, we can conduct a Normal test.

  • So yes, we have everything we need. Hooray!

Question

  • What is the expected value of \(\hat{\theta}\)?

  • What is the observed value of the test statistic?

  • What is the \(p\)-value of this test?

  • What is the critical value of this test (i.e., the value that ensures that type I error is less than or equal to \(0.05\))?