2024

Introduction

  • Null Universe: The neuron is not stimulus-driven. The Expected Value of \(X\) is 10 spikes per second.

  • Alternative Universe: The neuron is stimulus-driven.

  • We want to know which universe we are in.

  • We write this as:

\[ \begin{align*} H_0 &: \mu_X = 10 \\ H_1 &: \mu_X > 10 \end{align*} \]

We might decide wrong

  • Every hypothesis test we will perform in this unit comes down to choosing between two possible realities. In one reality, \(H_0\) is true and in the other reality \(H_0\) is not true.

  • Sometimes we will reject \(H_0\) even when \(H_0\) is in fact an accurate description of the universe. This is called a type I error and it makes us sad.

  • Simetimes we will fail to reject \(H_0\) even when \(H_0\) is a really bad description of the universe. This is called a type II error and also makes us sad.

What might happen when we choose

  • \(H_0\) might actually be true

    • We might fail to reject \(H_0\) (correct decision)

    • We might reject \(H_0\) (type I error)

  • \(H_0\) might actually be false

    • We might fail to reject \(H_0\) (type II error)

    • We might reject \(H_0\) (correct decision)

Decision matrix

  • We can summarize the possible outcomes of a hypothesis test in a decision matrix.
Decision \(H_0\) True \(H_0\) False
Fail to reject \(H_0\) Correct Decision (\(power\)) Type II Error (\(\beta\))
Reject \(H_0\) Type I Error (\(\alpha\)) Correct Decision (\(confidence\))

Specifiying the alternative

  • Our test procedure can only reject or fail to reject the Null.

  • It cannot accept the alternative.

  • This is in part because the alternative is actually a set of possibilities.

  • To compute power and the probability of a type II error, we need to specify a single alternative.

Null distribution

Critical value

Type I error (\(\alpha\))

Confidence

The alternative distribution

Type II error

Power

Power is a good thing!

  • Power is the probability of rejecting \(H_0\) when \(H_0\) is false.

  • It is the probability of finding an effect when there is really an effect there to be found.

  • Grating agencies typically want to see power of at least 0.8 when you design and propose experiments.

Power and sample size \(n\)

  • Power increases as sample size increases.

  • This is because the standard error of the mean decreases as sample size increases.

Power increases as sample size increases

Power increases as sample size increases

Power increases as sample size increases

Power increases as the effect size increases

Power increases as the effect size increases

Power increases as the effect size increases

Question

Suppose that you are given the following figure and told that:

  • \(X_{H0} ~ \mathbb{N}(\mu=7, \sigma=1)\) and \(X_{H1} ~ \mathbb{N}(\mu=5, \sigma=1)\)

  • What quantity does region I in the above plot correspond to?

  • What quantity does region II in the above plot correspond to?

  • What quantity does region III in the above plot correspond to?

  • What quantity does region IV in the above plot correspond to?

  • What is the numeric value (to two decimal places) of the power in this example?

  • Assuming a type I error rate of 0.05, what is the numeric value (to two decimal places) of confidence of this example?

  • Assuming that the illustrated distributions correspond to the distribution of sample means, will increasing the sample size increase the distance between the mean of H1 and H0 distributions?