• How big is an effect?

• The answer depends on the context and the statistical test being performed.

• In a single-sample test, we ask whether the true population mean of a variable is meaningfully different from a hypothesized value (e.g., \(C\)).

  \[ H0: \mu_{X} = C \\ H1: \mu_{X} < C \]

• The effect size quantifies the magnitude of that difference, using a standardized metric.

• This helps us assess whether a result is not just statistically significant, but also meaningful in practical terms.

• Effect size can be thought of as the area of overlap the raw data Null distribution and the raw data distribution estimated from the observed sample.

• Effect size is computed in the space of the raw data, not the test statistic.

• It answers the question: How far is the observed mean from the hypothesized mean, in meaningful units?

• In contrast, use test statistic distribution for:

  • Hypothesis testing (NHST decisions)
  • Power analysis

• Cohen’s d is a widely used measure of effect size.

• For a single-sample test (with known population standard deviation), it is:

\[ d = \frac{\bar{x} - \mu_0}{\sigma} \]

• This tells us how far the sample mean is from the hypothesized mean, in standard deviation units.

• For example, if \(\bar{x} = 1\), \(\mu_0 = 0\), and \(\sigma = 1\), then \(d = 1.0\).

• Effect size helps answer: Is the result meaningful, not just significant?

• Most scientific journals now require reporting effect size estimates along with p-values etc.

• We’ll learn different version of Cohen’s d and other effect size measures as we cover more statistical tests.