2025

Mixed ANOVA

  • A mixed ANOVA includes:
    • At least one between-subjects factor
    • At least one within-subjects factor.

Mixed ANOVA

  • As with all ANOVAs, it is an omnibus test:
    • Are there any population mean differences across different levels of any between-subjects factor?
    • Are there any population mean differences across different levels of any within-subjects factor?
    • Are there any interactions between any combination of between-subjects and within-subjects factors?

The logic behind a mixed ANOVA

  • The basic rationale behind a mixed ANOVA is the same as any other ANOVA:

    • If between-level means are equal, the variance between groups should be similar to the variance within groups.

    • If between-level means are not equal, the variance between groups should be larger than the variance within groups.

  • Getting into the math behind what really defines between- and within-group variance is beyond the scope of this course.

  • From here we will focus on implementing mixed ANOVAs in R.

Data from a mixed design

##             y      A      B subject
##         <num> <fctr> <fctr>  <fctr>
##  1: 0.8790487      1      1       1
##  2: 3.5396450      2      1       1
##  3: 9.1174166      3      1       1
##  4: 2.1410168      1      1       2
##  5: 4.2585755      2      1       2
##  6: 9.4301300      3      1       2
##  7: 2.9218324      1      1       3
##  8: 1.4698775      2      1       3
##  9: 4.6262943      3      1       3
## 10: 1.1086761      1      1       4
## 11: 6.4481636      2      1       4
## 12: 6.7196277      3      1       4
## 13: 2.8015429      1      2       5
## 14: 4.2213654      2      2       5
## 15: 4.8883177      3      2       5
## 16: 5.5738263      1      2       6
## 17: 4.9957010      2      2       6
## 18: 2.0667657      3      2       6
## 19: 3.4027118      1      2       7
## 20: 3.0544172      2      2       7
## 21: 3.8643526      3      2       7
## 22: 1.5640502      1      2       8
## 23: 1.9479911      2      2       8
## 24: 4.5422175      3      2       8
## 25: 0.7499215      1      3       9
## 26: 0.6266134      2      3       9
## 27: 7.6755741      3      3       9
## 28: 2.3067462      1      3      10
## 29: 1.7237261      2      3      10
## 30: 8.5076298      3      3      10
## 31: 2.8529284      1      3      11
## 32: 3.4098570      2      3      11
## 33: 7.7902513      3      3      11
## 34: 3.7562670      1      3      12
## 35: 5.6431622      2      3      12
## 36: 7.3772805      3      3      12
##             y      A      B subject

Checking that the data.table shows a mixed design: A

# For within-subjects factor A:
# Each subject should appear at multiple levels of A
d[, unique(A), .(subject)]

Checking that the data.table shows a mixed design: A

##     subject     V1
##      <fctr> <fctr>
##  1:       1      1
##  2:       1      2
##  3:       1      3
##  4:       2      1
##  5:       2      2
##  6:       2      3
##  7:       3      1
##  8:       3      2
##  9:       3      3
## 10:       4      1
## 11:       4      2
## 12:       4      3
## 13:       5      1
## 14:       5      2
## 15:       5      3
## 16:       6      1
## 17:       6      2
## 18:       6      3
## 19:       7      1
## 20:       7      2
## 21:       7      3
## 22:       8      1
## 23:       8      2
## 24:       8      3
## 25:       9      1
## 26:       9      2
## 27:       9      3
## 28:      10      1
## 29:      10      2
## 30:      10      3
## 31:      11      1
## 32:      11      2
## 33:      11      3
## 34:      12      1
## 35:      12      2
## 36:      12      3
##     subject     V1

Checking that the data.table shows a mixed design: B

# For between-subjects factor B:
# Each subject should appear at only one level of B
d[, unique(B), .(subject)]

Checking that the data.table shows a mixed design: B

##     subject     V1
##      <fctr> <fctr>
##  1:       1      1
##  2:       2      1
##  3:       3      1
##  4:       4      1
##  5:       5      2
##  6:       6      2
##  7:       7      2
##  8:       8      2
##  9:       9      3
## 10:      10      3
## 11:      11      3
## 12:      12      3

Verifying Design Structure with .N

# For within-subjects factor A: Each subject should appear
# at multiple levels of A.  So the sum of the number of
# unique levels of A per subject should be greater than 1.
d[, unique(A), .(subject)][, .N, .(subject)]

# For between-subjects factor B: Each subject should appear
# at only one level of B.  So the sum of the number of
# unique levels of B per subject should be equal to 1.
d[, unique(B), .(subject)][, .N, .(subject)]

Verifying Design Structure with .N: Factor A is within-subjects

##     subject     N
##      <fctr> <int>
##  1:       1     3
##  2:       2     3
##  3:       3     3
##  4:       4     3
##  5:       5     3
##  6:       6     3
##  7:       7     3
##  8:       8     3
##  9:       9     3
## 10:      10     3
## 11:      11     3
## 12:      12     3

Verifying Design Structure with .N: Factor B is between-subjects

##     subject     N
##      <fctr> <int>
##  1:       1     1
##  2:       2     1
##  3:       3     1
##  4:       4     1
##  5:       5     1
##  6:       6     1
##  7:       7     1
##  8:       8     1
##  9:       9     1
## 10:      10     1
## 11:      11     1
## 12:      12     1

Anatomy of a Mixed ANOVA Table

Effect DFn DFd SS MS F
\(B\) \(b - 1\) \(N - b\) \(\mathrm{SS}_B\) \(\mathrm{MS}_B\) \(F_B\)
\(A\) \(a - 1\) \((a - 1)(N - b)\) \(\mathrm{SS}_A\) \(\mathrm{MS}_A\) \(F_A\)
\(A \times B\) Interaction \((a - 1)(b - 1)\) \((a - 1)(N - b)\) \(\mathrm{SS}_{AB}\) \(\mathrm{MS}_{AB}\) \(F_{AB}\)
  • \(a\): Number of levels in within-subjects factor (\(A\))
  • \(b\): Number of levels in between-subjects factor (\(B\))
  • \(N\): Total number of subjects

ezANOVA makes mixed ANOVA very easy.

  • Use the within and between arguments to specify factor roles:
d[, A := factor(A)]
d[, B := factor(B)]
d[, subject := factor(subject)]
ezANOVA(data = d,
        dv = y,
        wid = subject,
        within = .(A),
        between = .(B),
        type=3)

ezANOVA makes mixed ANOVA very easy.

## $ANOVA
##   Effect DFn DFd         F            p p<.05        ges
## 2      B   2   9  1.003485 4.041906e-01       0.06945186
## 3      A   2  18 19.300951 3.326391e-05     * 0.58793204
## 4    B:A   4  18  4.058954 1.617267e-02     * 0.37503908
## 
## $`Mauchly's Test for Sphericity`
##   Effect         W         p p<.05
## 3      A 0.8677183 0.5669112      
## 4    B:A 0.8677183 0.5669112      
## 
## $`Sphericity Corrections`
##   Effect       GGe        p[GG] p[GG]<.05      HFe        p[HF] p[HF]<.05
## 3      A 0.8831724 8.310193e-05         * 1.082679 3.326391e-05         *
## 4    B:A 0.8831724 2.142083e-02         * 1.082679 1.617267e-02         *

Sanity check degrees of freedom

d[, length(unique(A))] # return number of levels of A
## [1] 3
d[, length(unique(B))] # return number of levels of B
## [1] 3
d[, length(unique(subject))] # return number of subjects
## [1] 12

ezANOVA Output for Mixed Designs

  1. Standard ANOVA table with F-statistics and p-values

  2. Mauchly’s Test for Sphericity reported only for within-subject factors with greater than 2 levels

  3. Sphericity Corrections: Greenhouse-Geisser and Huynh-Feldt corrections to be used when Mauchly’s test is significant

  4. Levene’s Test for Homogeneity of variance is only reported for purely between-subjects designs

Why not Levene’s test for mixed designs?

  • Tests homogeneity of variance across groups
  • Assumes one score per subject per group
  • In a pure between-subjects design, each person contributes one value
  • Easy to compare group variances

Why not Levene’s test for mixed designs?

  • In a mixed design, each subject contributes multiple scores (e.g., one for each level of a within-subject factor)

  • Example:

    • DV = Reaction Time
    • Between-subjects = Training (Trained vs Untrained)
    • Within-subjects = Stimulus Type (A, B)
    • Each participant now has two RT values