2024

Experiment Design in Psychology and Cognitive Science

  • This deck covers an essential element of experiment design in psychology and cognitive science: between-subjects vs. within-subjects designs, repeated measures, and mixed designs.

Between-subjects vs. Within-subjects

  • Between-Subjects Design:
    • Different participants are used for different conditions of the experiment.
    • Example: Comparing reaction times of two groups on two different tasks.
  • Within-Subjects Design:
    • The same participants are used across all conditions of the experiment.
    • Example: Measuring memory recall in the same group of participants with and without a cue.

Detecing Design From a data.table

  • Within-subejct design
##    participant condition score
##          <num>    <char> <num>
## 1:           1         A    10
## 2:           1         B    12
## 3:           2         A    11
## 4:           2         B    13
## 5:           3         A     9
## 6:           3         B    11
  • Notice the repeats in the participant column.

Detecing Design From a data.table

  • Between-subject design
##    participant condition score
##          <num>    <char> <num>
## 1:           1         A    10
## 2:           2         A    11
## 3:           3         A     9
## 4:           4         B    10
## 5:           5         B    11
## 6:           6         B     9
  • Notice the lack of repeats in the participant column.

Repeated Measures

  • Repeated Measures Design involves multiple measurements from the same source.

  • The source is usually a participant so a repeated measures design is usually – but not always – a within-subjects design.

  • E.g., A repeated measures design that isn’t also a within-subject design is collecting multiple mesurement from the same EEG sensors over multiple days, but each day with a different participant.

Mixed Designs

  • Mixed Designs combine elements of both between-subjects and within-subjects designs.

  • At least one variable is manipulated as a between-subjects factor, and another as a within-subjects factor.

Mixed Design data.table

##    participant condition  group score
##          <num>    <char> <char> <num>
## 1:           1         A      X    10
## 2:           1         B      X    12
## 3:           2         A      Y    11
## 4:           2         B      Y    13
## 5:           3         A      Z     9
## 6:           3         B      Z    11
  • Notice that the group column is a between-subjects factor, while the condition column is a within-subjects factor.

Choosing Between Between-Subjects and Within-Subjects

  • Carryover Effects:
    • Within-subjects designs suffer from carryover.
  • Statistical Power:
    • Within-subjects designs have higher power.
  • Cost:
    • Between-subjects designs can be more resource-intensive.
  • Generalizability:
    • Between-subjects more generalizable.

Example:

Question 1:

Consider the following data.table output from an experiment:

   participant condition score
1:           1         A    12
2:           2         A    15
3:           3         A    13
4:           4         B    14
5:           5         B    16
6:           6         B    12

What type of experimental design does this data represent?

  • Within-subjects design
  • Between-subjects design
  • Mixed design
  • Repeated measures design

Answer Correct Answer: B) Between-subjects design

Question 2:

Review this data.table output:

   participant condition score
1:           1         A    10
2:           1         B    12
3:           2         A    11
4:           2         B    13
5:           3         A     9
6:           3         B    11

Which experimental design is illustrated by this data.table?

  • Within-subjects design
  • Between-subjects design
  • Mixed design
  • Repeated measures design

Answer Correct Answer: A) Within-subjects design

Question 3:

Given the data.table output from a study:

   participant condition group score
1:           1         A     X    10
2:           1         B     X    12
3:           2         A     Y    11
4:           2         B     Y    13
5:           3         A     Z     9
6:           3         B     Z    11

What type of design does this dataset suggest?

  • Within-subjects design
  • Between-subjects design
  • Mixed design
  • Repeated measures design

Answer Correct Answer: C) Mixed design

Question 4:

Examine the output below from a data.table:

   participant condition score
1:           1         A    14
2:           2         A    15
3:           1         B    16
4:           2         B    14
5:           3         A    13
6:           3         B    15

Based on this output, which experimental design is being used?

  • Within-subjects design
  • Between-subjects design
  • Mixed design
  • Repeated measures design

Answer Correct Answer: A) Within-subjects design