Phase 1: Prepare
Step 1 Warm-up recall
From M1.L3: give one example of a measure that is reliable but not valid.
Private: nothing you type here is sent or stored.
Step 2 Learning goal
By the end of this lesson you can:
- pick out the sample, the design and the outcome measure in a short study description (Foundation);
- say what a correlational study can and cannot show, and why random assignment matters (Core);
- compute and interpret a standardised effect size, and explain why replication matters (Core).
Phase 2: Teach
Step 3 A familiar example
A headline says “People who keep a gratitude journal are happier.” Three stories fit the same fact. Journaling may make people happier. Happier people may be more likely to journal. Or something else, such as a calm routine, may cause both. A reader who stops at the headline cannot tell which.
Step 4 Clear explanation
Ask four questions of any study.
| Question | What to look for | Why it matters |
|---|---|---|
| Who? | Sample size, how people were chosen, their age, setting and country | Findings may not carry over to other groups. Much psychology has drawn on students in wealthy countries (Henrich, Heine and Norenzayan, 2010). |
| What was done? | Correlational (measured, not changed), longitudinal (the same people over time), or randomised (people assigned to groups by chance) | Only random assignment, done well, supports a claim that something caused a change. |
| How big? | An effect size. For two groups, Cohen’s d is the difference in group means divided by the standard deviation. Conventional labels: 0.2 small, 0.5 medium, 0.8 large. | A result can be statistically detectable and still tiny. The labels are rough anchors, not verdicts. |
| Would it happen again? | Replication: do other teams, with new samples, find a similar result? | A large project that tried to repeat many published psychology studies found that far fewer repeats produced significant results than the originals did (Open Science Collaboration, 2015). |
Design sets the ceiling on the claim a study can support:
| Design | Strongest claim it supports |
|---|---|
| Correlational | The two things are associated. No claim about cause. |
| Longitudinal | One thing changes before the other. Consistent with a causal link; not proof. |
| Randomised, well run | The treatment caused the difference in that sample, setting and period. |
Black: control group. Red: treatment group. Invented illustration. At d = 0.3 the curves still overlap heavily: many people in the control group do better than the average person in the treatment group.
Step 5 Worked example (4 marks, invented data)
120 students were randomly assigned to two groups. One group wrote three good things from their day every night for two weeks. The other wrote about their day in neutral terms. Life satisfaction (SWLS) was measured before and after. The first group’s mean score rose by 2.0 points and the second group’s by 0.5. The standard deviation was 5.0.
- Sample and design: 120 students, randomised two-group study, outcome the SWLS. (1 mark)
- Difference in means: 2.0 − 0.5 = 1.5. (1 mark)
- Divide by the standard deviation: 1.5 ÷ 5.0 = 0.30, between small and medium. (1 mark)
- Caveats: only two weeks, only students, one scale, and no information on whether it was repeated elsewhere. (1 mark)
Phase 3: Practise
Step 6 Learner practice (instant-marked)
2. A survey of 2,000 adults finds that people who walk more report higher life satisfaction. Can it show that walking raises life satisfaction?
It is correlational. Happier people may walk more, or a third factor, such as good health, may drive both.
3. A single study, with p below .05, reports a large effect. Should it be treated as settled?
One study can mislead by chance or by method. Confidence grows when other teams replicate it.
4. All participants were first-year psychology students at one university. What limit does this place on the findings?
The findings may not extend to older adults, non-students or other countries. The sample was chosen for convenience.
Phase 4: Close
Step 7 Recap
- Ask who, what was done, how big and would it happen again.
- Only well-run random assignment supports a causal claim.
- Cohen’s d = difference in means ÷ standard deviation. Labels are rough anchors.
- Replication is how a finding earns trust.
Step 8 Exit ticket (ungraded)
Find one health or happiness headline from this week and write which of the four questions its article answers and which it ignores.
Private: nothing you type here is sent or stored.
Step 9 Go further (Extension)
Read the abstract of the 2015 Open Science Collaboration report (Science) and write two sentences on what it does and does not show about any single finding.
Common mistakes
| Mistake | Why it fails |
|---|---|
| Reading “associated with” as “causes” | Association is compatible with several causal stories. |
| Reading “significant” as “important” | Significance is about chance. Effect size is about magnitude. |
| Treating the labels 0.2, 0.5, 0.8 as strict | They are conventions, and many real effects in this field are small. |
Limits. Cohen offered the labels as rough guides when little else existed. What counts as a meaningful effect depends on cost, risk and the outcome. Module 4 returns to this for positive activities.