The limits of an experiment on yourself

Step 1 Warm-up recall

From M3.L4: name two threats to a personal experiment.

Your answer

Private: nothing you type here is sent or stored.

Step 2 Learning goal

By the end of this lesson you can:

• name four limits of a personal experiment (Foundation);
• explain regression to the mean (Core);
• turn limits into careful report wording (Core).

Step 3 A familiar example

You start a new habit during your worst week of the term. A month later you feel better. Was it the habit, or would the week have passed anyway?

Step 4 Clear explanation

Limits you should look for in your own data:

• Expectation. You know the habit is supposed to help, so you may rate your days more kindly. You cannot blind yourself.
• Regression to the mean. Extreme periods tend to be followed by more typical ones. If your baseline happened to be a low stretch, ratings may rise without any habit.
• Concurrent events. Exams, weather, illness, news or travel affect ratings. Your daily notes help here.
• Sample of one, no comparison. You have no control group. You cannot say what your ratings would have been without the habit.
• Measurement. A single daily rating is noisy and depends on the scale you chose.

These limits do not make your experiment worthless. They set what you can honestly claim: that ratings were higher, lower or the same, with a dose and an adherence, and that the cause is open. An experiment with a better design (random order, comparison, blinding where possible) would be needed to claim more.

Step 5 Worked example (4 marks)

Your report says “the habit raised my mood by 0.7”.

1. “Raised” claims cause; the data show ratings were 0.7 higher. (1 mark)
2. No comparison, so you cannot rule out regression to the mean or concurrent events. (1 mark)
3. You knew the habit was supposed to help, so expectation may contribute. (1 mark)
4. Better wording: “My ratings in weeks 3 and 4 averaged 0.7 higher than baseline; I cannot say the habit caused this.” (1 mark)

Step 6 Learner practice (instant-marked)

Step 7 Recap

• Expectation, regression to the mean, concurrent events, sample of one and noisy measurement.
• Limits set what you can claim.
• Use “was higher than”, not “raised”.

Step 8 Exit ticket (ungraded)

Name the two limits most likely to apply to your own four weeks.

Your answer

Private: nothing you type here is sent or stored.

Step 9 Go further (Extension)

Design (on paper) a better experiment: how would you add a comparison period, and what would it cost you?