Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Online Courses / University Track / Study Smarter

Module 6 · Lesson 4 · Core

Evaluating study claims

Ask five questions of any claim about study methods and separate correlation from cause.

  • 50 minutes
  • Serves O5
  • Assumes M6.L3

Phase 1: Prepare

Step 1 Warm-up recall

From M6.L3: what does an interruption cost besides its own time?

Private: nothing you type here is sent or stored.

Step 2 Learning goal

By the end of this lesson you can:

  • ask five checking questions (Foundation);
  • separate correlation from cause (Core);
  • read a two-group comparison (Core).

Phase 2: Teach

Step 3 A familiar example

An advert says that students using a product score higher. Before you believe it, ask who was compared and who paid for the study.

Step 4 Clear explanation

Study advice comes from research, adverts and opinion. Five questions help you tell them apart.

Question to ask of a study claimWhy it matters
Who was studied, and how many?Small or unusual groups may not apply to you
Was there a comparison group?Without one you cannot tell what caused the change
Were people assigned at random?Otherwise groups may differ in other ways
What was measured, and when?A quick test may not show lasting learning
Who benefits from the claim?Sellers have a reason to overstate

A common error is mistaking correlation for cause. In an invented survey, students who sleep 7 hours or more average 72 on a test, and those who sleep less average 64.

Group (invented, 100 students each)Average test score
Sleeps 7 hours or more72
Sleeps under 7 hours64
Difference8 points

The difference is 72 − 64 = 8 points. That shows a link, not a cause. A hidden factor, such as a lighter workload or better health, could drive both sleep and scores.

Sleepmore or lessScoreshigher or lowerHidden factormay drive both
Correlation: two things move together. A third factor may drive both, so the link does not prove cause.

An experiment that assigns people at random to sleep conditions can show cause; a survey cannot. Also look for size: 8 points on a 100-point test may or may not matter, and a small group can produce a large gap by chance. The review by Dunlosky and colleagues (2013) is a good example of weighing many studies together instead of trusting a single one. See also retrieval practice, one technique whose support comes from many experiments.

Step 5 Worked example (4 marks)

Group A averages 78 and group B averages 69. What is the difference, and can you say A’s method caused it?

  1. Difference: 78 − 69 = 9 points. (1 mark)
  2. Was assignment random? Not stated. (1 mark)
  3. Without random assignment, a hidden factor may explain it. (1 mark)
  4. Answer: 9 points; the cause is not shown. (1 mark)

Phase 3: Practise

Step 6 Learner practice (instant-marked)

1. Correlation means:

2. 72 and 64 differ by:

3. The best way to show cause is:

4. A hidden factor is one that:

Phase 4: Close

Step 7 Recap

  • Ask who, how many, compared with what, measured when and who benefits.
  • A link is not proof of cause.
  • Weigh many studies, not one.

Step 8 Exit ticket (ungraded)

A survey finds students with tutors score 6 points higher. Name one hidden factor.

Private: nothing you type here is sent or stored.

Step 9 Go further (Extension)

Take one study-tip claim from a website and answer the five questions.

Common mistakes

MistakeWhy it fails
Treating a survey as proof of causeLook for random assignment.
Trusting advertsAsk who benefits.
Ignoring group sizeCheck how many.

Limits. Even a good experiment applies only to people like those studied; check that you resemble them.