Fluent but wrong: why assistants invent details

Step 1 Warm-up recall

From M1.L2: name two kinds of detail that must be checked.

Your answer

Private: nothing you type here is sent or stored.

Step 2 Learning goal

By the end of this lesson you can:

• describe the pattern of confident invention (Foundation);
• list warning signs in a reply (Core);
• apply a short verification checklist (Core).

Step 3 A familiar example

You ask for a reference for a claim, and receive a neat citation with authors, a journal and a year. You search for the paper and cannot find it. It looked right, but it was assembled to look right.

Step 4 Clear explanation

Because a model produces text that is likely to fit the request, it can fill a gap with something plausible even when it has no source. People call such invented content a hallucination. It is not a lie in the human sense: the system is not aware of the difference. It is a side effect of generating likely text.

Warning signs include: very specific details with no source (a precise percentage, a named expert); quotations attributed to a real person; citations you cannot find; and answers that match your request suspiciously well. Fluency makes all of these feel checked.

A short verification checklist helps: (1) Source: can I find the original? (2) Specifics: do dates, names and figures match a reliable source? (3) Sense: does it fit what I already know? (4) Stakes: what happens if this is wrong? The higher the stakes, the more of these you must complete before using the content.

Step 5 Worked example (4 marks)

An assistant gives you this invented sentence: “A 2019 study found that 73% of employees prefer remote work”, with no source.

1. Warning sign: a precise percentage with no source. (1 mark)
2. Search for the study; if you cannot find it, do not use the figure. (1 mark)
3. If you find a study, check the figure and the wording against it. (1 mark)
4. Rephrase without the figure if it cannot be checked. (1 mark)

Step 6 Learner practice (instant-marked)

Step 7 Recap

• Assistants can fill gaps with plausible inventions.
• Warning signs: unsourced specifics, unfindable citations, quotations.
• Check source, specifics, sense and stakes.

Step 8 Exit ticket (ungraded)

Write the four checks (source, specifics, sense, stakes) in your own words.

Your answer

Private: nothing you type here is sent or stored.

Step 9 Go further (Extension)

Ask an assistant for a source for a claim in your field and check whether the source exists.