The Model Lies. And Doesn't Know It.

A student sent me her seminar paper. Three perfectly formatted citations — author, year, publisher, page number. Everything looked correct. Everything was invented. The sources didn't exist. The authors were fictional.

The student was devastated. She had trusted the AI completely.

This is called hallucination. And it's not a bug that will eventually be fixed. It's structural.

Large language models don't "know" anything — they predict which text statistically follows the previous text. A plausible-sounding citation is statistically very likely, even if the source doesn't exist. The model doesn't lie intentionally. It has no concept of truth. It only has patterns.

This is a fundamental epistemological difference. And it's concealed by the elegance of the output: the more fluent the text, the more trustworthy it seems. But fluency and truth are two completely separate dimensions.

I wrote about this in detail here: From Hallucination to Hypothesis. The productive reframe: hallucinations aren't errors — they're hypotheses. Treat AI output like a first draft: interesting, worth checking, not ready to submit.

In practice:

AI as brainstorming partner: yes.
AI as research tool: with verification.
AI as authoritative source: no.

That sounds strict. It's just sober.

Diskutiere diesen Text, seine Begriffe oder Denker:innen mit Sociologica. Dialektisch (Lesart + Gegen-Lesart), mit Leitfrage zurück an dich.

Diskussion starten

Ask your AI three factual questions from your field. For each answer: find one specific claim you can verify independently. How many are correct?

When did you last trust an AI statement without verifying it — and what was the result?

Day 5
Day 7