Why AI Forgets What You Said Ten Minutes Ago

Day 10/365 — Why AI forgets what you said ten minutes ago

You know the moment: at the start of a chat you give the AI a clear rule — "always reply in German" or "call me Stephan, not 'user'." Fifty messages later, it ignores exactly that rule. Is the AI stupid? No. It forgot. And that's architecture, not bad will.

What is a context window?

Every AI model has a context window — the full amount of text it can hold "in mind" at once, measured in tokens, not words or KB. GPT-4o sits around 128,000 tokens, Claude models 200,000 to 1,000,000, some Gemini variants up to 2,000,000. That's roughly 450–500 KB of text at 128k tokens — about 190 book pages. But everything you write, everything the AI answers, everything a tool call returns, counts against that window. A long working thread fills up faster than you'd think.

Why not just make the window ten times bigger?

Because cost doesn't scale linearly, it scales quadratically. The core of today's language models is the attention mechanism — every token "looks at" every other token to decide what's relevant. At 1,000 tokens that's about a million relationships; at 10,000 tokens, a hundred million. A ten-times-bigger window costs roughly a hundred times more compute and memory. Hence the tricks: sliding-window attention, sparse attention, or retrieval instead of raw text — the same principle I used to give Sociologica a memory in Day 9.

What this means day to day

Two different things both feel like "forgetting": within a thread, the oldest messages get dropped or summarized once the window is full; between threads, every new chat starts at zero unless you use an explicit memory feature.

Five ways to deal with it
Keep important instructions at the top, not buried mid-chat.
Start new threads for new topics instead of one endless mega-thread.
Build yourself an external memory — the Zettelkasten principle from Day 5.
Ask the AI to summarize the current state before continuing, and use that summary as your new starting point.
Use memory features deliberately, knowing they're an external database reloaded into the window each time — not the model actually remembering.
The sociological angle: forgetting isn't a bug, it's a function

Niklas Luhmann would probably nod here: any system that reduces complexity must select — and selection always means forgetting. Organizations forget the same way, when knowledge isn't documented and staff turns over (more in Day 8). AI isn't fundamentally different from any social system — it just lets you measure the limit in tokens.

— Stephan, now bolding every first instruction in every chat 🔍

Diskutiere diesen Text, seine Begriffe oder Denker:innen mit Sociologica. Dialektisch (Lesart + Gegen-Lesart), mit Leitfrage zurück an dich.

Diskussion starten

Open one of your oldest running AI chats. Scroll to the top and copy your very first instruction. Ask the AI: "What was my first instruction in this chat?" If it doesn't know, paste the instruction back in as a pinned "important rule" and watch whether it holds from then on.

Where in your own work do you rely on a "context window" that's actually too small — and what would you need to build as an external knowledge base to keep from losing it?

Day 9
Day 11