An analyst spends an hour with an assistant building out a project plan. Halfway through, they ask it to revise the structure based on the constraints supplied at the start. The answer is wrong. It cites something never said and ignores a requirement stated clearly in the second message.
Nothing is broken. The start of the conversation no longer fits.
The working space of one conversation
A language model has no memory in the sense we usually mean. On every turn it receives the whole stack of text again and computes the next response from it. That stack has a ceiling, and the ceiling is the context window.
More goes into the stack than most people assume:
- your current question
- every earlier turn in this conversation, yours and the model's
- the contents of any files you attached
- hidden instructions from the provider about how the assistant should behave
- the space the response itself needs
All of it has to fit. When it does not, something has to go, and it is almost always the oldest material.
The unit here is the token: a fragment of text averaging three or four characters. A common word is usually one token, sometimes two. A page of prose runs to roughly five hundred.
What happens when it fills
Two behaviours, depending on the service.
The simplest approach drops the oldest turn outright. The model no longer has that part of the exchange in front of it and behaves as though it was never said. Hence the repetition: it asks something you already answered, or reintroduces a point that was settled.
The more sophisticated approach summarises the older portion and substitutes the summary for the original text. The shape of the conversation survives and the detail evaporates. That is nicer to use and more treacherous, because the model still sounds like it knows everything.
There is a third effect that gets less attention: models measurably underweight information sitting in the middle of a very long input. Beginning and end carry more. A requirement stated halfway through a long document can be technically inside the window and still get overlooked.
Forgetting is not erasing
Here is the confusion that matters for data protection.
When the model can no longer see the start of your conversation, that content has not been removed. It sits in your conversation history, on the provider's servers, under whatever retention period applies. What changed is only what the model can consult right now to produce an answer.
Those two get conflated constantly in conversations about AI and data. "The AI has forgotten it" sounds reassuring and says nothing about where your text lives. What actually happens to that history is covered in what a chat history actually is.
There is a further layer. The context window governs one conversation. Separately, most assistants now ship a memory feature that carries information between conversations, operating under entirely different rules. See how AI memory works.
Why a bigger window is not the fix
Windows grow every year. Where a conversation used to falter after a few pages, whole reports now fit comfortably.
That partly solves the memory problem and enlarges the data problem. While only a little fits, you trim by necessity. Once a full case file fits, people paste full case files, because that beats hunting for the relevant paragraph. The quantity of sensitive material leaving the organisation in a single action rises in step with the window.
Same dynamic as with attachments: see what happens when you upload a PDF to a chatbot.
What this means when you paste
Three habits that improve output quality and data exposure at the same time, which is rare enough to be worth acting on:
- Start a new conversation when the topic changes. Answers get sharper, and old data stops travelling into contexts where it does not belong.
- Paste the part, not the whole. If two paragraphs are what the task needs, supply two paragraphs. Better answers, and it removes exactly the data you did not intend to share.
- Put what matters at the start or the end. Content buried in the middle of a long input carries less weight.
Once this lands, a long conversation looks different. It is not an archive the assistant browses. It is a stack of text read aloud again on every single turn, which means everything you put in earlier is re-transmitted with every question that follows, long after it stopped being useful.
