Context rot

Context rot is the degradation in a model's answers as its context grows, through dilution of the relevant material, contradictions between old and new content, and accumulated dead ends.

It is the reason a long session gets worse rather than better informed. Everything that has ever been in the window is still in the window: the approach that was abandoned, the error that was fixed, the file that has since changed.

Dilution is the measurable part. Accuracy on retrieving a specific fact from a long context falls as the amount of unrelated material rises, and material placed in the middle of a long window is recovered least reliably.

The practical countermeasure is unglamorous and effective: end the session when the task ends. Nothing else recovers a window as completely as starting a new one.

All terms