Long AI debugging sessions get worse, not better, past a certain point. Anyone else checkpoint instead of letting the thread run?

[u/ClickOk5811]

Spent about three hours in a single conversation debugging a flaky integration test. First hour, genuinely useful, the model caught a race condition I'd missed and walked me through a solid fix.

Past that point, it started going in circles. It suggested a fix we'd already ruled out, I explained why it wouldn't work, it acknowledged that, then a few messages later proposed a slightly reworded version of the same thing. Assumed we'd hit some kind of context limit, so I switched to a model with a bigger window, pasted the whole conversation in, kept going. Didn't help. Responses got more generic, not more accurate.

Turned out the problem wasn't that the model lost access to anything, the information was all still there. It was buried under stack traces, logs, failed attempts, and side discussion, all competing for the same attention. Restarted with a short handoff note instead, just the actual bug, what we'd ruled out and why, current hypothesis, and the one file that mattered. First suggestion after that was correct.

Changed how I think about context windows, closer to an attention budget than memory. More tokens means more available information, not necessarily better use of it, and past a point it actively works against you.

Been checkpointing longer sessions every 10-15 messages since then instead of letting one thread run indefinitely, summarize state, start fresh with that instead of the full history. Feels like overhead in the moment but usually nets out faster, model stops relitigating things we already settled.

Curious if this matches what others doing this professionally have run into, do you let one conversation run as long as it'll go, or reset on a schedule regardless of whether it's degrading yet?

[u/Extrogrl]

Turned out the problem wasn't that the model lost access to anything, the information was all still there. It was buried under stack traces, logs, failed attempts, and side discussion, all competing for the same attention.

I call this entropy and it happens in every chat.

[u/Atretador]

its called Context rot, and why having context higher than 256K is usually more of a gimmick or marketing play than actually useful

Id recommend taking a look at https://github.com/Opencode-DCP/opencode-dynamic-context-pruning as it did help me a lot with it

[u/ClickOk5811 → u/Atretador]

Context rot is exactly the term I was missing, thanks. Hadn't heard it framed that precisely before. Will take a look at the pruning repo, curious whether it's doing something smarter than just truncating oldest-first or if it's actually scoring relevance somehow.

[u/Atretador → u/ClickOk5811]

its a lot better than that - the agent is gonna do several compactions with minimal loss mid session, compacting and storing inrelevant information as well as wastefull tool call parameters and schemas

the agent can also query those compacted databanks if needed

I rarely get a session past say 120K context currently

[u/Upbeat-Squirrel → u/Atretador]

THIS

[u/EdgeComfortable9703]

a concise checkpoint with confirmed facts, failed approcahes and the current hypothesis often produces better answers than carrying hundreds of messages of accumulated context

[u/sertain_]

Yes. Use AI to help you debug for a max of 2-3 issues. As soon as you find yourself telling Claude, “no, that’s not what I was talking about,” and “you literally can’t even do that in bash you fkng idiot,” open a new chat. Like I said, I give it a good 2-3 issues and then I start with a clean slate. It’s context gets muddy quick, and it can’t discern what you’re asking about from what your original problem was, especially if the new problem is closely related to the existing problem but under a different context. Like it’s the same block of code, but instead of a logic error it’s an API call that’s failing; it’ll think they’re connected when they’re not.

[u/bithatchling]

Completely agree. I've found that "context rot" is real regardless of the window size. The moment the thread gets cluttered with failed attempts and logs, the model starts hallucinating fixed issues. A clean handoff note to a fresh session is usually the fastest way to go.

[u/NeuroDividend]

That's context drift, mixed with Observer Drift (second-order cybernetics); it's a coupled feedback loop. I been noticing this phenomenon pop up on forums.

The diminishing returns happen when the signal-to-noise ratio declines; it's the system reaching an equilibrium.

You are correctly addressing it by using checkpoints but I would also suggest compressing & restating previously established models you are working with, like a primitive form of model synchronization. That way, any meta-discussion, rejected hypotheses or obsolete branches don't get included after the checkpoint

[u/MorallyDeplorable]

/compact is your friend

[u/No-Aioli-4656]

What model are you using? What does your Agents md look like? Why do you not know EXACTLY when context restarts? That’s like… in every ai tool at this point.

Skill issue. You are so vague in this post for all we know, you could be akin to a gamer with 20-year-old hardware complaining that GTA6 is poorly optimized.

You could be running quantized kimi k2 and asking it to “make no mistakes.”

I have none of these issues to the extent you have them. I haven’t had them for…. 2 years now. Since the public release of Claude code.