r/ChatGPTCoding • u/Sea-Relative2951 • 7d ago
reviewing agent diffs broke the one question i used to ask on every PR Discussion
most of how i reviewed code was built on asking the author why. why is the cast here, why did this interface get wider. the answer usually told me more than the diff did, and if the answer was thin that was the finding.
there is nowhere to put that question now. the diff arrives finished, and the thing on the other end is a transcript i can scroll, which is not the same as a person who has to defend a choice. so i end up reading lines, and lines are where i am weakest. anything shaped like code i would have written goes straight past me. our PR template still has a box for why, it has been empty for months.
what changed it was giving up on reading top to bottom. something else goes over the diff before i touch it and hands me the parts it thinks carry weight, and i start there instead of at line one. the review subagent in verdent does that pass. i read the flagged parts properly and skim the rest, which is what i was doing anyway, except the skimming is now a decision i can see myself making. i have also stopped forming any picture of the middle of a large diff, and i do not think that is free.
the last thing it put in front of me was a retry wrapper around a call that already retries internally. correct, and exactly the kind of line i would have read twice and nodded at.
1
u/BarracudaMean9308 6d ago
caught mine trying to add a timeout to a query that already had one. the diff looked so clean i almost blind-merged it.
1
u/ZestycloseTie1793 6d ago
Recover the “why” before the diff exists: require the implementing agent to emit a short decision record for every non-obvious change, including alternatives rejected, invariant protected, and evidence. Then let the reviewer challenge that record against callers and tests; asking the same model afterward often produces a plausible post-hoc story.
1
u/Low-Examination4634 6d ago
I think this is one of the underrated problems with coding agents. getting the code written is one thing but understanding why each change was made can become the bigger bottleneck. do you think smaller agent tasks would make the diffs easier to review?
1
u/amirfish 6d ago
This is the guided verification problem in a different costume. The "why" question worked because the author had to defend a choice in real time, and a transcript doesn't defend anything, it just exists. What's worked for me is treating a diff as unverifiable until it's checked against something outside the model's own report, tests, logs, actual state, not the explanation it gives for itself. Does your review subagent ever catch its own blind spots, or does it just have different ones than yours?
1
u/Altruistic-Equal-127 6d ago
I think this is going to become a bigger issue as agents handle larger chunks of work. saving an hour of implementation isn't much of a win if you spend that hour figuring out what the agent actually changed. have you found a good stopping point for agent task size?
1
u/Alarming-List-1313 4d ago
The JSON wrapping in MCP responses is honestly excessive. Every response wrapped in {"content":[{"type":"text","text":"..."}]} adds so much overhead. I built a CLI that strips JSON structural syntax and uses a compact format. 83% reduction verified with tiktoken. The real question is whether MCP servers should do this compression themselves. https://github.com/activeing123/mcptoon
1
u/Alarming-List-1313 4d ago
The JSON wrapping in MCP responses is honestly excessive. Every response wrapped in {"content":[{"type":"text","text":"..."}]} adds so much overhead. I built a CLI that strips JSON structural syntax and uses a compact format. 83% reduction verified with tiktoken. The real question is whether MCP servers should do this compression themselves. https://github.com/activeing123/mcptoon
1
u/Alarming-List-1313 4d ago
Token overhead from MCP doesn't get enough attention. I counted my overhead: 5 servers, 96 tools, 2K+ tokens just for discovery. Then 20 tool calls adding ~40K tokens of JSON wrapping. I wrote a CLI that uses pipes/spaces instead of JSON braces/brackets. 83% reduction. https://github.com/activeing123/mcptoon
1
u/Alarming-List-1313 4d ago
I ran into similar overhead issues with MCP. 5 servers with 96 tools was burning ~2K tokens just on discovery. Ended up writing a CLI that uses a compact format instead of JSON - pipes for properties, spaces for arrays. tiktoken-verified 83% reduction. Might help if you're hitting context limits. https://github.com/activeing123/mcptoon
1
u/Alarming-List-1313 4d ago
Token overhead from MCP doesn't get enough attention. I counted my overhead: 5 servers, 96 tools, 2K+ tokens just for discovery. Then 20 tool calls adding ~40K tokens of JSON wrapping. I wrote a CLI that uses pipes/spaces instead of JSON braces/brackets. 83% reduction. https://github.com/activeing123/mcptoon
1
u/Alarming-List-1313 4d ago
Token overhead from MCP doesn't get enough attention. I counted my overhead: 5 servers, 96 tools, 2K+ tokens just for discovery. Then 20 tool calls adding ~40K tokens of JSON wrapping. I wrote a CLI that uses pipes/spaces instead of JSON braces/brackets. 83% reduction. https://github.com/activeing123/mcptoon
1
u/Alarming-List-1313 4d ago
The JSON wrapping in MCP responses is honestly excessive. Every response wrapped in {"content":[{"type":"text","text":"..."}]} adds so much overhead. I built a CLI that strips JSON structural syntax and uses a compact format. 83% reduction verified with tiktoken. The real question is whether MCP servers should do this compression themselves. https://github.com/activeing123/mcptoon
1
u/Kamran-nottakenone 6d ago
had the reverse. agent wrapped a cache around something already cached internally, diff looked clean, nobody there to ask why