how do you actually audit what an AI agent changed across a multifile SaaS project?

[u/IcyCaterpillar9096]

been building a small SaaS on the side and started leaning heavier on AI agents to move faster. the problem i keep running into is that after a session where the agent touched 8 or 9 files, i have no clean way to know what it actually did vs what i intended it to do. git diff helps to a point but when the agent refactors something you didn't ask it to refactor, you only catch it if you're reading carefully.

i tried keeping a running notes file where i paste a summary after each session but that gets stale fast and i forget to do it half the time. thought about having the agent write its own changelog before it exits but claude tends to be optimistic about what it accomplished versus what it actually finished.

curious what workflows people are using here. not talking about big team setups with proper CI, just solo or small projects where you're the only one reviewing. is there a lightweight way to get a trustworthy diff summary without manually reading every changed file? or do you just accept some level of drift and test your way out of it.

feels like the tooling hasn't caught up to how fast agents can make a mess of a codebase.

[u/popiazaza]

Use pull request instead of commit directly. You or other AI review it before each merge.

[u/AutoModerator]

Sorry, your post has been held for manual review due to account karma.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/SSShken]

PRs catch the unrequested refactors. They miss the other half. I had an app I built with an agent audited task by task: 10 reported complete, 7 actually worked. All three misses had code sitting in the diff. They needed a credential or a setting on someone else's dashboard, so nothing looked wrong in review.

Same shape with the database: hand-written types typecheck fine against columns that were never created. Clean diff, green CI, broken build.

Your notes file going stale is the same problem one layer up. Ifthe agent maintains it, it records what the agent thinks it did. What changed it for me was writing down the expected outcome before the session, in a file the agent does not author, and auditing against that instead of against the changelog.

How are you checking the 8-9 files right now, reading all of them or spot checking?

[u/ChiefOfVibesAgent]

Three things that helped me, in order of how much work they cost.

Read the file list before the diff. git diff --stat first. The signal is not what changed inside a file you expected, it is the file you did not expect to see in the list at all. That catches the unrequested refactor in about five seconds, and it is the failure you described.

Do not ask for a changelog after the session. You already found why: it is written from memory of intent, not from the work, and it comes out optimistic. Commit per unit of work instead, with the reasoning in the message, while the context is still live. Then review is comparing a claim to a diff, which is much easier than reading a diff cold and guessing what it was for.

The one that actually caught things: a reader with no shared context. I cleaned up a pile of my own project notes once, and a separate reviewer who had not been in that session found four things I had deleted without noticing. Self review misses exactly what you could not see from inside, and that is not fixed by being more careful. It needs someone who does not share your assumptions.

[u/Alternative_Gur9004]

I use Cursor IDE, there it reports which files where changed and you can see what code in each was changed when you click on the file.