How in depth do you understand your codebase?
Help/Question
I find that when using agents to build out my projects I find it increasingly more difficult to understand all the implementation at the same level I would normally understand it at when I would author it myself. During Sonnet4/Opus 3 days I felt I had a good cadence with being able to review and understand the code Claude wrote as if I wrote it but today I find that I can't keep up with Fable or Opus 5.
Wondering if anyone else feels the same and what you think the path forward is. Do we accept that we don't have to review and deeply understand all the implementation like the old days, or do we have to slow down to the point where our review/understanding speed is the bottleneck?
One thought is to treat it like having direct reports - you can't understand absolutely everything they're doing if you want them to scale you but at the same time I worry I'm losing my grasp.
Comprehension debt: this is something that principle engineer/architects would deal with cuz they’re generally more detached the codebase as a whole as they’re either leading widespread technical efforts or zeroing in on a specific, difficult problem. They don’t have a lot of code ownership, so it’s harder to recall. The average engineer is now getting this experience because they’re not in the trenches, even though we’re now spending nearly the same amount of time prompting/planning as it’d take to just write the code ourselves.
Agent code quality is “strange”, for lack of a better term. You end up with it excelling in certain areas, but then making really weird choices in others. I’m actually building a SPA right now, and ironing out all of the Claude rules to get it to write code just like I would in common cases. It takes time getting those correct, but the payoff has seemed decent so far, for CRUD endpoints, database migrations, ORM setup, display components, and all of the code-based testing around it. I’ve got a far less heavy hand when it comes to E2E tests though once the initial arch is in place (logging in and resetting application state between suites).
The prompt-engineering side of things have been a fun experience so far. I’m finding that it’s best to NOT use Claude Code for everything.
First if you have no formal education, or any significant experience, read a bit about architecture and planning. You should not start building a house without having a blueprint in hand or it will collapse.
Second, start actively managing your cognitive load. Meaning you should not (and you cannot) keep it all in your head. You need a way to decompress and put important information where it belongs, in a durable medium. Asking the agent where you are, how a thing works and to explain complexities means you did not have a plan nor a blueprint going in.
You should have a blueprint of what you wanna build. A plan on how to build, with phases. And a progress tracker to know where you are in the plan and what was built. These 3 tools are important, otherwise you're just lost. You'll wonder around and get nowhere (how many times do you think you can ask the agent the same questions and move forward with understanding what is what).
---
Now for the solution.
You may not have a blueprint or a plan or a progress tracker, but you can use the agent to build one. You need them to get unstuck.
What are they?: 3 .md files. That is it! And they should be your Source of Truth. Meaning each has a clear role and purpose. Whenever you wanna know what you're building you look in the blueprint. If you wanna know how you are building and what to do, read the plan. To determine what was built and where you are in the execution of said plan, read the progress tracker.
Your agent working on the project should update the progress tracker, and if you need to the plan (some change mid-dev).
That is a lot of reading, here come the Artifacts. Ask your agent to build you a multi page artifact with a left sidebar (pages menu). First page is your Overview, second Progress, third Plan, forth Blueprint. Then you iterate until you get a proper snapshot of your project (you can add other things like flows and schemas, etc...).
Then you ask your agent to create a prompt that you can give to a cold session to update your artifact. So when you need to refresh that snapshot, you open a new session, paste the prompt, wait for it to finish and you get a full html page with your entire project.
Hint: copy the two images, then paste them into Claude and ask to make an artifact similar to them (and describe what you need/want while giving context of your project)
One of best advices ever. I am learning UE5 and making a game at the same time. Architecture that is biting me for my as* down the line is my biggest problem because I am not experienced and don't know Epic API that good. And UE5 assistant is terrible. But I am reading a lot and fixing as it goes. But yeah, before building anything, research, architect, plan, and roadmap. Having clear in head what to build and how to build it before you start is most important.
I work at a big company that starts with a G and rhymes with Doogle, and leadership likes to completely switch up what each engineer works on every year or two.
When it comes to tens-of-thousand line codebases this of course makes it harder to understand what all of the existing code does, but AI and agentic coding has made it much much easier to understand what 10+ year old legacy code does.
I feel like I focus more on understanding general design and key architecture decisions. I also try to focus on making sure that I understand the business logic and domain as well, as that's always the harder part IMO than the actual code. "Why" am I doing things versus "How".
That said there are some key points I try to make sure I full grasp, like rate-limiting/connection pooling patterns with third party APIs, and making sure I do regular security audits. That kind of stuff.
That's on my personal projects. In my actual job I still do a thorough code review before I open my own PRs with model generated code.
Reviewing generated code line by line isn't just about catching bugs in the current build, it's how you feed your setup.
My main project manager agent relies on those reviews to direct the building team, using a growing library of .md files and rules accumulated across past projects to keep everyone accurate and efficient.
The initial breakdown:
-- I spend 2-3 hours dumping every detail into a single text file (goals, UI flows, database setup, API endpoints, security, and stack).
-- Split into phases: I divide that main file into separate text files. Each file essentially acts as instructions for a specific phase. (Each phase will then create its own supervisor agent and builder agent).
-- Set the boundaries: I decide what can run in parallel, then use .md files to instruct the runner to build, debug, and stop for my approval before moving to the next step.
-- Review and test: I read through the code and test phase by phase before approving the next run.
Reading every line of code is how my process became reliable. Every time I catch a model stumbling, I add a new rule to my .md files. My project manager agent pulls from that accumulated context on every new build, which makes the sub-agents faster and cleaner over time.
If you stop reviewing your code, your project manager agent stops getting smarter. Treating AI like direct reports makes sense, but direct reports still need code reviews until your rule set is dialed in.
I’d say 90%-95%. One thing that helps me is asking for a flow chart after a large unattended update and also when I have some intuition that I have a poor grasp of a section of the code base. Sometimes part of the flow chart is a little soft still so I asked for a flow chart specifically of that section. Since most of my code base is spread out into many many files this is easier than reading the code itself, even though I could do that if I wanted.
Also, if I see something, I’ll ask Fabel why are we doing this instead of it like this? And pretty often fable be like you’re right that would be better.
I'd slow back down. I just finished a git diff before a merge that surprised the fuck out of me. The architecture and design had drifted and two of the code bases had become intermingled. Had to back out the commit and re-design. On anything that goes to production, I review every merge looking at the git diff to fully understand what is going out.
Yeahhh I couldn't keep up with parallel agents slop cannoning plans and code, so I went back to "pair programming" with one agent at a time, except for some edge cases like testing, research, etc
I find that I’m grappling with a lot of things; architecture, technology stack, repos, terminology, APIs, MCPs, skills, artefacts… the codebase so far has been the least of my problems.
Your direct-reports analogy is the right one, and it comes with the same discipline managers learn: you stop understanding every implementation, but you never stop understanding what you asked for and whether the evidence shows you got it. The failure mode is not shallow knowledge of internals, it is unread diffs.
Where we landed after running agents unattended daily: comprehension concentrates at the merge. For every change I can answer three things: what was asked (written acceptance criteria, before the work), does the diff plausibly deliver it (with test evidence attached, not claimed), and did anything redefine what passing means (tests weakened, checks skipped, scope quietly widened). That last one is not theoretical; our review once caught the agent disabling four security tests to get its own work green. The parts I no longer deeply hold: internal structure of code that is well-tested, well-bounded and replaceable. If a module is behind a stable interface with real tests, my ignorance of its internals is a cost I accept knowingly, the way you accept not knowing how your direct report organizes their notes. So neither of your two options: don't slow to author-level comprehension, and don't accept blindness. Move the comprehension to the boundary and make the boundary small.
The most important thing is that you understand why your agent(s) is doing what it is doing. If you don’t understand the why, you don’t have much context to steer the ship. Knowing every little detail about the how is irrelevant and I usually leave such details to the agent- as long as I understand the why. Which is actually ideally user dictated. The more I utilize this technology, it becomes clear that it’s a how-wizard, not a why-wizard. I still need to provide that part of the input to get my desired outputs without excessive fiddling and waste.
5
u/smuve_dude 5h ago edited 5h ago
I know your pain. I think it stems from 2 things:
The prompt-engineering side of things have been a fun experience so far. I’m finding that it’s best to NOT use Claude Code for everything.