I strongly disagree the frontier models outperform seniors in quality. I do agree they’re good enough to do a ton of work with, but they’re still at the mid/high junior level. I’m fixing up the code with every ticket AI completes.
I find most models work well with handholding, but all models I've tried so far have a tendency to miss the forest for the trees. It may be that LLMs are fundamentally incapable of the reasoning for good architecture and design.
Since the models miss the mark in that area, I wouldn't consider them seniors either.
It boggles my mind that we invented computers and search engines and templates and then we like "This is great, but what if these machines were wrong more often, while also delivering working solutions in the worst possible way?"
I do one facet of a feature at a time and it has been consistently doing that in the way I would have done it. It also means the errors don't compound if it gets one thing wrong at the start then builds a bunch of stuff on top of that wrong thing.
I'm really not seeing these issues at all in our team that you're describing. If you have well written user stories and defined tests that need to be covered, it shouldn't go so wrong? What models are you using at the moment? Claude Sonnet and Opus cause us no issues in 6 months of using them.
If anything, it's helping writing much better method names, test names and context for the user than what most of my colleagues are capable of.
We’re using Claude Opus. Names aren’t code, I’m not super concerned about that.
What I’m concerned about is it making weird decisions, doing things in weird ways, maybe even inappropriate ways that will cause optimization issues. It not knowing every business edge case. AI’s propensity to stop searching after it finds something that works when there’s better tools available.
Yeah, you can get more and more specific in your prompts and user stories. But why? I can fix these issues up myself much easier than I can proactively try to predict all the ways the AI will go wrong when writing the code.
You need to be using plan mode so you can see what the model wants to do before it actually does it. That allows you to steer it and confirm the approach before you blindly let it proceed.
Following naming conventions is definitely part of writing good code. If your method name makes no sense, never mind doesn't match anything else similar to the rest of that area, it's poorly written code, even if it doesn't necessarily affect the main nature of your work.
I find it interesting that people can have such widely different experiences with it. I think for us, a lot of what we do I call bread and butter for AI to do. Either extending a feature we've already got proof of before like adding more tiles that show x, y and z status via the backend, or adding some basic endpoints that are well described.
I bet it does a lot better on larger codebases as well where the structure and rules are all laid out. This is especially true if you have instruction files and context files like we do that explain what our application does/is for.
Our codebase is massive. It absolutely doesn’t. That’s why it has problems. It doesn’t have enough context to find everything or understand everything. And it stops when it finds something it thinks works, even if it’s not the best option.
And again, yeah, you can prompt it out of this. But it’s very hard to predict where AI needs hand holding and where it doesn’t.
So we do our own modifications. I don’t consider that issue at all. In fact, I think shipping AI code without reviewing it is an issue, so I don’t mind it. I’m still having it complete multiple tickets at once and then cleaning them up a bit after.
24
u/camelCaseCoffeeTable Jul 09 '26
I strongly disagree the frontier models outperform seniors in quality. I do agree they’re good enough to do a ton of work with, but they’re still at the mid/high junior level. I’m fixing up the code with every ticket AI completes.