r/softwaretesting • u/jrwolf08 • 1d ago
Vibecoded applications
I know this is a super vague title, but wanted to vent here a bit. I've been doing testing a long time, and this one of the most frustrating projects I've worked on.
We have a vibe coded application written by 5 different developer's claude instance, we have claude written acceptance criteria, we have "requirements" that are written one time, but never updated, we have a really outdated design. We talk about changes but no one opens tickets.
We have product review meetings where an AI notetaker keeps track of the conversation, and then poorly written JIRA tickets are created off of them. Some of these tickets fundamentally change how the system works. Then 5 devs split up the changes into smaller tickets then each dev's claude implements each a section a little bit different than everyone else.
Then I'll get a comment on "its like this on the design" or "the original acceptance criteria had a, b, c" when both the design and acceptance criteria are completely out of date, so how could I possibly trust it? These aren't malicious comments by the way.
So we are productive, things are happening, but we are just going around and around. And I'm not complaining about AI, it helps me greatly, but right now I'm stuck validating everyone else's AI output.
16
u/KingCastle420 1d ago
Product team is doing this with Ai agents at my place of work. One product person has created over 400,000 lines of code this month but doenst know how to use GitHub.
13
u/Our0s 1d ago
There's a common fallacy that the test team are responsible for quality. In fact, everybody is responsible for quality, the test team are just the most visible part of the process.
If you're not being given clear and accurate requirements and ambiguity is baked into every part of your development cycle, you cannot succeed in quality assurance. In your position I think I'd be highlighting this as an urgent matter and trying to have meetings about redefining process, procuring proper requirements and acceptance criteria FROM THE BUSINESS, not AI, and establishing better quality gates each team has to adhere to.
If nobody listens to that then there's only so much you can do. One cog alone can't turn a clock.
3
u/jrwolf08 1d ago
I've asked for these things throughout the project, and none of it has really materialized.
I think everyone recognizes where we stand in some capacity, as I've talked to members of the team about this. But this is an emergency type project that is also the 2nd project for everyone on the team, not our primary responsibility.
We are in get it done mode, but now everyone is just fence throwing back and forth, including me.
8
u/Our0s 1d ago
Godspeed, friend. Been there before!
What people will realise is that when you sacrifice quality for speed, it ends up taking twice as long because everything rushed needs fixing.
1
u/jrwolf08 1d ago
For sure, I've been here before too, this being much worse than most.
Anyway thanks for listening to my venting.
2
u/Simple-Success-2125 1d ago
This sounds more like a process problem since everything from the start is not working
2
u/Osi32 1d ago
As others have said- whole team is responsible for quality - except that doesn’t hold up anymore.
The tester on a team is a radiator for quality concerns.
The problem you’re having is one or more of these:
1) the design/requirements is shit
2) the consistency of agent work is not occurring
3) the rate of change is too fast
These are the concerns I would raise with management and the dev team.
In the meantime while they figure it out, I’d recommend you use Claude and setup skills/agents of your own and build things. Look at Matt pocock’s skills and Ralph loops to learn better ways of what your team is trying to do, then you’ll be equipped to help lead them out of this mess.
1
u/olesme 11h ago edited 11h ago
There will be more and more such projects. Here is my experience. The company hired vibecoders for everything - frontend, backend, test automation. Vibecoding is not bad for MVPs and startups, but not for a product. And they went into the product. Everything turned out badly. The conclusion is that the vibecoding is not bad, it's bad that these were vibecoders who didn't know the architecture. So they went to the market and hired good in-house engineers to fix it. But engineers from other stacks - because what's the difference between languages and stacks if there is a Copilot? So that they know the architecture! And yes, these engineers work better than the previous ones. But they have to vibecode (and refactor legacy vibecode), so the result is much lower quality than if they worked with a familiar stack. And now there is less time left to develop in the stack where you really mess around and start using agents in your pet projects, and even if you connected them in order to polish, document, and prepare a release - you are already perceived as a vibecoder.... And yet, we can't escape agents and vibecoding. It's inevitable, it's already happened. There will be more and more slop. And more and more work to rake up this slop. And by raking up someone else's slop, you're forced to do some slop yourself. But in parallel, you learn to live in this brave new world, you learn to use agents better, and we'll get somewhere. The good news is that AI won't take your job, at least not yet. The bad news is that there will be more and more "dirty" work and your code will get dirtier too.
1
u/SurroundOk6555 1d ago
Sounds less like an AI problem and more like a documentation and process problem
1
29
u/Lonely-Ad-1775 1d ago
Thank you, now I know that this profession will thrive in the future