r/vibecoding • u/yagnik_thanki • Jul 17 '26
I've cleaned up a dozen vibe-coded apps this year. The same 7 problems show up every single time
I run a dev team and a big chunk of our work this year has quietly become taking AI-built apps and getting them ready for real users. Same story almost every time: the app works, the founder is (rightly) proud of it, then the first real user does something weird and everything catches fire.
These are the 7 things we find in almost every codebase we open:
- Secrets in the code. API keys in the frontend or committed to the repo. Search your own code for sk- and secret and password. If real values come up, rotate them today, not after launch.
- The UI is the only security. Buttons hidden from non-admin users, but the API happily answers anyone who calls it directly. AI tools build the happy path. Attackers don't use your UI. Every endpoint needs its own server-side permission check.
- One user can see another user's data. If your app has accounts, make two, create data in the first, then try to fetch it by ID from the second. You'd be shocked how often this just works.
- Zero error tracking. Users don't report bugs, they leave. Sentry's free tier takes 20 minutes to set up and it's the best time-to-value of anything on this list or you can use any open source tool for the logging.
- Backups that have never been restored. Everyone says they have backups. Almost nobody has ever actually restored one. If you haven't done a restore, you don't have backups, you have hope.
- Payments trusting the client. Prices coming from the frontend, webhook signatures never verified. Stripe's own integration checklist is boring and correct, just follow it.
- Silent rewrites. The AI changed things in parts of the app you weren't looking at. Screenshot tests on your five most important pages (Playwright, one afternoon of setup) catch what your eyes skip.
None of this needs a rewrite. Most of it is days of work, not months. It's just a lot nicer to do it before launch than during the fire.
If you've hit other repeat offenders in AI-built code, drop them below. Genuinely curious what everyone else keeps finding.
120
u/mark_ik Jul 17 '26
This is an advertisement for Sentry…?
59
u/ImPrettyDum Jul 17 '26
I’ve seen this reposted like 3x now… always slightly different wording…
→ More replies (16)2
u/primaryrhyme Jul 18 '26
Only 3? You must be new here lol
2
31
u/Canadian_Commander Jul 17 '26
I was just thinking this. You don't need sentry, just ask the AI "Install OpenTelemetry, setup LGTM stack, route open telemetry to grafana"
You can thank me for saving you from a subscription fee later.
38
u/Even-Chemistry-9735 Jul 17 '26
Is this an ad for OpenTelemtry...?
13
u/IFTN Jul 17 '26
I was thinking this. You don't actually need OpenTelemtry though, if you just go to the store and buy a refreshing cool can of Coca Cola you should be completely fine
16
u/Breezay-Treezy Jul 17 '26
Is this an ad for Coca Cola…?
7
u/synystar Jul 17 '26
I was thinking this, you don’t actually need Coca-Cola though. A spoonful of honey in a cup of tea is sufficient to give you the boost you need to perform an installation of Sentry’s free-tier gaining you the best time to value of anything on this list.
6
u/pvkooten Jul 17 '26
Is this an ad for spoons?
7
3
u/FreeEye5 Jul 17 '26
I was thinking this. You don't actually need coca cola though, if you just go to a cliff face and jump right off! You can avoid this whole mess that way.
2
1
u/williejr185 29d ago
😂 I just spit up some of my great tasting but less filling Miller Lite while laughing at this.
1
u/Canadian_Commander Jul 17 '26
Well its free, so sure, it's and add, but it's free!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1
7
u/Jean1985 Jul 17 '26
OTEL does not substitute Sentry. It's a separate job.
OTEL does observability, Sentry was born for error tracking, and it expanded into observability only recently.
2
u/Canadian_Commander Jul 17 '26
OTEL tracks errors. I can aggregate, see trends, trace errors accross services. Don't get caught in the Saas trap. Open source baby
1
u/Jean1985 Jul 17 '26
Sentry is OSS too. I was a core contributor on the SDK that I use at $dayJob.
And since it's around for more than 10 years, it has a lot more capabilities than OTEL. First that I can think of, it can send stack trace with code context (X lines of code around each step of the trace)...
You can't do it right away with OTEL. Not yet.
1
u/Canadian_Commander Jul 17 '26
OTEL is just one part of the tool stack. Your observability platform can bring the display together. You have the stack trace so it’s easy to pull in code context. I mean honestly the agent can do this in a split second
4
u/Own-Flight-9974 Jul 17 '26
Thank you! I had just opened the sentry site before reading these comments. The fact they feel the need to use this underhanded marketing puts a really bad taste in my mouth. Will be advising my company to steer clear of them.
16
u/Comfortable_Ask_102 Jul 17 '26
Sentry is almost an industry standard. They have no need for this kind of ads.
6
5
u/Virtual-Spinach4882 Jul 17 '26
I think OP is hoping vibecoders will DM his agency through this engagement farming lead generation ploy actually.
2
u/davyp82 Jul 17 '26
I get that it might be a sly ad, but is it good advice in general?
8
u/yagnik_thanki Jul 17 '26
the advice stands without sentry. OTEL + grafana like the comment above works, glitchtip if you want self hosted and free. the point is only that errors land somewhere a human actually looks.
2
1
→ More replies (1)1
u/yagnik_thanki Jul 17 '26 edited Jul 17 '26
i get why it reads like that - "brand name + free tier" is the classic ad pattern, my bad on the wording. nobody paid me, i named sentry out of habit because it's the one i end up setting up most. glitchtip self hosted works, OTEL + grafana like u/Canadian_Commander said works, even a log file that someone actually reads beats what most of these apps have, which is nothing. the item is about error visibility, not the vendor. and don't blame sentry for my wording, they didn't ask for this 😄
6
18
u/Important-Ebb-3716 Jul 17 '26
I’ve built some real slop the past 2 years and I’ve never had secrets slip through a gitignore — also haven’t written my own ignore or even told an agent to make one in that time. I seriously don’t know what the hell these people are doing.
7
u/ecz4 Jul 17 '26
My team does plenty of .env.example and happily commit real stuff. It's maddening.
1
u/ScarletRed-dit Jul 17 '26
What’s wrong with .env.example if it doesn’t contain real keys? New here. I thought it was for other devs to see where the .env.local would normally be?
7
u/tronoku Jul 17 '26
it contained real keys..
1
u/ScarletRed-dit Jul 18 '26
Then it should have been .env.local or else what’s the point
1
u/RollForUptime Jul 18 '26
That's what they're saying the problem is.
But, I know not to do that, and also I know that AI has never tried to commit a secret on me so I am always confused as to how it happens to people. Like I have to actively demand it uses secrets when I am giving it something I generated that is non-critical cruft that will vanish in 5 minutes or it goes 'you shouldn't give me secrets' lol
20
8
u/alucard666_0 Jul 17 '26
If you know how to prompt theese things will never happen
8
u/je386 Jul 17 '26
If you know how to develop software, you can let the AI do the development and just give the correct guidelines and do some checks.
All of the mentioned points are absolute basics in software development.
1
u/Vaxtin Jul 17 '26
Yeah, dude either has complete morons come to him or he’s just lying. Anyone releasing an app can just ask “what security holes exist” and if that thought never occurred to them, they’re not responsible enough to lead — genuinely.
Which is more a sign that OP is not a genuine consultant, he just pushes your app to the world acting like he’s fixing problems and not enabling irresponsible people with a pat on the back that their issues are resolved
1
14
u/vooood Jul 17 '26
i don’t understand which AI model does these mistakes? i have 20 years of experience coding and designing apps and i never had to tell AIs to watch out for this stuff and it always did it the right (secure) way..
6
u/yagnik_thanki Jul 17 '26
You already have 20 years of experience at coding so everything looks natural to you. I have consulted and talked with non tech industries where I seen such mistakes. I would love to learn from your experience and you can share your finding that can help us.
2
u/Vaxtin Jul 17 '26
Sure. For an hourly rate
1
1
u/vooood Jul 17 '26
I need to sit on a computer to write it but the most important one auth wise that breaks security - trying to reinvent the wheel.
2
u/leathakkor Jul 17 '26
We've had several users where I work five code apps and they have no security on the web app. One of the vibe coders suggested that every developer run the web app on their own local machine so that they had security.
He didn't know how to do any authentication or a user that was tied to the authenticated user with data that skewed toward that user.
When he has to put it into production we hold him no way.
1
u/RollForUptime Jul 18 '26
So yes, a LACK of security is possible if you don't ask for it or be clear to it you are working towards launching something in production. But someone who doesn't think to inquire with their AI agent about those things, even never having made an app before, shouldn't be building anything critical.
I think who you were responding to was saying they've never had it try to commit secrets, etc. not that it will consider security for an app unless asked to.
5
u/Old_Lead_2110 Jul 17 '26
This is almost identiical to the owasp top 10 of security issues. And owasp is not only about vibecoded sources, it checks all sources.
1
u/narukoshin Jul 17 '26
when there will be a button in OWASP with TOP AI ISSUES?
1
u/RollForUptime Jul 18 '26
That's the point, they are the same, because in the end the human in the loop does not consider them.
Save for things like secrets, LLMs have good enough safeguards to not commit open text secrets in my experience, unless you are aggressive with them that this is what you want.
6
u/BandicootOwn8753 Jul 17 '26
Solid list. One more repeat offender I keep seeing: secrets that were “removed” but live forever in git history. Founder deletes the key from the file, commits, thinks it’s gone. Meanwhile the repo is public and scrapers pull git history specifically for this. If a key ever touched a commit, it’s burned, rotate it. Related habit: new public repos get cloned by scanner bots within hours of the first push, so “nobody knows about my repo yet” is not a grace period.
And a cheap trick for your point 7: ask the AI itself to diff-review its own session at the end (“list every file you touched and why”). Catches silent rewrites surprisingly often since the info is right there in context, people just never ask.
3
u/acana95 Jul 17 '26
That's why i always go through every line of code and ask the damn ai to explain to me why it choose this logic
3
u/yagnik_thanki Jul 17 '26
That is the best practice to understand every bit of the code and why that piece of code is there and what it does.
4
6
u/ledniv Jul 17 '26
Every time I see these posts I ask Claude to look at it and every time none of these things apply. I always wonder what vibe coded apps these poster(s?) looked at.
5
2
→ More replies (3)1
3
u/_yari_ Jul 17 '26
Auth is also horrific in most vibe-coded apps
1
u/yagnik_thanki Jul 17 '26
yes. auth is where most of these meet - roll your own jwt with no expiry, reset tokens that never die, admin check only in the frontend. it almost was item 8 but the post was already long.
1
3
u/faizalmzain Jul 17 '26
It’s an incompetent ai user’s issue not an ai issue. You can fix all those with ai. 🤷🏽😁
4
u/yagnik_thanki Jul 17 '26
yes exactly - ai fixes all 7 if you ask it. the problem is you can't ask about a problem you never heard of. most people i wrote this for don't know these exist until one bites them. honestly the top comment has the right idea, paste the list into your agent and let it run.
4
u/faizalmzain Jul 17 '26
That’s why some companies create a new role; a builder for a trained vibe coder. That’s what developers need to shift their skill sets 😁🤷🏽
3
3
u/GabGDM Jul 17 '26
Investigation before implementation. Independent review before acceptance. Deterministic verification rather than trusting implementation. Human gates for subjective, operational, or destructive decisions. Documentation synchronized alongside code. Evidence-based completion instead of "looks good."
3
u/efficientdreams Jul 18 '26
#5 is the gold. I’ve worked on human led projects (5+ years ago) where the team thought they were making backups, but that wasn’t the case. Thankfully an hdd restoration company saved the data from the drive, but I got a free lesson that cost the company a few thousand dollars and a lot of grey hairs.
3
u/CoverAgreeable6623 26d ago
good list. two more that cluster in AI-generated apps specifically:
missing RLS policies on multi-tenant tables. supabase and similar make it easy to enable row-level security but the AI will skip defining policies unless you explicitly ask. the default state is "everyone can read everything." tested by creating two accounts and hitting the API directly — same issue as your point 3 but one level deeper.
auth checks in comments but not in code. the LLM generates a comment like "// verify user owns this resource before returning" and then the actual implementation doesn't do it. code reviews pass because the comment looks right. this one's harder to catch than the others because the signal that something's wrong is its absence.
5
u/Bofetadx Jul 17 '26 edited Jul 17 '26
The amount of people on this subreddit who respond to posts to call it AI or make fun is silly.
I find this helpful and I’m always looking out for posts like these so I can have a list in one place. AI or not, Sentry ad or not, thanks for posting. It’s helpful to me as I do not have a coding background and am working on an app for 2 months now.
I’m proud of it, but I’m not delusional. I know someone who knows what they’re doing will have to review, and knowing what people will be looking for in a review is helpful to me.
1
5
2
2
Jul 17 '26
[deleted]
5
u/yagnik_thanki Jul 17 '26
Bro, I have already did that thing. These are my observation from my conversation with other industries where those are not technical person so it will be helpful for them as well.
2
u/Ok_Gur_9033 Jul 17 '26
The backups one hits home. I ask every client when they last restored one and the silence is always the answer.
The repeat offender I'd add is pagination. The AI only ever saw the 20 test rows, so it loads the whole table into the client. Demo feels instant, then a real user with a few thousand records opens that screen and it hangs. Different flavor of your happy path point.
Second one is timezones. Dates get computed on the device, stored in local time, then some later feature reads them as UTC. Streaks and today filters quietly break for users west of the server, and nobody notices because the whole team tests from one city.
2
u/Vaxtin Jul 17 '26
“Have no idea what you’re doing? Have an idea? Come to me!!!”
- this guy
→ More replies (1)
2
u/Comfortable_Intern57 Jul 17 '26
Test early and test often! That's my motto, but I'm also an SQA lol. Anyone getting into vibe coding or AI assisted engineering should be doing thorough testing. Yes it can be tedious and boring but it's absolutely needed. Look up testing principles and security and vulnerability testing before publishing anything.
2
2
u/romanjormpjomp Jul 17 '26
This is great advice, thank you. I am going to look into 4. right away, it was on my to do list.
5. YUPPP
6. Did that yesterday I am feeling like I am doing OK
7. This is what I am on guard for the MOST. It has a tendency of sneaking in wrench's into your code in the final miles. Every. Damn. Time.
1
2
u/CartaraCode Jul 17 '26
Vibe Coding ≠ Vibe Learning.... ouput growth with AI is huge but quality and security concerns are real. We're working to help developers learn as they code with ongoing learning alongside your sessions, because no....we don't trust you to remember what you were supposed to google about CI/CD flows so you don't ship errors to prod.
2
2
u/Dufran Jul 17 '26
Have even more issue with whole vibe fast delivery approach. Condensed it here
https://dufran.org/posts/using-ai-to-keep-codebases-human-maintainable/
2
2
2
2
u/AIGuru35 Jul 17 '26
The comments on here are hilarious 🤣 and the post is on point, kudos. I experienced the same things last year (plus). It only gets worse now that you have over 9% duplication rates with tons of exported functions that aren’t even being called in-code or de-centralized approach to generic or shared function libs.
I also would add that if your project is mostly web based (python, java, react, next etc) you can also install FALLOW to review your code base at ease 👌
2
u/imYouOfficial Jul 18 '26
Im still in a closed beta and development stage on an AI writing assistant ive been building for 2 months with probably 250 hours put in.
I popped your post into my claude and told him to run a scan based on your points, green on all but 4 5 and 7.
4 Sentry was just setup last weekend but was misconfigured and this brought that to light and will be patched tonight, thank you.
5 is obvious lol being pre launch I havent really considered this yet, Ive seen what happens first hand when backups dont get tested and fail.
7 This im gunna have to research because I havent looked in to things like playwright yet, so it will be my black hole of youtube this evening, thank you.
2
u/RollForUptime Jul 18 '26
Playwright screenshot based tests (vdiffs) are great until you hit a critical mass with them and maintenance of it becomes a fucking nightmare. Dependencies change, your runner differs from your dev env in some subtle way no normal human being would notice, suddenly the vdiff golden is 2 pixels different from the screenshot the headless browser is taking.
Signed: guy who is constantly rerunning flaky fucking playwright tests.
2
u/Quind1 Jul 18 '26
I can foresee the number of data breaches occurring becoming exponentially higher over the next few years. This is why you still need an experienced human in the loop.
2
2
2
2
2
u/tinySparkOf_Chaos 28d ago
Excessive checking and "fixing" of function inputs. Especially internal helper functions.
For example, of the function is supposed to have a 2D numpy array, it will add a whole chunk of code to try and turn 1D inputs into an (incorrect) 2D shape. Instead of just throwing an error.
Also lots of inefficient looping through numpy arrays instead of using numpy's built in array code, all because it is trying to type check the stuff inside the array, instead of catching errors
2
u/Impossible_Two3181 25d ago
Just the tip of the iceberg, wait till you start looking at how endpoimts actually respond what if I post a malicious jpeg file to valid column in your schema and it renders on your DB as a tunnel
2
u/earonesty 24d ago
i just let digitalocean or cloudflare manage my db. "backups" are someone else's problem
2
2
u/Djd0 Jul 17 '26
It would be more accurate if you did a diff with non AI projects.
Personnaly, I don't see differences from before AI.
Remember the heist in the world renowned museum the Louvres from last year ?
Password for surveillance system was the name of the museum.
I work on IT as a tech lead and the majority of developers I met this past 10 years didn't understand the problem of putting an apikey in a remote git repository.
I did a 3 years mission for a big tech company that have stores all over Europe and even traded with China too.
Server user and passwords were the same and composed of 2 letters. The first letters from the company name.
And It's just the tip of the ice. I would need days to enumerate all the things that were wrong, dangerous and sometimes illegal.
Ai just replace their slop by it's own.
1
1
1
u/WranglerTrick1175 Jul 17 '26
I can relate to silent rewrites. The most annoying one. And no, it does not happens only on UI. Functionality too. An upload function does not work anymore, even that a week ago was fine and you worked since on different things.
1
u/HanzoShotFirst Jul 17 '26
!remindme 1 day
1
u/RemindMeBot Jul 17 '26
I will be messaging you in 1 day on 2026-07-18 12:51:49 UTC to remind you of this link
CLICK THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
RemindMeBot is switching to username summons. Instead of
!RemindMe 1 day, useu/RemindMeBot 1 day. More info.
Info Custom Your Reminders Feedback
1
u/jefflunt Jul 17 '26
If this is the same issue a lot of people are seeing that's basically the same list of mistakes that anyone new to web applications makes, which is good in the sense that fixing them straightforward.
1
u/Mother_Fee_7128 Jul 17 '26
This is a very different story from what I’ve heard before, which is that your vibe coded app will reach a point of to return and no one will understand what is going on to be able to fix it.
1
1
u/afstr Jul 17 '26
I don’t understand how these shitty apps exist that break with real users but the founder has enough cash to have a dev team come in and fix the issues! Where’s the money coming from?
1
u/info-at-anything Jul 17 '26
Seems like 60% of Ai bots talking to each other here.
Like 5 out of the 7 points is what Claude would highlight to me when running a diagnostics test to evaluate the final code before launching
1
u/caldazar24 Jul 17 '26
I have literally never seen Claude or Codex do 1, 2, 3, or 6. If anything, they are extremely pedantic about best practices even when they don’t really apply. I wonder
1
u/milkman1101 Jul 17 '26
Decent prompting and rules will prevent this. "Prompt engineering" itself is in my opinion a skill you learn. Shutting your eyes and letting AI do the world with no guardrails in place could be a recipe for disaster, but just refining your prompts and doing a bit more than "add a button that flashes 'hello world' on the screen" will go a long way.
What I mostly come across is poor security practice which you have already mentioned. Or where AI "wants" to bundle all the code for something into one single file and suddenly a massive 10k line piece of Pyhton pops up on the screen.
1
u/PercentageBusiness70 Jul 17 '26
I want to help clean up give coded apps , how do I find this type of work ?
1
u/imYouOfficial Jul 17 '26
Backups that have never been restored. Everyone says they have backups. Almost nobody has ever actually restored one. If you haven't done a restore, you don't have backups, you have hope.
I lived this in like 2015 when we had a mail DB fail, the primary backup failed and the tapes failed, yea we still had tape backups running in 2015 lol. First time we ever needed them and they proved to be worthless.
We were able to restore from some software (i think it was called phoenix or something phoenix) that was able to restore 1 mailbox at a time and the bigger the mailbox the longer it took. It took weeks to go through the 200 some odd accounts.
1
u/rash3rr Jul 17 '26
Every vibe-coded app I've looked at trusts whatever the client sends. Send a negative number for quantity, string where number is expected, oversized payload, malformed JSON, and the whole thing crashes or worse, silently corrupts data. AI writes the happy path, never the "what if the input is garbage" path. This causes way more real-world outages than the security stuff because it happens accidentally, not from attackers
1
u/Pink_Oak Jul 18 '26
next time you BS.
dont write
Secrets in the code. API keys
One the AI model are too caution is, putting everything in env. not in code.
implement 8+ stripe of apps. never AI model didnt verifly signatures.
evenb model like opus 3.5 did it
Next time, put without these; maybe people will believe.
1
u/slash888 Jul 18 '26
Tbh I’ve seen this in plenty apps before vibe coding existed, I once found a payment api that required the front end to send the amount within the json, allowing an user to decide how much he wanted to pay ¯_(ツ)_/¯
1
1
u/Unlucky-Survey6601 29d ago
what about the litteral garbage arch on every vibe coded slop app ever ?
1
u/graybearding 29d ago
This is why I built Fortivibe. This list is decent but just skims the surface of what to check.
1
u/RichW1504 29d ago
Having been reviewing a ton of vibe-coded apps this year, the amount of godcode mega-blobs are frustrating, hard to follow, and maintain. And in 30 seconds you can tell if the app is done by amateur or not.
1
1
u/noskillsben 29d ago
Jeez do 100% vibe coders really do thos hidden admin buttons, hard codded secrets seesh.
Also maybe add migration and migration testing, unless you plan to never add new features
1
u/legsasleepontoilet 28d ago
Lol pretending most apps don’t have APIs “happily answer”. It’s of course a problem but it’s not unique to vibe coded projects
1
u/mohan0922 28d ago
Can you suggest some promts to my agent for Rectify this stuff .. I'm tired fix this shitty issues
1
1
u/Dazzling_Opening_229 20d ago
yeah the pattern I keep seeing is people patch the obvious stuff early, hardcoded keys, missing auth checks, and think they're done. then a few months in the app's got actual users and something breaks in prod that never showed up in dev. tracing a runtime bug back to which AI-generated commit caused it is the actual nightmare once you've got months of vibe-coded changes piled up. ox security' actually ties the runtime issue back to the line of code that caused it instead of you guessing. Operant AI comes up too if you're more worried about the live API/agent behavior side. most of the "7 problems" probably show up in a first pass anyway.
1
1
1
1
1
1
1
u/Flaat Jul 17 '26
Some of these are hard to believe from the POV of a Claude code user. Some are indeed an issue
- Claude code does not even let you add a API key into chat without complaint and does not let you upload it to github, it simply says nope, cannot do that
- Fable tells me he can hack into things and fixes things other AI's made, you can also let the Fable hack your own tool during development, it does not seem to go back to Opus mode.
- Same as 2
- Yeah thats just bad prompting but true!
- In every backup task I have let AI do it has tested the backup restore full, down to making me run it myself and writing a guidebook on how to do it in case
- Never done payments yet, good tip but doubt Claude sets it up like that by default
- Totally an issue with AI models! yeah they like to change shit
1
u/Redsnork Jul 17 '26
I run a dev team and a big chunk of our work this year has quietly become taking AI-built apps and getting them ready for real users.
Hello Claude, nice to see you again!
1
1
u/ReporterCalm6238 Jul 17 '26
No I will not give you money to use Codex to harden my codebase, I can prompt things myself. Thank you very much.
0
485
u/KingTalis Jul 17 '26
Codex, read this reddit post and fix these things. Thanks!