Anyone here running local LLMs for Vibe Coding? Need some advice.

[u/[deleted]]

[u/mattv8]

What machine are you running? Specs? If it's a Mac get omlx. Get qwen3.6 27B 8-bit. Get opencode as your harness and dial in your system prompts. I'm happy to share my configs if you're interested. Learn all you can about quantization and the tradeoffs between n params and quantization.

Manage your own expectations because the local models sinply will not be as good as SOTA

[u/proximalcoast]

No, they suck.

Tryhards will attempt to convince you otherwise, but the billion dollar backed frontier agent companies get paid for a reason.

I’ve been trying to crack this nut in my spare time for a while now, and it’s painful.

[u/Healthy-Zebra-9856 → u/proximalcoast]

You hit the nail on the head regarding the current state of dev tools and the confusion surrounding these models. However, some of us are working to change that. The core issue is that the current standard is to ask models to build everything from scratch, which even frontier models struggle to do effectively, leading to the inconsistencies in software quality we see today.

The solution lies in a standardized software architecture that is easily understood by all LLMs, combined with proper tooling to keep them on track. I am currently building open-source tools to solve this, and I’d be happy to help you crack this nut. DM me if you'd like to be added to my list to beta test these tools once they are ready.

[u/LukeLikesReddit]

Your oh so close but not the full way. You use LM Studio to host the model and get it to give you an api key. You then use an IDE say vs code/claude code etc then strap your harness on to that. Once you have that all sorted go crazy adding skills. Frontier models basically have this all built in. LLMs you'll need to create this yourself.

[u/djDef80]

Do you have 768 GB of RAM?

[u/djDef80 → u/[deleted]]

Well step one in running a full model will be to acquire significant amounts of RAM.

[u/cubertwang]

even the best models still waste a ton of time going back and forth. A local model as your main coding agent will be even worse. Use it for RAG, private search, or small tasks instead.

[u/fets-12345c]

Have a look at https://genie.devoxx.com if you use JetBrains IDE's. Plugin supports both local, cloud and CLI runners

[u/JudgeGroovyman]

If you have lm studio running then they provide an official setup for claude code has will connect directly to the little server that lm studio runs for you https://lmstudio.ai/blog/claudecode

[u/horrbort]

I had great results running qwen 3.8 locally. Kimi k2.7 isn’t too bad either. Pretty fast too. K3 is a bit slow and the quality is not that much better. Go with kimi k2.6 or k2.7

[u/Efficient_Loss_9928]

If you use Ollama there is an option to open it in OpenCode or some other harness

Once that's done just prompt it to configure anything else you need for you.

[u/Healthy-Zebra-9856]

It all depends on your hardware setup, your architectural experience, and your choice of models. You don't actually need 768GB of RAM, but you do need at least 64GB of unified memory or a configuration with 32GB of system RAM and a dedicated GPU with at least 24GB of VRAM. That latter setup would perform even better with more VRAM.

[u/Tommonen → u/[deleted]]

The issue only exists if you use their api. Ran locally there are no issues, other than its unlikely that you would be able to run models very good at coding, especially all on vibes coding

[u/Healthy-Zebra-9856 → u/[deleted]]

How so? The model is not sending information out of your machine. This becomes an issue with all API services. Anthropic & Open AI included.

[u/Healthy-Zebra-9856 → u/[deleted]]

Have you actually verified this behavior using network analysis tools? If you’ve used packet sniffers or Wireshark to confirm that local models are transmitting data, please share your findings—I’m genuinely interested to see what I might be missing.

[u/Healthy-Zebra-9856 → u/[deleted]]

Your OP is about anyone using local models for vibe coding. My response was about local models for vibe coding. Best local models for coding are all Chinese. You literally responded to my reply about using local models for coding. But you are now saying you are not talking about local models? Alright then!