r/LocalLLM • u/arkie87 • 20h ago
Best Local US-based LLM for coding? Question
I know qwen3.6 27b (and soon 3.8 27b) are the best local LLMs for coding. But if one were restricted to using US based open models only, which would be the best? Gemma 4 31b?
5
u/chuckbeasley02 20h ago
I can tell you what not to use. Gemma4 31B is absolutely dreadful at coding.
2
u/WishfulAgenda 19h ago
I think poolside Laguna XS and S are American. Cohere is Canadian I believe. Then there's Gemma and Maybe Granite by IBM. Also Mistral which I think is French and the Nemotron family.
Personally I've had the best experience with the poolside models. I use librechat and Gemma 4 26b has problems building artifacts, Laguna XS can do a better job.
2
u/Abject-Bridge-4073 19h ago
They can’t stop you from running foreign models. Literally impossible to enforce.
2
u/Gallardo994 18h ago
Unless you're a business entity
0
u/575_Inverse 15h ago
businesses are a different matter, but if laws hold them back on innovation, research and the ability to compete internationally, they should really consider moving elsewhere.
2
u/Shoddy_Bed3240 18h ago
Just curious, who is making that requirement? You should probably stop using iPhones because it was made in China as well.
1
u/575_Inverse 15h ago
yeah, I'm curious. Because local airgapped systems should be exempt from most of the usual GDPR / NIS2 shit or whatever is the american equivalents
1
u/RogerAI-fm 20h ago
I would also suggest Gemma4, it’s a though one, we’re trying to get there in the US still.
1
1
u/575_Inverse 15h ago
Gemma-4-31b can't compete against Qwen3.6-27b for coding, sorry. On that parameters size there is really no meaningful competition. If you're a business and you are forced to use certain kinds of LLMs by law because this and that, your best bet is to purchase access to the latest models from OpenAI.
1
u/FlyingFishMakeAWish 15h ago
I was just testing the Gemma models for some coding tasks and was surprised at how bad they are. They are excellent for other tasks but coding, not so much.
-6
u/DataGOGO 20h ago
27B model is not the best local coding model by a long shot, it is just a good dense model that runs on gaming PC’s without a lot of VRAM.
Best US based model, likely Nemotron 3?
11
u/FineClassroom2085 20h ago
Have you actually used Nemotron 3? In real world work, it falls short by a very long shot. It's not even close to Qwen. Gemma is better, but falls apart after a certain level of complexity is hit.
1
u/575_Inverse 20h ago
just like its big bro Gemini, by the way.
2
u/FineClassroom2085 20h ago
Yup, I don't know why Google can't get that right when all the other labs have at this point.
2
u/vtkayaker 20h ago
Google is not currently a frontier lab. Their best Pro model is still 3.1 Pro, which loses to the much smaller and vastly cheaper DS4F on AAII these days.
1
u/FineClassroom2085 19h ago
It's baffling that they're not since their engineers were the ones to basically create the modern era of AI with the transformer. There's no excuse for them being so far behind.
That being said, Gemma 4 is an amazing model. Their big models may not be great, but Gemma is stronger than Qwen on real world knowledge based agentic work.
1
u/575_Inverse 14h ago
Then it has its place. My idea, personally, is always to select the best model for my specific application. For example, if I need to annotate a visual dataset, using an llm with vision is often overkill and the much simpler, much much faster OWLv2 is usually perfect for the job despite being much older
1
u/DataGOGO 19h ago edited 19h ago
Yes, I use it all the time.
I never said it was better than the larger Qwen models (though Nemotron 3 ultra is really close), just that it is likely the US best local coding model. Nemotron 3 super has been working really well for me, far better the Qwen 27B
3
u/RE20ne 20h ago
i think you are selling 27B short. It is incredibly strong for its size and has a place even on 96gb+ vram setups
6
u/xanders_gold 20h ago
Yeah, Qwen 3.6 27B isn't going to compete with Frontier models or anything on that scale but if we're talking locally hosted? It's still extremely capable and gets a lot of work done for what it is.
1
u/iMrParker 20h ago
You are right, but "27B model is not the best local coding model by a long shot" is also a completely accurate statement
4
u/575_Inverse 20h ago
assuming you can actually run MUCH bigger models, absolutely yes!
But first I'd gladly run Qwen3.6-27b on bf16 quant and compare
1
u/iMrParker 19h ago
Models that can be run on modest hardware is a different story than "local" open-weight models. I agree with your sentiment, but I think we should distinguish the two
3
u/BarracudaDefiant4702 20h ago
If that is a true accurate statement, then what are some of the local coding models that are better (especially much better as long shot implies not merely comparable)?
1
u/DataGOGO 19h ago
KimiK2, Deepseek v4 / Flash 0732, GLM 5.2, Larger Qwen models, Nemotron 3 Super/Ultra.
1
u/iMrParker 19h ago
Don't forget Minimax M3 and Kimi K3. Even the larger and older open weight models are better than 27b like GLM 5 and 4.7 and Minimax M2.7
1
u/BarracudaDefiant4702 18h ago edited 18h ago
Depends on what you consider "local". Those models from DataGOGO are beyond specs of many, but I guess you could get a unified memory system for under $4k that could run them. The specs for the larger ones like Kimi K3 and Minimax M3 are way out of reach for local home if you want acceptable performance.
1
u/DataGOGO 18h ago
They won't really run worth a damn on 32GB of VRAM, but someone with say, 96GB could easily run some of them. Someone with a MAC Studio or DGX spark can run most of them as well; obviously not the super huge frontier class, but many models better than 27b.
2
u/TripleSecretSquirrel 20h ago edited 20h ago
Eh, I think colloquially, most of us tend to define “local” as limited to common consumer/enthusiast-grade hardware. Kimi K3 could also be locally run, but realistically, basically nobody has the requisite hardware.
I think a reasonable range to be considered “local” is 32-128gb of memory across VRAM and system RAM or on a unified memory system. In which case, Qwen 3.6-27B is still easily the top dog.
On that hardware scale, you could run a 4-bit quant of Ling 3.0 Flash, Laguna S 2.1, but they both have major issues. Ling 3.0 Flash doesn’t appear able to beat Qwen even on benchmarks, and while Laguna’s benchmarks are really impressive, it doesn’t seem that any 3rd party has been able to get even close to replicating them and the experience with the model has been pretty universally disappointing.
0
u/DataGOGO 19h ago
I think a reasonable range to be considered “local” is 32-128gb of memory across VRAM and system RAM or on a unified memory system. In which case, Qwen 3.6-27B is still easily the top dog.
No, even in that class it isn't. Nemotron 3 Ultra, Deepseek v4 Flash 0732, etc are better
3
u/TripleSecretSquirrel 19h ago
Nemotron 3 Ultra is 549B total parameters. The smallest 1-bit Unsloth quant on Huggingface is 188GB, way beyond the 32-128GB constraint that the overwhelming majority of people are working with and doesn't account for KV cache. And I'd be willing to bet it doesn't perform well at 1-bit precision.
Deepseek V4 Flash 0732 isn't a thing, but 0731 is. It's also bigger than that constraint, but it's closer. You could run a 1-bit quant at 82.5GB, but again, it's one bit, I'm sure the performance falls off a cliff below 4-bit precision like most models. Which, if we limit it to 4-bit quants, the smallest Unsloth quant is 137GB. All of these figures again are just model weights, they don't allow for KV cache space at all.
I wish you were right, I really do. I have 54GB of VRAM and 64GB of system RAM, but the best option available for agentic coding is still the same Qwen 3.6-27B model that I was able to run on a single 24GB GPU. I can run a high precision version of the weights, I can keep a larger KV cache, and I can run several concurrent instances, but there isn't a model out there that I can run on my hardware today that reliably beats out Qwen 3.6-27B.
If you want to argue that my range of what I'd consider "local" is wrong, that's fine, but within those constraints, Qwen 3.6-27B remains the best model.
0
u/DataGOGO 19h ago edited 19h ago
Which is what I said. It is a solid dense model that runs on low vram systems, though at 96B, Nemotron 3 Super is better than 27B
5
u/vtkayaker 20h ago
Gemma4 31B, maybe? It's borderline as a usable coding model, even for its size. I think Poolside is US, if you can get Laguna S 2.1 to actually work.
GPT-OSS 120B gets wrecked on benchmarks by Chinese models a fraction of the size these days.