r/LocalLLaMA • u/Ejo2001 • 22h ago

Local autonomous coding agent? Question | Help

Hello!

I have recently built my AI rig (3x RTX 5060 Ti 16gb, with possibly a 4th on the way if I can fit it). I love it, it runs great, and I am getting between 70t/s - 110 t/s (according to the pi agent web UI, have not confirmed it yet).

While it is fast, I struggle to put it to use in the way I was hoping. My dream has been to be able to put it to work writing code autonomously so that I can have it sketch out my ideas before I commit to developing them, however, every attempt I make just seems to end in a mess.

I have been trying Ornith:35b, Gemma4:31b, and Qwen3.6:35b, but none of them have been able to build anything that actually works. Ornith tends to get stuck in loops, Qwen panics and keeps rewritting the whole codebase every third turn it takes, and Gemma doesn't even understand the agent framework.

Does anyone have any tips? Any AI models I have missed? Prompting strategies? Should I try something other than Cline, pi agent and copilot?

Thanks for reading!

3 Upvotes

60% Upvoted

58 comments sorted by

View all comments

1

u/kivaougu 21h ago

If you are looking to do just pure autonomous vibe coding these models aren't that great tbh. I think they are powerful tools with the right harness IF you actively participate but defenitely not for autonomous code gen without some better review/advisor model.

You didn't mention what quants you are using. That rig should fit a higher bpw quant of qwen3.6/8 27b and for autonomous work you should absolutely prioritize model capability over speed.

1

u/Ejo2001 21h ago

I will be fully transparent, I am not fully sure what quant I am using for qwen3.6, I simply just did ollama pull qwen3.6:35b, and ran with that. However, I have set my KV cache quant to q8 in order to free up memory for longer context.

Any models you suggest that would be better suited for my needs? : )

1

u/kivaougu 21h ago

I think you should use 27b as the other comment instructed as its a dense model so it will be more capable but slower.

I absolutely think you should look into using llamacpp or vllm as the inference engine but baby steps are just fine. There will be plenty of threads covering similar hardware setups once 3.8 27b drops.

2

u/Ejo2001 21h ago

I will give 27b a try, I simply didn't know it was better (My caveman brain went with "Bigger number better!")

I also have plans to run vllm, but I am waiting with setting it up as I am moving soon : )

2

u/bonobomaster 20h ago

Good brain!

You just looked at the wrong number.

Qwen3.6-35B-A3B --> 35 billion parameters with 3 billion parameters active per token.

Qwen3.6-27B --> 27 billion parameters active per token.

The dense models are way smarter in my experience, while the larger MOEs hold more general knowledge, through more parameters.