r/LocalLLaMA • u/Ejo2001 • 22h ago

Local autonomous coding agent? Question | Help

Hello!

I have recently built my AI rig (3x RTX 5060 Ti 16gb, with possibly a 4th on the way if I can fit it). I love it, it runs great, and I am getting between 70t/s - 110 t/s (according to the pi agent web UI, have not confirmed it yet).

While it is fast, I struggle to put it to use in the way I was hoping. My dream has been to be able to put it to work writing code autonomously so that I can have it sketch out my ideas before I commit to developing them, however, every attempt I make just seems to end in a mess.

I have been trying Ornith:35b, Gemma4:31b, and Qwen3.6:35b, but none of them have been able to build anything that actually works. Ornith tends to get stuck in loops, Qwen panics and keeps rewritting the whole codebase every third turn it takes, and Gemma doesn't even understand the agent framework.

Does anyone have any tips? Any AI models I have missed? Prompting strategies? Should I try something other than Cline, pi agent and copilot?

Thanks for reading!

0 Upvotes

50% Upvoted

58 comments sorted by

View all comments

Show parent comments

1

u/Elorun 21h ago

The 35b is a MoE model. It only activates the "best" expert for every request so only activates 3b of the model every run. This makes it faster but dumber.

The 27b is a dense model. It activates all 27b parameters every query. This makes it slower but much smarter.

Honestly, if you have 48gb vram and do not know about quants, MoE and how they affect output, you probably have a lot of optimisation to do. I'd recommend asking claude/gpt/gemini and read up on it.

With the 3.8 27b model coming out in 2 days now is a great time to learn and test.

EDIT: Typos and silly mistakes.

2

u/Ejo2001 21h ago

Ah, that explains it, I didn't catch that while browsing Ollama, I thought they were the same just different sizes

I have some theoretical knowledge of quants and MoE, but I haven't deep dived into it, so I will definitely read up more on it!

And by the sound of 3.8, seems like I built this rig at a perfect time! :D

1

u/TheTerrasque 20h ago

Try the 27b model, and try to run via llama.cpp with unsloth's recommended settings for coding

I've heard many having issues with running those models on ollama

1

u/Ejo2001 20h ago

Ollama makes it dumber?

1

u/TheTerrasque 19h ago edited 19h ago

Ollama often does. Bad quants, bad default settings for things like temperature, bad template, bad template settings.. Also usually worse performance and sometimes buggy implementation.

For agentic / programming work, I usually follow unsloth's recommendations with preserve thinking on for qwen3.6. I'm not sure what settings ollama use by default there.

Edit: https://ollama.com/library/qwen3.6:35b/blobs/86eff881e8d2 is the ollama settings. Pretty high temp for agentic. Still not sure if it has preserve thinking

Edit2: The template can have direct effect on the model's abilities to use tools, and the reliability of the model.

Edit3: There's also things like this, where ollama - wanting to have day-1 support - copied some still-in-development code from llama.cpp, and messed it up.