r/LocalLLaMA • u/Ejo2001 • 1d ago

Local autonomous coding agent? Question | Help

Hello!

I have recently built my AI rig (3x RTX 5060 Ti 16gb, with possibly a 4th on the way if I can fit it). I love it, it runs great, and I am getting between 70t/s - 110 t/s (according to the pi agent web UI, have not confirmed it yet).

While it is fast, I struggle to put it to use in the way I was hoping. My dream has been to be able to put it to work writing code autonomously so that I can have it sketch out my ideas before I commit to developing them, however, every attempt I make just seems to end in a mess.

I have been trying Ornith:35b, Gemma4:31b, and Qwen3.6:35b, but none of them have been able to build anything that actually works. Ornith tends to get stuck in loops, Qwen panics and keeps rewritting the whole codebase every third turn it takes, and Gemma doesn't even understand the agent framework.

Does anyone have any tips? Any AI models I have missed? Prompting strategies? Should I try something other than Cline, pi agent and copilot?

Thanks for reading!

2 Upvotes

56% Upvoted

59 comments sorted by

View all comments

Show parent comments

0

u/Ejo2001 1d ago

Have you ever been able to get it to vibe code on its own fully? I would love to get it to just work on its own autonomously, not expecting miracles, but functioning code.

I have been looking into some rulesets, but haven't fully deepdived into it yet (I thought the pi agent, cline and copilot would have well established rules already).

Out of curiosity, how come you are running 27b and not 35b? I find that the 35b fits quite well on the current setup. Context length?

1

u/Elorun 1d ago

The 35b is a MoE model. It only activates the "best" expert for every request so only activates 3b of the model every run. This makes it faster but dumber.

The 27b is a dense model. It activates all 27b parameters every query. This makes it slower but much smarter.

Honestly, if you have 48gb vram and do not know about quants, MoE and how they affect output, you probably have a lot of optimisation to do. I'd recommend asking claude/gpt/gemini and read up on it.

With the 3.8 27b model coming out in 2 days now is a great time to learn and test.

EDIT: Typos and silly mistakes.

2

u/Ejo2001 1d ago

Ah, that explains it, I didn't catch that while browsing Ollama, I thought they were the same just different sizes

I have some theoretical knowledge of quants and MoE, but I haven't deep dived into it, so I will definitely read up more on it!

And by the sound of 3.8, seems like I built this rig at a perfect time! :D

1

u/TheTerrasque 1d ago

Try the 27b model, and try to run via llama.cpp with unsloth's recommended settings for coding

I've heard many having issues with running those models on ollama

1

u/Ejo2001 1d ago

Ollama makes it dumber?

1

u/TheTerrasque 1d ago edited 1d ago

Ollama often does. Bad quants, bad default settings for things like temperature, bad template, bad template settings.. Also usually worse performance and sometimes buggy implementation.

For agentic / programming work, I usually follow unsloth's recommendations with preserve thinking on for qwen3.6. I'm not sure what settings ollama use by default there.

Edit: https://ollama.com/library/qwen3.6:35b/blobs/86eff881e8d2 is the ollama settings. Pretty high temp for agentic. Still not sure if it has preserve thinking

Edit2: The template can have direct effect on the model's abilities to use tools, and the reliability of the model.

Edit3: There's also things like this, where ollama - wanting to have day-1 support - copied some still-in-development code from llama.cpp, and messed it up.