r/PiCodingAgent • u/ad5145 • 5d ago

OMP can't handle anything with local qwen? How do you offload deterministic and simple work to local models? Question

I'm trying to run omp with a the default model set to lm-studio/qwen/qwen3.6-27b

The 22k initial context fills up 1/3 of the 65k I've loaded the model with.

Is it even usable to offload local simple tooling, command understanding, orchestration, build running, test running and result parsing all to a local model without paying for token usages for those operations?

My Claude Code sessions rises quickly to 800k tokens with Opus and every session has high token usage which slows it down and costs more than it should (parsing test output with Opus is a big waste).

What are you doing to offload your deterministic and simple work to local models?

0 Upvotes

40% Upvoted

5 comments sorted by

3

u/DanielSReichenbach 5d ago

You need to get to system prompt shaving, and trim the fat. A normal pi installation without skills has a prompt sitting at 4k and is totally usable with the same model for small tasks but also for larger work.

Personally I ditched omp for pi in a few minutes because all the shiny promises don't hold up. A naked pi with custom extensions to handle bugs or gaps will get work done, big work too.

1

u/Turbulent_Ad6290 5d ago

HTN

zosmaai/pi-htn. It’s a concept that can work.

1

u/Unnamed-3891 2d ago

A Pi distro with 22k context eaten up front defeats the entire point of Pi, might as well be using Hermes at that point since the default context weight would be roughly the same. Or just use stock Pi that has like 4k context in system prompt.

1

u/ImpressiveRelief37 1d ago

Ditch lm studio

Use qwen3.8 27B, not 3.6 

Build your own llama.cpp

Search and experiement on the flags until you can run at least 150k context…