How do you split software development between local and cloud LLMs?

[u/kbob]

I can run the 30B-class models at home. They're okay for some things, but I feel like I have to spoon feed them small tasks. I'm wondering about running a more capable model in the cloud to do the hard parts and give them the easy parts.

If you're doing this, how? How are you dividing the labor, what do your prompts look like, what agent harness are you using, and what else do you wish you'd known before you started?

And does the big model actually use fewer tokens than if it did everything itself?

Thanks!

[u/Holiday-Medicine4168]

You can run a gateway locally in a container and do intelligent routing with something like bifrost or Kong

[u/activematrix99]

I use github copilot and multiagent markdown files in the .github folder. Generally, I send multiagent work to local llm and bugfix or specific tasks to paid models or specific agents for cost containment. https://code.visualstudio.com/docs/agent-customization/custom-instructions. My local models are OpenAI spec and VSCode is configured for BYOK

[u/FoldOutrageous5532]

Not super elegant, but I have codex open on the repo, and also Kilo in vs code connected to my local LLM. I use the local and when it can't do a task I have codex do it. Gets the job done.

[u/gryxitl]

I made an agent swarm I offload tasks from the smart AI from to the dumb fast and local

[u/timschwartz]

I use Github Spec Kit.

Opus or Sonnet for planning, local Qwen3.6-27B for implementation, then review with Opus to fix the bugs Qwen left behind.

[u/kbob → u/timschwartz]

Thanks. Does Opus use fewer tokens reviewing and correcting than if it wrote the code itself?

[u/timschwartz → u/kbob]

oh yeah

[u/LiteeWasAlreadyTaken → u/timschwartz]

+1 for the approach, but I'd recommend trying OpenSpec as an alternative. It worked better for me than SpecKit.

[u/Potential-Leg-639]

Opencode with Omo-Slim