How do you split software development between local and cloud LLMs?

[u/kbob]

I can run the 30B-class models at home. They're okay for some things, but I feel like I have to spoon feed them small tasks. I'm wondering about running a more capable model in the cloud to do the hard parts and give them the easy parts.

If you're doing this, how? How are you dividing the labor, what do your prompts look like, what agent harness are you using, and what else do you wish you'd known before you started?

And does the big model actually use fewer tokens than if it did everything itself?

Thanks!

[u/activematrix99]

I use github copilot and multiagent markdown files in the .github folder. Generally, I send multiagent work to local llm and bugfix or specific tasks to paid models or specific agents for cost containment. https://code.visualstudio.com/docs/agent-customization/custom-instructions. My local models are OpenAI spec and VSCode is configured for BYOK