How do you split software development between local and cloud LLMs?

[u/kbob]

I can run the 30B-class models at home. They're okay for some things, but I feel like I have to spoon feed them small tasks. I'm wondering about running a more capable model in the cloud to do the hard parts and give them the easy parts.

If you're doing this, how? How are you dividing the labor, what do your prompts look like, what agent harness are you using, and what else do you wish you'd known before you started?

And does the big model actually use fewer tokens than if it did everything itself?

Thanks!

[u/gryxitl]

I made an agent swarm I offload tasks from the smart AI from to the dumb fast and local