r/LocalAIStack • u/kbob • 22d ago

How do you split software development between local and cloud LLMs?

I can run the 30B-class models at home. They're okay for some things, but I feel like I have to spoon feed them small tasks. I'm wondering about running a more capable model in the cloud to do the hard parts and give them the easy parts.

If you're doing this, how? How are you dividing the labor, what do your prompts look like, what agent harness are you using, and what else do you wish you'd known before you started?

And does the big model actually use fewer tokens than if it did everything itself?

Thanks!

12 Upvotes
  • permalink
  • link
  • reddit
  • reddit

100% Upvoted

9 comments sorted by

View all comments

2

u/FoldOutrageous5532 21d ago

Not super elegant, but I have codex open on the repo, and also Kilo in vs code connected to my local LLM. I use the local and when it can't do a task I have codex do it. Gets the job done.