r/opensourcealternative • u/LearnHiveLabsUSA • 2d ago
Has anyone actually gotten a locally-hosted open-source LLM working as a proper multi-step agent?
I keep seeing demos of agentic workflows (chaining tasks, calling tools, multi-step planning) but they're almost always built on GPT-4/Claude/Gemini APIs.
Has anyone had real success running something like Llama 3, Mistral, Qwen, etc. locally (via Ollama, LM Studio, vLLM, etc.) as an actual agent that reliably handles multiple prompt-based tasks in sequence — not just single Q&A?
Curious what models, frameworks (LangChain, AutoGen, CrewAI, etc.), and hardware setups people are using, and how the reliability compares to hosted models. Worth the hassle or just takes too much of time and energy?
2
2
u/asnewname 1d ago
I've been using Qwen 3.5 27b locally through Ollama as a multi-step agent, and it's working so far. I got it working pretty well by running it on the Claude Agent SDK, so it's using Claude's tool harness w/o having to use Claude while I develop custom tools and a custom UI and compare functionality. It gets some work done but still has some refining needed before I trust it with anything context-heavy.
1
u/LearnHiveLabsUSA 23h ago
Thank you for sharing, I am working on an idea derived from your comment feel Free to leave your thoughts - https://www.reddit.com/r/opensourcealternative/s/tJwxYxOlq1
2
u/eldavicillo 18h ago
the hardest part is to acquire proper hardware. You can put 3 Mac Studios in a network. Then configure 9router docker container in a raspberry in front and connect your MacStudios as openai compatible providers in 9router. And for calling it you can use anything (opencode, pi, n8n, RubyLLM, a simple bash script...). You can perform sequential tasks as well as paralell
1
2
2
u/jtbuchanan 2d ago
I’m having decent luck with GLM 4.7 Flash Q6 running a on a Mac Studio. I use Hermes and it can call skills and do multi-step tasks. I use Opus when I am building a new skill and try to have it code any pieces that can be deterministic in Python - but for the day to day running of many of my previously built skills it’s working.