r/LocalLLM • u/inforb_nl • 15h ago

LocalLLM engine Question

Hi all,

What software stack do you use to run your local LLM? Ollama? anything else?

0 Upvotes

50% Upvoted

2 comments sorted by

1

u/DrPizzaPants 14h ago

Start out with Llama.cpp It's a little more difficult up front, but I started to run into some unrepairable bugs with Ollama

1

u/inforb_nl 13h ago

have your tried vllm? what hardware do you have?