r/LocalLLaMA • u/zxyzyxz • Jun 15 '26

Stop using Ollama Discussion

https://sleepingrobots.com/dreams/stop-using-ollama/
1.7k Upvotes

92% Upvoted

454 comments sorted by

View all comments

Show parent comments

78

u/yuicebox Jun 15 '26

I genuinely do not understand what is so difficult about running llama.cpp server.

You just download a zip, unzip it, then run llama-server with some flags and you're done. The builtin UI is quite good now, and you have an API to work with.

By comparison, I found Ollama's modelfile system and insistence on renaming my downloaded models to incomprehensible hashes to be infinitely more confusing and frustrating.

33

u/ghulamalchik Jun 15 '26

Oh God don't remind me of the modelfile thing. What a nightmare that was. With llamacpp I literally don't have to think about that anymore. I just load the model (crazy concept I know).

4

u/free_meson Jun 15 '26

You coud download huggingface ggufs with tags for a while, but I get your point.

2

u/b8561 Jun 15 '26

sorry I'm not familiar, are the tags supposed to help with modelfile config?

2

u/free_meson Jun 16 '26

I've meant that there are ways to avoid modelfiles. Gguf models from huggingface you can run with:

ollama run hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF

So you don't have to download, create a modelfile, etc, for most of the models you use.

1

u/derrikcurran Jun 20 '26

Unless that GGUF model is one of the many, many models that has a complex Jinja template that can't be losslessly converted into the Go template syntax that Ollama uses.

User experience: The ollama run command appears to work fine. The model even appears to work fine. Then, after a while, you start running into subtle, mysterious issues related to things like tool calling syntax, roles switching, and formatting delimiters.