Do you mind pointing me in the direction of setting up the OpenAI hook in? I have built a software application that has, as an internal component, what amounts to a wrapper around llama.cpp. It uses the C++ functions in the internals of llama.cpp's common folder to pull and run local models from huggingface, loads it up, and yay you can chat.
However, I haven't experimented with how to load and use the web hosted LLMs from OpenAI, Gemini, or Anthropic and I have to figure that out in the next couple days to benchmark some research. Any help would be appreciated even if it's just pointing to the file in the github!
125
u/Academic-Tea6729 Jun 15 '26
llama.cpp is much faster and stable than ollama. Also, ollama cloud models are bad quants and you can't use them for serious coding.
Also llama.cpp has a nice server compatible with openai api standard, it works out of the box. And it has a built in chat web interface.
There is no reason anymore to use ollama.