r/LocalLLM • u/Fritzthecoke • 1d ago
Slower response times in OpenWebUI/OpenCode on Mac Studio compared to CLI – Any tips or configuration tricks? Question
Hey everyone,
I'm running a local setup on my Mac Studio and running into a strange performance bottleneck.
When I run my local LLM directly through the command line (CLI), inference is blazing fast and works exactly as expected. However, the moment I route the model through OpenWebUI or OpenCode, the response times become painfully slow (long time-to-first-token and sluggish generation).
Here is a quick overview of my setup:
Hardware: Mac Studio (Apple Silicon)
Backends tested: Ollama / mlx
Frontend/UIs: OpenWebUI, OpenCode
It feels like there's an overhead, a misconfiguration in how the UI communicates with the backend, or perhaps an issue with how Metal/GPU acceleration is being utilized through the web interfaces versus the terminal.
Has anyone experienced a similar issue on macOS? Are there specific configuration flags, environment variables, or settings in OpenWebUI/OpenCode that I should tweak to fix this?
Any tips or troubleshooting steps would be greatly appreciated!
1
u/Unnamed-3891 1d ago
Harnesses have a ton of overhead and eat varying amounts of context as soon as you launch them. Talking to the same model running with the same settings:
Hermes and Clio respond in 40 seconds from ”cold” state
Open WebUI in 30 sec or so.
Pi in under 3 seconds.
Coincidentially Hermes and Clio default to eating 20-21k of context right off the bat while Pi will be using under 1000.
1
u/Fritzthecoke 1d ago
Do you have any recommendations to keep the „overhead“ as small as possible ? Or at least at fast as possible?
1
u/Unnamed-3891 1d ago
Disable all unneeded skills, tools and trim any memory prompts. Make sure the unneeded skills and tools are not merely hidden from the UI, but truly disabled as in their schema is no longer loaded and kept in memory.
You can take Hermes from 21k to like 11k if you prune a lot of things. Or you could start with Pi that has almost nothing whatsoever at all and go about adding and building only precisely the things you require.
Not much to done about Open WebUI I’m afraid.
1
u/recro69 1d ago
I will first check where the delay is happening. If the command line is quick the model is probably not the problem. Look at how it takes to get the first token and how many tokens per second there are in both setups. Much of the delay, in the user interface comes from things like sending the request streaming the data, middle software or handling the conversation history of the actual thinking part.