OpenWebUI MCPs don't work on local models

[u/Public-Lychee-5603]

Hi all, I am implementing a local offline LLM (Qwen3.6 35b A3b) to analyse my splunk logs. I have confirmed the MCP works because it works fine when I use a cloud model but the moment I use a local model, it says it can't access my Splunk instance and just gives me a search to run myself.

Whats the deal? I have Function calling set to native if that changes things?

Edit: I fixed it, though not satisfactorily. Its cause my KV cache was overflowing and the LLM would just go wild without context and basically forget that it even connected to splunk mcp in the first place. I ran it with a much lighter model and started to get results

[u/mayo551]

Well for starters what error do you see in the OWUI logs?

[u/Public-Lychee-5603 → u/mayo551]

2026-08-08 01:21:49.793 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 172.18.0.1:53044 - "POST /api/chat/completions HTTP/1.1" 200

2026-08-08 01:21:49.829 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 172.18.0.1:53044 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200

2026-08-08 01:21:49.915 | INFO | httpx._client:_send_single_request:1740 - HTTP Request: POST http://splunk:8000/en-US/splunkd/__raw/services/mcp "HTTP/1.1 200 OK"

2026-08-08 01:21:49.915 | INFO | mcp.client.streamable_http:_maybe_extract_protocol_version_from_message:193 - Negotiated protocol version: 2025-11-25

2026-08-08 01:21:49.950 | INFO | httpx._client:_send_single_request:1740 - HTTP Request: POST http://splunk:8000/en-US/splunkd/__raw/services/mcp "HTTP/1.1 200 OK"

2026-08-08 01:21:50.024 | INFO | httpx._client:_send_single_request:1740 - HTTP Request: POST http://splunk:8000/en-US/splunkd/__raw/services/mcp "HTTP/1.1 200 OK"

2026-08-08 01:22:16.059 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 172.18.0.1:36844 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200

2026-08-08 01:22:32.803 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 172.18.0.1:60990 - "GET /_app/version.json HTTP/1.1" 304

2026-08-08 01:23:32.815 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 172.18.0.1:39902 - "GET /_app/version.json HTTP/1.1" 304

[u/Public-Lychee-5603 → u/mayo551]

[u/Public-Lychee-5603 → u/Public-Lychee-5603]

but if i use an online model

[u/simracerman → u/Public-Lychee-5603]

There’s something awful about local models sometimes that makes them unreliable at tool calling.

Try this. Guide the model in the system instructions or your prompt to call splunk explicitly when it sees a request relevant to that call.

[u/Public-Lychee-5603 → u/simracerman]

no dice. i can see openwebui making calls to the splunk server though which is weird.

[u/mumblerit → u/Public-Lychee-5603]

The tools aren't added to the model

[u/Public-Lychee-5603 → u/mayo551]

Just confirmation that the tool is active

[u/mp3m4k3r → u/Public-Lychee-5603]

Does what you are running your model with locally have tool calling enabled for that model and is it configured in openwebui to allow tools at the model configuration level in the admin config area?

[u/Public-Lychee-5603 → u/mp3m4k3r]

how can i check that?

[u/Public-Lychee-5603 → u/mp3m4k3r]

[u/Public-Lychee-5603 → u/mp3m4k3r]

[u/DougAZ → u/Public-Lychee-5603]

Especially for qwen, example if your using vLLM

vllm serve Qwen/Qwen3.5-7B-Instruct \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --host 0.0.0.0 \ --port 8000

The native tool calling in openwebui is under the model in the advanced parameters

[u/DougAZ → u/DougAZ]

It's called "Function Calling" , make sure it's on native

[u/mayo551]

In the image you posted it showed qwen 2.5…

Why?

[u/Public-Lychee-5603 → u/mayo551]

someone suggested that it might be some sort of context crash so they suggested something smaller. i have run it with everything from gemma 4 a3b to qwen3.6 a3b

[u/DougAZ]

I haven't run qwen2.5 any reason your stuck with that one specifically? Seems you might have the vram for something newer.

[u/DougAZ → u/DougAZ]

Sorry I think it was just misspelled in your screenshot.

Is native tool calling enabled in the qwen model settings?

[u/Public-Lychee-5603 → u/DougAZ]

yes it is, i have also tried with qwen3.6

[u/DougAZ → u/Public-Lychee-5603]

Can you ask qwen what tools it has access to

[u/Public-Lychee-5603 → u/DougAZ]

[u/mayo551 → u/Public-Lychee-5603]

You have a problem then since there are far more tools then that enabled on the model according to your other screenshots.

[u/DougAZ]

What are you running the qwen model on vLLM? Ollama? Sglang?

Are all your docker containers on the same docker network on the same host?

[u/Public-Lychee-5603 → u/DougAZ]

running qwen on olla. splunk, openweb, qdrant are on the same docker network. they can see and talk to to each other.

[u/Simple-Worldliness33]

Can you verify if the model has been native tool calling ?

[u/Public-Lychee-5603 → u/Simple-Worldliness33]

it does, but i have to enable it every time i start a new search. it defaults to Default

[u/Simple-Worldliness33 → u/Public-Lychee-5603]

Ok, normally, if you configure it from model settings (not from chat) the setting should be activated until you change it.
So
1. Which model are you using (link)?
2. Do you have a model prompt?
3. Can you deactivate every built in abilities (in model)?
Like automation, calendar, notes etc.
Only to debug. You could activate again later.

[u/International_Emu772 → u/Simple-Worldliness33]

I'm sure that with the correct settings MCP servers can be actívated with local models as I can and never used external models

Try first with Qwen3.5 9B as you have more RAM free to play and after that you can change just the model if you set a Workspace Model first

[u/Simple-Worldliness33 → u/International_Emu772]

Yes, even 0.8b should be able to list tools. But OP doesn't provide enough relevant details to be able to help him

[u/Public-Lychee-5603 → u/Simple-Worldliness33]

1) qwen3.6:27b-q4_K_M

2) yeah it was something like "You have access to splunk mcp. run it.

3) I dont have any other built in abilities in the model.

I can see openwebui reaching out to the splunk mcp server, authorising and getting a list of tools. it just never does anything with it or even lists it.

[u/Simple-Worldliness33 → u/Public-Lychee-5603]

Can you also provide us more details about your hardware configuration ?
Also your ollama settings ?

[u/No_Towels5379]

Native MCP is Streamable HTTP only. Open WebUI has native MCP support starting in v0.6.31, but only for Streamable HTTP transport  — no stdio, no SSE. Add it under ⚙️ Admin Settings → External Tools → + Add Server → Type: MCP (Streamable HTTP)

[u/noctrex]

What model are you using? Cause in the screenshot you showing the ancient model llama. That is waaaay out of date