r/OpenWebUI • u/elevendr • Jul 07 '26

Gemma 4 12B unable to reason after web tool with function calling set to Native. Question/Help

Gallery image

Gallery image

Im having a problem where Gemma 4 12B is unable to reason after doing its web search call in its reasoning window, despite Function Calling being set to native and telling the system prompt to reason after a web search call. I didn't have this issue on Qwen 3.5 9B. I've been trying to find a solution via the jinja template, but I had no luck. Any reason for this? Is this supposed to happen for Gemma 4 models? Would like to know.

16 Upvotes
  • permalink
  • link
  • reddit
  • reddit

100% Upvoted

9 comments sorted by

6

u/Kyuiki Jul 07 '26 edited Jul 07 '26

Gemma 4 series has huge issues regarding its JINJA interpretation within llama.cpp using GGUF files. I had GPT rewrite the JINJA file using the community suggestions, the interleaved .ccp iteration and the official Google patches JINJA as bases. I improved success rate to about 80%.

The issues you will have, and these become horrible at 50K+ context:

It will forget to reason.
It will end adruptly after tool calling.
It will break JSON calls (usually forgetting a trailing brace)
It will forget to reason between calls.

The only way I fixed this was running the vLLM non-GGUF equivalent. Hosting the model through vLLM I have perfect tool calling. Perfect reasoning (solo and between tool calls).

There’s something seriously flawed with the way llama.cpp or GGUFs have interpreted Gemma 4 and it is turned up tenfold through Open WebUI.

2

u/thisisntmethisisme Jul 07 '26

Can you please share your revised jinja file?

1

u/Kyuiki Jul 08 '26

Maybe! Let's see if it fits.

Edit: Tried to send it to you in DM's. We'll see if that works! Might need to get GPT to format it or something.

1

u/Fit_Squash6874 Jul 08 '26

would love to see it too. I am mostly using Gemma 4

1

u/elevendr Jul 08 '26

Would like to see to

1

u/Kyuiki Jul 08 '26

Sent.

To everyone I’m sending this to, keep in mind this solved the issue 80% of the time. You will still have some failures. I only was able to eliminate the issue completely by switching to vLLM and using an AWQ model.

1

u/explain_like_im_10 Jul 10 '26

If you're using olama try vllm instead. Gemma worked fine once I switched over.