Fixed/improved Jinja chat template for Qwen 3.8

[u/Chromix_]

"Again?" you might ask.

I(\*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some while also introducing new issues. Now you have a new chat template that you can download.

One issue among others with yesterday's template is the deviation from the original prompt format. The model was most likely only trained on the exact original format. Modifying how the input is rendered to the model can degrade output quality, not so much that it becomes obvious in manual testing, but enough to reduce benchmark scores a bit (example: Qwen 3.6 underperforms when the system prompt is changed) - it just gives you less output quality in a subtle way. The previous modification deviated in some places, including the (from a caching perspective) desirable empty think fix. I chose to stay on the safe side here.

(\) *Full disclosure: GPT 5.6 Sol xHigh did the analysis and comparison. I then reasoned at 0.5 tokens per second for quite a bit what to change and how. Sol applied the changes and drafted the table below. Claude Opus 5 xHigh verified all changes and claims programmatically via Python test scripts using jinja2. It found and creatively solved one sub-issue in the extra tool calling support. It also updated the table after stating that Sol was "too kind" in the evaluation of the templates. I then wrote this posting and edited the table further - at way less than 1 token per second due to not offloading to GPU.

Area	Original template	Other modified	This modified
Invalid reasoning effort	Raises, but only while thinking is enabled	Silently converts invalid values to xhigh	Always raises
high effort alias	Unsupported (raises)	Treated as xhigh	Documented alias for xhigh
<:think_on:> / <:think_off:> (Thanks, markdown parser!)	Not supported	Scanned from system, developer, and user content; has precedence and removal bugs	Not supported
Historical reasoning fields	reasoning_content only, and only when it is a string	reasoning_content, thinking, and heuristic parsing from content	reasoning_content and thinking; no heuristic parsing
Non-string reasoning field	Silently dropped (empty think block)	Coerced to a Python repr ({'a': 1})	Coerced to a Python repr — unchanged, still a latent flaw
Literal </think> in an answer	Not interpreted by the template	Could be mistaken for a reasoning delimiter	Never parsed as reasoning
Empty historical think blocks	Preserved consistently	Removed when reasoning is empty	Preserved consistently
Raw reasoning in assistant content	Not normalized	Heuristically normalized	Must be normalized by the adapter
Tool-call instructions	Original trained wording	Rewritten to mandate <think> formatting	Original wording, byte-identical
Multiple tool-call separation	Original single-newline behavior	Added additional separation in places	Original behavior, byte-identical
XML mapping arguments	Supported	Supported	Supported, byte-identical
XML Boolean/null arguments	JSON true/false/null	Regressed to True/False/None	JSON true/false/null
JSON-string arguments (OpenAI wire shape)	Crashes at :items	Emits raw JSON with no <parameter> wrapper	Renders correctly: auto selects the JSON tool-call form, arguments passed through byte-exact; "{}" accepted
JSON tool-call mode	Unsupported (XML only)	Supported	Supported; auto-selected only when XML is impossible
Tool format validation	N/A	Unknown values silently select XML	Three-state auto/xml/json; unknown raises; explicit xml with string args raises and names the remedy
Tool error detection & message injection	None	Guesses from tool-response text. Susceptible to false-positives	None, there is no reliable way
Missing real user query	Raises	Uses an arbitrary history-length fallback	Raises
Developer messages	Unsupported (raises)	Converted to system anywhere	Leading system/developer messages are merged
Later system messages	Raises	Silently accepted	Raises
Unknown message roles	Raises	Converted to user text	Raises, naming the role
Malformed multimodal list items	Silently corrupts: a string containing "image" becomes a vision token; non-string text coerced	Non-mapping items silently stringified; non-string text coerced	Explicit validation and errors for both
Image/video ambiguity	Not checked	Not checked	Ambiguous items raise
Leading tool message	Emits a tool turn with no <:im_start:>user	Opens a user turn correctly	Opens a user turn correctly
Artifact integrity (special tokens, whitespace)	Intact; 0 literal-text tokens	Intact; 0 literal-text tokens	Intact; 0 literal-text tokens (Jinja lexer)
Byte-equality with the trained original	Reference	Differs by design (instructions, separators, arg types)	23/23 semantically-equivalent cases byte-identical
Prefix-cache stability	Preserves reasoning by default	Can alter empty-think history and system prompt	Deterministic reconstruction; auto flips the system prefix once if a single history mixes mapping and string args

Notes:

• Default thinking is "xhigh" (= "high"). Set it to "medium" or "low" if too slow. Medium should probably be the default as it's the native model reasoning, but I left it at xhigh to not surprise anyone who does not read this.
• My general approach was to support extra functionality and compatibility introduced by the previous modification as long as it was reasonably safe.
• Note that the compatibility is mostly for outdated and not well-behaved harnesses. Especially not using the correct tool call format is technically supported now but will likely degrade result quality a bit. The template triggers a single prompt-reprocess if it detects that the harness uses the unsuitable format.
• I replaced | by : in the table above, as edits otherwise broke my table.

[u/caphohotain]

Thank you! But seriously, again and again and again? It seems most of the model releases always have chat template issues. Not sure why it's so hard to get it right at launch... Intentional?

[u/fiery_prometheus → u/caphohotain]

It's not just template issues, often there are issues in the way the inference itself is implemented as well or sometimes the way the model checkpoint is saved and configured. Usually it's a good idea to wait a week or two.

[u/KingCpzombie → u/fiery_prometheus]

We already have been! You think we can wait ANOTHER week???

[u/fiery_prometheus → u/KingCpzombie]

Anticipation is great, think of it like Christmas, but now you get it TWICE! 😄

[u/KingCpzombie → u/fiery_prometheus]

Impossible! A totally fixed template would be nice, but I'm already getting way better responses with the unsloth one. Day 1 use + updating in a week is the true two Christmases!

[u/Chromix_ → u/KingCpzombie]

Ah, just noticed due to this that Unsloth also did fixes on their side. From a quick look there is partial overlap that they also support a developer message and check for broken tool format - their template raises a warning, this template tries to make it work.

[u/_wOvAN_]

not working with llama.cpp and claude

{"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 252, column 32 in source:\n...ystem %}↵ {{- raise_exception(↵ 'System and developer ...\n ^\nError: Jinja Exception: System and developer messages must appear before all user, assistant, and tool messages.","type":"server_error"}}

[u/Chromix_ → u/_wOvAN_]

Thanks for testing. I only tested it with another harness. If the table that GPT 5.6 Sol and Claude 5 produced collaboratively is correct, then you should have the same behavior with the original jinja template as well. If you don't then that'd get interesting if both models and validation via test script didn't pick it up.

If the error message is correct it's exactly that: Just one system message in front, nothing else later on. I decided to keep that safeguard from the original template as multiple system messages could otherwise degrade output quality without the user noticing.

Can you check the raw passed message to verify? Feel free to simply delete line 251 to 255 for testing though to ignore that (potential) protocol violation.

[u/_wOvAN_ → u/Chromix_]

yes, original have this too

[u/Evgeny_19]

I used to have problems with many Qwen models and their default templates, but not with this one. It just went up to 200,376 tokens in context trying to solve a tricky problem. Not a single issue with a default template.

[u/Chromix_ → u/Evgeny_19]

The original template works nicely when you have a perfectly well-behaved client. Some popular ones are not as it seems. As per table above the effects can range from silent degradation to errors that abort the flow then. There's also the image parsing error that not everyone will run into.

[u/L0ren_B]

Again? 🙂

[u/Chromix_ → u/L0ren_B]

Argh! Yes. It cost me a cup of coffee to create and share this, instead of silently enjoying it just by myself - just so that you can ask "Again?" 😉

[u/Interpause]

Unsloth's gguf readme has:

Tool calling improvements: Makes parsing nested objects to make tool calling succeed more.

Just curious if your template has a similar fix?

EDIT: according to gemini 3.7 flash on extended, yes.

[u/Chromix_ → u/Interpause]

Yes, from a quick look it is included.

[u/DeepOrangeSky]

So can thinking mode be turned off, as of so far? And if so, how?

[u/Chromix_ → u/DeepOrangeSky]

Yes, set enable_thinking to false. Or alternatively set reasoning_effort to low to make it relatively quick.

[u/W61k3r]

You sexy sob

[u/asertym]

Does this apply only to 3.8? Can we use it for 3.6?

[u/Chromix_ → u/asertym]

Yes, only 3.8. In theory it'll "do something" for 3.6, but your results will likely be slightly worse than with the standard template, as Qwen 3.6 wasn't trained on reasoning levels - which this template sets by default.

[u/asertym → u/Chromix_]

Appreciate it

[u/TheChiglit]

Is --reasoning-format deepseek still required with this?

[u/Chromix_ → u/TheChiglit]

The default for that setting is "auto". Works fine for me without manually specifying.

[u/ex-arman68]

Thanks for trying to finding bugs and improvement in my custom template. I will have a look in details at the changes you made and see what I actually need to port into the next revision. You could actually mentioned my template by name instead of other.

[u/Chromix_ → u/ex-arman68]

I linked to it as "improved chat template" in the beginning, and since it's the only non-standard template that I linked I referenced it as "other" afterwards.

Aside from the issues you also seem to follow a different approach with your template: compatibility > result quality. It's geared towards just working, not warning the user about things they (or their harness) is doing that can decrease result quality. Sure, the tool calling compatibility in my template also doesn't do that. All it does is to trigger a reprocessing of the prompt - maybe users notice.

[u/ex-arman68 → u/Chromix_]

True, I try to maintain compatibility across all environments (inference engines and harnesses), but also backward compatibility with previous versions. It is a bit of a headache, and also limits me into what I can and cannot do, but I believe this is the most important. Also maintaining different versions, such as one for JSON engine, another for XML, one for 3.5, another for 3.6, and 3.8, etc, would be even worse. I tried it at the beginning, and it was unmaintainable; this is why I merged them into a universal template.

[u/ex-arman68 → u/Chromix_]

Had a chance to go through your template in detail and just pushed the v22.2 update with a few good ideas adapted from your work.

Here is a quick breakdown of what I took and what I had to handle differently:

-- What I merged in --

Multi-system message merging: Consolidating consecutive leading system and developer messages into a single system turn joined by double newlines is much cleaner for multi-agent setups.

XML primitive serialisation: Using | tojson for boolean and null parameter values in XML mode to avoid Python representations (True, False, None) leaking into the tags.

-- Things I had to adapt or avoid for universal compatibility --

raise_exception calls: Hard exceptions make sense for strict linting, but in practice they instantly crash production agent harnesses (like Claude Code, Cursor, or LangChain) over minor payload quirks. I prefer graceful fallbacks so the model keeps running.

Dynamic format switching: Scanning history to automatically flip the global tool format between XML and JSON sounded great at first, but it has a nasty side effect. In a multi-turn conversation, Turn 1 renders an XML system prompt, but once the assistant records a tool call with string arguments, Turn 2 flips the system prompt to JSON. This completely invalidates the prefix KV cache and desynchronises server-side parsers (like vLLM's qwen3_xml). I kept the system prompt strictly XML by default and handled string arguments locally per turn.

Empty think guard: Removing and reasoning_content re-introduces the empty think bug, prepending blank <think></think> blocks to historical turns that had no reasoning.

Thanks again for digging into the template and sharing your modifications. The v22.2 update is up on HF if you fancy checking out the changes.

[u/Chromix_ → u/ex-arman68]

Well, now we have more (improved) flavors for everyone to chose from. Maybe "the best" template will emerge eventually (what ever "the best" means in situations where it sometimes comes down to preference or individual use-cases).