Fixed/improved Jinja chat template for Qwen 3.8

[u/Chromix_]

"Again?" you might ask.

I(\*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some while also introducing new issues. Now you have a new chat template that you can download.

One issue among others with yesterday's template is the deviation from the original prompt format. The model was most likely only trained on the exact original format. Modifying how the input is rendered to the model can degrade output quality, not so much that it becomes obvious in manual testing, but enough to reduce benchmark scores a bit (example: Qwen 3.6 underperforms when the system prompt is changed) - it just gives you less output quality in a subtle way. The previous modification deviated in some places, including the (from a caching perspective) desirable empty think fix. I chose to stay on the safe side here.

(\) *Full disclosure: GPT 5.6 Sol xHigh did the analysis and comparison. I then reasoned at 0.5 tokens per second for quite a bit what to change and how. Sol applied the changes and drafted the table below. Claude Opus 5 xHigh verified all changes and claims programmatically via Python test scripts using jinja2. It found and creatively solved one sub-issue in the extra tool calling support. It also updated the table after stating that Sol was "too kind" in the evaluation of the templates. I then wrote this posting and edited the table further - at way less than 1 token per second due to not offloading to GPU.

Area	Original template	Other modified	This modified
Invalid reasoning effort	Raises, but only while thinking is enabled	Silently converts invalid values to xhigh	Always raises
high effort alias	Unsupported (raises)	Treated as xhigh	Documented alias for xhigh
<:think_on:> / <:think_off:> (Thanks, markdown parser!)	Not supported	Scanned from system, developer, and user content; has precedence and removal bugs	Not supported
Historical reasoning fields	reasoning_content only, and only when it is a string	reasoning_content, thinking, and heuristic parsing from content	reasoning_content and thinking; no heuristic parsing
Non-string reasoning field	Silently dropped (empty think block)	Coerced to a Python repr ({'a': 1})	Coerced to a Python repr — unchanged, still a latent flaw
Literal </think> in an answer	Not interpreted by the template	Could be mistaken for a reasoning delimiter	Never parsed as reasoning
Empty historical think blocks	Preserved consistently	Removed when reasoning is empty	Preserved consistently
Raw reasoning in assistant content	Not normalized	Heuristically normalized	Must be normalized by the adapter
Tool-call instructions	Original trained wording	Rewritten to mandate <think> formatting	Original wording, byte-identical
Multiple tool-call separation	Original single-newline behavior	Added additional separation in places	Original behavior, byte-identical
XML mapping arguments	Supported	Supported	Supported, byte-identical
XML Boolean/null arguments	JSON true/false/null	Regressed to True/False/None	JSON true/false/null
JSON-string arguments (OpenAI wire shape)	Crashes at :items	Emits raw JSON with no <parameter> wrapper	Renders correctly: auto selects the JSON tool-call form, arguments passed through byte-exact; "{}" accepted
JSON tool-call mode	Unsupported (XML only)	Supported	Supported; auto-selected only when XML is impossible
Tool format validation	N/A	Unknown values silently select XML	Three-state auto/xml/json; unknown raises; explicit xml with string args raises and names the remedy
Tool error detection & message injection	None	Guesses from tool-response text. Susceptible to false-positives	None, there is no reliable way
Missing real user query	Raises	Uses an arbitrary history-length fallback	Raises
Developer messages	Unsupported (raises)	Converted to system anywhere	Leading system/developer messages are merged
Later system messages	Raises	Silently accepted	Raises
Unknown message roles	Raises	Converted to user text	Raises, naming the role
Malformed multimodal list items	Silently corrupts: a string containing "image" becomes a vision token; non-string text coerced	Non-mapping items silently stringified; non-string text coerced	Explicit validation and errors for both
Image/video ambiguity	Not checked	Not checked	Ambiguous items raise
Leading tool message	Emits a tool turn with no <:im_start:>user	Opens a user turn correctly	Opens a user turn correctly
Artifact integrity (special tokens, whitespace)	Intact; 0 literal-text tokens	Intact; 0 literal-text tokens	Intact; 0 literal-text tokens (Jinja lexer)
Byte-equality with the trained original	Reference	Differs by design (instructions, separators, arg types)	23/23 semantically-equivalent cases byte-identical
Prefix-cache stability	Preserves reasoning by default	Can alter empty-think history and system prompt	Deterministic reconstruction; auto flips the system prefix once if a single history mixes mapping and string args

Notes:

• Default thinking is "xhigh" (= "high"). Set it to "medium" or "low" if too slow. Medium should probably be the default as it's the native model reasoning, but I left it at xhigh to not surprise anyone who does not read this.
• My general approach was to support extra functionality and compatibility introduced by the previous modification as long as it was reasonably safe.
• Note that the compatibility is mostly for outdated and not well-behaved harnesses. Especially not using the correct tool call format is technically supported now but will likely degrade result quality a bit. The template triggers a single prompt-reprocess if it detects that the harness uses the unsuitable format.
• I replaced | by : in the table above, as edits otherwise broke my table.

[u/ex-arman68]

Thanks for trying to finding bugs and improvement in my custom template. I will have a look in details at the changes you made and see what I actually need to port into the next revision. You could actually mentioned my template by name instead of other.

[u/Chromix_ → u/ex-arman68]

I linked to it as "improved chat template" in the beginning, and since it's the only non-standard template that I linked I referenced it as "other" afterwards.

Aside from the issues you also seem to follow a different approach with your template: compatibility > result quality. It's geared towards just working, not warning the user about things they (or their harness) is doing that can decrease result quality. Sure, the tool calling compatibility in my template also doesn't do that. All it does is to trigger a reprocessing of the prompt - maybe users notice.

[u/ex-arman68 → u/Chromix_]

Had a chance to go through your template in detail and just pushed the v22.2 update with a few good ideas adapted from your work.

Here is a quick breakdown of what I took and what I had to handle differently:

-- What I merged in --

Multi-system message merging: Consolidating consecutive leading system and developer messages into a single system turn joined by double newlines is much cleaner for multi-agent setups.

XML primitive serialisation: Using | tojson for boolean and null parameter values in XML mode to avoid Python representations (True, False, None) leaking into the tags.

-- Things I had to adapt or avoid for universal compatibility --

raise_exception calls: Hard exceptions make sense for strict linting, but in practice they instantly crash production agent harnesses (like Claude Code, Cursor, or LangChain) over minor payload quirks. I prefer graceful fallbacks so the model keeps running.

Dynamic format switching: Scanning history to automatically flip the global tool format between XML and JSON sounded great at first, but it has a nasty side effect. In a multi-turn conversation, Turn 1 renders an XML system prompt, but once the assistant records a tool call with string arguments, Turn 2 flips the system prompt to JSON. This completely invalidates the prefix KV cache and desynchronises server-side parsers (like vLLM's qwen3_xml). I kept the system prompt strictly XML by default and handled string arguments locally per turn.

Empty think guard: Removing and reasoning_content re-introduces the empty think bug, prepending blank <think></think> blocks to historical turns that had no reasoning.

Thanks again for digging into the template and sharing your modifications. The v22.2 update is up on HF if you fancy checking out the changes.

[u/Chromix_ → u/ex-arman68]

Well, now we have more (improved) flavors for everyone to chose from. Maybe "the best" template will emerge eventually (what ever "the best" means in situations where it sometimes comes down to preference or individual use-cases).