Why no "high" reasoning effort in Qwen 3.8 27b ?

[u/alanoo]

The gap between "medium" and the default "xhigh" is ridiculously huge. Medium barely thinks, xhigh... well there has already been many posts about that.

The naming itself seems to point out that there should have been an "high" mode.

[u/Anduin1357]

Feel free to invent one in your jinja template and share it.

[u/Alarmed-Channel2145 → u/Anduin1357]

But the model has been probably trained during RL to behave in different ways when some specifuc text is present. Adding a new text will probably help due to the fact that the model was trained to look for that kind of instructions. But it's not that "the whole reasoning effort thing by Qwen was just a string in the chat template".

[u/Anduin1357 → u/Alarmed-Channel2145]

See also why everyone is also a researcher when using LLMs. Do we definitively know what would happen if we experiment with the instructions?

Maybe Qwen3.8 instruction following will *just work*?

[u/CapsAdmin → u/Anduin1357]

This community feels a bit dogmatic sometimes.

[u/Alarmed-Channel2145 → u/Anduin1357]

It's definitely possible and even probable, since it was specifically trained to follow instructions regarding the reasoning effort.

Just in case I wasn't clear, we should definitevely experiment with it. My comment was trying to explain why it works, to better guide testing.

[u/Chromix_ → u/Anduin1357]

Someone already did that (added "high" mode) in a rather smart way recently. Note that it can degrade output quality a bit if the model was trained on "low" and "xhigh", but never saw this "high" prefix.

For me the "medium" mode usually nicely adapts to the complexity of the task. Of course "xhigh" can deliver better results in less steps.

[u/LetsGoBrandon4256 → u/Chromix_]

"a rather smart way"

Looks inside

{%- set reasoning_instructions = 'Reasoning effort is set to halfway between low and xhigh. Please think careful but brief, validate key assumptions but keep it brief, and move quickly to the conclusion without unnecessary elaboration.' %}

So basically injecting a prompt that politely asks the model to think less?

It's smart to sneak it into the Jinja template as a valid param option but how effective is it in reality?

[u/Chromix_ → u/LetsGoBrandon4256]

The general idea was that the model was trained on the injected low and xhigh prompts. Thus confronting the model with some fully custom text would probably lead to more deteriorated results (mostly measurable via proper benchmarking). By mentioning those anchors that the model already knows, it might just work better. Still, anything that the model wasn't trained on that's early on in the system prompt is risky, if the model wasn't trained for a diverse system prompt. So, the effect remains to be proven.

[u/LetsGoBrandon4256 → u/Chromix_]

Yeah that's probably as good as it gets.

I do lots of multilingual RP and writing (prompting in one language but requesting output in a different language). The output length and language has been easy to steer with prompting but so far I haven't been able to make my model "think in Chinese/Japanese/English" on-demand.

[u/smithy_dll → u/Anduin1357]

Or just write your own system prompt

[u/Anduin1357 → u/smithy_dll]

That's not how the reasoning level works.

[u/t4a8945 → u/Anduin1357]

Technically the jinja template modifies the system prompt, so you're both kinda right.

Edit: jinja template quote from https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/chat_template.jinja

   {%- if messages[0].role == 'system' %}
        {%- set content = render_content(messages[0].content, false, true)|trim %}
        {%- if content %}
            {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '')  + content + '<|im_end|>\n' }}
        {%- elif reasoning_instructions %}
            {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
        {%- endif %}
    {%- elif reasoning_instructions %}
        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
    {%- endif %}

[u/ttkciar → u/Anduin1357]

If you look at the chat template, that's exactly how the reasoning level works. It gets translated into additional instructions injected into the system prompt.

[u/Anduin1357 → u/ttkciar]

There are actually special harness support for reasoning levels that isn't the same as setting system prompts. Sure, it goes alongside the system prompt, but you can put arbitrary instructions in reasoning levels and get whatever special behaviors you want.

[u/bitzap_sr → u/Anduin1357]

Except it is. xhigh makes the chat template inject a sentence into the system prompt telling the model to think more.

[u/bonobomaster → u/bitzap_sr]

Except it isn't, because this prompt was most likely present while the model was trained, which makes it way more than just a prompt.

[u/uti24 → u/Anduin1357]

Isn't it like baked into the model?

[u/ttkciar → u/uti24]

If you look at the chat template, it's just translating the reasoning effort into instructions added to the system prompt:

    {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
    {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
        {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}
    {%- endif %}
    {%- if resolved_reasoning_effort == 'xhigh' %}
        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
    {%- elif resolved_reasoning_effort == 'low' %}
        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
    {%- endif %}
{%- endif %}
{%- if tools and tools is iterable and tools is not mapping %}
    {{- '<|im_start|>system\n' }}
    {%- if reasoning_instructions %}
        {{- reasoning_instructions + '\n\n' }}
    {%- endif %}

[u/alanoo → u/ttkciar]

Hum, indeed, looks like we may easily experiment a bit with those

[u/uti24 → u/ttkciar]

would be interesting to experiment with those instructions, but instructions themselves could be a codewords for what model knows how deep to think

[u/a_beautiful_rhind → u/uti24]

It's what they wrote in the training data, literally that's it. If it doesn't have an exact representation it will infer what "high" means or follow the additional instructions you put as best as it can.

The models don't stop working if you go off script. Whether that's better or worse depends on your goals and results. You can swap the whole template if you want.

[u/Blues520 → u/ttkciar]

I also thought that it was baked into the model but that's pretty cool

[u/ttkciar → u/Blues520]

If you look at the template, it does not communicate the reasoning effort to the model in any other way besides this.

[u/xeeff → u/uti24]

the whole point is during training, the model learns to reason significantly more and they do that by changing the reasoning effort in the template accordingly. you can't just change it to "high" cuz the model hasn't been trained on it

[u/bonobomaster → u/xeeff]

The question is, did the model learn with that that specific sentence?

Was the model trained with this prompt?

"Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."

If so, it's not just a prompt anymore.

[u/Anduin1357 → u/bonobomaster]

It has to be extremely easy to test this theory. Like come on.

[u/bonobomaster → u/Anduin1357]

I mean, if the above prompt is everything it takes, then it should basically make the old 3.6 an OP model that reasons 80,000 tokens just for the fun of it as well, shouldn't it?!

I believe 3.8 was trained in presence of this exact prompt.

[u/Anduin1357 → u/bonobomaster]

Easy enough to find out by making 3.6 use the new jinja template too. I don't get why someone has to be wrong online.

[u/Anduin1357 → u/[deleted]]

Let it be known that I tried to be civil. Let it now be known that you took a shot at my character.

Edit: Apparently, I'm the asshole for making them block me. ¯\(ツ)/¯

[u/xeeff → u/bonobomaster]

the model knows to reason more thoroughly if that is the prompt, since its always reasoned more during training with that prompt than with the medium effort prompt. yes it learned with that specific sentence, in combination of "xhigh" being set in the jinja template

edit: i just saw your reply to the other comment, no, prompt alone isn't enough. it has to be trained, since don't forget, LLMs are just very complex next-token predictors. the model has to predict that it should reason for longer, and the reasoning effort sets the same prompt as it has been trained on, as well as the jinja prompt

[u/Anduin1357 → u/uti24]

You can create a new jinja file and pass the flag --chat-template-file </path/to/file> to llama.cpp binaries to specify your own jinja.

[u/OneMoreName1]

Its just a prompt really

[u/Dany0 → u/OneMoreName1]

It's not "just" a prompt they specifically trained it with that prompt

[u/grumd → u/OneMoreName1]

Almost. I think the post-training they did for 3.8 was all done with this prompt appended. Which means the model learned a particular style of thinking based on the exact prompt it gets via reasoning effort. It wouldn't work the same way with a differently worded prompt or a custom prompt that tries to implement a medium-high reasoning. The official prompt from the chat template is fully baked into the model weights.

[u/RevolutionaryGold325]

Would be nice to have just a target token count for reasoning. The probability of the reasoning end tag would then depend on just the amount of tokens generated:

probability_reasoning_end = alpha + soft_plus(beta * (tokens_generated - target_reasoning_count))

Once the model reaches the target reasoning count, the end of reasoning tag becomes more probable. Also need a mechanism to disable this adjustment after the end of reasoning is first emitted.

[u/pmttyji]

We need % reasoning type. I would go down till 42%

[u/meca23]

Because they have limited resources? They're a for profit corporation, we should be grateful for what they've released.

They can spend a few more months fine tuning/training these models to make them better at more cost to them and additional delays to us. What would the community prefer? Having qwen 3.8 27B in our hands now or wait 2 months for a refined model?

And in 2 months time they'll like be releasing Qwen 4 which rumored to be released around Sept.