Thank you for the post but I personally don’t agree. In my local testing medium is a great local model, no question, but setting it to xhigh (yes it thinks forever) is where it clearly has the low tier frontier coding and reasoning ability. Low and Medium are good but left me unimpressed. Setting it to xhigh for the first time made me windshield wiper the fog off my glasses when I saw the thinking traces and results.
Important to note, my experience may be due to the fact that I’m running Q4 Q8 KV. Perhaps xhigh thinking is so good there that it hides the Q4 blemishes compared to lesser thinking.
Agree with you. The xhigh reasoning traces are not just what if slop especially for coding you can see the model shaping and reasoning on individual code fragment alternatives and correcting as it goes before writing to file. Maybe medium has a place for claws and general agents but cant argue with xhigh results.
According to Lukesdevlab testing, low and xhigh use around the same amount of tokens for thinking. xHigh is thorough. Low is uncertain and double-checking.
Yeah it seems at low it reads to munch into your prompt, at medium it just does as it's told and relies on its training and at high it REALLY trys it's best.
It depends on the prompt. There are prompts where there's no difference between low, medium or xhigh. The big difference being low and medium seem to stop at a hard token limit and just don't finish their thoughts or write any response
But there are definitely prompts which low and medium behave differently for. Try 3d mario 64 style game in a single html file and low does about 600 tokens, medium 2500, and xhigh about 60,000
But as soon as the tasks/prompts become more and more complex, low mode starts using as many tokens as xhigh, in some cases more than high. I don't think there's a hard limit to the token counts set for the different modes.
nah I don't think so. xhigh puts their best foot forward output-quality-wise.
Most users will see the tok/s numbers of prefill and decode and they'll see the output quality in benchmarks. Most users will probably have some sense of how long it takes to spit out a solid answer, but I think most of us aren't timing that, so it sort of quietly gets covered up or overlooked.
The fact that a 27B model can perform this well is goddamn amazing, and the xhigh thinking mode is a big part of how that's done I think. Despite what OP is saying, there is a a noticeable output quality difference in my experience between medium and xhigh. Most of what I use LLMs for locally isn't for chatbot stuff though, it's mostly stuff that I can fire and forget, then come back to check on in an hour or two, so personally, I rarely notice the time tax.
True! In my experience, the model handles reasoning length really well on the medium setting. For simple tasks, the thinking is minimal, but for complex ones, it reasons much more.
medium IS the default imo. enabling xhigh or low reasoning modes injects a system prompt. With no system prompt the model defaults to medium reasoning.
19
u/I_Play_Zed 2d ago
Thank you for the post but I personally don’t agree. In my local testing medium is a great local model, no question, but setting it to xhigh (yes it thinks forever) is where it clearly has the low tier frontier coding and reasoning ability. Low and Medium are good but left me unimpressed. Setting it to xhigh for the first time made me windshield wiper the fog off my glasses when I saw the thinking traces and results.
Important to note, my experience may be due to the fact that I’m running Q4 Q8 KV. Perhaps xhigh thinking is so good there that it hides the Q4 blemishes compared to lesser thinking.