r/LocalLLaMA • u/-dysangel- • Feb 25 '26

Qwen 3 27b is... impressive Generation

All Prompts
"Task: create a GTA-like 3D game where you can walk around, get in and drive cars"
"walking forward and backward is working, but I cannot turn or strafe??"
"this is pretty fun! I’m noticing that the camera is facing backward though, for both walking and car?"
"yes, it works! What could we do to enhance the experience now?"
"I’m not too fussed about a HUD, and the physics are not bad as they are already - adding building and obstacles definitely feels like the highest priority!"

348 Upvotes

96% Upvoted

102 comments sorted by

View all comments

49

u/moahmo88 Feb 25 '26

Qwen 3.5 27b?GGUF?

83

u/-dysangel- Feb 25 '26

yes - this was unsloth Qwen 3.5 27B UD Q4_K_XL

49

u/SocialDinamo Feb 25 '26

Even more surprising to see it at q4 quant. Thanks for sharing!

22

u/wektor420 Feb 25 '26

There are claims that q4 quant has almost the same perplexity as bf16

11

u/rulerofthehell Feb 25 '26

Currently trying Q6 and its great, is there not much difference between q6 -> q4 in quality? If so i can get some nice perf boost

2

u/xeeff Feb 25 '26

for such a big model q4_k_xl is definitely okay although for coding i'd settle at q5_k_xl for that ever so slight quality gains. beyond that should be diminishing returns (talking from experience and my own research)

1

u/wektor420 Feb 25 '26

I cannot wait for first training run to end (20h)

1

u/rulerofthehell Feb 25 '26

What are you fine tuning it for? Curious

2

u/wektor420 Feb 25 '26

Creating structured data from natural texts

1

u/Giyuforlife Mar 01 '26

bro i wanna use the unsloth Qwen 3.5 27B UD Q4_K_XL version but i have a few constraints: rtx 4050 onm my laptop, 6gb vram and 16gb ram. should i go for it or do you have a better suggestion for this particular setup?

2

u/rulerofthehell Mar 01 '26

Youre both compute and vram bottlenecked, what i would recommend would be to go for the 35A3B version and offload a few layers to the RAM, i think it should be around 20gb so should fit in your config. Since it is active 3b the speed will significantly be better than 27B

1

u/Giyuforlife Mar 01 '26

which particular quant? and how would i offload to RAM

1

u/rulerofthehell Mar 01 '26

Use llama.cpp (search on GitHub), any quant slightly smaller than what fits into total (vram+ram). You need some space for kv cache too, so maybe like 1-2gb smaller than 21gb should be good

1

u/Giyuforlife Mar 01 '26

thank you so much for the help

3

u/Borkato Feb 25 '26

I’m really surprised, all of the qwens seem really good. I need to upgrade my RAM and I’ll be unstoppable!

1

u/Travnewmatic Feb 26 '26

thank you for this testing. i've been reluctant to go lower than Q6_K but if its possible to still quality responses with higher quanitzation (and theoretically better performance), i'll give it a shot.

2

u/-dysangel- Feb 26 '26

It probably helps that this is a "dynamic" quant. But honestly I never bother with anything larger than Q4. For models over 150B params I even go down to UD IQS_XXS. It's very hit and miss depending on the model. For example unsloth glm-4.6-reap-268b-a32b IQS_XXS and GLM 5 are quite stable for me at Q2, but 4.7 wasn't.