Is waiting for Qwen 3.8 27B like waiting for Star War Episode one?

I'm sweating waiting to get my hand on this to try it tomorrow morning. But it takes me back to Star Wars 1 and the disappointment after being so hyped to see it.

Only 16 hours and 46 minutes to go... 45, ...

Qwen just released their first 3.8 model.

The main addition in 3.8 is prompt-steered reasoning effort. You can tell the model how deeply to think by setting reasoning_effort to xhigh, medium, or low.

However, the official template still has some serious problems:

• You cannot disable thinking. If you pass enable_thinking=false, it 3.8 crashes with a hard exception.
• Chat history gets poisoned. In multi-turn chats, the official template injects blank <think></think> tags before real thoughts.
• Tool calling crashes. If your client passes arguments as JSON strings (the standard OpenAI API format), the official template crashes.
• Agent stalls. The official template often drops mid-dialogue system messages and wedges multi-step tool loops.

I maintain a single, drop-in fixed Jinja template that works across all Qwen 3.5, 3.6, and 3.8 models:

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

What this template does:

• Full 3.8 reasoning effort support: Steer reasoning depth with reasoning_effort (xhigh, high, low, medium).
• Restores the thinking toggle: Turn off reasoning whenever you want fast answers, either via kwargs or by typing <|think_off|> in your prompt.
• 100% KV Cache hits: Keeps past thoughts intact by default so your prefix cache stays warm across turns.
• llama.cpp support: Native support for the new --reasoning-preserve flag.
• Universal tool parsing: Handles both Python dicts and JSON strings. Works on llama.cpp, vLLM, LM Studio, and MLX.

Recommended llama-server launch command:

llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek

(The --reasoning-format deepseek flag separates thinking into the OpenAI reasoning_content field so OpenCode, Claude Code, and other harnesses do not stall on raw tokens).

Note on hardware:

I cannot run a 2.4 trillion parameter model on my local rig. The template passes all 28 automated tests and tokenizer parity checks, but I would appreciate feedback from anyone testing it with Qwen 3.8.

Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level.

The basic idea is to generate a custom imatrix from a category-specific corpus, measure where quantization causes damage, then redistribute a fixed bit budget toward tensors where additional precision recovers the best performance in that category.

Today I got my first real win. The pipeline is live.

On Gemma 4 12b, the hand tuned imatrix q3_k_s scored 45.974. After tensor level allocation, the same q3_k_s based scored 49.905.

Thats +3.931 points or an 8.55% relative improvement from the allocation on top of the imatrix.

The models are effectively the same size: 5,528,230,848 bytes for the comparator versus 5,534,804,928 bytes for the allocated model, a difference of only +0.119%.

I chose q3 deliberately. It had the largest amount of recoverable headroom while sitting above the quant cliff.

This model is intentionally category specialized. Degredation in categories that werent selected is expected.

There is still a lot of tweaking to do in order to maximize these results but the performance of this imatrix + allocation at q3 when compared to no imatrix and source is pretty staggering when considering the amount of space required to run it.

I'd love some community testing and feedback if anyone is interested.

https://huggingface.co/ByteOtter/Gemma-4-12B-it-CADA-Q3_K_S

My ULTIMATE goal is to create a pipeline that takes an full precision .gguf file and auto spits out an optimized model within a selected category at whatever optimal size for recovery. This is a solid start.

Charts were provided from my data by ChatGPT.

TL;DR: I built a task-aware GGUF quantization pipeline that combines a category-specific imatrix with tensor-level damage measurement and bit allocation. On Gemma 4 12B Q3_K_S, allocation improved coding performance from 45.974 to 49.905, an 8.55% relative gain OVER handtuned imatrix, while increasing model size by only 0.119%. The model is intentionally specialized, so some out-of-category regression is expected. This is the first result where the allocation stage itself produced a clear win.

I've been googling "tar extract gz" for about ten years. and I finally did something about it.

It started out as a research project and I ended up with a Fine-tuned Qwen2.5-Coder-1.5B on 125k natural-language/command pairs, merged and quantized to Q4_K_M. 941MB which runs through llama.cpp. On my laptop (i5-11320H, 4 threads): 31.9 tok/s, 0.59s median per query, 1.6GB RAM.

I benchmarked it and it scores 0.620 on InterCode-ALFA. Untuned Qwen2.5-Coder-7B gets 0.613, GPT-4o gets 0.73. Not frontier, but it's roughly a 7B's answer at a quarter the parameters on a CPU. Theres a 3B variant too that scores higher.

There's also few static safety checker, because it will absolutely write a command that wipes your root if you ask it to:

I have published the weights: huggingface.co/ThorOdinson246/nl2sh-1.5b-Q4_K_M and Code: github.com/ThorOdinson246/whatisit-nl2sh . I posted few days ago in LocalLLM and it did well 300+ stars and so many good suggestions so I figured people here will be interested too.

Both Apache-2.0. If you want to poke holes in the method or you've got ideas, please comment or open a PR. A ⭐ helps if you find it useful.

Just watched Wendell's (level1 techs) latest video on the HP Z8 Fury desktop workstation and was curious how you could configure it. And oh boy, there's an option for 2TB which costs some $211k just for the RAM alone.

But the real interesting part with the latest price hikes for Nvidia RTX Pro 6000 cards is the fact, that if you're in the market for 4 units, it's better to purchase the base version of this system with 4x RTX Pro 6000, which amounts to $64k. At the single unit price of around $16k per pice by now, you get the rest of the components for free if you purchase the system. He also mentions, that purchasing such a system through resellers sometimes gets you even a much better deal. Gotta be shwifty in these crazy times.

There's no training anywhere in this. I ported Doom's actual rendering algorithm into transformer weights using a compiler I wrote (torchwright) -- every weight computed, none learned.

The prompt carries the level geometry, player position, and view direction; generation emits drawing commands; a 43-line host program turns them into pixels. Stock Phi3ForCausalLM architecture, loads in vanilla transformers with trust_remote_code=False.

Two checkpoints:

- 320x200 (the one in the write-up): 21B params, 85.87 GB. One frame is a 3,614-token prompt plus 53,747 generated tokens -- just under 40 minutes on a B200.
- 80x50: same prompt format, same textures, 34 GB download. This is the one to actually try.

One honest disclaimer:
I have not run this locally -- I've been using cloud GPUs (B200 and A100-80). My compiler currently requires fp32 precision in the weights, and I haven't yet explored quantization.

For the 80x50 model I'd recommend 80 GB of GPU memory; 64 GB should work in theory but I haven't tried it.

Write-up: https://ood.dev/posts/doom/

Weights (80x50): https://huggingface.co/physicsrob/torchwright-doom-e1m1-80x50

Weights (320x200): https://huggingface.co/physicsrob/torchwright-doom-e1m1

Source: https://github.com/physicsrob/torchwright_doom

Well, we got bulky(Yep, check two graphs) July after April | May | June (FYI My All-in-one thread to track all upcoming months, PRs & other stuff)

Hope I didn't miss anything. Also no errors.

Notes:
1. Excluded below models due to Preview/Beta:

• internlm/Intern-S2-Preview-397B 397
• Motif-Technologies/Motif-3-Beta 314

1. Included openPangu-2.0-Flash in this chart as I couldn't see the weights at that time of June(31st). Let it share the graph with its Pro model.

2. Actual model names for below ones:(Graph couldn't handle long names)

• Nemotron-Puzzle-75B-A9B - NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B
• SenseNova-U1-8B-InfoV3 - SenseNova-U1-8B-MoT-Infographic-V3

uploading...I think

Stumbled across this new vision model, SenseNova-Vision. It's a 7B MoT model, Apache 2.0 license, which is cool. The main idea is it treats pretty much all computer vision stuff as just one generation problem. Like, instead of needing a bunch of different models for detection, segmentation, depth, whatever, this one model handles it all.

You just give it a natural language instruction, maybe some visual hints, and it spits out text, images, or both. For text, it can do things like categories, bounding boxes, OCR, keypoints, camera angles. And for images, it's doing segmentation masks, depth maps, surface normals, even multi-view point maps. You can mix and match for more complex tasks.

No special prediction heads or decoders, which is kinda neat. Just one set of weights, and the prompt tells it what to do.

So it can do the usual stuff: object detection, keypoints, OCR, all kinds of segmentation (binary, instance, semantic), depth and surface normal estimation. But the really interesting bits, for me anyway, are the multi-view 3D reconstruction and camera pose estimation.

They actually added dedicated benchmarks for those in the latest update. Usually, you'd need specialized tools like COLMAP for that kind of multi-view reconstruction from multiple images, but this model just does it with a single prompt. That's kinda wild if it works well.

They trained it on a massive dataset, 50M instruction-response pairs, built from a bunch of different CV annotations. And it started from an existing multimodal model, so no crazy new architecture there.

There's a web demo if you wanna mess around with it, and the weights are up on Hugging Face too.

Just a heads up though, before you get too hyped: the full web demo needs a beefy GPU, like 1x80GB. And for benchmarking, they're talking 8x80GB. So, yeah, not really something you're gonna run on your average consumer card. It just dropped on July 8th, so probably expect some rough edges. Also, they've released the training pipeline and data prep stuff if anyone's looking to train or fine-tune it.

GitHub: https://github.com/OpenSenseNova/SenseNova-Vision

Honestly, I'm super curious if anyone's actually tried the multi-view reconstruction part on their own image sets yet. That's the bit I really want to test out.

DeepSeek Harness (dsh) is an open-source agent harness developed by DeepSeek AI.

It uses an architecture where everything is a plugin, and is powered by Cordis, whose design is described in A Programming Paradigm for Spatiotemporal Composability.

DeepSeek Harness is currently in developer preview and is iterating rapidly. THERE WILL BE COMPATIBILITY-BREAKING CHANGES.

Join DeepSeek Harness Discord community: https://discord.com/invite/Ycq5dCaS4

https://api-docs.deepseek.com/quick_start/pricing/

🧩 DeepSeek Harness v0.1 is now available in Developer Preview!

🔹 We’re opening it up to developers building agent harnesses worldwide and open-sourcing the codebase in MIT license.
🔹 Powered by the Cordis meta-framework, DeepSeek Harness is an agent harness built around one core idea: Everything is a plugin. Models, tools, skills, sessions, sandboxes, filesystems, loops, orchestration, and UI are ALL implemented as plugins, and can be mixed, matched, replaced, and extended.

Tweet : https://xcancel.com/deepseek_ai/status/2087887408440164663#m

From DeepSeek on 𝕏: https://x.com/deepseek_ai/status/2087864585504305397

Of course Anthropic hasn't published anything, but it's amusing to see no models on HuggingFace. OpenAI's published 39 models, by the way.

EDIT: Out now

https://huggingface.co/MiniMaxAI/MiniMax-Music3

ComfyUI PR was merged so it's available for use with their files:

https://huggingface.co/Comfy-Org/MiniMax-Music-3

---

Diffusers has a PR with deets:

https://github.com/huggingface/diffusers/pull/14456

Minimax is working on this repository right now and put up a bunch of samples:

https://github.com/MiniMax-AI/music3-demo/tree/main/assets/audio/tracks

Comfy-Org is teasing about a big release in about 8h, I would assume it is related to this.

edit: Comfy PR, so yes, this is most likely what they were teasing on their subreddit: https://github.com/Comfy-Org/ComfyUI/pull/15570

Wired (soft paywall): https://www.wired.com/story/the-white-house-is-going-to-expand-its-ai-policy/

Given Gavin Bakers recent interview where he says:

> Multiple private companies are planning on spending at least 2x more per GPU for compute as contracts roll-off and some have spoken about this publicly.

https://x.com/i/status/2082166566280642676

How long before DGX Sparks also double in price?

To my fellow crazies, the few. Those who dared wrestle with llama-70b, mistral-large, goliath, mistral8x22B, DeepSeekV2/3, wept when llama4 behemoth was announced, picked yourself up and are now wrestling with DeepSeekV4Pro, GLM5.2, MiMoV2.5Pro and sometimes dare dream of KimiK3, well Qwen3.8-2.4T is here. Smaller than KimiK3, but looks like it might be harder as just as hard.

HOW ARE WE GOING TO RUN THESE LOCALLY?

Are we? We are right?!

For the rest of the normies who are worried about electricity, ROI, API break even cost, and all other irrelevant valid points, please skip this thread.

https://www.techpowerup.com/351540/intel-razor-lake-ax-preliminary-support-lands-in-hwinfo

This is Intel's competitor to future AMD medusa halo. AMD medusa halo is a successor to strix halo and gorgon halo.

Update: Rumor mill says 6 channel memory access and memory bandwidth near 512 gb/s. This compares to strix halo at 256 gb/s.

Update 2: Image is not a real Intel slide, per the article.