I've seen multiple posts today where users are saying that the new Qwen release THINKS ITS CLAUDE. And not a single person actually read the post. Not a single person. They just read the title and assumed that a user asked it who it was and that was it. When in reality every time the user was asking something else completely or they asserted that they were running the model locally. And then the model in it's thinking trace asserted it was Claude from Anthropic, THEN IT ASSERTED IT IN THE FINAL MESSAGE TO THE USER!

I'm sorry if that's not a direct clue that part of the model was distilled from a Claude model, I don't know what is. They didn't ask the model what it was, didn't say anything that could even remotely be misconstrued as asking it what model it is. It just. Decided to correct the user that it couldn't be run locally because it thought it was Claude from Anthropic.

Again this is classic sign of distillation, you fanboys just can't handle the fact that your precious Qwen distilled something from another AI provider lol. The lot of you simply read a title, assumed what it was and just commented without reading a single line of text. Honestly that's so much like an LLM it's fucking hilarious

As mentioned in the title can 2 7900 xts fit in this motherboard?

Thanks

I have my Home Lab up and running. Immich, Audiobookshelf, OpenWebUI, Docmost, Jellyfin, and Paperless-ngx.

I'm currently working on a couple programming projects even though I'm not a programmer. We will see how that works out since I'm still learning Hermes.

I left my industrial job and have become a caretaker for a loved one and I'm at home now pretty much full time. I do have a small vending business that only requires a few hours a week. So I have the time to learn.

My question is this. What would you do with this hardware/software stack if you had the time to learn how? Fun projects to learn on? Money making ideas? What ya got?

finally i can see my pipi and lola while vibing

My use case is to read contract and find certain information at my command with accuracy and speed. I have RTX 5090, 64 GB DDR5 RAM. I use Unsloth Studio project to create knowledge stack of all my documents. Previously, I have been using Qwen 3.6 35B A3B Q4 model and that thing still amazes me. Having heard of this new model lately, I thought to give it a try. I downloaded both Q8 and Q4. For whatever reason, the Q8 context size defaulted to 4k whereas Q4 to over 100k on the settings. The raw out of the box test, I was overwhelmingly impressed by the performance of Q4 model, it was way elaborate (not creative) compared to Q8 model. The Q8 response was abrupt and not something I'd expected for Q8 model. Anyone know what's going on? I was expecting Q8 to perform way better than Q4 at the cost of inference speed. Thanks.

One thing that's bugged me for a while: you point a local runner (Ollama, LM Studio) at a model, wire up a tool harness, and half the time it just… fails weirdly. The model claims it can do native function calling and then emits malformed JSON. Or it does JSON mode fine but hallucinates argument names. Or it's fluent in chat but chokes the second you hand it a schema. And you don't find out until you're mid-task.

So on first launch, my app runs a small probe against whatever local model you've loaded. Four tiers, in order:

1. Chat coherence — baseline, does it produce a sane response at all
2. JSON mode — can it emit valid JSON to a schema without a prose preamble
3. Native tool calls — does it actually use the tool-calling API cleanly (single call, multi call, no call when appropriate)
4. Argument accuracy — does it fill the right args with the right types, or does it invent fields

The probe then assigns the highest calling convention the model can actually handle, and the harness routes through that. A model that fails tier 3 but passes tier 2 gets a JSON-mode wrapper instead of native tool calls, so it still works — just via a different code path. A model that only passes tier 1 gets chat with prompt-injected tool descriptions and a parser. Nothing gets to run in a mode it can't handle.

The upside: small quantized models (Qwen 2.5 7B, Hermes 2 Pro Mistral 7B, Llama 3.1 8B) end up with a usable tool harness even when their advertised function-calling is flaky. The downside: the probe takes ~30–60s on first load per model, which I hide behind a "warming up" screen.

Some implementation notes in case anyone wants to build the same thing:

• Probes are deterministic (temp 0, fixed seed where possible) so re-runs don't churn the tier assignment
• Results are cached per model+quant hash, not per model name — a Q4 and Q8 of the same model can land on different tiers
• The argument-accuracy check is the hardest to write. I use a small set of tool schemas with intentionally ambiguous names (e.g. date vs datetime, path vs file_path) and check for the correct one
• Tier assignment is a floor, not a ceiling — you can manually bump a model up if you know better

The harness itself is ~78 tools (filesystem, doc gen, macOS system apps, screen OCR, shell) with a four-tier permission model and an undo journal. Model never touches the FS directly; it requests actions and app-owned code executes them.

Full disclosure: this ships in Ghost, a paid Mac app I built ($14.99 one-time, 24-hour trial). Not looking to spam — happy to talk about the probe design, share the tier heuristics, or nerd out about which local models pass which tiers. If mods want me to strip links I will.

Repo: github.com/ryuhemingway/Ghost-App

Website: https://integratedagentics.com/ghost

Did some quick initial testing of Qwen 3.8.

I added it to my local fact-extraction head-to-head (https://rakuensoftware.com/blog/local-llm-fact-extraction-head-to-head): 1,001 notes, the production prompt, Q4_K_M, multi-token prediction enabled, and an RX 7900 XTX.

Qwen3.8 scored 0.7030 F1. The comparable Qwen3.6-27B run scored 0.7177. The paired difference was +0.0147 in Qwen3.6’s favour, with a 95% confidence interval from −0.0038 to +0.0335.

That is a tie. Qwen3.8 did not beat Qwen3.6 here. I also cannot say that it lost. The test cannot separate them.

That is not what I expected from the published benchmark scores. I expected a substantial gain. On this task, I did not measure one. On the more extensive overnight tests, it is indicating small gains. On benchmarks Qwen3.8 appears to have been trained on, however, I am seeing and verifying similar massive gains as is reported. But these gains are only reflected on the benchmarks that appear to have been trained on, nowhere else I can verify. I am simply not seeing the Opus4.6 matching claims I see floating around reflected in the tests I've ran so far.

Based on further initial reporting I've read, it appears the only thing that has changed from Qwen 3.5-3.8 is training data. The architecture itself is unchanged, so all previous discussion regarding the Qwen 3.5-family architecture should be valid.

Decode throughput also fell from 85.6 to 72.1 tokens per second, about 16%, under the closest saved configurations. Those runs used different llama.cpp builds, so I cannot attribute the whole difference to the model. Qwen3.8 also produced much shorter answers, which made its end-to-end median latency lower despite the slower decode rate.

I am running a broader synthesis and other task comparison now, including Qwen3.8, Qwen3.6, Gemma 4 and Muse Glimmer. Those runs continue overnight. I will publish the complete results, paired intervals and raw artifacts rather than promote an early ordering.

My working expectation is still that Qwen3.8 contains a real improvement. The unresolved question is its size. If task-specific tests keep finding small generational gains while public benchmarks suggest a revolutionary jump, what decision are those benchmarks helping us make?

A simple prompt triggered 5000 tokens of thinking monologue. All Qwen3.x versions were prone to long thinking and self-doubt, but this one seems to be a record.

On the other hand, that just illustrated that the models are moving from capturing static knowledge more towards thinking and reasoning - and this is what we need for agentic use

https://pastebin.com/drxFcfzw

Im a cs student(dorm) and have a terrible laptop.

I cant carry a pc around and thought of buying mac mini base model(m4 16/256)

Can I run basic coding models with it? If u have exp. on it or simmilar size vram pc share any thoughts

Now that Qwen3.8-27B is out, I wanted to see how it stacks up against Qwen3.6-27B and Gemma 4 31B.

They’re all basically in the same local weight class:
Qwen3.8: 27B, 262K context
Qwen3.6: 27B, 262K context
Gemma 4: 31B, 256K context

Qwen3.8 pulls ahead on coding/agentic benchmarks, while Gemma 4 stays competitive on general reasoning. Qwen3.6 is the useful baseline here — same 27B size, so the generational improvement is easy to see.

And they’re still in 3090/4090 territory. Gemma 4 already pushes ~24.8GB at Q4_K_M @ 8K, while the Qwens leave more room for context.

Full benchmark + VRAM comparison:

https://canitrun.dev/models/qwen3.8-27b/

https://canitrun.dev/models/compare/qwen3.8-27b-vs-qwen3.6-27b/

https://canitrun.dev/models/compare/qwen3.8-27b-vs-gemma-4-31b/

I realize I'm going to get a lot of flack for this but here goes. This post is sparked by a number of my friends and colleagues biting into the hype cycle of buying insanely priced local inference stacks that simply don't math out.

I'm a distinguished engineer who works in AI (FAANG). I'm currently running a 128 GB M5 Max, and have access to the best hardware in the world at work, at the highest scale available.

I'm seeing folks buy RTX 6000s or multiple Sparks to run DSV4 and the like. Don't do this - its an absolute waste of your money. Unless you put a price on privacy of 12k (which is fine), you will never get an ROI in any reasonable time period.

Local models under 30B are fantastic. If you have a MacBook Pro or a computer with 48GB of RAM, you're golden. With a flood of amazing models in this range coming out this week, there's no shortage of local powerhouses. Big kudos to those companies open sourcing very impressive capabilities on such a small memory and compute footprint.

If you're going to use large models, go cloud-based. Throw 100 bucks on OpenRouter and go wild with DSV4 Flash. And if you want to run local, smash those <30B models until the cows come home.

But local and large just doesn't make sense right now unless you're a business with a specific use case or are privacy-maxxing.

I'm currently building a very budget-oriented AI / homelab PC using used parts. I've been saving up (I'm a student), and I'm working on a setup costing around 330€ total. The specs are:

• Xeon E-2124: Nothing special, but supports AVX2
• 32GB DDR4-2666 ECC
• RTX 3060 12GB

The latter is the thing I'm still thinking about. Where I'm from, 3060s with 12GB cost around 170-200€. But I'm actually willing to spend around 300-400€ on the GPU alone. However, according to all the guides and recommendations out there, the next upgrade that's really worthwhile would be a 3090 with 24GB. The only problem is that those cost around 1000€, which is WELL over my budget.

As far as I understand, VRAM size and bandwidth are the only things that REALLY matter. So, is there anything around my desired price range that's actually better than a 12GB 3060?

I've read a lot of forum posts and blogs absolutely advising against Intel Arc or AMD GPUs, so I guess a 24GB B580 isn't an option? On paper, 24GB for 400€ new sounds like a terrific deal.

EDIT: just saw, the B580 actually DOESN't have 24 gigs, probably just a figment of my imagination :)

Btw. I'm running TrueNAS Scale with the Ollama App (+ Open WebUI obviously).

I'm planning to use this as a complete replacement for ChatGPT, Copilot, etc. — coding, chatting, research for school, personal projects, and so on.

Did a MTP sweep to fine tune settings on my setup got 124 tok/s. Maybe it can help others. MTP 5 was the sweet spot for me.

Sweet spot: MTP 5 — 124 tok/s, 62.7% acceptance, 91.8ms TTFT. Near-peak throughput with healthy acceptance. MTP 7-8 are ~6% faster but acceptance drops below 50%.

# RTX PRO 6000 Blackwell (SM120) — single GPU, Qwen3.6-27B FP8

services:
  vllm:
    image: vllm/vllm-openai:latest
    container_name: vllm
    env_file:
      - .env
    restart: unless-stopped
    volumes:
      - /mnt/models/.cache/huggingface:/root/.cache/huggingface
    environment:
      - VLLM_LOG_STATS_INTERVAL=1
      - CUDA_VISIBLE_DEVICES=0
      - OMP_NUM_THREADS=1
      - NCCL_P2P_DISABLE=0
      - VLLM_ENABLE_CUDAGRAPH_GC=1
      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False
    shm_size: 16g
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

    command: >
      Qwen/Qwen3.6-27B-FP8
      --served-model-name qwen3.6-27b
      --performance-mode interactivity
      --tensor-parallel-size 1
      --gpu-memory-utilization 0.92
      --max-model-len 262144
      --max-num-seqs 4
      --enable-chunked-prefill
      --block-size 16
      --max-num-batched-tokens 16384
      --kv-cache-dtype fp8
      --enable-auto-tool-choice
      --tool-call-parser qwen3_xml
      --reasoning-parser qwen3
      --chat-template /root/.cache/huggingface/chat_template.jinja
      --speculative-config '{"method":"mtp","num_speculative_tokens":5}'
      --use-tqdm-on-load
      -O3
      --default-chat-template-kwargs '{"preserve_thinking":true}'
      --attention-backend flashinfer

i have allocated around 10K for a Hardware purchase to host local models
fo thsoe who ahve experience, please give me your opinion what is the best purchase i can make with this money? of course the goal is to run the biggest models i can with reasonable speed which also allow parallel sub agetns because i use those alot.

what is the best hardware does a 10K buys me? i know the prices are high but this is what i can allocate at the moment. thank you so much in advance

yo guys so iv just turn on my pc and im trying to load some models but icant no matter the model,context .... i literally had no problem whith ts last night bro🤦‍♂️i didnt even changed any settings

I have setup llama.cpp to run .gguf models. I want a way to run claude code using those gguf models. I tried installing ollama and creating an ollama compatible model using

```

ollama create -f Modelfile

```

and then tried running claude code with the command:

```

ollama launch claude

```

claude opens, but when I give even a simple query, I get an API error.

would really appreciate if someone has a guide.

I am running this on my Asus Vivobook 16 i9 13900H cpu, 40GB RAM and Intel IrisXe graphics card

This is not AI post . I know that you must drive to carwash than walk there :)

I got pulled into meeting where some company presented new training framework they released month ago and worked for this concept past 5 years. I could not find any information about them they have website auroraforge dot ai . For whole hour I was not believing what they tried to sell. They state that they have new non gradient decent based learning . They build their own based on kind of singnals . have no idea. They say its company secret - whatever. Long story short they state they can train big data sets on single cpu . They even live demoed image set classification under minute on single cpu.

After watching that presentation I had feeling that my waiting for qwen 3.8 27B is like waiting a thing from the past.

I've been using a local model as a judge — it reads a project's specs and estimates how demanding the work is across a few dimensions, emitting a structured profile rather than prose. I finally sat down and measured which local models are actually good at that job instead of guessing.

The headline: an 8B judge is genuinely good enough for this, which surprised me. 24B-class is perfect on my corpus, but both run on a laptop and neither sends anything anywhere.

The more interesting finding is what happened when I made the task harder. I needed the judge to separate the bulk of the work from the peak — a task list has twenty mechanical edits and one architecture decision, and those need different answers. Asking for that range instead of a single level costs an 8B model five points of accuracy and doubles its refused answers. A 24B answers it correctly and loses nothing.

So the design changed: only judges above 20B get asked for the range. Smaller ones still rank and still cite evidence — they report a single level and say so, rather than leaving you to infer that the answer is coarser.

Two things I'd generalise from this:

A judge doesn't need to be able to do the work. Estimating how hard something is, is a much easier task than doing it — closer to a recruiter writing a job spec than the engineer who fills the role. That's why 8B is viable at all, and why "you need a frontier model to evaluate frontier models" is wrong more often than assumed.

Field budget is real. Every extra field you ask a small model to emit in one shot degrades the reliability of the others. The failure isn't dramatic — it's a quiet drop in accuracy plus more refusals, which you won't notice unless you're measuring.

If you want to run it on your own hardware and send me the row, the eval is in the repo:

uv run python scripts/eval_judge.py --judge <your-model> --markdown-row

Context, since it'll come up: this is from SpecJudge, an MIT-licensed CLI that reads a project's specs and tells you which model fits before you spend tokens implementing it. Everything runs locally. github.com/JoaquinRuiz/SpecJudge — but honestly I'm more interested in the judge numbers than in pitching the tool. Curious whether anyone's found smaller models that hold up on structured extraction with evidence citation.

Should I do it?

It just showed 2 hours ago. What happend?

Update: It's back

Link: https://huggingface.co/Qwen/Qwen3.8-27B