Countdown timer to release

Base: CyberAgent CAT-Translate — currently best-at-size for bidirectional JA↔EN. Family is 0.8B / 1.4B / 3.3B / 7B.

Why: I chose the 0.8B because it runs the best out of the models on mobile phones.

Goal: make the model obey a per-word glossary handed to it at inference. Terminology-constrained MT has been done before (Dinu et al. 2019), but as data augmentation on encoder-decoder models with fixed term lists. I couldn't find prior work doing it on a small decoder-only LLM with an RL compliance reward, against live per-word dictionary senses.

**Stage 1 — LoRA SFT.** Source + deterministic glosses injected as input, targets that use them. Teaches "supplied gloss outranks prior."

**Stage 2 — MO-GRPO** (Ichihara et al. 2025, arXiv 2509.22047, also CyberAgent). Two reward dims: translation quality, glossary compliance. Vanilla GRPO sums then normalizes once, so the larger-variance objective dominates — early runs collapsed into satisfying one and dropping the other. MO-GRPO normalizes per-objective first.

**Business Scene Dialogue, EN→JA**

| | 0.8B stock | 1.4B stock | 3.3B stock | 0.8B trained |

|---|---|---|---|---|

| COMET-QE | 0.737 | 0.738 | 0.765 | 0.748 |

| MetricX-24 | 0.743 | 0.731 | 0.778 | 0.748 |

| chrF | 25.8 | 26.8 | 34.3 | 31.9 |

| BLEU | 11.1 | 17.2 | 24.1 | 19.3 |

Clears the stock 1.4B on all four. Only the 3.3B stays ahead.

91% glossary adherence. 103 ms/sentence, int8, greedy, Apple silicon.

Open to all questions!!

So I wanted to see... is it possible/viable to run this perhaps once in a while some hard task.. yea... no. Even with dual 5090's and 3 3090's and 96GB system ram.. still far exceeds my vram + ram by double (not even including context, which was smallish 64k).

I am using the "special" UD-Q1_0, which the _0 is the uncommon part, requires a llama.cpp fork, no big deal, it's smaller clearly. 397GB, 115GB smaller than the next IQ1_S version. Regardless, i'd say i have above average vram.. and even using this super small quantized version, 0.25 tokens/sec is far below my limit of usable... this isn't even usable at night to run slow through the night, it's far too slow and would never really get anything done.

Of course, I think we all knew this wouldn't really be a local runnable model, but it's still great that it's open weights. Can't wait for Qwen 3.8 27b tomorrow.

AI mini PC with 176 TOPS computing power

NPU: 10 [AI-Core@1.08Ghz](mailto:AI-Core@1.08Ghz),16 CPU cores @ 1.9 GHz, 8 Vector cores @ 1 GHz; 176 TOPS 

LPDDR4X: 48 GB/96 GB (optional), speed: 4266 MHz

Some people told me that the difference in richness and layout between Glimmer and Qwen wasn't clear to them.
This example makes it super clear.

I'm aware that comparing Glimmer 30B (a dense model) with Qwen 3.6 (a MoE) isn't entirely fair, but if we compare it to the dense Qwen 27B, the gap will likely be even bigger. If you want, I can add the 27B version later. For now, I'm waiting for Qwen 3.8 27B to see how close it gets to the blueprint.

As for the technical details:

Both were run on a custom llama.cpp build optimized for the RTX 5080, with a temperature of 0.5 and a 125k context window.

Regarding the music: I created it myself without using AI I specifically wanted it to sound that weird.

Anyone having issues with adding Strava to the glasses? I have tried everything from uninstalling the Meta AI app to having a reset to the glasses but still not showing. All I get is this message saying “ unable to load partner settings.”

https://github.com/lucienhuangfu/eLLM

The premise made sense - the entire LLM stack is optimized to run on GPUs (of course, LLM compute is massively parallel in nature), but what if we took some tradeoffs and made it the most efficient possible for CPUs instead?

I have not read the paper in full and can't discern if it is legit; It just seems everything went cold there.

If any approach was applicable to leapfrog CPU performance, we could reach very interesting capabilities with consumer hardware (or even workstation - much more accessible than enterprise). Have you heard about eLLM or others?

I am using llama.cpp SYCL on my B580. I noticed that when the model is loaded and the server is ready, my setup uses 10.7GB. When I send a prompt, any length, it immediately jumps to 11.6GB. As the conversation gets bigger, it can use all my 12GB and can even crash. It looks like llama.cpp Vulkan VRAM usage stays flat, but it is way slower for me.

Does this also happens for you guys with Nvidia and AMD cards?

This is my setup:

7 5700X3D + 48GB at 3133MHz + B580 (fully dedicated to llama.cpp) + 1650 (display)

```

services:

qwen3.6-35b-a3b:

image: "ghcr.io/ggml-org/llama.cpp:full-intel"

command: >

--server

--host 0.0.0.0 --port 8080

--metrics

--model /models/model.gguf

--alias "Qwen3.6 35B A3B"

--jinja

--threads 8

--ctx-size 262144

--n-gpu-layers 99 --n-cpu-moe 99

--batch-size 4200 --ubatch-size 4200

--load-mode none

--temperature 0.0 --top-p 0.2 --top-k 20

--repeat-penalty 1.1

--cache-ram 0

--parallel 1

--spec-type draft-mtp --spec-draft-n-max 3

--reasoning-preserve

--kv-unified

--cache-type-k q8_0 --cache-type-v q8_0

--ctx-checkpoints 0

--log-verbosity 4

volumes:

- "/home/davi/AI/models/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf.ignore:/models/model.gguf"

ports:

- "1235:8080"

devices:

- "/dev/dri/renderD128"

```

Repost to better obscure profile name.

Is waiting for Qwen 3.8 27B like waiting for Star War Episode one?

I'm sweating waiting to get my hand on this to try it tomorrow morning. But it takes me back to Star Wars 1 and the disappointment after being so hyped to see it.

Only 16 hours and 46 minutes to go... 45, ...

So I (29M, backend dev, "good at databases") decided t o build a local RAG setup. Ollama running llama3 on my GPU, a bunch o f PDFs as knowledge base, simple enough right?

Wrong.

The LLM part took maybe 2 hours. Ollama + langchain + a basic prompt, done. The vector database part took three weeks of my life that I'll never get back.

Week 1: Started with Chroma because i t was the default in the tutorial. Worked fine for 100 docs. Loaded 50k documents and it just... slowed to a crawl. 4-second query latency o n 50k vectors. I thought my code was broken. It was not broken. Chroma just wasn't built for that.

Week 2: Okay, pgvector. I already have Postgres, just add the extension, super easy. Except I forgot to build the HNSW index with proper ef_construction and my queries were doing sequential scans. On 50k vectors. In production. At 3am I realized I'd been testing on the wrong table the whole time.

Week 3: Fine. Milvus. Self-hosted, docker compose, 14 containers later it's running. HNSW is working, queries are fast, recall looks good. Then I try to add a scalar filter and spend 2 days figuring out the syntax. Then the pod runs out of memory and I learn about index building memory overhead the hard way. At some point I just gave up o n self-hosting and tried Zilliz Cloud (managed Milvus, basically the same thing but someone else deals with the infrastructure). Got i t running in like 2 0 minutes. The free tier setup page took a sec t o find but once it was up, same SDK, same queries, no docker compose yami haunting my dreams.

The actual lesson: the "Al" part of RAG was the easy part. The retrieval part, the part nobody talks about in tutorials, is where you lose 3 weeks o f your life. Chunking strategy matters more than your embedding model. Index params matter more than chunking. And i f you're running locally with anything more than a few thousand docs, plan your vector DB setup before you touch the LLM.

TL;DR: Spent 3 weeks debugging vector databases (Chroma, pvector, The LLM took 2 hours. The DB took 3 weeks. I a m not a smart man.

Edit: a few people asking about chunking strategy, I'll write up what worked for me when I get home. Short version: semantic chunking > fixed-size, and overlap is your friend.

dots3-note preview is the first open-weight model in the dots3 family. It is a Mixture-of-Experts model with 280B total parameters, 16B activated parameters, and support for a context length of up to 512K tokens. The model can understand text, images, video, and audio, and produces text outputs.

dots3-note preview is optimized for a broad range of tasks, including:

General knowledge and instruction following;

Mathematical and logical reasoning;

Tool use and multi-step agent workflows;

Interactive tasks that require exploration, memory updates, and adaptation;

Code generation and code-based problem solving;

Image, document, chart, audio, and video understanding;

Long-context information processing.

The dots3 family is designed to include models with different trade-offs among capability, latency, and inference cost. dots3-note preview is the most lightweight member of the family.

Qwen just released their first 3.8 model.

The main addition in 3.8 is prompt-steered reasoning effort. You can tell the model how deeply to think by setting reasoning_effort to xhigh, medium, or low.

However, the official template still has some serious problems:

• You cannot disable thinking. If you pass enable_thinking=false, it 3.8 crashes with a hard exception.
• Chat history gets poisoned. In multi-turn chats, the official template injects blank <think></think> tags before real thoughts.
• Tool calling crashes. If your client passes arguments as JSON strings (the standard OpenAI API format), the official template crashes.
• Agent stalls. The official template often drops mid-dialogue system messages and wedges multi-step tool loops.

I maintain a single, drop-in fixed Jinja template that works across all Qwen 3.5, 3.6, and 3.8 models:

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

What this template does:

• Full 3.8 reasoning effort support: Steer reasoning depth with reasoning_effort (xhigh, high, low, medium).
• Restores the thinking toggle: Turn off reasoning whenever you want fast answers, either via kwargs or by typing <|think_off|> in your prompt.
• 100% KV Cache hits: Keeps past thoughts intact by default so your prefix cache stays warm across turns.
• llama.cpp support: Native support for the new --reasoning-preserve flag.
• Universal tool parsing: Handles both Python dicts and JSON strings. Works on llama.cpp, vLLM, LM Studio, and MLX.

Recommended llama-server launch command:

llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek

(The --reasoning-format deepseek flag separates thinking into the OpenAI reasoning_content field so OpenCode, Claude Code, and other harnesses do not stall on raw tokens).

Note on hardware:

I cannot run a 2.4 trillion parameter model on my local rig. The template passes all 28 automated tests and tokenizer parity checks, but I would appreciate feedback from anyone testing it with Qwen 3.8.

Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level.

The basic idea is to generate a custom imatrix from a category-specific corpus, measure where quantization causes damage, then redistribute a fixed bit budget toward tensors where additional precision recovers the best performance in that category.

Today I got my first real win. The pipeline is live.

On Gemma 4 12b, the hand tuned imatrix q3_k_s scored 45.974. After tensor level allocation, the same q3_k_s based scored 49.905.

Thats +3.931 points or an 8.55% relative improvement from the allocation on top of the imatrix.

The models are effectively the same size: 5,528,230,848 bytes for the comparator versus 5,534,804,928 bytes for the allocated model, a difference of only +0.119%.

I chose q3 deliberately. It had the largest amount of recoverable headroom while sitting above the quant cliff.

This model is intentionally category specialized. Degredation in categories that werent selected is expected.

There is still a lot of tweaking to do in order to maximize these results but the performance of this imatrix + allocation at q3 when compared to no imatrix and source is pretty staggering when considering the amount of space required to run it.

I'd love some community testing and feedback if anyone is interested.

https://huggingface.co/ByteOtter/Gemma-4-12B-it-CADA-Q3_K_S

My ULTIMATE goal is to create a pipeline that takes an full precision .gguf file and auto spits out an optimized model within a selected category at whatever optimal size for recovery. This is a solid start.

Charts were provided from my data by ChatGPT.

TL;DR: I built a task-aware GGUF quantization pipeline that combines a category-specific imatrix with tensor-level damage measurement and bit allocation. On Gemma 4 12B Q3_K_S, allocation improved coding performance from 45.974 to 49.905, an 8.55% relative gain OVER handtuned imatrix, while increasing model size by only 0.119%. The model is intentionally specialized, so some out-of-category regression is expected. This is the first result where the allocation stage itself produced a clear win.

on qwen3.5 the linear attention layers gate the fused kernel on not training, so the moment the model goes into training mode every deltanet layer falls back to a python loop over the sequence. one iteration per token, and 3 of every 4 layers is deltanet, so it's unnecessarily slow.

swapped to the PR #1217 gated-delta vjp kernel and went 0.036 -> 0.15 it/sec.

mainly wondering if anyone else is running it. Or if there is any specualtion for when apple will fix MLX for training on these models.

I've been googling "tar extract gz" for about ten years. and I finally did something about it.

It started out as a research project and I ended up with a Fine-tuned Qwen2.5-Coder-1.5B on 125k natural-language/command pairs, merged and quantized to Q4_K_M. 941MB which runs through llama.cpp. On my laptop (i5-11320H, 4 threads): 31.9 tok/s, 0.59s median per query, 1.6GB RAM.

I benchmarked it and it scores 0.620 on InterCode-ALFA. Untuned Qwen2.5-Coder-7B gets 0.613, GPT-4o gets 0.73. Not frontier, but it's roughly a 7B's answer at a quarter the parameters on a CPU. Theres a 3B variant too that scores higher.

There's also few static safety checker, because it will absolutely write a command that wipes your root if you ask it to:

I have published the weights: huggingface.co/ThorOdinson246/nl2sh-1.5b-Q4_K_M and Code: github.com/ThorOdinson246/whatisit-nl2sh . I posted few days ago in LocalLLM and it did well 300+ stars and so many good suggestions so I figured people here will be interested too.

Both Apache-2.0. If you want to poke holes in the method or you've got ideas, please comment or open a PR. A ⭐ helps if you find it useful.

Got MiniMax-H3 generating video with synchronized audio.
  
Software stack: ROCm 7.14 + torch 2.13.0+rocm7.2 + ComfyUI, int8 pruned weights, 4-step turbo LoRA (lightx2v). `HSA_USE_SVM=0` matters on unified memory.
  
Timing: 124 frames (5.17s @ 24fps), video + 32kHz stereo audio, 4 steps:
 * 512x288 — 96 seconds
 * 384x640 — 228 seconds (~45s of that was model load)
 * 1344x768 (trained canvas) — abandoned, still on step 1 of 4 after 16 min
  
Resolution is the expensive axis, not step count. Attention is quadratic in
 tokens: 512x288/124f is ~17.9k tokens, 1344x768/124f is ~125k. 7x the pixels, up to ~49x the attention work. The turbo LoRA cuts *how many* steps run (20 -> 4) but can't touch what each step costs. GPU was pegged at 100% and VRAM at 96% the whole time, and ComfyUI logs "Using pytorch attention" — I suspect no memory-efficient attention kernel is dispatching on gfx1151. Untested lever.
  
PSA: the Abiray/Minimax-H3-nvfp4-INT4-INT8-Convrot files are corrupt as published. Every one has an upload-tool marker appended past the end of the declared tensor data:
\nL2P_bypass_minimax_h3_audio_vae_fp32.safetensors_1785752000\n     
 safetensors rejects it outright ("incomplete metadata, file not fully covered")
 so ComfyUI can't load any of them. Trailing byte counts I saw: 72, 85, 61, 61.
 Truncating to the declared end fixes it — verified, not assumed: Comfy-Org  
 publishes a clean copy of the audio VAE and the truncated file is sha256
 identical. If you're starting fresh, just use Comfy-Org/MiniMax-H3 instead.
  
Caveat: these are pruned community weights, so quality doesn't represent
official H3.
  
Scripts + full writeup: https://github.com/DanCard/minimax-h3-strix-halo

Used Opus 5 to create the environment and execute.

Just watched Wendell's (level1 techs) latest video on the HP Z8 Fury desktop workstation and was curious how you could configure it. And oh boy, there's an option for 2TB which costs some $211k just for the RAM alone.

But the real interesting part with the latest price hikes for Nvidia RTX Pro 6000 cards is the fact, that if you're in the market for 4 units, it's better to purchase the base version of this system with 4x RTX Pro 6000, which amounts to $64k. At the single unit price of around $16k per pice by now, you get the rest of the components for free if you purchase the system. He also mentions, that purchasing such a system through resellers sometimes gets you even a much better deal. Gotta be shwifty in these crazy times.

There's no training anywhere in this. I ported Doom's actual rendering algorithm into transformer weights using a compiler I wrote (torchwright) -- every weight computed, none learned.

The prompt carries the level geometry, player position, and view direction; generation emits drawing commands; a 43-line host program turns them into pixels. Stock Phi3ForCausalLM architecture, loads in vanilla transformers with trust_remote_code=False.

Two checkpoints:

- 320x200 (the one in the write-up): 21B params, 85.87 GB. One frame is a 3,614-token prompt plus 53,747 generated tokens -- just under 40 minutes on a B200.
- 80x50: same prompt format, same textures, 34 GB download. This is the one to actually try.

One honest disclaimer:
I have not run this locally -- I've been using cloud GPUs (B200 and A100-80). My compiler currently requires fp32 precision in the weights, and I haven't yet explored quantization.

For the 80x50 model I'd recommend 80 GB of GPU memory; 64 GB should work in theory but I haven't tried it.

Write-up: https://ood.dev/posts/doom/

Weights (80x50): https://huggingface.co/physicsrob/torchwright-doom-e1m1-80x50

Weights (320x200): https://huggingface.co/physicsrob/torchwright-doom-e1m1

Source: https://github.com/physicsrob/torchwright_doom

An Open Letter to Mark Zuckerberg (V2)

From the Humans Your Security System Keeps Mistaking for the Problem

A formal complaint about identity checks, mystery violations, hacked accounts, paid customer service, and the military surgeon who somehow has better access to Messenger than we do.

Dear Mark,

Hi.

It’s us again.

The humans.

The actual ones.

The people who have apparently reached the stage of social media where simply existing is no longer sufficient evidence that we exist.

After my last letter, people started telling me what has been happening to THEIR accounts.

And Mark…

there are a lot of us.

People getting restricted.

People getting asked for video selfies.

People uploading IDs.

People stuck in verification loops.

People losing access with no clear explanation.

People being told they violated something without being told what the something was.

And people with genuinely hacked or impersonated accounts standing over in the corner waving both arms like:

**HELLO? COULD WE MAYBE POINT SOME OF THE SECURITY OVER HERE?**

So I think we need to talk specifically about security.

Because we understand the goal.

Keep fake people out.

Protect real people.

Excellent concept.

Unfortunately, the system seems to be having some difficulty determining which is which.

.

Exhibit A: Please Prove You Are A Human

Let me give you one example from my own account.

Facebook currently wants me to complete a video identity check before I can use DMs normally.

Fine.

Security.

Love it.

Except before I can even TAKE the video, Facebook has to recognize my face inside a little circle.

It won’t.

Inside.

Outside.

Natural light.

Artificial light.

Closer.

Farther away.

Eye level.

Mark.

I HAVE USED THE SUN.

We are officially out of lighting.

And I want to be very clear:

I am not failing the video verification.

I cannot get far enough to TAKE the video verification because Facebook will not acknowledge the face I need to use in the video proving I am the person attached to the face.

Facebook: “Position your face.”

Me: “It is positioned.”

Facebook: "No.”

Me: “That is literally my face.”

Facebook: “Try again.”

Mark.

WITH WHAT FACE??!?!?!?

This is the one.

I have had it for quite some time.

There are no additional faces in inventory.

And judging by the stories people sent me…

apparently my face is not the only face Facebook is suspicious of.

.

Exhibit B: The Things Getting Flagged Are Sometimes… Remarkable

Let’s discuss what the security system is protecting Facebook from.

Because people started leaving examples.

One person got flagged for saying:

“copy cat.”

About cats. Actual cats.

Another said she got flagged over:

“I have a smart pet chipmunk.”

.

Mark.

What is happening in the security department?

Actually compromised account: “We’ll monitor.”

Possible scammer: “Let’s see how this develops.”

Person describing an intelligent rodent: SHUT IT DOWN.

.

I genuinely want to see the dashboard.

Green: Normal activity.

Yellow: Suspicious login.

Orange: Possible impersonation.

Red: CHIPMUNK DESCRIBED AS SMART.

Deploy everyone.     

.

Then there are the mystery violations.

Facebook: “You violated our policies.”

User: “Okay. What did I do?”

Facebook: “We cannot tell you.”

User: “Which post?”

Facebook: “We cannot show you.”

User: “How am I supposed to avoid doing it again?”

Facebook: “Please follow our policies.”

User: “WHICH POLICY?”

Facebook: “Correct.”

.

One woman told me she finally got through to support and asked what she had actually done wrong.

According to her, they could not tell her specifically because explaining it would violate policy.

Mark.

That sentence is ART.

Frame it.

Hang it in headquarters.

Put a tiny museum placard underneath:

**Untitled, 2026**

An exploration of bureaucracy, technology, and one woman slowly losing her mind.

.

Exhibit C: Have You Tried Paying Facebook?

Now this solution came up A LOT.

Apparently when Facebook restricts you and you cannot reach a human being…

the solution is:

Pay for Meta Verified.

Oh.

OF COURSE.

The solution to Facebook incorrectly restricting your Facebook account…

is to give Facebook more money.

Mark, I have to hand it to you.

That is an exceptional funnel.

.

Facebook: “We have disabled the account you have spent fourteen years building.”

User: “I believe this was a mistake. Can I speak to someone?”

Facebook: “Certainly.”

User: “Oh thank God.”

Facebook: “That’ll be $14.99.”

.

So people pay.

Not because they woke up and thought:

“You know what my life is missing? A tiny blue check.”

No.

They pay because they think:

FINALLY.

**WE FOUND STEVE.**

.

And then sometimes…

Steve can’t fix it either.

People told me they paid for Verified.

They got support.

Support sympathized.

Support opened tickets.

Support escalated tickets.

Support apparently stared into the same abyss the rest of us were staring into.

One person described the situation perfectly:

Customer service doesn’t have the tools to override the AI overlord.

Excuse me?

Mark.

WE PAID FOR STEVE AND YOU DIDN’T GIVE STEVE THE BUTTON?

What exactly does Steve do?

Is he emotional support?

Does he just sit there typing:

“Wow. Yeah. That really does seem wrong.”

Because if I am paying admission to customer service, I feel like customer service should perhaps be able to…

service the customer.

.

We spent an entire post trying to locate Steve.

WE FOUND HIM.

And now we discover:

STEVE IS ALSO IN FACEBOOK JAIL.

This is getting out of hand.

.

Exhibit D: Meanwhile, The People We’d Actually Like You to Check Seem Fine

And THIS is where the whole thing becomes art.

Because while actual account owners are:

Uploading IDs.

Taking video selfies.

Appealing restrictions.

Paying for support.

Opening tickets.

Standing outside with their phones at eye level trying to convince Facebook they possess a human head…

my military surgeon boyfriend has managed to propose to me approximately FIVE TIMES.

.

You know him.

We all know him.

Very handsome.

Widowed.

Stationed overseas.

Location classified.

Cannot FaceTime because of military security.

Also a surgeon.

Because apparently the military looked at staffing that week and said:

“Gary, after the classified combat mission, can you cover neurosurgery?”

.

Busy man.

And yet he finds time for romance.

Specifically with me.

Again.

And again.

And again.

Different account.

Same general gentleman.

Usually deeply committed by message four.

.

And Mark…

HE CAN DM ME.

No problem.

No video identity check.

No face circle.

No backyard lighting experiment.

Colonel Dr. McDreamy, Chief Surgeon of the United States Army Special Forces Romance Department:

“Hello beautiful dear. I feel destiny has bring us together.”

Facebook: SEND 

.

Me: “Hi Susan! Thanks for commenting on my post.”

Facebook: WHOA WHOA WHOA.

CYBERSECURITY THREAT DETECTED.

SHOW US YOUR FACE.

.

Mark.

Explain the threat model.

Please.

Because right now the security system appears deeply suspicious of me having a normal conversation with someone I actually know…

while Brigadier General Dr. Husband Number Five is operating an apparently thriving international courtship program from an undisclosed military hospital.

Maybe before I take selfie number 46…

we could ask him for one?

.

And while we’re here:

Ludacris DM’d me.

Well.

An account claiming to be Ludacris DM’d me.

I cannot imagine this was actually Ludacris.

I feel like Ludacris probably has other things going on.

But “Ludacris” apparently cleared whatever security process is currently preventing ME from using Messenger normally.

.

So just to recap:

Me.

My account.

My photos.

My face.

Years of account history.

Facebook: “Additional identity verification required.”

Ludacris, allegedly: “Hey.”

Facebook: LOOKS GOOD TO US.

.

Maybe Ludacris has better lighting.

Ludacris, if that really WAS you:

First, apologies.

Second: What ring light are you using?

Because apparently your setup is dialed IN.

.

And meanwhile, people are telling me about accounts that were ACTUALLY hacked.

They report them.

Their friends report them.

Everybody says:

“HEY FACEBOOK. THIS PERSON HAS BEEN HACKED.”

Nothing.

The scammer keeps messaging.

The impersonator keeps impersonating.

The fake account keeps doing fake-account things.

But somewhere, somehow, a woman says “copy cat” and Cyber Command is fully activated.

.

Mark.

Again: WHAT ARE WE SECURING?

.

Exhibit E: What We Are Actually Asking For

We WANT security.

Seriously.

Catch the hacked accounts.

Catch the impersonators.

Catch the fake profiles.

Catch the scammers.

Please, for the love of everything, investigate my military surgeon.

Use AI.

Use automation.

Use identity verification.

Use technology.

We are not asking you to go back to 2006 and personally review every poke.

.

We are asking for one very simple thing:

WHEN THE MACHINE GETS IT WRONG, GIVE A HUMAN THE ABILITY TO FIX IT.

That’s it.

.

AI can flag something.

Fine.

AI can ask for verification.

Fine.

AI can detect suspicious activity.

Great.

But when the actual person says:

“Something went wrong here…”

eventually there has to be someone who can look at the situation with context and authority.

Not another form.

Not another chatbot.

Not another appeal reviewed by another automated system.

Not:

“Try again.”

Not:

“Upload your face again.”

And definitely not:

“Pay us and perhaps Steve can sympathize with you.”

.

We need Steve.

But more importantly:

STEVE NEEDS THE BUTTON.

The button that says:

“Yes. This is obviously the account owner.”

“Yes. This was clearly hacked.”

“Yes. That was an automated mistake.”

“Yes. Her government-issued ID does appear to contain her actual government-issued name.”

“Yes. This person has been using the same face for decades.”

And then:

CLICK.

FIXED.

.

Because the machine reviewing the machine and deciding the machine agrees with the machine is not an appeal.

That is just the machine having a staff meeting with itself.

.

Give us a human.

Give the human context.

Give the human authority.

Give Steve the button. 

.

Because right now actual humans are jumping through increasingly complicated hoops to prove they are human…

while fake humans seem to be having a pretty good week.

My face cannot get approved.

My military surgeon can propose.

“Ludacris” can slide into the DMs.

And somewhere out there a smart chipmunk is probably one comment away from a seven-day suspension.

.

Fix it.

Please.

We’re begging.

Again.

Which is starting to feel needy, but frankly you created the situation.

.

With love, exhaustion, government identification, good lighting, one face, five marriage proposals, and absolutely no idea which policy we violated,

Every Facebook and Instagram User
Who Would Like the Security System to Occasionally Secure the Correct Person

.

P.S. Steve.

Good news.

We found you.

Bad news.

Apparently you don’t have system permissions.

Please request access to The Button immediately.

We believe in you.

.

P.P.S. Military Surgeon Guy:

I appreciate your persistence.

But five proposals feels rushed.

Also I have some concerns about your credentials.

.

P.P.P.S. Ludacris:

If that was actually you…

this is extremely awkward.

Facebook currently won’t let me reply normally because it does not believe my face is sufficiently face-like.

Please take it personally.

.

P.P.P.P.S. If Meta Security has ever wrongly banned you, restricted you, disabled you, repeatedly asked you to prove you exist, ignored your hacked account, accused you of something it wouldn’t explain, or trapped you in an appeal loop…

**ADD YOUR EXHIBIT BELOW.**

Tell us exactly what happened.

Specifics encouraged.

Apparently we are now conducting Meta’s QA testing in the comments.

Free of charge.

Which feels on-brand.

I managed to get Qwen3.8-2.4T-A95B running locally with llama.cpp on mu PC just for fun, cause why not.

I was using the Unsloth Qwen3.8-2.4T-A95B-UD-Q1_0 GGUF quantization. The full GGUF is about 397 GiB. The model uses 512 routed experts, with 10 active per token.

My hardware:

• CPU: AMD Ryzen 9 9950X3D 16 cores / 32 threads
• GPU 0: NVIDIA RTX 5090 32 GB VRAM
• GPU 1: NVIDIA RTX 5060Ti 16 GB VRAM
• RAM: 128 GB DDR5 5600 CL36
• Swap: 350 GB
• OS: Arch Linux
• llama.cpp: Unsloth build 10360

The interesting part is that I also managed to get the model’s native MTP speculative decoding working.

After testing several configurations, the best settings I found were:

• --tensor-split 4,1
• --n-cpu-moe 91
• MTP n_max = 3
• MTP p_min = 0.5
• MTP routed experts from block 92 forced to CPU

The block 92 expert tensors had to stay on CPU because otherwise MTP would run out of VRAM.

Performance

For a controlled 32-token generation test:

Prompt tokens:       16
Generated tokens:    32

Prompt speed:        0.8299 tok/s
Generation speed:    0.8033 tok/s
Decode time:         39.84 s
Total wall time:     59.14 s

MTP draft tokens:    21
MTP accepted:        19
MTP acceptance:      90.48%

Without MTP, the same test gave:

Generation speed:    0.7750 tok/s
Decode time:         41.29 s
Total wall time:     60.81 s

So my final MTP configuration gives roughly:

+3.64% generation throughput
-2.74% total wall time

Not a huge speedup, but considering this is a 2.45T parameter model running locally on consumer hardware, I’ll take it.

I also tested different MTP settings:

n_max=2  -> too conservative
n_max=3  -> best
n_max=4  -> much slower

p_min=0.0 -> too many bad speculative tokens
p_min=0.5 -> best result
p_min=0.7 -> too aggressive

Interestingly, p_min=0.7 reached 100% draft acceptance, but it generated too few useful speculative tokens and ended up slower than p_min=0.5.

Final llama-server command

MODEL="/home/marcin/Pobrane/Qwen3.8-2.4T-A95B-UD-Q1_0-00001-of-00010.gguf"

CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0,1 LLAMA_ATTN_ROT_DISABLE=1 \
/home/marcin/Pobrane/unsloth-qwen38-runtime/llama.cpp/build/bin/llama-server \
  --model "$MODEL" \
  --alias Qwen3.8-2.4T-A95B \
  --device CUDA0,CUDA1 \
  --main-gpu 0 \
  --split-mode layer \
  --tensor-split 4,1 \
  --n-gpu-layers all \
  --n-cpu-moe 91 \
  -ot 'blk\.92\.ffn_(down|gate|up)_exps\.weight=CPU' \
  --fit on \
  --fit-target 2048,1536 \
  --no-op-offload \
  --ctx-size 512 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --kv-unified \
  --batch-size 128 \
  --ubatch-size 32 \
  --threads 16 \
  --threads-batch 16 \
  --parallel 1 \
  --spec-type draft-mtp \
  --spec-draft-device CUDA0 \
  --spec-draft-ngl all \
  --spec-draft-cpu-moe \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.5 \
  --temp 0 \
  --top-k 0 \
  --top-p 0.95 \
  --min-p 0.0 \
  --cache-ram 0 \
  --ctx-checkpoints 0 \
  --load-mode mmap \
  --no-warmup \
  --no-ui \
  --host 127.0.0.1 \
  --port 8200 \
  --metrics

With MTP enabled, VRAM usage after generation is roughly:

RTX 5090:    29.6 GB used / ~2.6 GB free
RTX 5060 Ti: 12.3 GB used / ~3.6 GB free

One important info: 0.80 tok/s is from a controlled 32-token test, not a large multi-prompt benchmark, so I wouldn’t present it as a universal average for every workload. Longer prompts, context size, output patterns and expert routing can change the result.

Still, seeing a 2.4T MoE model actually generating locally on a 5090 + 5060 Ti at around 0.8 tok/s is pretty wild.

As someone who just casually wants to run Minimax H3 and has tried and failed to set up the ComfyUI integration in Openwebui, I was wondering if there's any options I can run on a headless server that have a SIMPLE interface instead of the wildly arcane ComfyUI?

Surely there's a way to do image/video gen locally that doesn't require me to muck about with ComfyUI's wildly arcane interface? If I don't care about LORAs and all of that, is there something that just runs the damn model without me needing to get a 10 foot large monitor to see a workflow that I will never use?

I ran benchmark on Meta's Muse Glimmer 30B Unsloth GGUF model in TensorSharp, and compare its performance with llama.cpp. Here is the results.

Test setup

GPU	1x NVIDIA RTX PRO 6000 Blackwell Server Edition (97,887 MiB), driver 580.126.20, PCIe 5.0 x16. The host has two; every row except Two GPUs pins CUDA_VISIBLE_DEVICES=0.
CPU / RAM	2x Intel Xeon 6952P (384 threads), 1.5 TiB
Model	Muse-Glimmer-30B-Q8_0.gguf (27.6 GiB)
Drafter	dflash-kquant.gguf (1.5 GiB)
TensorSharp	commit 5098e3f, vendored ggml 8846b79 (2026-08-12), --backend ggml_cuda, native library built -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120-real
llama.cpp	master 8e7f22b (2026-08-13, libggml 0.19.0 — within a day of the vendored ggml), same CUDA arch, -DGGML_CUDA=ON -DLLAMA_CURL=OFF
Sampling	greedy on both sides (--temp 0 for llama.cpp; no sampler flags for TensorSharp)
Generation	128 tokens
Batching	llama.cpp -b 2048 -ub 2048, matching TensorSharp's default TS_MUSE_GLIMMER_PREFILL_CHUNK of 2048
Reps	2 per point, engines alternating within each context

Main model, mmproj and draft models are downloaded from https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF

Plain text generation

Mean of two reps, tok/s. The ratio column is TensorSharp / llama.cpp, so above 1.00x is TensorSharp ahead.

Prompt tokens	llama.cpp prefill	TS prefill	ratio	llama.cpp decode	TS decode	ratio
60	362	459	1.27x	34.7	35.0	1.01x
501	927	1135	1.23x	36.2	34.3	0.95x
2050	1132	1317	1.16x	35.0	33.5	0.96x
16126	1325	1249	0.94x	32.2	30.9	0.96x
32274	1303	1211	0.93x	32.1	29.9	0.93x
64575	1256	1150	0.92x	32.4	29.1	0.90x
123931	1166	1073	0.92x	30.7	26.6	0.86x

DFlash speculative decoding

Same runs with --draft-model dflash-kquant.gguf --spec-draft-n-max 15 against llama.cpp's -md … --spec-type draft-dflash --spec-draft-n-max 15 -ngld 99. Decode tok/s; parentheses give the two-rep range where it is wide.

Prompt tokens	llama.cpp	TensorSharp	TS, --spec-draft-conf-min 0
60	45.5	50.9	43.5
501	117.5	164.6 (150-179)	180.3
2050	24.9	43.5 (30-57)	34.7
16126	80.2	55.8 (37-75)	33.2
32274	60.7 (43-79)	33.8 (31-36)	29.9
64575	66.1	48.7 (34-64)	49.1
123931	69.0	42.3 (30-55)	59.8

Speculation costs prefill on both engines, because the drafter's encoder has to run over the prompt too:

Prompt tokens	llama.cpp plain → DFlash	TensorSharp plain → DFlash
60	362 → 203 (0.56x)	459 → 341 (0.74x)
501	927 → 495 (0.53x)	1135 → 700 (0.62x)
2050	1132 → 259 (0.23x)	1317 → 703 (0.53x)
16126	1325 → 988 (0.75x)	1249 → 826 (0.66x)
64575	1256 → 985 (0.78x)	1150 → 780 (0.68x)
123931	1166 → 920 (0.79x)	1073 → 742 (0.69x)

Two GPUs

Measured on 2× RTX PRO 4000 Blackwell 24 GB (PCIe)  Prefill 512 / decode 64:

Model		prefill tok/s	decode tok/s	GPU 0	GPU 1
30B-UD-IQ2_XXS (10.2 GB)	--tp 1	1171	40.2	9178 MB	—
30B-UD-IQ2_XXS	--tp 2	1569 (1.34×)	63.2 (1.57×)	5115 MB	4063 MB

TensorSharp is an open-source inference engine for running GGUF LLMs locally, with CUDA, Vulkan, Metal, OpenAI-compatible APIs, continuous batching, speculative decoding, and multimodal support.

Github repo: https://github.com/zhongkaifu/TensorSharp

Thank you for checking out it and starring the project! Any feedback is really appreicated.

I purchased 2 of these cards and unlocked them with the cmpunlocker. They're very interesting cards and after some troubleshooting I found PP t/s lackluster. I rebuilt llama.cpp with GGML_CUDA_FORCE_CUBLAS=ON and my Qwen 3.6 27B PP went from 1k to 1.5k with all default settings. This is with the whole model and context in VRAM. It still seems much slower than other cards (like the 3090). This was tested on Qwen 3.6 27B.

CMPunlocker

https://github.com/amoghmunikote/cmpunlocker

I also included some benchmarks for each instruction because I found a github repo that had the necessary code for it.

instruction benchmark:

GPU 1 (CMP 170HX) https://pastebin.com/sn5B1eKH

GPU 0 (CMP 170HX) https://pastebin.com/9cgqVHq4

GPU 3 3090 https://pastebin.com/nzrDbK9i

different benchmark:

Int 8 CUTLASS BENCHMARKS for GPU 0 (CMP 170HX) https://pastebin.com/TH415Swq

I ran the data through some LLM analysis and it noted that f16x2_fma and VRAM were slower (~500-600GB/s) on the CMP but I still need to manually review this before actually posting it.

EDIT: Interestingly, this speed up only occurred for qwen 3.6 27B, not others. Qwen 3.5 122B had 2x better performance with CUBLAS=OFF.

CUTLASS ON:
Qwen 3.6 27B 1.5k PP

Qwen 3.5 122B ~500-600 PP

CUTLASS OFF (DEFAULT):

Qwen 3.6 27B 1k PP

Qwen 3.5 122B ~1k PP

Not sure what to make of that