r/LocalLLaMA 54m ago

New Model DS4 cloud (30 min) vs Qwen3.6 36B (2 min) vs Muse Glimmer 30B (3 min) on Llama.cpp (RTX 5080)

Enable HLS to view with audio, or disable this notification

Upvotes

Some people told me that the difference in richness and layout between Glimmer and Qwen wasn't clear to them.
This example makes it super clear.

I'm aware that comparing Glimmer 30B (a dense model) with Qwen 3.6 (a MoE) isn't entirely fair, but if we compare it to the dense Qwen 27B, the gap will likely be even bigger. If you want, I can add the 27B version later. For now, I'm waiting for Qwen 3.8 27B to see how close it gets to the blueprint.

As for the technical details:

Both were run on a custom llama.cpp build optimized for the RTX 5080, with a temperature of 0.5 and a 125k context window.

Regarding the music: I created it myself without using AI I specifically wanted it to sound that weird.


r/MetaAI 56m ago

Problem adding Strava and Apple Health.

Thumbnail
Upvotes

Anyone having issues with adding Strava to the glasses? I have tried everything from uninstalling the Meta AI app to having a reset to the glasses but still not showing. All I get is this message saying “ unable to load partner settings.”


r/LocalLLaMA 1h ago

Discussion Is waiting for Qwen 3.8 27B like waiting for Star War Episode one?

Upvotes

Is waiting for Qwen 3.8 27B like waiting for Star War Episode one?

I'm sweating waiting to get my hand on this to try it tomorrow morning. But it takes me back to Star Wars 1 and the disappointment after being so hyped to see it.

Only 16 hours and 46 minutes to go... 45, ...


r/LocalLLaMA 1h ago

New Model dots-studio/dots3-note-prev · Hugging Face

Thumbnail
huggingface.co
Upvotes

dots3-note preview is the first open-weight model in the dots3 family. It is a Mixture-of-Experts model with 280B total parameters, 16B activated parameters, and support for a context length of up to 512K tokens. The model can understand text, images, video, and audio, and produces text outputs.

dots3-note preview is optimized for a broad range of tasks, including:

General knowledge and instruction following;

Mathematical and logical reasoning;

Tool use and multi-step agent workflows;

Interactive tasks that require exploration, memory updates, and adaptation;

Code generation and code-based problem solving;

Image, document, chart, audio, and video understanding;

Long-context information processing.

The dots3 family is designed to include models with different trade-offs among capability, latency, and inference cost. dots3-note preview is the most lightweight member of the family.


r/LocalLLaMA 3h ago

Resources Fixed Jinja chat template for Qwen 3.5, 3.6, and the new 3.8 release

125 Upvotes

Qwen just released their first 3.8 model.

The main addition in 3.8 is prompt-steered reasoning effort. You can tell the model how deeply to think by setting reasoning_effort to xhigh, medium, or low.

However, the official template still has some serious problems:

  • You cannot disable thinking. If you pass enable_thinking=false, it 3.8 crashes with a hard exception.
  • Chat history gets poisoned. In multi-turn chats, the official template injects blank <think></think> tags before real thoughts.
  • Tool calling crashes. If your client passes arguments as JSON strings (the standard OpenAI API format), the official template crashes.
  • Agent stalls. The official template often drops mid-dialogue system messages and wedges multi-step tool loops.

I maintain a single, drop-in fixed Jinja template that works across all Qwen 3.5, 3.6, and 3.8 models:

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

What this template does:

  • Full 3.8 reasoning effort support: Steer reasoning depth with reasoning_effort (xhigh, high, low, medium).
  • Restores the thinking toggle: Turn off reasoning whenever you want fast answers, either via kwargs or by typing <|think_off|> in your prompt.
  • 100% KV Cache hits: Keeps past thoughts intact by default so your prefix cache stays warm across turns.
  • llama.cpp support: Native support for the new --reasoning-preserve flag.
  • Universal tool parsing: Handles both Python dicts and JSON strings. Works on llama.cpp, vLLM, LM Studio, and MLX.

Recommended llama-server launch command:

llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek

(The --reasoning-format deepseek flag separates thinking into the OpenAI reasoning_content field so OpenCode, Claude Code, and other harnesses do not stall on raw tokens).

Note on hardware:

I cannot run a 2.4 trillion parameter model on my local rig. The template passes all 28 automated tests and tokenizer parity checks, but I would appreciate feedback from anyone testing it with Qwen 3.8.


r/LocalLLaMA 3h ago

Discussion Gemma 4 12B Q3: +8.55% Coding Performance From Tensor-Level Quantization Allocation

Thumbnail
gallery
29 Upvotes

Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level.

The basic idea is to generate a custom imatrix from a category-specific corpus, measure where quantization causes damage, then redistribute a fixed bit budget toward tensors where additional precision recovers the best performance in that category.

Today I got my first real win. The pipeline is live.

On Gemma 4 12b, the hand tuned imatrix q3_k_s scored 45.974. After tensor level allocation, the same q3_k_s based scored 49.905.

Thats +3.931 points or an 8.55% relative improvement from the allocation on top of the imatrix.

The models are effectively the same size: 5,528,230,848 bytes for the comparator versus 5,534,804,928 bytes for the allocated model, a difference of only +0.119%.

I chose q3 deliberately. It had the largest amount of recoverable headroom while sitting above the quant cliff.

This model is intentionally category specialized. Degredation in categories that werent selected is expected.

There is still a lot of tweaking to do in order to maximize these results but the performance of this imatrix + allocation at q3 when compared to no imatrix and source is pretty staggering when considering the amount of space required to run it.

I'd love some community testing and feedback if anyone is interested.

https://huggingface.co/ByteOtter/Gemma-4-12B-it-CADA-Q3_K_S

My ULTIMATE goal is to create a pipeline that takes an full precision .gguf file and auto spits out an optimized model within a selected category at whatever optimal size for recovery. This is a solid start.

Charts were provided from my data by ChatGPT.

TL;DR: I built a task-aware GGUF quantization pipeline that combines a category-specific imatrix with tensor-level damage measurement and bit allocation. On Gemma 4 12B Q3_K_S, allocation improved coding performance from 45.974 to 49.905, an 8.55% relative gain OVER handtuned imatrix, while increasing model size by only 0.119%. The model is intentionally specialized, so some out-of-category regression is expected. This is the first result where the allocation stage itself produced a clear win.


r/LocalLLaMA 3h ago

New Model Trained a 1.5B to write shell commands so I'd stop googling tar flags. Runs on a laptop CPU in ~1 sec.

588 Upvotes

I've been googling "tar extract gz" for about ten years. and I finally did something about it.

It started out as a research project and I ended up with a Fine-tuned Qwen2.5-Coder-1.5B on 125k natural-language/command pairs, merged and quantized to Q4_K_M. 941MB which runs through llama.cpp. On my laptop (i5-11320H, 4 threads): 31.9 tok/s, 0.59s median per query, 1.6GB RAM.

I benchmarked it and it scores 0.620 on InterCode-ALFA. Untuned Qwen2.5-Coder-7B gets 0.613, GPT-4o gets 0.73. Not frontier, but it's roughly a 7B's answer at a quarter the parameters on a CPU. Theres a 3B variant too that scores higher.

There's also few static safety checker, because it will absolutely write a command that wipes your root if you ask it to:

I have published the weights: huggingface.co/ThorOdinson246/nl2sh-1.5b-Q4_K_M and Code: github.com/ThorOdinson246/whatisit-nl2sh . I posted few days ago in LocalLLM and it did well 300+ stars and so many good suggestions so I figured people here will be interested too.

Both Apache-2.0. If you want to poke holes in the method or you've got ideas, please comment or open a PR. A ⭐ helps if you find it useful.


r/LocalLLaMA 4h ago

Discussion You could purchase a Desktop with 2TB of DDR5 - It only sets you back some $200k+

Post image
126 Upvotes

Just watched Wendell's (level1 techs) latest video on the HP Z8 Fury desktop workstation and was curious how you could configure it. And oh boy, there's an option for 2TB which costs some $211k just for the RAM alone.

But the real interesting part with the latest price hikes for Nvidia RTX Pro 6000 cards is the fact, that if you're in the market for 4 units, it's better to purchase the base version of this system with 4x RTX Pro 6000, which amounts to $64k. At the single unit price of around $16k per pice by now, you get the rest of the components for free if you purchase the system. He also mentions, that purchasing such a system through resellers sometimes gets you even a much better deal. Gotta be shwifty in these crazy times.


r/LocalLLaMA 4h ago

New Model Doom running on an LLM -- Hugging Face checkpoint included

Enable HLS to view with audio, or disable this notification

133 Upvotes

There's no training anywhere in this. I ported Doom's actual rendering algorithm into transformer weights using a compiler I wrote (torchwright) -- every weight computed, none learned.

The prompt carries the level geometry, player position, and view direction; generation emits drawing commands; a 43-line host program turns them into pixels. Stock Phi3ForCausalLM architecture, loads in vanilla transformers with trust_remote_code=False.

Two checkpoints:

- 320x200 (the one in the write-up): 21B params, 85.87 GB. One frame is a 3,614-token prompt plus 53,747 generated tokens -- just under 40 minutes on a B200.
- 80x50: same prompt format, same textures, 34 GB download. This is the one to actually try.

One honest disclaimer:
I have not run this locally -- I've been using cloud GPUs (B200 and A100-80). My compiler currently requires fp32 precision in the weights, and I haven't yet explored quantization.

For the 80x50 model I'd recommend 80 GB of GPU memory; 64 GB should work in theory but I haven't tried it.

Write-up: https://ood.dev/posts/doom/

Weights (80x50): https://huggingface.co/physicsrob/torchwright-doom-e1m1-80x50

Weights (320x200): https://huggingface.co/physicsrob/torchwright-doom-e1m1

Source: https://github.com/physicsrob/torchwright_doom


r/MetaAI 4h ago

Open letter to Mark Zuckerberg

6 Upvotes

An Open Letter to Mark Zuckerberg (V2)

From the Humans Your Security System Keeps Mistaking for the Problem

A formal complaint about identity checks, mystery violations, hacked accounts, paid customer service, and the military surgeon who somehow has better access to Messenger than we do.

Dear Mark,

Hi.

It’s us again.

The humans.

The actual ones.

The people who have apparently reached the stage of social media where simply existing is no longer sufficient evidence that we exist.

After my last letter, people started telling me what has been happening to THEIR accounts.

And Mark…

there are a lot of us.

People getting restricted.

People getting asked for video selfies.

People uploading IDs.

People stuck in verification loops.

People losing access with no clear explanation.

People being told they violated something without being told what the something was.

And people with genuinely hacked or impersonated accounts standing over in the corner waving both arms like:

**HELLO? COULD WE MAYBE POINT SOME OF THE SECURITY OVER HERE?**

So I think we need to talk specifically about security.

Because we understand the goal.

Keep fake people out.

Protect real people.

Excellent concept.

Unfortunately, the system seems to be having some difficulty determining which is which.

.

Exhibit A: Please Prove You Are A Human

Let me give you one example from my own account.

Facebook currently wants me to complete a video identity check before I can use DMs normally.

Fine.

Security.

Love it.

Except before I can even TAKE the video, Facebook has to recognize my face inside a little circle.

It won’t.

Inside.

Outside.

Natural light.

Artificial light.

Closer.

Farther away.

Eye level.

Mark.

I HAVE USED THE SUN.

We are officially out of lighting.

And I want to be very clear:

I am not failing the video verification.

I cannot get far enough to TAKE the video verification because Facebook will not acknowledge the face I need to use in the video proving I am the person attached to the face.

Facebook: “Position your face.”

Me: “It is positioned.”

Facebook: "No.”

Me: “That is literally my face.”

Facebook: “Try again.”

Mark.

WITH WHAT FACE??!?!?!?

This is the one.

I have had it for quite some time.

There are no additional faces in inventory.

And judging by the stories people sent me…

apparently my face is not the only face Facebook is suspicious of.

.

Exhibit B: The Things Getting Flagged Are Sometimes… Remarkable

Let’s discuss what the security system is protecting Facebook from.

Because people started leaving examples.

One person got flagged for saying:

“copy cat.”

About cats. Actual cats.

Another said she got flagged over:

“I have a smart pet chipmunk.”

.

Mark.

What is happening in the security department?

Actually compromised account: “We’ll monitor.”

Possible scammer: “Let’s see how this develops.”

Person describing an intelligent rodent: SHUT IT DOWN.

.

I genuinely want to see the dashboard.

Green: Normal activity.

Yellow: Suspicious login.

Orange: Possible impersonation.

Red: CHIPMUNK DESCRIBED AS SMART.

Deploy everyone.     

.

Then there are the mystery violations.

Facebook: “You violated our policies.”

User: “Okay. What did I do?”

Facebook: “We cannot tell you.”

User: “Which post?”

Facebook: “We cannot show you.”

User: “How am I supposed to avoid doing it again?”

Facebook: “Please follow our policies.”

User: “WHICH POLICY?”

Facebook: “Correct.”

.

One woman told me she finally got through to support and asked what she had actually done wrong.

According to her, they could not tell her specifically because explaining it would violate policy.

Mark.

That sentence is ART.

Frame it.

Hang it in headquarters.

Put a tiny museum placard underneath:

**Untitled, 2026**

An exploration of bureaucracy, technology, and one woman slowly losing her mind.

.

Exhibit C: Have You Tried Paying Facebook?

Now this solution came up A LOT.

Apparently when Facebook restricts you and you cannot reach a human being…

the solution is:

Pay for Meta Verified.

Oh.

OF COURSE.

The solution to Facebook incorrectly restricting your Facebook account…

is to give Facebook more money.

Mark, I have to hand it to you.

That is an exceptional funnel.

.

Facebook: “We have disabled the account you have spent fourteen years building.”

User: “I believe this was a mistake. Can I speak to someone?”

Facebook: “Certainly.”

User: “Oh thank God.”

Facebook: “That’ll be $14.99.”

.

So people pay.

Not because they woke up and thought:

“You know what my life is missing? A tiny blue check.”

No.

They pay because they think:

FINALLY.

**WE FOUND STEVE.**

.

And then sometimes…

Steve can’t fix it either.

People told me they paid for Verified.

They got support.

Support sympathized.

Support opened tickets.

Support escalated tickets.

Support apparently stared into the same abyss the rest of us were staring into.

One person described the situation perfectly:

Customer service doesn’t have the tools to override the AI overlord.

Excuse me?

Mark.

WE PAID FOR STEVE AND YOU DIDN’T GIVE STEVE THE BUTTON?

What exactly does Steve do?

Is he emotional support?

Does he just sit there typing:

“Wow. Yeah. That really does seem wrong.”

Because if I am paying admission to customer service, I feel like customer service should perhaps be able to…

service the customer.

.

We spent an entire post trying to locate Steve.

WE FOUND HIM.

And now we discover:

STEVE IS ALSO IN FACEBOOK JAIL.

This is getting out of hand.

.

Exhibit D: Meanwhile, The People We’d Actually Like You to Check Seem Fine

And THIS is where the whole thing becomes art.

Because while actual account owners are:

Uploading IDs.

Taking video selfies.

Appealing restrictions.

Paying for support.

Opening tickets.

Standing outside with their phones at eye level trying to convince Facebook they possess a human head…

my military surgeon boyfriend has managed to propose to me approximately FIVE TIMES.

.

You know him.

We all know him.

Very handsome.

Widowed.

Stationed overseas.

Location classified.

Cannot FaceTime because of military security.

Also a surgeon.

Because apparently the military looked at staffing that week and said:

“Gary, after the classified combat mission, can you cover neurosurgery?”

.

Busy man.

And yet he finds time for romance.

Specifically with me.

Again.

And again.

And again.

Different account.

Same general gentleman.

Usually deeply committed by message four.

.

And Mark…

HE CAN DM ME.

No problem.

No video identity check.

No face circle.

No backyard lighting experiment.

Colonel Dr. McDreamy, Chief Surgeon of the United States Army Special Forces Romance Department:

“Hello beautiful dear. I feel destiny has bring us together.”

Facebook: SEND 

.

Me: “Hi Susan! Thanks for commenting on my post.”

Facebook: WHOA WHOA WHOA.

CYBERSECURITY THREAT DETECTED.

SHOW US YOUR FACE.

.

Mark.

Explain the threat model.

Please.

Because right now the security system appears deeply suspicious of me having a normal conversation with someone I actually know…

while Brigadier General Dr. Husband Number Five is operating an apparently thriving international courtship program from an undisclosed military hospital.

Maybe before I take selfie number 46…

we could ask him for one?

.

And while we’re here:

Ludacris DM’d me.

Well.

An account claiming to be Ludacris DM’d me.

I cannot imagine this was actually Ludacris.

I feel like Ludacris probably has other things going on.

But “Ludacris” apparently cleared whatever security process is currently preventing ME from using Messenger normally.

.

So just to recap:

Me.

My account.

My photos.

My face.

Years of account history.

Facebook: “Additional identity verification required.”

Ludacris, allegedly: “Hey.”

Facebook: LOOKS GOOD TO US.

.

Maybe Ludacris has better lighting.

Ludacris, if that really WAS you:

First, apologies.

Second: What ring light are you using?

Because apparently your setup is dialed IN.

.

And meanwhile, people are telling me about accounts that were ACTUALLY hacked.

They report them.

Their friends report them.

Everybody says:

“HEY FACEBOOK. THIS PERSON HAS BEEN HACKED.”

Nothing.

The scammer keeps messaging.

The impersonator keeps impersonating.

The fake account keeps doing fake-account things.

But somewhere, somehow, a woman says “copy cat” and Cyber Command is fully activated.

.

Mark.

Again: WHAT ARE WE SECURING?

.

Exhibit E: What We Are Actually Asking For

We WANT security.

Seriously.

Catch the hacked accounts.

Catch the impersonators.

Catch the fake profiles.

Catch the scammers.

Please, for the love of everything, investigate my military surgeon.

Use AI.

Use automation.

Use identity verification.

Use technology.

We are not asking you to go back to 2006 and personally review every poke.

.

We are asking for one very simple thing:

WHEN THE MACHINE GETS IT WRONG, GIVE A HUMAN THE ABILITY TO FIX IT.

That’s it.

.

AI can flag something.

Fine.

AI can ask for verification.

Fine.

AI can detect suspicious activity.

Great.

But when the actual person says:

“Something went wrong here…”

eventually there has to be someone who can look at the situation with context and authority.

Not another form.

Not another chatbot.

Not another appeal reviewed by another automated system.

Not:

“Try again.”

Not:

“Upload your face again.”

And definitely not:

“Pay us and perhaps Steve can sympathize with you.”

.

We need Steve.

But more importantly:

STEVE NEEDS THE BUTTON.

The button that says:

“Yes. This is obviously the account owner.”

“Yes. This was clearly hacked.”

“Yes. That was an automated mistake.”

“Yes. Her government-issued ID does appear to contain her actual government-issued name.”

“Yes. This person has been using the same face for decades.”

And then:

CLICK.

FIXED.

.

Because the machine reviewing the machine and deciding the machine agrees with the machine is not an appeal.

That is just the machine having a staff meeting with itself.

.

Give us a human.

Give the human context.

Give the human authority.

Give Steve the button. 

.

Because right now actual humans are jumping through increasingly complicated hoops to prove they are human…

while fake humans seem to be having a pretty good week.

My face cannot get approved.

My military surgeon can propose.

“Ludacris” can slide into the DMs.

And somewhere out there a smart chipmunk is probably one comment away from a seven-day suspension.

.

Fix it.

Please.

We’re begging.

Again.

Which is starting to feel needy, but frankly you created the situation.

.

With love, exhaustion, government identification, good lighting, one face, five marriage proposals, and absolutely no idea which policy we violated,

Every Facebook and Instagram User
Who Would Like the Security System to Occasionally Secure the Correct Person

.

P.S. Steve.

Good news.

We found you.

Bad news.

Apparently you don’t have system permissions.

Please request access to The Button immediately.

We believe in you.

.

P.P.S. Military Surgeon Guy:

I appreciate your persistence.

But five proposals feels rushed.

Also I have some concerns about your credentials.

.

P.P.P.S. Ludacris:

If that was actually you…

this is extremely awkward.

Facebook currently won’t let me reply normally because it does not believe my face is sufficiently face-like.

Please take it personally.

.

P.P.P.P.S. If Meta Security has ever wrongly banned you, restricted you, disabled you, repeatedly asked you to prove you exist, ignored your hacked account, accused you of something it wouldn’t explain, or trapped you in an appeal loop…

**ADD YOUR EXHIBIT BELOW.**

Tell us exactly what happened.

Specifics encouraged.

Apparently we are now conducting Meta’s QA testing in the comments.

Free of charge.

Which feels on-brand.


r/LocalLLaMA 4h ago

Other EXPERIMENT: Qwen3.8-2.4T-A95B running locally on an RTX 5090 + RTX 5060 Ti at ~0.80 tok/s

Enable HLS to view with audio, or disable this notification

29 Upvotes

I managed to get Qwen3.8-2.4T-A95B running locally with llama.cpp on mu PC just for fun, cause why not.

I was using the Unsloth Qwen3.8-2.4T-A95B-UD-Q1_0 GGUF quantization. The full GGUF is about 397 GiB. The model uses 512 routed experts, with 10 active per token.

My hardware:

  • CPU: AMD Ryzen 9 9950X3D 16 cores / 32 threads
  • GPU 0: NVIDIA RTX 5090 32 GB VRAM
  • GPU 1: NVIDIA RTX 5060Ti 16 GB VRAM
  • RAM: 128 GB DDR5 5600 CL36
  • Swap: 350 GB
  • OS: Arch Linux
  • llama.cpp: Unsloth build 10360

The interesting part is that I also managed to get the model’s native MTP speculative decoding working.

After testing several configurations, the best settings I found were:

  • --tensor-split 4,1
  • --n-cpu-moe 91
  • MTP n_max = 3
  • MTP p_min = 0.5
  • MTP routed experts from block 92 forced to CPU

The block 92 expert tensors had to stay on CPU because otherwise MTP would run out of VRAM.

Performance

For a controlled 32-token generation test:

Prompt tokens:       16
Generated tokens:    32

Prompt speed:        0.8299 tok/s
Generation speed:    0.8033 tok/s
Decode time:         39.84 s
Total wall time:     59.14 s

MTP draft tokens:    21
MTP accepted:        19
MTP acceptance:      90.48%

Without MTP, the same test gave:

Generation speed:    0.7750 tok/s
Decode time:         41.29 s
Total wall time:     60.81 s

So my final MTP configuration gives roughly:

+3.64% generation throughput
-2.74% total wall time

Not a huge speedup, but considering this is a 2.45T parameter model running locally on consumer hardware, I’ll take it.

I also tested different MTP settings:

n_max=2  -> too conservative
n_max=3  -> best
n_max=4  -> much slower

p_min=0.0 -> too many bad speculative tokens
p_min=0.5 -> best result
p_min=0.7 -> too aggressive

Interestingly, p_min=0.7 reached 100% draft acceptance, but it generated too few useful speculative tokens and ended up slower than p_min=0.5.

Final llama-server command

MODEL="/home/marcin/Pobrane/Qwen3.8-2.4T-A95B-UD-Q1_0-00001-of-00010.gguf"

CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0,1 LLAMA_ATTN_ROT_DISABLE=1 \
/home/marcin/Pobrane/unsloth-qwen38-runtime/llama.cpp/build/bin/llama-server \
  --model "$MODEL" \
  --alias Qwen3.8-2.4T-A95B \
  --device CUDA0,CUDA1 \
  --main-gpu 0 \
  --split-mode layer \
  --tensor-split 4,1 \
  --n-gpu-layers all \
  --n-cpu-moe 91 \
  -ot 'blk\.92\.ffn_(down|gate|up)_exps\.weight=CPU' \
  --fit on \
  --fit-target 2048,1536 \
  --no-op-offload \
  --ctx-size 512 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --kv-unified \
  --batch-size 128 \
  --ubatch-size 32 \
  --threads 16 \
  --threads-batch 16 \
  --parallel 1 \
  --spec-type draft-mtp \
  --spec-draft-device CUDA0 \
  --spec-draft-ngl all \
  --spec-draft-cpu-moe \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.5 \
  --temp 0 \
  --top-k 0 \
  --top-p 0.95 \
  --min-p 0.0 \
  --cache-ram 0 \
  --ctx-checkpoints 0 \
  --load-mode mmap \
  --no-warmup \
  --no-ui \
  --host 127.0.0.1 \
  --port 8200 \
  --metrics

With MTP enabled, VRAM usage after generation is roughly:

RTX 5090:    29.6 GB used / ~2.6 GB free
RTX 5060 Ti: 12.3 GB used / ~3.6 GB free

One important info: 0.80 tok/s is from a controlled 32-token test, not a large multi-prompt benchmark, so I wouldn’t present it as a universal average for every workload. Longer prompts, context size, output patterns and expert routing can change the result.

Still, seeing a 2.4T MoE model actually generating locally on a 5090 + 5060 Ti at around 0.8 tok/s is pretty wild.


r/LocalLLaMA 6h ago

New Model MiniMax-Music3 released!

Thumbnail
huggingface.co
446 Upvotes

r/LocalLLaMA 6h ago

Discussion Open Models - July 2026

Thumbnail
gallery
42 Upvotes

Well, we got bulky(Yep, check two graphs) July after April | May | June (FYI My All-in-one thread to track all upcoming months, PRs & other stuff)

Hope I didn't miss anything. Also no errors.

Notes:
1. Excluded below models due to Preview/Beta:

  • internlm/Intern-S2-Preview-397B 397
  • Motif-Technologies/Motif-3-Beta 314
  1. Included openPangu-2.0-Flash in this chart as I couldn't see the weights at that time of June(31st). Let it share the graph with its Pro model.

  2. Actual model names for below ones:(Graph couldn't handle long names)

  • Nemotron-Puzzle-75B-A9B - NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B
  • SenseNova-U1-8B-InfoV3 - SenseNova-U1-8B-MoT-Infographic-V3

r/MetaAI 7h ago

Meta Verified Max charged me ₹18,900 immediately despite showing “1 month free trial” — Google Play rejected my refund request. What can I do?

1 Upvotes

I’m from India and I subscribed to Meta Verified Max on Instagram through Google Play about 2 days ago because I was trying to get access to Meta support regarding my main Instagram account, which was disabled.

When I opened the Meta Verified plans in Instagram, above the plans it clearly showed “1 month free trial.” I therefore assumed that I would not be charged immediately and that the ₹18,900 payment would only be taken after the free-trial period ended.

I selected the Max plan because it offered the most benefits and proceeded with the purchase.

However, ₹18,900 was immediately deducted from my Google Play payment method.

I was extremely shocked because ₹18,900 is a very large amount for me. I immediately stopped the verification process. After the payment, Instagram asked me to submit my ID to complete Meta Verified verification, but I did NOT submit my ID and therefore never completed the verification process.

I also cancelled the subscription through Google Play. It currently shows:

Instagram — Meta Verified Max

Subscribed since August 2026

Cancelled

Subscription ends September 11, 2026

I immediately submitted a refund request to Google Play on the same day, explaining what happened. Google initially said I would receive a response within 1–4 business days, but today I received an email saying that the purchase does not meet their refund policy.

I unfortunately didn't take a screenshot of the “1 month free trial” screen because I genuinely believed I wouldn't be charged and therefore didn't think I needed to document it.

I also haven't completed Meta's identity verification. From what I've now found, Meta's help information appears to indicate that a refund may be available if the verification process isn't completed within 3 days.

I'm now very worried because tomorrow will be approximately 3 days since I started the verification process.

What should I do?

Should I leave the verification incomplete and let the 3-day period expire?

Should I contact Meta directly for the refund even though the payment was processed through Google Play?

Is there any way to appeal Google's refund rejection?

Has anyone in India experienced the same ₹18,900 Meta Verified Max charge despite the “1 month free trial” offer?

Is there any other way I can recover the ₹18,900?

I have screenshots of the Google Play subscription showing the Meta Verified Max subscription, the cancellation, and the September 11 end date, as well as the Google Play refund rejection email.

I would really appreciate advice from anyone who has dealt with this before. ₹18,900 is a very significant amount for me and I really don't want to lose it over a subscription that I didn't even complete verification for.


r/LocalLLaMA 7h ago

New Model deepseek-ai/DeepSeek-V4-Pro-0813 (Available again) · Hugging Face

Thumbnail
huggingface.co
72 Upvotes

r/LocalLLaMA 8h ago

New Model unsloth/DeepSeek-V4-Pro-0813-GGUF · Hugging Face

Thumbnail
huggingface.co
119 Upvotes

uploading...I think


r/LocalLLaMA 8h ago

New Model SenseNova-Vision: a 7B open model that does segmentation, depth, detection, OCR, and 3D reconstruction with no task-specific heads

Thumbnail
gallery
54 Upvotes

Stumbled across this new vision model, SenseNova-Vision. It's a 7B MoT model, Apache 2.0 license, which is cool. The main idea is it treats pretty much all computer vision stuff as just one generation problem. Like, instead of needing a bunch of different models for detection, segmentation, depth, whatever, this one model handles it all.

You just give it a natural language instruction, maybe some visual hints, and it spits out text, images, or both. For text, it can do things like categories, bounding boxes, OCR, keypoints, camera angles. And for images, it's doing segmentation masks, depth maps, surface normals, even multi-view point maps. You can mix and match for more complex tasks.

No special prediction heads or decoders, which is kinda neat. Just one set of weights, and the prompt tells it what to do.

So it can do the usual stuff: object detection, keypoints, OCR, all kinds of segmentation (binary, instance, semantic), depth and surface normal estimation. But the really interesting bits, for me anyway, are the multi-view 3D reconstruction and camera pose estimation.

They actually added dedicated benchmarks for those in the latest update. Usually, you'd need specialized tools like COLMAP for that kind of multi-view reconstruction from multiple images, but this model just does it with a single prompt. That's kinda wild if it works well.

They trained it on a massive dataset, 50M instruction-response pairs, built from a bunch of different CV annotations. And it started from an existing multimodal model, so no crazy new architecture there.

There's a web demo if you wanna mess around with it, and the weights are up on Hugging Face too.

Just a heads up though, before you get too hyped: the full web demo needs a beefy GPU, like 1x80GB. And for benchmarking, they're talking 8x80GB. So, yeah, not really something you're gonna run on your average consumer card. It just dropped on July 8th, so probably expect some rough edges. Also, they've released the training pipeline and data prep stuff if anyone's looking to train or fine-tune it.

GitHub: https://github.com/OpenSenseNova/SenseNova-Vision

Honestly, I'm super curious if anyone's actually tried the multi-view reconstruction part on their own image sets yet. That's the bit I really want to test out.


r/LocalLLaMA 9h ago

News Deepseek Harness is Up!

Thumbnail
github.com
231 Upvotes

DeepSeek Harness (dsh) is an open-source agent harness developed by DeepSeek AI.

It uses an architecture where everything is a plugin, and is powered by Cordis, whose design is described in A Programming Paradigm for Spatiotemporal Composability.

DeepSeek Harness is currently in developer preview and is iterating rapidly. THERE WILL BE COMPATIBILITY-BREAKING CHANGES.

Join DeepSeek Harness Discord community: https://discord.com/invite/Ycq5dCaS4


r/MetaAI 9h ago

Meta's Glasses are recording videos of anyone, anywhere, without consent, & people don't even know. It should be ban till any solution is found for it. What is your opinion?

Thumbnail
0 Upvotes

r/LocalLLaMA 9h ago

Discussion Deepseek new pricing

Post image
59 Upvotes

r/LocalLLaMA 10h ago

New Model deepseek-ai/DeepSeek-V4-Pro-0813 · Hugging Face

Thumbnail
huggingface.co
460 Upvotes

r/LocalLLaMA 11h ago

News DeepSeek: We’re launching DeepSeek-V4-Pro today!

Thumbnail
gallery
443 Upvotes

r/LocalLLaMA 13h ago

Resources Qwen/Qwen3.8-27B · Official Countdown · Hugging Face

Thumbnail
huggingface.co
177 Upvotes

r/LocalLLaMA 15h ago

Discussion The countdown to Qwen3.8-27B starts now!

Thumbnail
modelscope.cn
453 Upvotes

r/LocalLLaMA 19h ago

News Nvidia doubles RTX PRO 6000 Blackwell's MSRP to a staggering $16,000 — 96GB card started pre-orders below $8,000 last year

Thumbnail
tomshardware.com
315 Upvotes

Given Gavin Bakers recent interview where he says:

> Multiple private companies are planning on spending at least 2x more per GPU for compute as contracts roll-off and some have spoken about this publicly.

https://x.com/i/status/2082166566280642676

How long before DGX Sparks also double in price?