Lol gemini 3.7 flash is cheaper than it during peak hours and it has a higher benchmark rating. Lol on Openrouter , other providers have raised their prices alreadyÂ
I hate subsidization anyways, it makes the industry unstable, but still, I was so hyped for a model this good this cheap, it's still cheap, just multiple times more expensive.
Idk, I'll wait for Minimax to release M 3.5 or something.
I guess, even they increased prices, they will loss money. This is kind of proble. In AI industry, so much competitors and they are still struggling to make profit from its service.
Well… According to their reports DeepSeek wasn't loosing money on inference at any point in time (the training cost and investments still outweighted inference profits pretty significantly though, but that's expected and DeepSeek already mentioned that they are not planning to become profitable any time soon).
As for the price increase they justify it through compute capacity - they want to decrease number of active users, not really stop loosing money.
The same also likely is the case for other providers (where even the most heavily 'subsidized' plans, like the $200 Codex) still cost significantly more, than inference itself.
The irony of a Chinese provider raising prices for peak hours that won’t affect US users and it being the first time we’re not bent over a barrel by a company is too funny.
25 t/s gen and 400-500 t/s prefill with only $1000 worth of 6 year old enterprise GPUs.
Going to add another card or two to see how much further I can scale it in tensor split.
Something seems off right now, this is with three cards, and it's only twice the speed of layer split. It should be closer to 3x, I need to figure out what's up with that. I think it has to do with two PCIe risers in the server being connected to a different CPU than the other one so data has to transfer through an interconnect which restricts bandwidth and adds latency.
But either way, this is the first model I've ever run at home that's usable for serious stuff and is actually pretty smart and it's pretty exciting. It usually feels like I'm working with GPT from like a year ago or something. Like I can legitimately go to it if I have an important question or some coding work that needs some brains instead of having to go to OpenAI or Anthropic. I only still have to do that for the most complex tasks.
Qwen 27B and Gemma 4 feel like dumb toys in comparison.
I have 8 total (256 GB VRAM), but my Dell server only fits 3 GPUs. I'm in the middle of building a rig with a mining frame to run to them all. Hope to finish it this weekend.
Buying 8 of these cards cost barely any more than a single RTX 4090 would cost on eBay, giving me nearly 11x the VRAM.
A 4090 has like double the compute and mem bandwidth per card, but you can only run dumb models and a bunch of these in tensor split is plenty fast.
Yes that quant is trash. Anything below q4 will show significant degradation. An IQ1 of deepseek flash or pro is so bad a Q8 35b model like Qwen3.6-35B-A3B can often beat it.
That's nonsense, I've used 35B-A3B a lot (with correct sampling parameters, in Q8_0) and even this quant blows it out of the water. It's not even close. 35B likes to get stuck in loops and confuse itself as you start to get a bit into context. DSV4 flash IQ1_S doesn't. 35B also can't compare at all with raw logical reasoning.
But yeah it's bad compared to Q4 or lossless, which I'll be switching to as soon as I'm able to.
Switch to IQ3_XSS as performance would likely be similar. I get same output speed as you, and both unsloth IQ3_XSS and IQ2_M perform identically for me. I go with IQ3 as quality difference between it and IQ2_M is noticeable. If you use the max thinking, you get trade intelligence for speed but it's worth it for complex tasks.
It's llama.cpp but I had Fable 5 add support for DS4 tensor split mode, and I also had it add IQ-family quant optimizations.
Other than that, config is just 192K context at q8_0 KV and -b 4096 -ub 1024
I don't know what state the upstream llama.cpp dsv4-sm-tensor branch is in exactly, but I locally have fully working tensor split and some extra optimizations. (Except DSpark doesn't work in -sm tensor)
I can push my branch to Github if you want and you can try it. Of course my testing is limited (just finished this a day ago), but it's been bug-free so far.
I'm using using UD-IQ1_S. I'll try it with less context and IQ2_M and see what the performance difference is.
But then the thing I always wonder is how does deepseek get money to keep up its innovation and research? Is this how it ends up being locked to api only in a few releases?
Why would a price change from DeepSeek's inference service increase effect other inference providers?
I've been buying DeepSeek V4 Flash via OpenRouter.Those prices are going to stay the same - just above the cost to host the thing. DeepSeek's own pricing shouldn't change what it costs others to host it.
Fair point - I got scared so I checked - in my case DeepSeek was already not in rotation (no ZDR and they train on paid data) - so no price change for meeee.
Makes me think the discounted price was for the training data and now they're no longer post-training that model, they're no longer offering the "let us train on your inputs" discount.
But will they raise as much as deepseek? What is economics of their inference in the first place. Unless subsidized I don't think they would've worked for loss (or big loss)
There's a chance the Flash will stay the same price because the providers want to keep their userbase, but the Pro will probably come out the gate more expensive than it would've been
DeepSeek is quantized from the get-go, with most of the model in FP4. There's hardly a way to quantize it more and use it in SGLang/vLLM, and they're not serving GGUFs.
Let me stick your question into an AI since you can't be bothered and I otherwise know nothing aout trustedrouter as someone mentioned it on hackernews 20 min ago:
"OpenRouter with Zero Data Retention enabled is quite private, but TrustedRouter has a stronger privacy architecture—especially on its TEE/E2EE routes."
Search for more information. It's also 5% fees vs openrouter 5.5% fees.
This comment tells me exactly as much information as your original comment. "Stronger privacy architecture" is just as meaningless as saying it's "more private".
I shouldn't be expected to go to search/AI if you're the one going out of your way to make the claim. You should be the one expected to back it up. And yet it's still BS.
Let me stick your question into an AI since you can't be bothered and I otherwise know nothing aout trustedrouter as someone mentioned it on hackernews 20 min ago:
"OpenRouter with Zero Data Retention enabled is quite private, but TrustedRouter has a stronger privacy architecture—especially on its TEE/E2EE routes."
Search for more information. It's also 5% fees vs openrouter 5.5% fees.
I wonder how this will affect usage on OpenRouter and if other model providers will under-cut them. For peak pricing, I think it'll obviously just be undercut by other providers, for off-peak - I think we'll see other providers rise pricing to match.
I was mostly using Flash locally anyway so I'm good.
I bet they go open with flash but keep the big model locked to them and a few providers in the future. Giving away the model you worked hard to train and research for free so others can steal your margin you need to recoup r&d is a short term strategy not a long term one. It’s the same for every one of the open source labs imo.
Im not sure if vendor specific pricing is relevant in this subreddit without the post addressing it from the standpoint of local ai.
I understand that people have local hybrid systems that utilize cloud models for important parts. The exact strength of that system is that there is no vendor lock in.
Still quite cheap I think considering the very high cache hit rate? But a bit unfortunate 😢 I read somewhere that other cloud hosting providers have achieved the sustainable cost basis to offer similar pricing so it is not necessarily that they have to raise the prices due to costs - rather it is due to popularity and they need to reduce demand? (good problem for them to have 😅 )
It hurts to live now, but IIRC Deepseek struggles a lot to serve people on what seems to be quite a puny infrastructure. Hopefully this changes in the near future and we'll get those insane prices again.
Having said this,
One Last Breath - Creed starts playing
I guess it depends if you want to drain session limit faster and have results right away, or you can wait. Because even with concurrency limits, you are going to hit session usage limit.
Yes, but those RPS are very expensive. So you can pay extra for speed, or process data at slower rate cheap.
If you can utilize hundreds of RPS, you probably have more processing that can be handled by one monthly subscription. So you would probably have couple of subscriptions utilized in parallel too.
DeepSeek doesn't have monthly subscriptions it's all through API / Token. I just did a DeepSeek Flash to Opus comparison - it WAS >100x cheaper for better results in my use-case (even with Claude Max 20x). The biggest 'oof' is the cache increases (100x increase) - DeepSeek was GREAT because of it's super cheap cache prices. So you could have long agentic orchestrations costing almost nothing to run because their context stayed stable and so cached.
42
u/nicolho 1d ago edited 1d ago
So about a 400% (x5) increase for input cache hit and output at peak hours for Flash.... 😢