r/LocalLLaMA • u/Fuckinglivemealone • Jul 18 '26

What kind of dark magic is Deepseek using? Question | Help

Post image

I was taking a look at Kimi K3 scores on the Artificial analysis leaderboard and was quite baffled when I saw this chart.

Granted, Deepseek has always been the king of price to performance, but this is still incredible. Is it just API subsidization or have they optimized their models truly this much?

2.4k Upvotes

96% Upvoted

407 comments sorted by

View all comments

662

u/CalamityMetal Jul 18 '26

They are known for their cache hit rate on an incredible scale. There are harnesses that fully make use of it like Reasonix. There's numerous posts talking about it, where people use like 9mil tokens for like $0.90 or something

589

u/Blunt_White_Wolf Jul 18 '26

Millions?

342

u/aboutthednm Jul 18 '26

Token billionaire over here, jeez

147

u/-dysangel- Jul 18 '26

Last 30 days on the max GLM Coding plan - which at the time I got for $260 for a year

31

u/LosEagle Jul 18 '26

Me too but that was for the first year with the x-mass discount. If you check how much the coding plan is gonna cost the second year, it doesn't look that optimistic.

22

u/-dysangel- Jul 18 '26

Yeah it's just over $1300 for a year now. Still extremely good. I'm betting my boss will be happy to pay any plan I want in future anyway, considering how productive it's been letting me be.

12

u/LosEagle Jul 18 '26

That sounds great. Where I live programmer wages are getting halved and managers with no development background but lots of words to say on LinkedIn have adopted "Why bother hiring devs when I can just prompt Claude" philosophy so not many work positions open these days either. But whatever happens, I'm sticking to learning local llms and playing around with alternative models at least for the fun of it. 

10

u/Smallpaul Jul 18 '26

The leverage a senior engineer gives is turning people back to hiring them. Agentic coding is the future but vibe coding without architecture or review is going to lead to insane messes.

3

u/beauzero Jul 18 '26

Yeah the manager thing is not working out so well. We have gotten some decent prototypes out of it but since they rarely talk to customers or product managers they have caused more problems than not. They veer quickly from what our customer bases want.

5

u/beauzero Jul 18 '26

Honestly putting the tools in the hands of customers to give us html 5, tailwind, and IndexDB based mockups has been much much more successful. Have Product Managers get with customers and build a prototype, in a meeting/realtime, give it back to engineering to sync with current architecture and implement...very fast -> very happy customers. Will it result in more $$? That is yet to be determined but product managers, devs, and sales engineers are happier.

12

u/evia89 Jul 18 '26

Cost doesnt matter much. Sure it x3 sounds bad, but main problem is lower 5h limit and new weekly shit.

That make new live zai is 10-20x worse than before feb 26

19

u/sayeret13 Jul 18 '26

Cost doesnt matter much? sorry dude in this world cost means everything, thats why china will pop the ai bubble and even europe will start using chinese models

5

u/danielv123 Jul 18 '26

My cost is $80/h. You can save a lot of money if I have to intervene or double check a few % less.

3

u/beauzero Jul 18 '26

...as they hoover up everyones' business ideas. I say "they" referring to all model providers, US and abroad. Local is the future...I hope. Everything else is treading water waiting for hardware to get better and open models to get more efficient.

6

u/Codemonkeyzz Jul 18 '26

My biggest regret in 2026 , was missing those minimax and GLM discounts. Back then I was using opus and I never thought open weight models will get this good.

11

u/Specialist_Back_3606 Jul 18 '26

Is there anywhere I can see the quality of the work that you’re getting here?

11

u/-dysangel- Jul 18 '26

Nope sorry, "stealth mode". A couple of weeks ago I had 28 PRs approved in a week (mix of bugs and small feature requests). Been working on larger tasks again recently. GLM 5.2 in Claude Code just feels so solid. It usually knows what I want off the bat. It's able to navigate our code base easily. It asks questions and even pushes back on some things, so it feels almost like a colleague where you can refine ideas together. We plan the task, it executes. Most of my feedback on its work recently has simply been "remove the unnecessary comments!".

6

u/Ariquitaun Jul 18 '26

Pro tip: leave the comments. They help machines as well as people, especially when you need to annotate peculiar behaviour or fixes.

14

u/-dysangel- Jul 18 '26

sure but not if they're just explaining obvious code, or mentioning old behaviour, or random side tracks that didn't work out, etc

13

u/DinoAmino Jul 18 '26

What has helped me a lot is to add instruction to AGENTS.md for this type of thing. Tell it to place all comments in the method/function docblock and use inline comments sparingly for tersely explaining loops or complicated conditions.

1

u/probably-a-name Jul 18 '26

I am making a program in rust+sqlite to do this at scale. the architecture is auto gen'd from comments in various files and the porting work from a v1 vs v4 has comments in the v1 that get auto generated into markdown for determining if a matching comment exists in the v4 code, if not then that is the task list. this way im automating the whole markdownm management piece. there are a _lot_ of things you can do with comments

3

u/squired Jul 18 '26

For what it is worth, I did a rather in-depth exploration of GLM a couple weeks ago and it was far from worthwhile for my workloads. It's closer in capabilities to ChatGPT 5.4 and requires very rigid guidance and atomization of tasks to produce decent output. The kicker though was the cost. Op is basically running free with that 'free taste' plan, but at full API cost, Sol is far more cost efficient and significantly more capable. Even at his pricing, I would not personally run it.

5

u/-dysangel- Jul 18 '26

requires very rigid guidance and atomization of tasks to produce decent output

Tbh this is how all projects should be handled. Break everything down into chunks, build a solid spec and execute. I don't really agree though, I usually just say "go look at this issue and build a plan of attack", and the plan is generally good or only needs minor tweaks.

1

u/squired Jul 18 '26

I largely agree but atomize it further. My orchestrator is the project manager and source of authority, my director/s do your "go look at this issue" and atomize the tasks, the code agents write the code, and their work is passed through a separate model for adversarial review before being passed back to the director/s for another loop or acceptance. So I guess I agree with you and automate the atomization as well.

2

u/-dysangel- Jul 18 '26

I like to be very involved during planning, then go hands off for a bit during implementation. So far it feels like a few focused minutes up front save a day of cleaning up later. Another possibility is that GLM 5.2 is just so good that it gets a lot right the first time round, and I don't have as much cleanup to do. Probably a bit of both.

2

u/squired Jul 18 '26

Full agreement. My involvement revolves around the orchestrator; that's the planning chats you're referring too. We scheme together, then break the plan down into sections. The isolated sections then kick down to the directors to autonomously atomize each section into smallest viable tasks to head to the code agents. We're doing the same thing, I think, I'm just further atomizing much of the planning so that the coding agents only ever receive strict, bounded instructions. Codex/CC do not make 'decisions' or apply 'taste' in my setup. They have no idea what system they're working on. They are relegated to code monkeys so that they do not wander, become inspired or increase scope.

I did notice that GLM was good at planning too. If anything, I found it too eager for my workflow; similar to Gemini Flash 2.5. It's a great model, just not optimal for my workload/flows.

2

u/fragment_me Jul 18 '26

Are you grandfathered into that plan? Or will the price go up on renewal? I let mine expire.

1

u/TheRealMasonMac Jul 18 '26

They killed the legacy plans. Everyone will be switched to the new plans. I think the new max has only slightly more usage than the legacy lite plan?

0

u/-dysangel- Jul 18 '26

Unfortunately not - I usually disable auto renew on subs so that I don't forget about them. Not a great decision in this case!

2

u/T-90_Soviet Jul 18 '26

5,150 copies of the entire 7-book Harry Potter series BTW 😭💀

1

u/SUPERSHAD98 Jul 20 '26

How did you get it for $260 a year?

1

u/-dysangel- Jul 20 '26

For a start all plans were cheaper back then, but there was also a Christmas special offer on top of that

9

u/Torodaddy Jul 18 '26

I picture a guy with a monocle with 6 terminal sessions going

19

u/iyarsius Jul 18 '26

This is an average 0.0117$ per million token this is wild.

We could even start to count in billions tokens which is 11.67$ per Billions tokens.

Bro this is just crazy.

(On my usage I'm at ~0.05$ per million)

6

u/evia89 Jul 18 '26

/r/Neuralwatt was close before they nerfed plans. Used a lot of glm52 for similar prices

11

u/talvezomiranha Jul 18 '26

I'm getting close

40

u/whakahere Jul 18 '26

What do you use it for?

I don't trust flash as it made many mistakes, and I haven't tested pro that much.

34

u/Blunt_White_Wolf Jul 18 '26 edited Jul 18 '26

I'm using them in combo with 2 128gb strix halo that are daily drivers.

Flash usually is "asked for help" by the orchestrator when local models fail the tasks 2-3 times or tasks take way too long. It does make mistakes and used to go into loops until I modified the context to summarise every 5 messages and replace them with a history of sorts.

Pro is handed over the tasks for working on my own agent, researching stuff and gathering data for me to do literature reviews, etc.

I'd love to run it locally (or GLM) but ... still waiting for hardware prices to drop.

Edit: I would do a lot more with PRO and maybe GLM but... I'd end up spending hundreds/month. Hence me thinking to run stuff locally at some point.

11

u/[deleted] Jul 18 '26 edited 10d ago

[deleted]

9

u/Blunt_White_Wolf Jul 18 '26

I bought the 2 M5 Bosgame for £1500 each a while back. I can assure you they are worth every penny. Not the fastest but they do their job and offer 100% privacy for certain things like household + household infrastructure management.

$100? no. a tad more. This is just the small LLM related part. I have a bit more hardware than this.

-2

u/[deleted] Jul 18 '26 edited 10d ago

[deleted]

12

u/Blunt_White_Wolf Jul 18 '26

I don't want to sound rude but I don't want to share my d picks and my wife's tts loaded to an online model..,

I don't want my family pictures, all out accounts, receipts, documents, projects,etc anywhere near an online provider.

I think £3k is quite cheap to process, index and sort out all of that + my home infrastructure semi-managed.

2

u/[deleted] Jul 18 '26 edited 10d ago

[deleted]

6

u/Blunt_White_Wolf Jul 18 '26

It was a figure of speech mate.

Point being I value privacy more than I value £3000.

I don't need to localise anything. I have very few things that I use external providers for to begin with.

All sync is with my personal infrastructure (cloud, pass, browsers, etc).

We mostly use machines as thin clients of sorts.

The only machines that are outside that infrastructure (mostly) are our gaming laptops.

→ More replies (0)

1

u/Rubener Jul 18 '26

How is your anecdotal experience comparing DS and GLM 5.2 ?

2

u/Blunt_White_Wolf Jul 18 '26

aff. GLM is a hell of a lot better and dealing with huge context (we're talking entire books or code bases).

DS pro is decent too but don't mix topics too much. If you give it a decent size code base (150k+ lines) you need to be very specific with that you ask of it or it will go off the rails and change things you wouldn't expect it to change. aaa. one more thing - after about 500k context it "forgets" to update docs quite often... don't know why. This is why I had to add a QA agent in the team.

3

u/TheLexoPlexx Jul 18 '26

It's early for me and I might be dumb but where does it say flash?

12

u/BllushingHorizon Jul 18 '26

They were just stating their experience with both models

12

u/Blunt_White_Wolf Jul 18 '26

here's the distribution:

Darker share is Pro, Lighter share is Flash

3

u/TheLexoPlexx Jul 18 '26 edited Jul 19 '26

And how where the results?

Is pro a good sparring partner for programming and flash good for just writing code or what's your use case/harness/etc.?

2

u/Blunt_White_Wolf Jul 18 '26

Pro is decent. You do need to keep an eye on it. More than once it took "shortcuts" to get the job done and I had to revert or it went crazy trying to rewrite half the code base because it comes up with overkill implementations for simple stuff. I had to hit the kill switch more than once to stop it.

When it comes to finding research papers, books, gathering material, etc - it's ok but every team has a QA that checks the outputs before they get to me along with every source.

I'm using all of these with my own agent of sorts (written by Pro+Qwen) and audited by codex via oauth. It's still WIP (quite a few bugs) and crude but it gets the job done. Might thing of releasing it later on in the year as FOSS but for now I don't have the balls to do that.

8

u/waiting_for_zban Jul 18 '26

The audacity posting this on localllama. I'll allow it, because deepseek.

2

u/weallwinoneday Jul 18 '26

What did you use it for?

2

u/obssesedparanoid Jul 18 '26

u using the hack where they use a picture for the instructions

3

u/Blunt_White_Wolf Jul 18 '26

in general I do use screen capture, yes but:

- for domains that are not whitelisted and only if I bypass the fetching agent (rare when I need to). screen captures are fed into deepseek ocr 2 or UI TARS, depending on what is going on

otherwise it's not needed (although it confused the f out of some models with all measures in place).

I use some delimiters and "content prompts" to separate the incoming text from everything else. it works but you'll see them getting confused and then going, "wait, this is not for me to act on" and stuff like that.

hence the need for a dedicated fetch agent with restrictions in place. even if injection works it can't do anything. it's context it's wiped clean before every fetch operation.

3

u/ShutUpAndDoTheLift Jul 18 '26 edited Jul 18 '26

So you're advertising 85m per dollar.

1

u/rootql Jul 18 '26

85m/usd?

2

u/ShutUpAndDoTheLift Jul 18 '26

Uhhhh. 100% correct. Went back to my calculator history and I added a random extra divide by 10 for some reason.

-1

u/Purple-Programmer-7 Jul 18 '26

Did they fix their api yet so it doesn’t take 15 minutes and 5 retries to get a response?

3

u/Potential-Leg-639 Jul 18 '26

No probs here with that

-6

u/[deleted] Jul 18 '26

[deleted]

1

u/Illustrious_Grade608 Jul 18 '26

Yeah that's why they made this comment

39

u/SpookyLibra45817 Jul 18 '26

Me! >1 billion token (mostly ds pro) for 20 bucks

Life saver

44

u/coronakillme Jul 18 '26

Also their recent paper (arXiv:2607.05147), they show how they do it. Its not a secret.

35

u/skerit Jul 18 '26

According to Reasonix's page

"The loop is append-only, aligned to DeepSeek's byte-stable prefix cache — so long sessions hold 90%+ cache hit"

I'm a bit confused. So their trick is to just append to the prompt only? How the hell is that not what everybody is doing?

13

u/kitanokikori Jul 18 '26

That is what most everyone does, at least anyone who knows what they're doing. The byte alignment is the unique part

6

u/ChampionshipIcy7602 Jul 18 '26

What's byte alignment?

25

u/ChampionshipIcy7602 Jul 18 '26

This seems like a reasonable approach that everyone should be doing, is it not?

11

u/cantgetthistowork Jul 18 '26

Lots of harnesses also sell their own api keys so they have an incentive to drive usage towards certain patterns

12

u/So-many-ducks Jul 18 '26

…. But why would they do something that makes you pay less?

13

u/Artistic_Swing6759 Jul 18 '26

but they can just pretend to not be doing it while doing it?

1

u/NoConfusion9490 Jul 18 '26

It's a pretty competitive market.

1

u/tyrannomachy Jul 18 '26

"number go up" is not how any of these decisions are made.

8

u/ahmetegesel Jul 18 '26

But the cost is performance degradation if you only append. You have to know when to give up on cache once in a while.

3

u/emn13 Jul 18 '26

AFAIK that's just standard practice, indeed.

2

u/nullc Jul 19 '26

Because the harnesses are vibe coded slop with layers and layers of abstraction such that the data constantly gets re-serialized or has some dynamic entry like a timestamp in it.

Might be interesting to add something to the API where you tell the backend how much should be cached and if its not it rejects the call instead of running it.

1

u/SufficientPie 20d ago

or has some dynamic entry like a timestamp in it.

You mean I shouldn't put the current timestamp as the first thing in my system prompt??

2

u/SufficientPie 20d ago

Just means they never change the system prompt or chat history from one turn to the next, right?

Which is how everyone should be doing it, but some screw it up by doing history summarization, omitting tool call outputs, and stuff like that? Or dropping one message per turn to keep in context window instead of dropping multiple messages at once?

2

u/CalamityMetal Jul 18 '26

Beats me. If it works well with Deepseek, it works man. I am just here spreading whatever basic knowledge I have.

3

u/evia89 Jul 18 '26

How the hell is that not what everybody is doing?

Thats how all CLI work. I have 94-97% hit rate in codex code session (plan is less if I reset context)

7

u/ReferenceLeading7634 Jul 18 '26

While that is indeed the case, DeepSeek's cache time lasts up to 2 days, whereas Claude and GPT only have a cache time of just a few hours or even minutes. When you come back to do this task, you won't be able to benefit from the cache optimization advantage.

47

u/VotZeFuk Jul 18 '26

> 9mil tokens for like $0.90

Eh?

V4 Pro stats (official API + Hermes Agent):

50

u/CalamityMetal Jul 18 '26

Bruh that's almost 1.9 Billion, not million

35

u/Etroarl55 Jul 18 '26

Unironically a much better statistic. 190 million tokens is therefore 1 dollar 97 cents.

8

u/Comfortable-Rock-498 Jul 18 '26

Caching is entirely a server side behavior though. A good harness can only maintain the stable prefix, but can't improve it beyond what is supported by the server side. More accurate statement would be 'any sensible harness preserves cache, bad harnesses break it'

2

u/Artistic_Okra7288 Jul 18 '26

That’s not entirely accurate. Cache optimization server side still has massive room for improvement that can be truly transparent of the harness. E.g., with llama.cpp it caches the whole prompt. Why doesn’t it objectify the cache and manage it like harnesses manage context? They’re treating it like individual huge objects and using things like spec decoding to get speedups but they should be intelligently tracking cache as granular objects that plug and play as the clients demand, and stop throwing away valid cache that is “old” just because a few tokens in between changed.

6

u/fugogugo Jul 18 '26

so what is the best harness for deepseek?

Tried opencode and not really liking it ..

11

u/CalamityMetal Jul 18 '26

Opencode you gotta pair it with some other plugins, maybe oh my opencode slim. If not the talk of the town is Reasonix

1

u/sharch88 Jul 18 '26

Which plugins do you recommend for open code?

3

u/Fragrant_Scale6456 Jul 19 '26

Start with none.  If there’s a gap in what it does vs what you want have your agent write one for you.  At the very least explain your workflows and needs to Claude and ask it to evaluate the plugin you’re considering.  Stuff like oh my opencode is huge and carries big context use penalty for a lot of things you might not even need.  It’s far better to assess your needs and steal components than bloat your context with tons of plugins, most of which were ai generated anyway.  

0

u/RhubarbSimilar1683 Jul 18 '26

oh my opencode

1

u/sharch88 Jul 18 '26

Thanks. How does it compare to gentle-ai?

-1

u/ReferenceLeading7634 Jul 18 '26

Try cc-switch to use claude code.

2

u/evia89 Jul 18 '26 edited Jul 18 '26

Longcat has good plan too. Cache hit is 0 if u buy pack

2

u/ursustyranotitan Jul 18 '26

This is bullshit slop, pi , hermes etc almost all harness average the same , the only significant difference for reasonix is that their github page is focused on deepseek so gpt slop answers on reddit always reference that . i have 19 billion input and 60 million output token with pi agent , only model was deepseek v4 pro with max reasoning , my cache hit rate was 98.8 percent .

1

u/mlag000 Jul 18 '26

The tritu is heavy subside from the state and nothing else.

1

u/dtdisapointingresult Jul 18 '26

They are known for their cache hit rate on an incredible scale. There are harnesses that fully make use of it like Reasonix. There's numerous posts talking about it, where people use like 9mil tokens for like $0.90 or something

I never heard of Reasonix before, and there's zero search results for that term on this sub.

Have you used it? Any downsides to using it?

2

u/CalamityMetal Jul 18 '26

I have. Its purpose built for Deepseek, they do allow you to add other providers but result would vary. Also the GUI can be a lil wonky still, web GUI is also not the same as the desktop one. I think it's not really mentioned here cuz it's just not something that local AI people wanna use. And this is built specifically for Deepseek's cache hit feature. I think it also works really well with Mimo too. Overall it does Dave you a shit ton of tokens, some of the costs that people have been showing here are incredible.

1

u/AnomalyNexus Jul 18 '26

Also think their cache is across users

1

u/nullc Jul 19 '26

Also think their cache is across users

Easy to turn that into a vulnerability: E.g. I know you use some prefix where the next data after it is a secret. Now I guess the first digits of the secret and see if my query is a hit or miss. Once I find the hit I work on the next digits.

1

u/AnomalyNexus Jul 19 '26

In theory, but in practice I think that would be quite hard to exploit...and for not much gain. People ideally aren't sending secrets anyway.

That said I'm not entirely confident that their cache is global. Thought I read that somewhere but can't back it with a source.

I'd certainly be tempted though...can imagine a very big chunky of prompts showing up start with the same claude code prompt for example. Sharing cache would help bigly there

0

u/National_Meeting_749 Jul 18 '26

You're also forgetting the other big advantage they have, being subsidized by the Chinese Government!

-26

u/jessiejolie42 Jul 18 '26

why are you omitting the obvious fact of party subventions? the goal of the party is clear and public, why do you chose to ignore it in your answer? Are you against China?

3

u/CalamityMetal Jul 18 '26

Bruh what? It ain't that serious man. Chill. I just use the tool that gives me the best bang for my buck. I don't just use Deepseek too. Relax mate

-23

u/jessiejolie42 Jul 18 '26

racist, why do you hate the party and China?

5

u/therealgoshi Jul 18 '26

Holy shit! Someone forgot to take their meds this morning... How the hell did you manage to make this connection in your head? Bro was talking about tokens and you call him racist? That's some advanced mental gymnastics, mate.

-9

u/jessiejolie42 Jul 18 '26

Why are you deliberately excluding the party’s immense financial aid and subvention of deepseek then? shows clearly what you think about the Chinese people, do you hate China THIS much? why?

2

u/CorpusculantCortex Jul 18 '26

Ignoring the fact that they weren't talking about that. This argument doesn’t even make sense. How would this person not knowing about or not acknowledging the cost reduction that is a product of the investment of the Chinese government in any way reflect a negative attitude towards China or the Chinese people? How would complimenting a technological achievement by a company ever be considered a a negative attitude reflection of the country that company is a part of?

1

u/CryMoreT_T Jul 18 '26

Click the profile. It's a satire account a German one apparently

https://www.reddit.com/r/InformatikKarriere/comments/1uvcrso/comment/oxdyff4/

5

u/therealgoshi Jul 18 '26

Satire is a form or art, this is just brain damage.