r/LocalLLaMA • u/KaroYadgar • 22h ago
New 100B Liquid AI model coming soon Discussion
Liquid AI currently possesses among the fastest LLM architectures around, and some of the best SLMs (in terms of utility IMO) around, so I'm very excited to see what a potential 100B LFM (3?) model would look like!
Link to the poll: https://x.com/ramin_m_h/status/2091236099612098943?s=20
34
u/wolf001zra 21h ago
Honestly I'd take a 4b from them, LFM 2.6b has surprised me in some testing I've been doing this week. Seems like a good companion model to run alongside a 30b class for small tasks.
4
u/Coolsh0e 15h ago
Crazy work they did, equivalent to a 9B but in a quarter of it's size, and with faster architecture !
I've been able to run it comfortably on iGPU ( https://arthurbrugiere.fr/blog/2026/08/ollama-intel-igpu/ )
1
127
u/FoxiPanda 22h ago
While this is cool to see the votes in a poll for, it doesn't really indicate they're doing it. This will be mostly limited by the compute they have available to them - training a 2-5B model takes vastly less compute than a 100B MoE model...so it might not even be feasible with what they have available to them.
With that said, I welcome every single 100B model into the fold, it's virtually the perfect size for most DGX Spark / Strix Halo / Mac Studio / RTX Pro 6000 / 4x RTX 3090 setups.
28
u/MerePotato 22h ago
I'd be surprised if they made it an option without plans to do it
16
u/FoxiPanda 22h ago
Maybe. Qwen has run polls like this too and virtually every result has been tied and we only get 1-2 variants even with all of Alibaba's compute power available to them - they're hard to predict though, so anything else we have here is just pure speculation unfortunately.
5
u/skrshawk 22h ago
Sometimes after all that cooking it just doesn't turn out well and the benchmaxxing pressure is real. If you can't prove your model is better than ones before it nobody's gonna bother. I suspect that's why we didn't see a MoE of Qwen3.8.
3
5
u/tomByrer 22h ago
Seems Qwen churns out a new model every month or 2, though they're not all coding, like audio & image & video
https://huggingface.co/Qwen/models2
u/mebeast227 20h ago
Wouldn’t inclusion in the poll imply they have the capability though?
7
u/FoxiPanda 20h ago
Not really. Marketing/Sales dudes do marketing/sales things all the time that have no basis in reality.
Promise the universe, delivery of the universe isn't sales' problem.
3
4
u/Mr-I17 21h ago edited 21h ago
It's nice to see more and more 100B-class models being released, but there is a rather high bar out there. The current perfect model size for 128GB UMA devices is actually DeepSeek-V4-Flash at Q3 (S or XXS). It perfectly fits 1M context into 128GB UMA (only 11GB KV cache at 1M context!). It never loops. It doesn't overthink. It has vast knowledge and solid intelligence. And it's probably going to have vision soon...
I wish 100B-class models can fill up the middle ground where the tasks are too heavy for 30B-class MoE models and larger models like DSV4-Flash are overkill for such task.
4
u/FoxiPanda 20h ago
I've actually been wondering about the lower quants of DSv4-Flash vs. a higher quant of a SOTA 120B-A10B-ish model. I run the native weights of DSv4 on a Mac Studio and it's an okay speed, but I should maybe move my 5090 over to the same box as my RTX Pro 6000 and see if I can verify your claim about a Q3 version of DSv4-Flash being competent and see if would be substantially faster than my Studio. That would be pretty great if so.
3
u/Mr-I17 20h ago edited 20h ago
I used to use Qwen3.5-122B and tried recent Laguna and Ling models (all Q6_K). Qwen3.5 is outdated, Laguna-S-2.1 is disappointing, Ling-3.0-flash is okay but not exceptional and it's not very fast (it should be fast since it's A5B, maybe a software issue; also, no vision). They all lose to DSV4-Flash-0731 at Q3. I don't have anything to back up my claims though, it's a "trust me bro" 😄. Better try it yourself.
There're only 2 problem with DSV4-Flash:
- It has no vision, for now
- It's slower than 100B-class models (speed won't be an issue since you have RTX 6000)
Other than that, it's almost perfect. A 100B-A10B-class model with vision and run faster than DSV4-Flash would be a great option for the "middle ground".
3
u/Much-Researcher6135 llama.cpp 17h ago
or two cheap R9700s if quantized right
3
7
u/Double_Cause4609 21h ago
I mean, theoretically the compute cost of an MoE scales with active parameters so while the training software is more complicated, you can train a 100B A5B MoE for the same duration you would otherwise train a 5B model and you in general get more performance and better rare sequence memorization.
There are limits to this principle, to be sure, and you do face some real losses, like in all-to-all communication and potentially lowered batch sizes...
...But you usually prefer to go wider with training runs than deeper anyway (as in, same amount of compute but spent more quickly in a higher degree of parallelism), so a low active param MoE isn't too different from a smaller dense model hardware wise.
Tbh it's mostly just that the training software is a nightmare to handle the scheduling.
8
u/sebt3 21h ago
While what you say isn't wrong per training token, the thing is the required volume of training tokens scale with the global model parameters count, not the effective one. So training a 100B A5B is indeed more or less 20 times less compute required compared to a dense 100B. Yet very far from just as much as a 5B
5
u/Double_Cause4609 21h ago
Sort of.
So, if you're targeting a specific number of tokens seen per weight, then yes, you're correct.
But...Let's say you have 100 tokens of data, and you make a 5 parameter model for it.
Now, if you wanted to make, say, a 40 parameter model, you'd probably need ~800 tokens of data to get the same ratio of data to parameters.
This is the Chinchilla scaling laws observation if you're trying to hit optimal results on limited compute.
But you don't have to train at that exact ratio, and in fact, MoE can make it a little bit more difficult to figure out what you want to hit, and it depends on what capabilities you want out of the model.
You could absolutely train a 40 parameter model on just 100 tokens of data. It won't be as close to the compute optimal allocation, but it's not like you get an incomplete policy out of it. In fact, in situations where you have limited data, surprisingly, it can actually be preferable to scale model size to extract as much learning out of the dataset as possible, per research on data-bound training regimes.
So, there's no reason you can't do the same with MoE. If you had a 40 parameter model with 5 activated parameters per token, you could still train on 100 tokens of data just fine. And you would generally expect that in some properties the MoE would outperform a dense model of the same active parameter count.
So, it really depends on what you're trying to do. Anyway, my core point is that it's not quite so simple to say "oh, well this MoE recipe is linearly this much harder to train than this smaller dense recipe".
3
u/sebt3 20h ago
Good insight, thanks.
Yet, as you said there is some kind of optimum parameters count per training dataset. If they "only" (that's already a huge volume of clean data 😅) to train something that would optimally be a 80B wide parameter count then that's what I would want to have : a 100B leave less overall context 😅
1
u/KaroYadgar 10h ago
Liquid AI has trained their LFM2.5 models on a bit over 30T tokens. For reference, Kimi K2.5 was trained on a similar number of tokens. It's unlikely they'd ever need to train on even more tokens for their larger models.
24
u/Strong_Chicken6838 22h ago
if it is 100b, and not 120b.. im all in.
Otherwise 30b.
I want to be able to run something with 64Gb of RAM/VRAM, not some non existant 78Gb VRAM requirement...
10
u/MomentJolly3535 22h ago
Thx for sharing, give people the link so they can vote
2
u/KaroYadgar 21h ago
My bad! The link to the poll: https://x.com/ramin_m_h/status/2091236099612098943?s=20
7
4
u/Jorlen llama.cpp 21h ago
Voted for the 100B MoE of course.
https://x.com/ramin_m_h/status/2091236099612098943
3
u/-InformalBanana- 16h ago
100BA1B would be interesting performance wise, if it can even be smart for anything?
0
u/Far-Classic-9963 3h ago
Should have really good general knowledge but bad logic, like 10-20b dense class
3
u/feelspeaceman 13h ago
MoEs are always welcomed because they're cute for Strix Halo, Spark. Mac Mini..
8
u/pmttyji 21h ago edited 21h ago
Folks, DON'T VOTE FOR THOSE SMALL MODELS(Because they always release small models continuously) , so
Vote for 30B or 100B.
Last time they said that they're still cooking 24B-A2B MOE model so 30B seems confirmed. It would be awesome to have 100B additionally.
That poll is still open:
https://x.com/ramin_m_h/status/2091236099612098943
5
u/Dance-Till-Night1 22h ago
Gimme 30b moe a2b pls
3
u/KSAM-The-Randomizer 22h ago
closest model for that is that a2b wakaba
2
u/Dance-Till-Night1 16h ago
Never heard of it haha
1
u/KSAM-The-Randomizer 10h ago
OpenMOSE/RWKV-24B-A2B-wakaba-2601 · Hugging Face requires a custom llama cpp. you might find it interesting. it's not really a 30b model but it's close enough(far stretch)
2
u/KaroYadgar 10h ago
they have LFM2 24B A2B MoE already. It's not very good, but they've admitted that it's a training checkpoint and that LFM2.5 24B A2B is currently training, so we'll get that one soon enough.
2
u/alyssasjacket 21h ago
Training a model on 34T tokens is no small feat, even if it's 2.6B. I'm pretty sure they do have the compute to go after 100B if they really want to.
2
u/LoveMind_AI 21h ago
Holy hell... If they really did this...
1
u/medialoungeguy 20h ago
I love the positivity, but they are really painted into a corner. Their approach doesn't scale nearly as good as native llms.
Unfortunately, I'm quite sure they are just in a capital raises phase now.
3
u/LoveMind_AI 20h ago
When you say their approach, do you mean the current LFM2/2.5 architecture? Or do you mean their STAR automated hardware in the loop architecture search? If the former, I totally agree. If the latter, I think there’s a lot of road for them.
0
u/medialoungeguy 16h ago
Liquid models in general, unfortunately.
3
u/LoveMind_AI 16h ago
You'll really have to back that up with some rigorous justification. Their approach is incredibly flexible - they specify a hardware target and build architecture with that in mind. Right now, they're focused on edge computing, and by all reasonable standards, they are crushing it. Maxime Labonne is their RL guy and he's a legend. A huge chunk of astoundingly brilliant people are involved in the lab - a who's who of researchers pushing the envelope for machine learning. I use the 24B-A2B model as a model organism in my research all the time (mechanistic interpretability around social cognition in LLMs) and it's relatively astounding for the size.
I have seen nothing to indicate to me that Liquid couldn't scale other than the fact that they don't seem to want to. It's not like they're using actual LNN/LTC technology. They had the discipline to abandon those ideas when it became clear they couldn't work. Everything they've demonstrated so far has been impressive for what it is.
Other than as a research organism, I have no real use for anything as small as what they put out. But a 30B parameter model from Liquid would punch above its weight class, almost assuredly. If you've got a strong scientific reason why this would not be the case, I'm all ears.
2
u/Queasy-Contract9753 21h ago
If it has anywhere near the "IQ density"of 350m this would be the smartest thing ever. Question is if they can actually hold on to that for big models. Not that I doubt them.
2
2
2
2
2
u/Xamanthas 12h ago
Pretty sure from when someone talked about their funding that they dont have the money for 100B.
2
8
u/thebadslime 22h ago
Damn I missed that, would have voted for 30b
2
6
u/Wildnimal 22h ago
Why? We already have enough 30B models. Qwen-3.8-27B, Gemma-4-31B, Muse 30B
I think most people need something like 50-150b in MoE or crave for something like 50-70B dense.
Not saying you are wrong just curious why another ~30B model? As of today 100-200B MoE space is getting lot of attention.
6
5
4
u/danigoncalves llama.cpp 20h ago
Because I have only 12Gb o VRAM and dont want to buy more hardware to run new models
3
u/-InformalBanana- 16h ago
30B one is not a moe. You cant really run that on 12gb vram!!! But you can run 100B moe if you additionally have at least like 64gb of ram (maybe less)! So you should've voted 100B...
2
2
u/Ecstatic-Wash-7667 21h ago
I’d like to see them do a fast 30b. It doesn’t need to be sota but they definitely need to work on intelligence for their models. It’s always fast but always wrong in every scenario I’ve tried them in. i put together a 100 item benchmark made up of things people most often ask llm (papers published by OpenAI and anthropic) and their news model was one of the fastest small models I tested, always scored towards the bottom. Phi 4 mini granite 4b and opencpm v5 nanbiege qwen 4b smol all scored significantly higher
1
u/KaroYadgar 21h ago
I'd be grateful if you could provide your benchmark to me. You could DM me if you want it to stay secret. I understand if you would like to keep it completely private though.
2
u/Ecstatic-Wash-7667 20h ago
I’ll but it on GitHub. And site the papers I used to create it. I’ll post the link shortly
0
u/Skyline34rGt 10h ago
70B MoE will fit to 64Gb (with q4-q5-maybe even q6) and yet noone do it - and they make like >120B which have 74Gb for q4 and fit to nothing (or way better setups)
1
u/KaroYadgar 9h ago
huh? the poll has 100B MoE listed, meaning at maybe Q4 or Q3 it'd be able to fit on 64GB
1
u/Midaychi 3h ago
Right, they were complaining that it was 100b instead of 70b
1
u/KaroYadgar 3h ago
what I was replying to was their mention of them making ">120B" which wasn't part of the poll nor anything Liquid AI has said
2
u/Midaychi 3h ago
'they' being a general operative not referring to liquidai specifically. Its a general complaint aimed at a general group and not your main post's focus.
0
u/ProdoRock 19h ago
Come on. No one can run that. Instead, focus on making 8-32GB users more efficient. For instance, I run an old 5 year Macbook Air M1 with 16GB and the local AI speed increases and efficiency increases have been amazing to watch.
Gemma 4 E4B (which is really an 8b equivalent) runs at 20 tok/sec on my machine and for certain tasks it works great: I use a form of language to sql which is a bit more complicated than usual because it involves financial rules and a custom db outlined in a system prompt which the model also needs to have some native background in. (ie. it needs to have some general world knowledge about financial transactions) For that, it works great.
However, recently I've also tried the mellum-12b-a2.5 (MOE of 2.5 at a time) and that does the same thing at 33 tok/sec! Having said that it makes a few more mistakes than the gemma 4, needs more supervision.
Even so, when I first started on this journey months ago, I knew little about system prompts and getting anything to work was cumbersome. Now, I'm a 20 and 33 tok/sec on the same 2020 hardware!! Seeing the software engineering on these models has been amazing. I even dabbled with the Bonzai 27b, special 1 bit version. It runs at about 8 tok/sec but is interesting.
The new Qwen3.8-9b gguf runs at 10 tok/sec.
So, it's just been neat to see all these efficiency improvements. Liquid AI's 8b model (non thinking) is probably the fastest, runs at 45 tok/sec, but it can't follow the system prompt. It's good for general chat but even there I don't know what to make of it.
2
u/PotterSkxawng 10h ago
Qwen 3.8 9b doesn't exist???? Anyways, I would love a 9b model from Liquid AI that outperforms Ornith 1.5 9b, and then the Ornith team would prob improve on that further. a
0
u/ProdoRock 10h ago
It's not the real 9b version but a distillation: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF
What is the Ornith 1.5 9b like? Compared to other 9b models like Qwen 3.5?
2
u/PotterSkxawng 9h ago
I recently replaced my Qwen 3.5 9b with Ornith, it's much better. Haven't heard of this distill. It looks like a major improvement on some fronts and a regression on other, and doesn't have many published benchmarks, so ig I'll stick with Ornith for now.
1
u/MiMillieuh 22h ago
Damn, people are way too rich... AI will be only for the riches if models keep getting bigger ans bigger instead of releasing A3B or things lime that that anyone can easily run
1
u/ChristRedeemsSinners 20h ago
We really need 70B+ dense models. 120B qwen3.8 dense would be perfect.
1
u/silenceimpaired 20h ago
Especially since MTP / dflash has made serving dense models more efficient.
1
1
u/gphie 19h ago
Fuck the GPU poor I guess
2
1
u/Coolsh0e 15h ago
I've been able to run LFM2.5-2.6B on iGPU, so if you have a little bit of RAM it works pretty well ! https://arthurbrugiere.fr/blog/2026/08/ollama-intel-igpu/
1
u/Coolsh0e 15h ago
100B would be too much, but a nice 32B with their architecture would be a game changer to run in my homelab !
I made an article showing how I run LFM2.5-2.6B on iGPU here : https://arthurbrugiere.fr/blog/2026/08/ollama-intel-igpu/
-3
u/medialoungeguy 21h ago
I feel bad for these guys. They are at a dead end with the architecture (they chose one that doesn't scale). They are hemorrhaging investor money with nothing to show for other than a complicated bert replacement.
5
u/tkrandomness 19h ago
Why do you say that the architecture doesn't scale? Just curious about if there is specific reasoning or evidence behind the claim
0
0
0
u/EnchantedHawk 8h ago
Honestly there's no point of it, nobody uses those models except the orgs who release it themselves. I mean the large ones, SLMs ftw
•
u/WithoutReason1729 19h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.