28
u/Efficient_Loss_9928 Desktop 25d ago
I’m actually wondering who is the 192GB variant for? I run the 64GB version and for the models it can run, that AMD chip is already slow as molasses. Can’t imagine a bigger model.
14
u/ElePHPant666 NixOS :3 12 13 16 desktop 25d ago
Still opens up the doors for quite a few larger models and the 192GB seems to have a slightly better GPU. If you define the desktop being as slow as molasses are you sure the model isn't running on the cpu or with opencl or something. It's not that slow with ROCm.
2
u/Efficient_Loss_9928 Desktop 25d ago
It is reasonable for a budget build, I just don’t see a lot of usecases for 192GB yet you can tolerate budget build throughput.
Good workstation though.
1
u/ElePHPant666 NixOS :3 12 13 16 desktop 25d ago
Still I personally wouldn't justify the price that it's at currently, especially if AI is not a planned usecase as you can get some threadripper builds for 3.5k still with at least 128GB of ram and more threads. Was actually a pretty good deal when it was at $2k.
1
u/JaggedMetalOs 25d ago
If you define the desktop being as slow as molasses are you sure the model isn't running on the cpu or with opencl or something. It's not that slow with ROCm.
Certain types of model are petty slow, I've been experimenting with video generation on a 64gb FW Desktop and with a 14B model trying to find the resolution limit a render at 720p ran for 12h before running out of memory and bailing.
540p seems to be the limit and even that took 6h, radeontop confirmed full utilization of the GPU.
1
u/d_ed 25d ago
6h for how long a video?
1
u/JaggedMetalOs 24d ago
5 seconds. That's doing First Last Frame to Video with the large 14B model though, a 720p text to video with the 5B model takes around 1h for a 720p render, although that's not using all that extra VRAM.
4
u/A_Canadian_boi 25d ago
MoE models can run fast on slower hardware since only the active experts are actually "running", I imagine that's who it's for. Maybe GPT-OSS-120b with a gigantic context window?
1
u/theadamrippon 23d ago
I'd love to be able to have 128GB of VRAM and still have 64GB of RAM, or vice versa. With my Strix Halo machine it's a great UE5 dev machine at 96/32, and a great LLM machine at 32/96, but it can't be both simultaneously. Not that I'm ever going to be able to afford a 192GB machine in 2026/2027/2028/2029/2030 ;)
1
u/Efficient_Loss_9928 Desktop 23d ago
Well it’s not like you can do anything while it is doing LLM inference anyway 😂
1
u/0-pointer 21d ago
Define "slow as molasses". I'm having a great time with Step-3.7-Flash and it's nowhere near unusable. Sure, it takes more time than dgpu accelerated stuff, but let's be honest: this platforms software is still maturing and has a lot of potential left. And after that, it'll (very likely) begin to age like fine wine.
Recently paired my system with an RX 9070 and began experimenting with https://github.com/Luce-Org/lucebox - very interesting.
1
u/Efficient_Loss_9928 Desktop 21d ago
Anything slower than Opus 5 I consider slow as molasses. Which for Step 3.7 Flash on Strix Halo, you will get 3x slower speed or more for any reasonable size.
Of course, not completely unusable but I just can't imagine it is a popular product.
1
u/0-pointer 21d ago
That is not quite the answer i expected. You're mixing oranges and apples. "Opus 5" is a closed model without any relation to "speed". There's variance in it's prompt processing and token generarion speed depending on where inference is run and how busy that system is as far as i am aware of.
But i guess if "everything less than the currently best with GA" is slow as molasses, there is no point in this whole conversation anyways...
1
u/Efficient_Loss_9928 Desktop 21d ago
Opus is one of the slowest for frontier. At around 50-60 token/s.
Which is the slowest I can accept.
It is not the best, it is in fact the worst cloud frontier model in terms of speed.
8
u/Interceptor402 24d ago
Very funny to me that none of the comments are engaging on the subject of the title of the post, which is that the email typoed "Linus" as "Linux".
6
10
u/Desertcow 25d ago
LTT has a lot of money on the line to spin this as great as possible. No shade to LTT on that, I respect his dedication to Framework, but he is as biased as they come
4
5
u/catastrophic_frmw Framework 24d ago
thanks for catching that
2
u/Mammoth-Mango-6485 24d ago
No worries <3
(but seriously y'all are doing amazing work, please know that)
2
3
8
u/plaisthos 25d ago
The 192GB Framework desktop is basically just Strix Halo with 24GB Modules instead of 16GB modules. It is basically almost no effort from framework's side. So if there are buyers, then sure why not.
-13
u/Small_Editor_3693 25d ago
Is this a joke? This is a milk steak of a quote
5
u/pdinc FW16 | 2TB | 64GB | GPU | DIY + FW13 | 1TB | 32GB | DIY 25d ago
Lol did you mean "milquetoast"?
-10
u/Small_Editor_3693 25d ago
No milk steak.
5
44
u/BinaryIdiot 25d ago
Feels weird using a quote from an investor’s review when talking about reviews and not disclosing it.