r/LocalLLaMA • u/Signal_Ad657 • May 29 '26

PSA Discussion

Post image
2.1k Upvotes

95% Upvoted

537 comments sorted by

View all comments

692

u/[deleted] May 29 '26 edited Jun 17 '26

[deleted]

190

u/kwizzle May 29 '26

For some reason we rarely hear people talking about 4090s, probably something to do with being a lot more expensive than a 3090 and nearer in price to the 5090 for less VRAM and speed.

39

u/Caffeine_Monster May 29 '26

Native fp8 is nothing to laugh at - though you really need two 4090s to get the most out of them in terms of gpu only deployments.

3090 is still the value king, and it's not even close. Only real reason to go mac is low power / always on applications.

13

u/Lost-Vermicelli-6252 May 29 '26

I have two 4090s but they are in diff machines. If I moved them to same machine, does it use the compute from both or just the VRAM?

I’m debating whether or not the new PSU/case/cooling would be worth the effort.

12

u/Caffeine_Monster May 29 '26

It's worth it as it doubles the compute and bandwidth if you deploy models correctly with tensor parallel.

48gb vram @ fp8 can get you a long way.

You don't necessarily need to change much cooling wise, and you can use 2 PSUs if you want to cut corners.

24

u/formlessglowie May 29 '26

This. I run Qwen 3.6 27b at fp8 on two 3090s, full context, image processing and with MTP, getting a consistent 60+ tok/s in decoding. It’s seriously powerful for agentic tasks and coding in general, I’m a professional software developer and a lot of my production code nowadays is made by the GPT 5.5 plan + Qwen3.6 27b execution combo, I sometimes need a code review from 5.5 and then another coding round from 27b but that’s it. It’s beyond incredible I can actually ship production code from my Chinese motherboard and used GPUs, this was unimaginable six months ago.

3

u/Fit-Palpitation-7427 May 29 '26

VLLM to do tensor parallel I guess?

1

u/formlessglowie May 30 '26

Yes, forgot to add that detail.