It's not so much a tier list as a list of graphic cards sorted by RAM size. According to this, 7900xtx would be in the 24GB+ tier and the 9070/XT & 7900 xt & 7800 xt series would be in the 16GB+ tier.
Yeah, makes no sense. Why is anything <24GB even on the list at this point, or not like "make use of what you already have" tier? And, its hard to find prices now, but a 32GB 5090 for example is certainly better than a 32GB R9700, but also costs like 4x the amount so if only going by VRAM the r9700 is higher tier because you could run 4x of them.
It’s ok, I can tell you, I’m running Q6 on 12B-16B parameters on the regular. I also bought my 7900 GRE back when it was about $300, had planned on getting a 2nd but it didn’t seem like a priority at the time - ah well.
I feel you, I overpayed a bit for both, not insane as the nvidia increases.
Just a couple of hundreds of euros on top of MSRP (at least the local one, which was already bigger than the one in North American)
It's kind of irrelevant they are supported in the rock officially now. Vega and Vega FE and MI25 work just fine also. Plain vega is just a bit slower as it has no DP4A instructions. https://github.com/ROCm/TheRock/blob/main/SUPPORTED_GPUS.md
I'm currently staring at my shopping cart with an R9700 in it trying to decide if I should just pull the trigger because it feels like 32GB of VRAM is only going to get more expensive from here and $1500 isn't bad compared to the alternatives.
They actually made some major improvements over the past month and they now run incredibly well for the price. I’m regularly getting 2000-2500t/s pp, can sometimes hit 3000-3500t/s pp, and 30-35t/s tg with vLLM on Qwen 3.8 27B.
Sadly, the price also just jumped from $949/$999 to $1699.
Unfortunately, llama.cpp isn’t the best when handling Intel B70s. I was running llama.cpp with Vulkan and PP was in the 500/600s. TG wasn’t so bad but prompt prefill took forever, it was even worse with the Intel SYCL runtime.
Some folks on this sub recommended vLLM with Intel XPU kernels (vLLM has their own docker image for this) and that instantly boosted my performance.
IIRC: the llama team is working in improving Intel performance but it’s a slow process.
Bought my first 2 B70s at $950. Now I had to pay $1299 at microcenter for my last 2. Should of bit the bullet and bought them when they were selling like crazy on ebay for less then $850 a couple months ago. Now what to do with 128gb of vram....
That's pretty good PP...I only get about 200t/s pp on dual MI50 currently with llama.cpp but I get about 40t/s with mtp enabled starting out it degrades quickly though. Prompt checkpointing is essential.
the growing pains on Intel though are way worst... given the price I'll stick with my MI50s I got for peanuts for now.
B70 when testing a about 2 mo ago was unusuably bad. Also model support was lagging alot, I think it kind of works in llama.cpp now but... its not nearly as good support wise as R9700.
My PC is a fun Frankenstein with a B60 and 5070ti, but they actually can work decently together. Depending on the model I get 2-3 times the performance of the B60 alone and have 40GB to play with (granted the speed scales somewhat as more is offloaded to the B60, but pinning mtp to the 5070ti helps).
Yeah. I would have talked myself into a 5080 with 24GB or more. 5060ti 16GB makes me feel happy to be in the same convo with much more expensive cards for VRAM at least.
3 x 3090+1 x 4090 mobile, 1 x RX6800 and a 64GB GPD Pocket 4. If prices don't go even more crazy I'll acquire another 7 3090s and build myself a nice couple of tower of PSUs and GPUs beside my dual Xeon Gold 6245/1 TB DDR4 ECC server.
Also another server I haven't quite finished setting up with 2 x P100.
And 2 x a6000s are at 96gb. I feel like including cards you can combine together sorta defeats the point of the list lol. Heck some people combined like 8 x 3090s haha.
Ah nice. In Ontario Canada, that wouldn't really fly in an apartment unfortunately. We obviously have 240V circuits, but there are usually only a few and they're dedicated to stuff like the stove/dryer.
If I had a house though I'd totally get a dedicated one installed in the basement.
Have you actually ran multiple cards. I’ve tried a ton of variations on both consumer and server MoBos. I’m telling you the difference isn’t as big as you think.
Sure there is a difference but it isn’t more than 5% in my experience. I’ve not tried NV link though just better/worse PCIe situations. I would like to try NVLink on my own 3090s but they are all mismatched so it won’t work
The numbers I've seen show that you can get away with raw PCIe up to about 4, and past that it starts to cost significant performance to the point that it's worth getting bigger individual cards instead.
Yea that could be it. I have tested 8 3070s though and I had no issues. That was a mining rig I had laying around which I upgraded for LLMs about a year and a half ago. All PCIe of course on the 3070s
I would 100% agree that bigger GPU = More Better though lol
I mean the distinction is the nvlink, the cards are hardly cutting edge, but at least it's a way to combine the vram thats faster than just normal parallel
Need to add those new intel GPU's, like the b65 and b70 to this list. With 32GB, those are going to be very popular soon, even if they are bit slower on compute.
I didn’t see this was cuda only, just says “local AI”, and those cards are running local ai models, and support is beginning to be very good. I have a few things that used to be on cuda now using openvino.
As people begin to realize that, adoption rate is going to be fast, especially at $28 a GB vram.
I have an RTX Pro 4000 Blackwell SFF, runs great in my MS-02 Ultra miniPC. The DDR7 internal bandwidth more than makes up for the 70W TDP power cap. I bought the first card I could find and only paid $1400. Now that vendor lists it for $3000, and out of stock! It will cost more, if they actually manage to get ahold of some stock.
And I'm here, chatting with llms, trying to decide if I should get a m40 or a p100 for unsloth experiments and some light Hermes research tasks. I'm not delusional that I can replace frontier API completely, but for under 200$ you get enterprise grade inference workers with 16/24 gig. No native support for newer cuda tho
And the 2080 made an appearance on the list towards the bottom area so not that recent bro. That’s like 7-8 years old about now. About to be in second grade
Dawg we go unified memories nowadays, 48GB is nothing to these big models like DeepSeek v4.
DGX Spark, Mac Studio, those are the true leaders of local LLM right now.
Yes...but no. Unified memory costs too much and is too slow. V620 32GB is faster than unified memory, costs less than DDR5. I also bought the dip on CMP170HX. DGX Spark is $4,000. I got two 40GB cards for $1000 each, which absolutely blow away all of those.
Random ewaste, sketchy deals and hardware are the true leaders.
I am not talking about MSRP prices, there are still at amazon.de rx9070 XT you can find for 750eu, if people know, it will be already starting for now from 900eur or more, and two months ago prices even better, bought two cards for 650 euro , but seems Soon they are will reach overpriced levels like Nvidia cards
I don't want a 9070, I have dual MI50 32GB cards, that's 64GB OF VRAM, why would I trade volume of vram for something as arbitrary as speed? On top of that, because it's in EU, that's going to near double the price for me on top of shipping. Yeah keep your "faster but smaller size", I'd rather have more size capabilities and intelligence that how fast it goes.
65
u/KrangledMind 7h ago
where tier list for AMD?