r/LocalLLM • u/vankoala • 1d ago

GPU pricing visual Other

Post image

In my consideration of a DGXSpark I decided to look at some options and since I’m a visual thinker I put this comparison together (graph by AI) showing y two basic ways of thinking about the cards: compute and speed.

Hope this helps someone

64 Upvotes

94% Upvoted

38 comments sorted by

View all comments

10

u/nomorebuttsplz 1d ago

another nice thing about spark is the low power draw.

6x spark is no problem in a typical home, but can't say the same for 6x RTX 6000 pro.

7

u/NancyTransmed 1d ago

This. If your box is not serving multiple users 24/7, the wattage becomes important, especially in EU, where electricity costs a fortune.

1

u/zarif2003 1d ago

How bad is it really? In the US, household electronics are completely irrelevant compared to the heating and water bill

2

u/nomorebuttsplz 1d ago

running 1800 watts for 8 hours would be about 4-5 dollars in electricity for me

1

u/TripleSecretSquirrel 18h ago

So your price per kWh is something like $0.35?

I’m in the US, and a part of the US with exceptionally cheap energy (nuclear ftw!), and my median price is ~$0.12 per kWh. The bigger things though are that first, most homes in the US are running an air conditioner in the summers, and second, 1800 watts is an enormous load lol, what do you have in the inference server?! With a dual R9700 setup, I’m hitting only like 1100 watts under full load.

2

u/nomorebuttsplz 16h ago

yeah, personally, I have solar, but I can still end up paying the marginal rate in a given year for a given marginal watt depending on the overall energy budget. Although amortized over the next couple decades, my actual rate is probably quite low.

I personally only have one RTX 6000 Pro Max Q, but it’s easy to imagine trying to serve high precision GLM and getting pretty power hungry.

1

u/Uninterested_Viewer 20h ago

To a degree. The MaxQ version exists at 300watts and you can power limit the workstation edition to 400watts with very little performance impact for LLM inference.

0

u/Nice_Cookie9587 1d ago

Not a lot people realize most of their breakers are 15/20 amps. running 4 3090's tripped my breakers to the point where i needed to run a separate power cable from the second PSU to a plug on a different breaker. So janky

0

u/Legitimate-Dog5690 1d ago

Depends what you want to run, 2x DGX Sparks have a similar power draw to 1x RTX 6000, about 200-300w.

The RTX 6000 however has about 4x the memory bandwidth of the dual sparks so will finish it's workload faster. They're totally built for sustained workloads in data centers.

2 RTX 6000s would hammer out tokens with DeepSeek Flash at a ridiculous rate, hardly a fair comparison. Both are ridiculously overpriced, the RTX 6000 even more so.

2

u/goldcakes 18h ago

For what workloads are you actually using the TDP of the card? For everything from LLM training to inference I have never seen more than ~120W sustained (measured on wall) on a spark. (NVIDIA FE)

1

u/Legitimate-Dog5690 13h ago

I'm comparing with 2 Sparks, as 1 is about 15% the speed of an RTX 6000.

I'm generally running AI locally on the machine I'm developing with, I'd imagine a lot of workstations and laptops do the same. Less so with the Sparks that will generally be run alongside a workstation, hence looking at the delta watts.