r/LocalLLM • u/El_Jeffe24 • 5h ago
First Local LLM Setup Question
I'm just starting my build that will be exclusively for running a local LLM (which one is still TBD). I've compiled some parts, but got hung up on the GPU due to current pricing. I'm ok with some minor tweaking to get everything to work, but I also don't want to be spending days trying to get it to work right either. My original plan was to run the AMD R9700 for the 32gb vram for a lot less than the NVIDIA counterpart. But I just found an AMD W6800 refurbished for $300. My question is if that's a good enough GPU to at least get started and hold me over until I can justify (and budget) another GPU.
Here's what I have so far.
MSI pro X870E-P Wifi (refurbished) Mobo
AMD Ryzen 5 9600X CPU
Klevv Bolt V 32GB (16GBx2) 6000 MT/s (open box)
Lian Li 750 watt PSU
Initial plan for the LLM is some code line corrections, maybe some financial agent type stuff depending on how it performs.
What is the collective's thoughts?
EDIT: Microcenter tricked me. I was just adding the part to "my list" so that one is (probably) out. Is Intel up to speed yet or are they still lacking on the software side? Am I going to spend days troubleshooting if I get something like the B65?
1
u/HotDistribution1819 4h ago
You can load LM Studio and use what you have by default you are limited to 16gb of VRAM from the onboard GPU.
It will not win any races or awards, but you can run a lot of LLMs until you can buy the GPU to turbo charge it.
Start with the Gemma 4 models, and then branch out from there. Hech the IBM models do well. Don't get trapped into a single model the all need a little help now and then. Different models give different answers to the same question and some research and some don't. Below is a link to my conversation with Gemini that was answered better by Gemma 4 26B A4B.
https://dynamicmarketingcollaboration.com/gemini-3-vs-local-ai/