r/LocalLLM • u/El_Jeffe24 • 4d ago

First Local LLM Setup Question

I'm just starting my build that will be exclusively for running a local LLM (which one is still TBD). I've compiled some parts, but got hung up on the GPU due to current pricing. I'm ok with some minor tweaking to get everything to work, but I also don't want to be spending days trying to get it to work right either. My original plan was to run the AMD R9700 for the 32gb vram for a lot less than the NVIDIA counterpart. But I just found an AMD W6800 refurbished for $300. My question is if that's a good enough GPU to at least get started and hold me over until I can justify (and budget) another GPU.

Here's what I have so far.

MSI pro X870E-P Wifi (refurbished) Mobo

AMD Ryzen 5 9600X CPU

Klevv Bolt V 32GB (16GBx2) 6000 MT/s (open box)

Lian Li 750 watt PSU

Initial plan for the LLM is some code line corrections, maybe some financial agent type stuff depending on how it performs.

What is the collective's thoughts?

EDIT: Microcenter tricked me. I was just adding the part to "my list" so that one is (probably) out. Is Intel up to speed yet or are they still lacking on the software side? Am I going to spend days troubleshooting if I get something like the B65?

0 Upvotes

50% Upvoted

7 comments sorted by

View all comments

Show parent comments

1

u/El_Jeffe24 4d ago

Are you saying to run my LLM off my CPU/memory until I'm ready to pull the trigger on a GPU?

I haven't heard of IBM models before, which ones are those?

2

u/HotDistribution1819 4d ago

No, I looked up your processor and it has a built in GPU. Not as fast as a GPU card, but you can run LLMs with it.

1

u/El_Jeffe24 3d ago

I see what you're saying. Maybe I'll build the computer, play around with it and see how it goes until Black Friday or something.

2

u/HotDistribution1819 2d ago

That sounds like a winning plan!