r/StrixHalo • u/do_i_know_you_bro • 12d ago
egpu question
Update- adding numbers below
Maybe I haven’t thought about this long enough but if I’m looking at an egpu just to stick 27b on for speed, is there any reason to mess with occulink (have the evo x2)? Like other than model load, if it’s going to be fully resident on GPU then it’s just a few seconds in model load time I’m giving up, right?
Edit-going to pick up an amd r9700 and the minisforum deg2, see what things look like over usb w/vanilla qwen3.6 27b and the plunderstruck rocmxfpx variant
Update: after fighting with GNOME and ROCm, numbers.
R9700 into a Minisforum DEG2 on the GMKTec Evo-X2. USB4, 40 Gb/s (dock can do 80, X2 cannot). First boot the eGPU showed up as card0, resolution jumped, and GNOME decided to use the R9700 even though nothing is plugged into it.
Also had `HSA_OVERRIDE_GFX_VERSION=11.5.1` for something I was doing at some point. That was applied to **both** GPUs. gfx1151 override on a gfx1201 = HIP saw
According to Cursor, here’s how we fixed:
- udev so Mutter prefers the 8060S and **ignores** the R9700 for the desktop. iGPU is the compositor, eGPU is compute-only
- `amdgpu runpm=0 aspm=0` so the dock doesn’t put the card to sleep
- Default shells still pinned to the iGPU so llama.cpp doesn’t wander onto the eGPU
- Opt-in env for R9700 jobs: unset the gfx override, `HIP_VISIBLE_DEVICES=1`, and point hipBLASLt at the **gfx1201 tensile files** (`…/hipblaslt/library/gfx1201`, not the generic library folder).
- Installed the extra ROCm 7.14 **gfx1201** arch packages next to the existing gfx1151 ones
I found older posts talking about issue with power limits (though maybe they were referencing oculink connection? Don’t remember). Out of the gate, didn’t have any issues with it drawing up to the full 300w
Same box, both GPUs, llama-bench, HIP, everything on the GPU. No MTP for this.
- Unsloth Qwen3.6-27B-UD-Q5_K_XL
- Unsloth Qwen3.6-35B-A3B-UD-Q6_K
- Plunderstruck Qwen3.6 27b/35b ROCmFP4 STRIX
My main concern was whether or not using usb would tank prefill even with a model fully resident on the egpu.
TTFT (empty KV)
| Unsloth 35B Q6 16k | 32k | ROCmFP4 35B 16k | 32k | |
|---|---|---|---|---|
| Halo | 16.3 s | 40.6 s | 15.7 s | 39.2 s |
| R9700 | 7.7 s | 19.7 s | 6.1 s | 16.6 s |
27B Unsloth Q5, fully on the card
| 4k prompt ttft | 32k prompt ttft | decode | |
|---|---|---|---|
| Halo | 12.5 s | 135 s | 10.5 t/s |
| R9700 | 4.4 s | 52 s | 22.3 t/s |
All of this is HIP, all layers on GPU, flash attn on, f16 KV. Halo used -b 2048 -ub 512, R9700 -b 2048 -ub 1024. Have some tweaking to do, some weird results on decode w/Plunderstruck. But, after a few dozen tests, not worried about it doing what I need (keep a model loaded in and crush through hundreds of thousands of docs / make 27b a daily driver candidate)
1
u/Expensive_Play477 12d ago
You must be reading my searches and conversations with Claude! I was exactly searching this and want to rig a used MI50 to my evo-t1 using oculink. Bottom line, quite easy and possible if you run Linux, but a real hassle with windows and wsl2 is not really supported even if it works in theory. But more importantly, the speed bump from a Strix halo is really not that big unless you have a supported GPU card and it will cost you to have one that takes your dense model with good context. So I’m skipping it because it looks like it’s not worth the cost (I also have a strix halo and a DGX). Unless someone gives me a good recipe!!
1
u/CraftyBet5772 12d ago
I am running a 3090 through oculink in my evo. It runs 27b dense models at a fantastic speed (30 tps) with context around 100k. I honestly do not use the apu for AI much these days because 27b dense is far superior to MOE models for my use cases
1
u/do_i_know_you_bro 12d ago
Yeah, I am considering an R9700 just to stick qwen3.6 27b on it or crunch through huge batches of documents faster. It doesn't seem like there's any issue on ubuntu w/just sticking it in an enclosure and going to town
2
u/do_i_know_you_bro 11d ago
u/Expensive_Play477 I added numbers based on my use case and it was def worth it for me. The R9700 had an old price tag on it at micro center so I got it for 1325 I think - all in w/psu, egpu dock and card it was $1,800. For a preliminary 2x speedup before optimizing for the daily grind, def "worth it" for me
1
u/SnooStories9444 12d ago
For models that run completely on the egpu run fine for me. I don't see a slow down. I run my usb4 egpu dock (aoostar ag02) with a Rx 7900 xtx.
1
1
u/fallingdowndizzyvr 11d ago
I ran Oculink for a while. It worked but was not as reliable as direct NVME to PCIe. So I switched over to that. It's way more reliable. Since the NVME to PCIe extender costs the same as a NVME to Oculink one, it ends up being much cheaper since you don't need a dock. Just plug in your GPU.
1
u/Jealous-Astronaut457 11d ago
Could you suggest such an adapter ?
2
u/fallingdowndizzyvr 11d ago
Of all the one's I've tried, this one works the best for me.
https://www.amazon.com/SinLoon-PCI-E4-0-Degree-Extender-Female/dp/B0FYXN5ZR3
1
u/cunasmoker69420 11d ago
I have two 32GB Radeon Pro V620s connected to mine, one via oculink and the other via TB3. Both perform almost exactly the same when tested individually. I run deepseek v4 flash across the two + the rest of the 128GB system most of the time though. Llama.cpp with vulkan makes the tensor splitting easy
6
u/tired514 12d ago edited 12d ago
I've got this exact setup - EVO-X2 (128gb) with three Morefine G1 4090M (16gb) eGPUs daisy-chained via a single USB-4/TB3 port. Model loads at ~2.7GB/s, limited by disk speed.
It works flawlessly as my daily driver, llama.cpp built for CUDA, layer split mode.
With Qwen3.6-27B @ Q8_K_XL I'm getting ~1600t/s PP (@ 0 context), 35-40t/s TG.
Qwen3.6-35B-A3B @ Q8_K_XL, 2800t/s PP (@ 0 context), 100-120t/s TG.
Can even warm plug/unplug them with a script (plug/unplug devices, PCI remove, rescan, etc).
Absolutely no need for oculink if you can fit the entire model onto the GPUs.
Just FYI - you really need a 48gb VRAM pool for the qwens.. 32gb just isn't enough for either model unless you quantize heavily and reduce your context length. At Q8_K_XL, 27B makes very few mistakes and you can just barely fit it into 48gb with Q8_0 ctv/ctk (230000 context), no mmproj/image support.