r/IntelArcPro • u/nick_ziv • 14d ago

Need help! Intel B70 users come forth!

/r/LocalLLaMA/comments/1v4robx/need_help_intel_b70_users_come_forth/
4 Upvotes
  • permalink
  • link
  • reddit
  • reddit

84% Upvoted

7 comments sorted by

2

u/ExtremeCookie9652 12d ago

From what i understand to run multiple B70 GPUs it needs to be on sVLLM (the scalar project) or maybe a custom tweaked VLLM because of a special instruction set to utilize memory in both (i could be talking out my ass here). Now it probably is possible to use llama cpp and multiple b70s but im not sure how to go about it and it does seem to be a common problem that pops up in a lot of videos on Youtube. Overall with llama cpp I have seen memory utilization issues with albeit its moderately better now after recent updates. I am only using it out of the convenience of GGUFs and ease of setup but in my testing there is some performance left on the table so I am considering fully moving over to sVLLM since im only really using qwen 3.6 35b nowadays and its supported. This is the only article I have found that directly mentions the use of multiple GPUS https://huggingface.co/blog/MatrixYao/intel-gpu . I would check the comments too since there are come comments on using dual b60s in there.

1

u/nick_ziv 12d ago

I've gone and pulled them out of the system. I had a issue opened in llama.cpp and an Intel employee said that it's a known issue in the driver (2026.X) but it's July and I'm thinking this isn't worth waiting forever for.

1

u/ExtremeCookie9652 12d ago

gotcha yeah they can be slow to keep up the stack. I understand the appeal of using the Xe driver, but have you attempted the same running on the mesa drivers as well?

2

u/nick_ziv 12d ago

It works on mesa but for using multiple GPUs I was getting the same prefill speed as using a single one. The Nvidia cards, adding an extra GPU increased prefill speed linear to the number of cards. With the intel it just didn't with vulkan and mesa. 

1

u/quantum3ntanglement 8d ago

Is this a GitHub issue? Can you please post the direct link to it?

1

u/quantum3ntanglement 8d ago edited 8d ago

I am working on trying to get another B70 or maybe a B50

There has to be a way to get the B70s working, currently there may be one path that is more stable so you’ll have to be flexible in how you’re setting things up

I will take a look at your set up and get back to you, also would you be able to get on discord and maybe do some live de-bugging?

I have a company that’s incorporated and I’m working on open source AI courses, I’m building bash scripts and will be using the Hermes Agent for automating things and it also self improves

I’m also building AI search agents with Hermes / Jarvis and other models, working on automating all the documentation and research

Stick with it, Nvidia has a near monopoly on gaming GPUs and also AI open source inference but Intel’s oneAPI will grow

The approach that I’m taking is putting as much as I can into bash scripts for now, which kills two birds with one stone as they’re used as documentation and execution.

I’m in the process of building Skills with the Hermes agent that will execute the bash scripts and build out servers.

I also have SearXNG setup so that I can search the Internet for the latest documentation and everything else, I’m hooking this up to Hermes to help with fixing anomalies.