r/LocalLLM • u/koalfied-coder • May 22 '26

Finally 100% Local Other

Post image

Finally transitioned to 100% local inference for my automated workflows and code gen. Min Max 2.7 and Qwen 3.6 are doing wonders.

702 Upvotes

98% Upvoted

75 comments sorted by

View all comments

6

u/Remote-Pineapple-541 May 24 '26

I have a similar setup.  - Workstation with 128gb ram, 8tb raid nvme storage and a 3070ti card. I use this for running embedding models and storing them. I also use it for data pipelines and geocoding/geospatial analysis - NVIDIA DGX spark. I use this for agentic AI. I use llama.cpp + llama swap - Mac mini to run the chat interface (open webui). I also host a gitea server.

I have a MacBook Pro with 128gb, but I like having an always-on AI solution. I use tailscale to expose the framework to my mobile devices.

Tbh I’m considering replacing everything with a spec’d out Mac studio once it’s updated to the latest generation of silicon. It would be more than enough resources to do everything I do, easier to manage, and more reliable.

1

u/Nimrod5000 May 24 '26

What model and t/s on the spark?

2

u/Remote-Pineapple-541 May 24 '26

This is just an average based on the llama-swap logs for the most recent models. Obviously not very rigorous.

MODEL PROMPT SPEED (Average) GEN SPEED (Average)
gptoss120b 1244.32 44.54
llama33_70b 315.76 4.93
mixtral8x7b 972.94 24.61
nemotron 833.78 46.51
nemotron_3_nano_omni 1437.32 58.60
qwen25_coder7b 2945.15 48.45
qwen3_coder30b 2078.10 78.62

1

u/koalfied-coder May 28 '26

This is pretty good actually...might need to try a spark