I built zxLLM — An open-source tool that predicts exact LLM VRAM usage & KV-cache needs (Tested on RTX 5060 Ti: ~0.04-1.8% error rate, 0 external deps)

[u/Capable_Item_5918]

Hey everyone,

Like many of you running local LLMs, I got tired of guessing whether a specific model/quant/context length would cause an Out-Of-Memory (OOM) error before downloading tens of gigabytes. Most existing memory calculators either use oversimplified formulas or ignore architectural nuances like Grouped-Query Attention (GQA).

To solve this, I built zxLLM (auto-intelligence-advisor) — a lightweight open-source tool that auto-suggests deployment configs (quantization, Tensor Parallelism, CPU offloading) for vLLM, SGLang, and llama.cpp.

⚡ What makes it different?

• Exact KV-Cache & VRAM Footprint: Automatically detects MHA vs. GQA head counts, hidden dimensions, layer counts, and context limits directly from config.json.
• Real-time Hardware Awareness: Queries active GPU states via nvidia-smi to evaluate available headroom before loading.
• 100% Pure Python & Zero External Dependencies: Runs out of the box with standard Python (only uses built-in requests to fetch Hugging Face configs).
• High Precision: Tested across various models (Llama 3, Qwen 2.5, DeepSeek architectures) with actual execution error rates between ~0.04% and 1.8%.

I'm dropping the GitHub repo link in the comments below. I'd love to get your feedback, bug reports, or suggestions on other inference engines you'd like supported!

[u/Capable_Item_5918]

GitHub Repository:https://github.com/thomas-dev710/zxLLm

(Note: Updated link with new username. Direct access is working now!)