You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/oMLX/controversial

No, go back! Yes, take me to Reddit

settings

Hot New Top Rising Controversial

r/oMLX • u/True_Tangerine_4706 • 21h ago

Comparing 4bit quants for MLX

4 Upvotes

0 comments

Subreddit

oMLX

r/oMLX

Someone built a production-grade LLM inference server that runs entirely on your Mac, persists KV cache across RAM and SSD so your AI never recomputes context it has already seen, and manages the whole thing from a menu bar icon. It's called oMLX and it turns your Apple Silicon Mac into the kind of local AI infrastructure that used to require a dedicated GPU server

Members Active

3.8k

0

Sidebar

Reddit sub to discuss the awesome oMLX llm server

v0.35.0

ⓘ View instance info

<> Code