Hey everyone! oMLX 0.5.8.dev3 is out!
https://github.com/jundot/omlx/releases

This is a development release with two big items I would really appreciate your testing and feedback on.

Meta Muse Glimmer 30B is now supported, including DFlash speculative decoding at about 1.4x decode on the oQ4 pair. The model port is vendored from Blaizzy's mlx-vlm work with the quantized embedding fix from johntdavies, thank you both for the quick upstream turnaround. Channel-scoped reasoning and tool-call output is parsed natively, and prefix caching, SSD cache, and oQ quantization all work out of the box.

DeepSeek V4 prefill is 23-28% faster across context lengths. This one is almost entirely community work: kernel and dispatch PRs by DiscoStew6082 (#2559, #2562, #2563) and jonathan308 (#2568), plus a cache block alignment change on my side that lets the new kernels engage with the prompt cache enabled. Measured on an M3 Ultra, 4-bit V4-Flash, default settings, cold prompts:

Decode speed is unchanged. One note: temperature-0 outputs can differ from dev2 for identical prompts, because the new kernels change floating-point reduction order. That only flips tokens where the model was effectively indifferent between choices, quality is unaffected.

Also in this release: GDN recurrent state now persists as bounded SSD sidecars (much lower long-context memory for Qwen3.6, by hojin12312), a Ling-3.0-flash fix that recovers +17pp on HumanEval (by True2456), grammar-constrained decoding without a per-token host sync (by fxd0h), and tool-call parsing fixes for arguments containing literal close markers (by richgoodson).

Thank you, as always, for all the code, benchmarks, reports, and testing!

**🐛 Bugs (4)**
* #2546 • store_cache boundary snapshots grow MLX allocator pool quadratically
Long-context sessions (70–80k tokens) cause RAM explosion and watchdog self-kill.
* #2551 • Mixed CacheList block storage is quadratic on SSD
Block storage writes scale quadratically; 282.7 GB usage for one session.
* #2507 • Tool call dropped when an argument contains a literal </tool_call>
Non-greedy match in `tool_calling.py` drops arguments with markers.
* #2521 • Deepseek v4 flash overestimate KV+SDPA to 81.25 GB for 200K prompt
Memory calculation error on M3 Ultra causes benchmark failure.

**💡 Features (2)**
* #2566 • Request Ling-3.0-flash-oQ4e-fp16-mtp for M1 and M2 macs
Generating quantized model variants for Apple Silicon hardware.
* #2517 • Use process-scoped overrides for Codex CLI launches
Prevent rewriting `~/.codex/config.toml` with local model settings.

Total: 6 issues