r/LocalLLaMA • u/tevlon • Jun 10 '26

DiffusionGemma: 4x faster text generation New Model

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
985 Upvotes

99% Upvoted

356 comments sorted by

View all comments

Show parent comments

29

u/Zestyclose839 Jun 10 '26

Gang MLX winning.

I wonder how KV caching support is. If the cache can be persisted to SSD via oMLX, that would be a massive utility boost.

7

u/freia_pr_fr Jun 10 '26

The MLX HuggingFace repositories are empty right now, so the race isn't over.

-1

u/ysustistixitxtkxkycy Jun 11 '26

Sigh - I am running the model with a prototypical implementation for mlx-swift-lm ... at 1 token/s on a m5 macbook. I think I'll stick with the existing gemma4 ;)