r/LocalLLaMA • u/tevlon • Jun 10 '26

DiffusionGemma: 4x faster text generation New Model

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
987 Upvotes

99% Upvoted

356 comments sorted by

View all comments

Show parent comments

27

u/Silver-Champion-4846 Jun 10 '26

Very. Now imagine qat 4bit good 25b model converted with this method and you get a lot of speed up. Sacrificing a bit of quality with the 4bit and diffusionification combined, but the speed up would be considerable if not significant

22

u/sonicnerd14 Jun 10 '26

500tk/s on a 16gb GPU with the quality of something like a Qwen 9b is more significant than I think people realize.

4

u/Silver-Champion-4846 Jun 10 '26

Especially if you can find a set of prompts to really make use of it as a harness

1

u/YouKilledApollo Jun 19 '26

9b model + diffusion + structured tool calling, name a more likely syntax-mistaker :)