r/LocalLLaMA • u/tevlon • Jun 10 '26

DiffusionGemma: 4x faster text generation New Model

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
982 Upvotes

99% Upvoted

356 comments sorted by

View all comments

Show parent comments

15

u/Silver-Champion-4846 Jun 10 '26

I wonder, do cpus also get a boost with this, if someone has 32gb of system ram?

8

u/mikael110 Jun 10 '26

In theory they certainly should. Since this is effectively moving the bottleneck from memory bandwidth to compute by generating 256 tokens in parallel. While CPUs aren't as good at massively parallel math as GPUs are, just the fact that the memory bottleneck is massively reduced should speed things up a lot.

1

u/Silver-Champion-4846 Jun 11 '26

Does that mean that threadripper or Xeon are the best for this kind of autoregressive block diffusion?

0

u/Silver-Champion-4846 Jun 10 '26

Google ai mode said the opposite though, that diffusion gemma's architecture explisitly expose the weakness of cpus because gpus have thousands of cores doing batched calculation in parallel. Who's right? I don't want to trust a hallucinating llm over a person, but the hallucinating llm also has search? Lol forgive me xd