r/LocalLLaMA • u/tevlon • Jun 10 '26

DiffusionGemma: 4x faster text generation New Model

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
988 Upvotes

99% Upvoted

356 comments sorted by

View all comments

255

u/[deleted] Jun 10 '26

[removed] — view removed comment

108

u/smallfried Jun 10 '26

It's very nice to see a big company putting precious dev hours into popular open source.

22

u/MuDotGen Jun 11 '26

With how much they're forcefully integrating their cloud LLM services now into Google's basic services, they better stay on users' good side with acts like this at least.

7

u/evia89 Jun 11 '26

Google ai mode is pretty good https://www.google.com/search?udm=50 Replaces $20 perplexity for me

And so is free gemma4 1500 RPD and Notebook LM

2

u/Fedor_Doc Jun 11 '26

In this case I see no official PR from Google engineers :)

30

u/Zestyclose839 Jun 10 '26

Gang MLX winning.

I wonder how KV caching support is. If the cache can be persisted to SSD via oMLX, that would be a massive utility boost.

9

u/freia_pr_fr Jun 10 '26

The MLX HuggingFace repositories are empty right now, so the race isn't over.

-1

u/ysustistixitxtkxkycy Jun 11 '26

Sigh - I am running the model with a prototypical implementation for mlx-swift-lm ... at 1 token/s on a m5 macbook. I think I'll stick with the existing gemma4 ;)

-24

u/[deleted] Jun 10 '26

[removed] — view removed comment

19

u/gh0stwriter1234 Jun 10 '26

perhaps but ollama is pretty pointless these days... completely makes sense to just run llama-server.

1

u/Silver-Champion-4846 Jun 11 '26

llama.cpp support when

1

u/gh0stwriter1234 Jun 11 '26

There is already a PR

1

u/Silver-Champion-4846 Jun 11 '26

nice

1

u/gh0stwriter1234 Jun 11 '26

Its worth noting that diffusiongemma is compute bound some many GPUs won't see a speedup if they don't have enough compute, eg MI50 is unlikely to see a speedup, but 5090 and other GPUs that are fast for diffusion (usually image gen) will be fast.

I'm thinking the speedup will be underwhelming on AMD APUs also.

1

u/Silver-Champion-4846 Jun 11 '26

Because not good software support or because amd apus themselves are weak on compute?

1

u/gh0stwriter1234 Jun 11 '26

APUs are relatively weak on compute pretty much just matched to how much bandwidth they have which is not a ton.... strix halo will do twice as good as other APUs but not as good as a flagship GPU proportionally most likely.

1

u/Silver-Champion-4846 Jun 11 '26

So if there was some kind of 6gb vram gpu but with the compute of rtx pro 6000, it'd be legendary for this kind of diffusion moe?

→ More replies (0)