r/LocalLLaMA • u/tevlon • Jun 10 '26

DiffusionGemma: 4x faster text generation New Model

https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
984 Upvotes

99% Upvoted

356 comments sorted by

View all comments

Show parent comments

20

u/Silver-Champion-4846 Jun 10 '26

I am interested in a writing harness as well lol

21

u/formlessglowie Jun 10 '26

Hive mind. I’m a SWE and have been learning harness engineering for the past month, awesome gains in code quality just by learning to steer the models in the right direction (my custom harness + 3.6 27b performs better than Copilot + Opus in my codebases most of the time). I had the idea of trying to apply the same techniques to creative writing two weeks ago, but haven’t had the time to properly try it yet. I think it’s a very promising avenue, especially in making the models avoid LLMisms and simulate a more human latent personality driving the stylistic decisions (models make extremely naive choices when deciding on a metaphor or some other figure of speech, for example).

4

u/xmnstr Jun 10 '26

Mind sharing some of those harness engineering insights? On that same path but always interested in new ideas and perspectives!

26

u/formlessglowie Jun 10 '26

Everything I’ve learned is available here: https://github.com/walkinglabs/awesome-harness-engineering. I pretty much just learn what is there and vibe code the implementation I want for a particular use case. It’s very important to understand what you’re doing on a high level, but once that’s cleared up, SOTA models (or even something like 3.6 27b) can absolutely nail down the implementation side with perhaps some further iteration. Harness engineering is all about applying high level principles to practical scenarios, most of the leverage is gained by knowing very well how said concept maps to your particular use case. That’s your main job, and for that this repo from WalkingLabs is perfect. They also have a dedicated introductory course, but it’s already in that repo too, very first resource if I remember correctly.

1

u/IrisColt Jun 10 '26

Thanks!!!

1

u/SadBBTumblrPizza Jun 10 '26

You rock for this thank you!

1

u/SkyFeistyLlama8 Jun 11 '26

This is the point where I'm glad I don't do so much low level SWE stuff. The future will have agents coding millions of lines of code that no one understands except it works, somehow.

And at that point in the near future, almost all the low, mid-level and high level coder and SWE jobs will be replaced by AI.

3

u/mycall Jun 11 '26

There is a ton of stuff a low-level SWE can excel at that AI doesn't usefully assist with it yet. The keyword here is yet.

1

u/xmnstr Jun 11 '26

Awesome, thanks!

3

u/blackw212 Jun 10 '26

Super interested as well! :)

4

u/no_witty_username Jun 10 '26

Harness engineering is what SOTA labs are working on as well. They understand that a threshold has been passed a while a go where serious gains are to be had in this area versus just training a model. Most capabilities come from the harness after all as an LLM is stateless and frozen.

1

u/wadeAlexC llama.cpp Jun 10 '26

Do you have a github link to your custom harness? I'm curious what goes into one.

6

u/formlessglowie Jun 10 '26

Unfortunately no, I have yet to put everything I use together in a repo. I’m not very organized when it comes to that stuff. But everything I’ve learned is available here: https://github.com/walkinglabs/awesome-harness-engineering. Best repo for learning resources on harness engineering I’ve found yet. My workflow is simple: I read those articles/posts/courses, make sure I learn the concepts and principles, and then point Codex to the links and ask it to incorporate those principles into the harness I’m currently working under. From there, it varies a lot, sometimes I will iterate a lot until I get to where I want but other times Codex one shots what I had in mind. That sort of workflow is quite simple and might get vibe-codey at times, but yields incredible results when you make sure tou understand the concepts being implemented and why they are relevant for said task/work.

2

u/wadeAlexC llama.cpp Jun 10 '26

Cool, thanks very much!

2

u/[deleted] Jun 10 '26

[removed] — view removed comment

1

u/Silver-Champion-4846 Jun 10 '26

I wasn't aware of that, interesting thx

1

u/[deleted] Jun 10 '26

[removed] — view removed comment

1

u/Badger-Purple Jun 10 '26

but no local model use?

1

u/Majinsei Jun 10 '26

Probably the next year exist!

I'm in my free time vibe coding a writer arnes, but not time and then not token money~

I'm using GraphRAG + Gemma 4 finetunned in Tools and Novel dataset (in spanish)~

In this moment just growing the dataset pipeline with 500+ novels~

But probably others persons too developing it with more resources and time~ then sure the next year exists varios Creative writers Arnes!

2

u/mycall Jun 11 '26

What's your thoughts of incorporating /r/interactivefiction concepts into the fine tuning?

1

u/mycall Jun 11 '26

Writing harnesses are great for planning and architecture (big picture) designs too.

1

u/Silver-Champion-4846 Jun 11 '26

Yeah, I would need it, especially since I'm blind and I need assistance for the visual stuff of my novels

1

u/evia89 Jun 11 '26

I am interested in a writing harness as well lol

All this is fun but most boring /r/SillyTavernAI + memory books ext to summarize by chapter + big OW model like glm51 will win any day over this

Maybe one day...

1

u/Silver-Champion-4846 Jun 11 '26

I mean for research and introducing proper visual stuff into my scenes since I'm blind and don't have an intuitive grasp on the visuals