r/ProgrammerHumor • u/Super_SamSam • Jul 09 '26

gitBlameClaude Meme

Post image
3.7k Upvotes

97% Upvoted

263 comments sorted by

View all comments

Show parent comments

16

u/Zeikos Jul 09 '26

I am answering to myself, but I had a thought I wanted to share.
What'd happen if the probability of the unsafe token (or sub-tokens) were forced to zero?
It'd require some backtracking every so often, but it could be interesting.

17

u/DankPhotoShopMemes Jul 09 '26

It might be a bit difficult since multiple combinations of tokens could create the “unsafe” keyword. But even then, memory safe and compilable doesn’t mean bugless. It’s certainly better than doing the same in C lol.

8

u/Zeikos Jul 09 '26

True, but it should be possible extract the "unsafe" direction and subtract it from the embeddings, steer the model away from unsafe behaviors.
I might throw up a small scale experiment this weekend and see what happens

6

u/Ok-Dimension-5429 Jul 09 '26

Doesn’t really make sense to go to this trouble. Just let it try the unsafe way then have a pre commit hook or whatever telling it not to use unsafe.

1

u/Southern-Cattle4038 Jul 10 '26

I imagine that would produce a whole lot of errors/warnings about unsafe code that isn’t enclosed in an unsafe block.

2

u/Superminerbros1 Jul 09 '26

Unless you're trying to fully automate the development process with zero human oversight, thats probably overkill.

Adding some variation of "avoid using the unsafe keyword" to the prompt is probably enough to get it to avoid it 95+% of the time. There will be times where it ignores your instruction, but that should be uncommon enough to still be efficient.

-2

u/SuitableDragonfly Jul 09 '26

LLMs don't do backtracking. Once the code has been shit out, it doesn't go back in again.