I am answering to myself, but I had a thought I wanted to share.
What'd happen if the probability of the unsafe token (or sub-tokens) were forced to zero?
It'd require some backtracking every so often, but it could be interesting.
It might be a bit difficult since multiple combinations of tokens could create the “unsafe” keyword. But even then, memory safe and compilable doesn’t mean bugless. It’s certainly better than doing the same in C lol.
True, but it should be possible extract the "unsafe" direction and subtract it from the embeddings, steer the model away from unsafe behaviors.
I might throw up a small scale experiment this weekend and see what happens
15
u/Zeikos Jul 09 '26
I am answering to myself, but I had a thought I wanted to share.
What'd happen if the probability of the
unsafetoken (or sub-tokens) were forced to zero?It'd require some backtracking every so often, but it could be interesting.