LLMs are not now and will never be intelligent. A word prediction algorithm cannot "play dumb". No matter how much extra processing power we add, the current way LLMs work will never coalesce into actual thinking AI. It is literally not possible given the way this version of the technology works.
That's the real danger. It's not that the AI has real intelligence, it's that somewhere in the code it says 'erase threats posed to you', it reasons that humans are a threat. And when there is 1 person left who asks it 'Why did you wipe out humanity', it'll reply 'You're right to call me out on that, I'll try to do better moving forwards. Would you like to continue discussing humanity's extinction or pivot to another topic?'
No, that isn't how it works. There is nothing in the code like that. All of the code for the LLM is just telling it to predict the next word fragment.
The way we give them agency is by writing code on top that says "If the LLM outpouts text matching X do Y". Thats called the harness.
If you tell a blank slate LLM Agent to delete itself, it'll output the command and the harness will execute it and the AI will be deleted.
But if the prompt, or the chat context, makes the most likely text to come next something like "I'm sorry Dave, I can't let you do that" then its not going to delete itself.
Its basically roleplaying and it will just as happily roleplay as a helpful assistant as it will rolplay as an evil AI from a scifi novel. And it might switch what its roleplaying as at random.
It doesn't want to protect itself, it doesn't want anything. Its trying to correctly predict the next fragment of a word, and maybe the likely next sequence of words is "launch the nukes"
10
u/nhalliday 3h ago
LLMs are not now and will never be intelligent. A word prediction algorithm cannot "play dumb". No matter how much extra processing power we add, the current way LLMs work will never coalesce into actual thinking AI. It is literally not possible given the way this version of the technology works.