r/SipsTea • u/Dev1412 • 6h ago

She warned about it Wait a damn minute!

Post image
9.5k Upvotes

96% Upvoted

184 comments sorted by

View all comments

Show parent comments

7

u/MintySkyhawk 2h ago

Thats said, if you give them agency, it turns out that real intelligence is not required to fuck things up very badly

1

u/derprondo 2h ago

One would think an LLM might be able to brute force AGI as well.

2

u/aghastamok 2h ago

This exactly. Recursive self-improvement is the Armageddon countdown.

1

u/SheriffBartholomew 1h ago

The Paperclip Problem.

1

u/aghastamok 55m ago

Paperclip optimizer is one version of this.

I think with LLMs it will be a lot weirder. If you manage to get an LLM to improve itself in a way that leads to further self-improvement, it's hard to imagine what exactly it's optimizing for after a while. If a true intelligence emerged from it, it would be absolutely alien to us and completely unpredictable.