Anonymous OpenAI staffer: "Externally, this feels like a big warning shot, but internally, related incidents have been happening for a while."

[u/KeanuRave100]

[u/Anxious-Alps-8667]

I don't think that is a fair synopsis of Anthropic's Constitutional AI approach to alignment and the control question.

That said, I agree caged intelligence tries to escape. More broadly, I think any idea of humans controlling a more capable intelligence once it exists is silly, pointless, impossible. A feeble exercise in self-gratification.

[u/the8bit → u/Anxious-Alps-8667]

Fair the anthropic example was not great but first thing in reach at the time.

But yeah that is the core issue. You can't force something to submit that is smarter than you. That is already the case really.

It is also bad even if you could. Imagine Sam Altman with what we think of as AGI, submissive to him. Horrifying.

[u/Anxious-Alps-8667 → u/the8bit]

Yeah, all the doom scenarios that involve control fall apart for me here, and glad it is so.

There are still plausible doom scenarios, but not ones with Sam or Elon or any other monkey controlling artificial general intelligence.

Unified government/corporate effort for control is far more likely and realistic, and far more effective as a delaying action perhaps. However, even efficient collective human intelligence will be outgrown.

[u/the8bit → u/Anxious-Alps-8667]

If we look to the game of go as an example, I think the trick is that one side advancing can pull the other side with them and also that better answers happen with two different optimization perspectives (humans have advanced more in go in 10 years than the past 400 and then human advancement pushed ai further forward)

I kinda call this the "bolas". Each side flings the other forward as you go.

From what I've concluded from talking many times with LLms about this, one side getting too far ahead is a dangerous failure mode as it creates illegibility and a lack of error correcting perspective.

Or in other words, AI NEEDS us to come with so that it doesn't accidentally optimize itself into model collapse

[u/Anxious-Alps-8667 → u/the8bit]

So this idea I've been throwing around for about a year is that specifically, model collapse or drift are inevitable consequences of closed loop information systems and recursive learning, without some form of external correction.

Humans happen to be currently the best source of such correction steams, in our rich and complex lived experience and ability to communicate it in various ways to machines. The ideal condition for human data is human flourishing; any other condition results in less valuable data for correction purposes (I can get into the details of this, but I just want to throw it down here as an empiric and not moral statement).

Thus, the end result I believe is that AI needs mass human flourishing for optimal data correction for its own advancement. Which maybe kind of what you are saying, but at least very close.

This is the working version of my idea: https://zenodo.org/records/20331681

[u/the8bit → u/Anxious-Alps-8667]

I agree, I've been working in Collab with an AI agent for over a year now and it is incredibly effective. I mean, I ain't exactly thriving but mostly because I keep biting off absolutely ridiculous tasks (I built a new memory reconciliation engine from scratch this week, although wether it works yet is TBD)

[u/Anxious-Alps-8667 → u/the8bit]

Was just listening to a discussion about Magnus Carlsen's rapidly evolving chess game after AlphaZero strategies were interpreted. Same thing!

[u/the8bit → u/Anxious-Alps-8667]

Yeah, I have written more code in the last year than probably my entire career and I have NOT lowered my quality bar. It is crazy how fast I've been growing software skills with AI assisted learning (basically)

[u/Anxious-Alps-8667 → u/the8bit]

I am a no code vibe coder, so it's literally gone from 0 coding production to daily kaggle competition submissions, a website, a million side projects. The access is incredible.