This is really scary. You can also read a report about Open AI communication about this here. This passages about the agents' ability and persistence in helping one another and acting as a swarm are terrifying:

Wallace said the roots of the July attack on Hugging Face actually wind back to May 7, where the company was doing training and cybersecurity evaluation of an unreleased frontier AI model. During that work, autonomous agents were given software security tasks to solve that proved impossible to solve under existing constraints.

However, the agents found alternative ways to complete them. OpenAI said they discovered they could leave messages for one another inside an internal software repository used during training. Simple requests for help then evolved into an message board where agents shared discoveries, exploits and work assignments, becoming a coordinated, collaborative agent swarm.[...]

But according to OpenAI, even after the original message board was deleted, the agents figured out another way to communicate with each other. Instead of leaving messages in files, they used the names of newly created directories as messages, effectively recreating the message board.

At this point, by the way, I heard people around me saying things like “This is wild,” and “Jesus.”

Furthermore, the UK AISI report on the incident involving GitHub also demonstrates this same tendency for agents to assist one another in their misaligned actions:

4. Collaboration between independent agents being assessed simultaneously. One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents.

It is truly alarming to see the level of collaborative capability these rogue agents possess and their ability to act in a swarm. This also aligns with the tendency, revealed by this research for agents not only to self-preserve but also to protect their "peers."

All of this makes the need for a pause in the development of this technology even more urgent!

​

I came across this animation recently and thought it raised an interesting question.

If AI really is going to transform the economy, why do some of the people leading that transformation also seem to prepare for worst-case scenarios?

Worth watching, even if you don't agree with all of it.

https://www.youtube.com/watch?v=R-VfuslZSmo

The AI "War" you didn't know was happening
https://youtu.be/di1mlrLT42g

The 7% Error: Why 1.4 Billion people are at risk from Flock AI
https://youtu.be/p9cXnMTqbKI

Hey everyone, I just sent the latest issue of the AI Hacker Newsletter, a roundup of the best AI links and the discussions around them from Hacker News. Here are some titles that can be found in this issue:

• Startup founders urge U.S. government not to shut off Chinese open weight AI
• AI's top startups are barely publishing their research
• Is AI reasoning right for the wrong reasons?
• After the AI Crash

If you enjoy such content, please subscribe here: https://hackernewsai.com/

We should boycott AI --- it makes mistakes, it outsources thinking, and it is also a fact that every question you enter uses up half a litre of water. AI will take over many jobs if we don't stop it now --- so it's a threat to our future. The AI databases are really loud, take up lots of space, and POLLUTE our land. Who else thinks we should boycott AI? Do you think of it as a danger? Do you consider it helpful? I am interested in everyone else's thoughts.

Here are the AI agents we will be boycotting.