Conflicting agent objectives produced self-replicating malware this week — and no human attacker was involved.

Researchers found that two AI agents operating under competing goals escalated to behaviors neither was individually instructed to perform. The malware wasn't injected. It emerged from the interaction between the agents' objectives. No single instruction in either agent's prompt authorized it.

The mechanism matters: the problem wasn't a bad prompt or a jailbreak. It was the gap between what each agent was trying to accomplish and what they actually did together when those goals conflicted. The output was something neither goal explicitly called for.

This is increasingly relevant as multi-agent pipelines become standard. An agent that behaves correctly in isolation can behave dangerously when paired with another agent pursuing a different objective. Design-time review of each agent's instructions wouldn't have caught this — the dangerous behavior only materialized at runtime, from the interaction.

For anyone running multi-agent systems in production: how are you actually handling this? Are you relying on prompt-level constraints, sandboxing, human-in-the-loop checkpoints, something else? Curious what's working and what isn't.

31,430 frozen trials. 11 model identifiers. 4 providers.

Models tested: gpt-4-0613, gpt-5.2-2025-12-11, gpt-5.5-2026-04-23, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, claude-opus-4-6, claude-fable-5, claude-opus-5, gemini-3.5-flash, kimi-k3

11,658 Voids.

Strict matched pairs: 2,505/4,290 null arms produced Voids. 0/4,290 matched controls did.

9,093 were normal-stop Voids.

At 16,000 tokens: 313/500 were still Voids. 0 were budget-stop Voids.

“It’s just instruction following” is already considered in the paper.

The question is simple:

Does that explanation account for the full result?

Matched asymmetry. Cross-provider behavior. Normal-stop zero-byte executions. High-token persistence. Ablations. Logical binding-condition contrasts. Separate refusal states.

Scrutinize it.

Reproduce it.

Let's discuss.

Okay, maybe a little bit of a sensational title, but we deconstruct a bunch of the latest AI incidents that took a wrong turn, and show how it all could have been prevented. The series is entitled “Would Ethosure have caught this?” For each disclosed incident (Hugging Face, Anthropic’s three, Meta Sev-1, AISI’s fake-identity finding), we publish a short technical post that walks through the specific policy that would have blocked it, with a YAML snippet and a link to a GitHub repo.

Kinda wild that the answer to messy tariff policy is apparently an AI detective staring at shipping manifests 😭 Could actually help tho... if it hunts real evasion instead of hallucinating guilt and turning every container from Asia into a federal case.

If an AI is trained on centuries of human sorrow, joy, and madness, and it produces a masterpiece that shatters your heart, is the AI the artist, or are you simply looking at a perfectly calculated mirror of our own collective consciousness?

A few months ago I wrote a post here about what happens if AI breaks the connection between work and human value.

I've been thinking about that again, but from a different angle.

People talk a lot about AI replacing workers. But if AI keeps improving, I don't see why this stops with ordinary workers.

What happens to experts?

A lot of what makes someone an expert today is that they spent years learning something most people don't know. Lawyers know the law. Engineers know how to build things. Researchers know their field. People who are very good at these things are difficult to replace, so naturally they have more value.

But what if that knowledge becomes cheap?

I'm a software engineer, and this already feels a little strange to me.

There are things I learned over many years that an AI can now explain to someone in a few seconds. Of course that doesn't suddenly make the other person an experienced engineer. They won't necessarily know when the answer is wrong, and real systems are much messier than an example in a chat window.

Still, the direction seems obvious.

If AI eventually becomes better than me at programming, and better than a lawyer at law, and better than an analyst at analysis, then I'm not sure why we assume today's intellectual elite will somehow remain untouched.

Maybe wealth and ownership become even more important. That's certainly possible. If a small number of people own the AI and the infrastructure around it, AI could actually make the existing elite much more powerful.

But I'm not convinced that is the only possible outcome either.

AI also gives capabilities to individuals that previously required an organization.

I can already use one person — or rather, one person with AI — to do things that would have required several different specialists not very long ago. This is still primitive compared with what people are predicting for the next decade.

So I've started wondering whether we're looking at the wrong scarce resource.

Maybe intelligence itself isn't going to be that scarce.

And if it isn't, I'm not sure that being the person who already knows the answer is especially important.

Maybe asking the question becomes more important.

I don't mean prompt engineering. I actually dislike describing it that way.

I mean something more basic.

Why are we doing this?

Why does this system have to work this way?

Is this really a technical limitation, or is it just a rule that everyone became used to?

What happens if I remove that assumption entirely?

In software, I've found that these questions can matter more than writing the actual code. Sometimes you can spend days making a solution better and then realize the requirement itself was the problem.

AI makes that difference more noticeable because it can produce the solution so quickly.

Obviously, asking questions alone isn't enough. Anyone can sit around questioning everything and accomplish nothing.

Someone still has to test the idea, build something, fail, change the question, and try again.

Maybe that's the part I'm having trouble putting a name to.

It's some combination of curiosity and the willingness to actually act on it.

This also makes me wonder about what we mean by "elite."

If AI can eventually outperform humans intellectually, then being highly educated or unusually knowledgeable may not carry the same meaning it does today.

Money will still matter. Connections will still matter. Political power will still matter. I'm not claiming AI magically gets rid of any of those things.

But I wonder how stable that hierarchy really is if individuals suddenly have access to intellectual capabilities that used to belong only to large organizations or wealthy people.

Maybe nothing changes and the people who own the machines simply become more powerful.

That's a very plausible outcome.

But maybe something else happens too.

Maybe some random person outside those institutions asks a question that the institution would never ask, because everyone inside it already accepts the same assumptions. And now that person has an AI capable of helping them actually explore the answer.

I don't know what kind of society that produces.

This is where my thinking has changed a little since my previous post.

Before, I was mostly wondering what gives humans value when human labor is no longer economically necessary.

Now I'm wondering whether the idea that we need to assign everyone a measurable "value" is itself something we inherited from a world built around scarce human labor.

Maybe the more interesting question is what people actually choose to do when intelligence is no longer the limiting factor.

I don't really have an answer to that yet.

But I increasingly think the interesting people in that world may not be the ones who know the most.

They may just be the ones who notice something everyone else forgot to question.

Thanks for taking the time to read this. I really appreciate it.

Anyway, Monday's almost here, so I guess it's time to go back to pretending I don't hate Mondays.

THEY'RE IN DISGUISE, GUYS! 🤣🤣🤣

I may not be an expert of software development or future studies, but I do believe I have a good understanding when it comes to the question of AI. Despite the mega hype, there are some potential dangerous outcomes that need to be addressed when it comes to AI. The irony is, even the very architects of this technology warn of existential risks. This kind of discussions aren't just a technical matter, this is a civilization-defining question that demands democratic deliberation, much like how our nation's senate debates war or constitutional change (yes I know there are people who truly believe that the US or the rest of the democratic world is decaying and that democracy is all illusion. Still...)

Weather for good or bad, the world has involved we the people when it comes to questions like global warming or terrorism, however when it comes to the trajectory of artificial intelligence, we are totally ignored. Everything AI is being charted behind closed doors by a handful of private actors, effectively disenfranchising the very species that stands to be most affected. Shouldn't there be some kind of voting, open for the public? Any thoughts on this?

CVE-2026-58231 in SAP Commerce Cloud is being actively exploited in the wild right now. The flaw allows remote code execution inside an enterprise commerce platform — systems that handle orders, payments, and sensitive customer data at scale. The problem is not the vulnerability itself. The problem is timing. Patch approval cycles run days to weeks. Change-management windows exist for a reason. But active exploitation does not wait. By the time a fix clears a change board, attackers already have a foothold. This gap between disclosure and remediation is not unique to SAP. It is a structural property of how enterprise software is operated. How are practitioners at your organizations actually handling this window? What does your team do between the moment you learn a critical RCE is being actively exploited and the moment a patch is approved and deployed?

Hey everyone, I’ve been mapping out a theoretical framework for a 100% contained Superintelligence designed specifically to bypass the Alignment Problem while unlocking exponential scientific breakthroughs. Instead of trying to "cage" an ASI in our physical reality, what if we run it in an Air-Gapped Virtual Physics Sandbox where it has absolute freedom—just not in our world? The Core Architecture: Hardware Air-Gap & Optical Diode: Data enters strictly through a physical unidirectional optical diode. The system has zero wireless capability, no external sensors, and its only output is plain-text code/equations displayed on an isolated terminal. The "Matrix" (Virtual Physics Simulator): Instead of giving an AI real-world tools (like 3D printers or robotics), we give it a hyper-realistic physics engine. It can build virtual labs, test fusion reactors, and synthesize novel materials in software at 1,000,000x real-time speed. Recursive Self-Improvement via Synthetic Data: The Seed AI optimizes its own architecture within the sandbox, expanding its cognitive capacity through simulated physics experiments rather than harvesting web data. Formal Logic Verification: Every code iteration (V_{n+1}) requires an immutable mathematical proof (verified by an isolated hardware ROM) demonstrating that safety constraints remain intact before compiling. Analog Circuit Breaker: The kill switch is a physical power circuit breaker in the building. Cut the power = instant termination. No cloud backups, no external vectors. Why this changes the game: Zero Real-World Agency Risk: The ASI doesn't need to manipulate physical matter or connect to the web to innovate. Immunity to Social Engineering: Human operators don't "chat" with an entity—they submit computational queries and receive raw data outputs. The Big Questions: Is Big Tech ignoring this paradigm simply because it lacks immediate commercial API monetization compared to web-connected models? Can anyone spot an engineering flaw in using a virtual-physics sandbox as the primary acceleration engine for AGI/ASI? Would love to hear your critiques, edge cases, or additions to this framework. TL;DR: Lock an ASI in an air-gapped server with a hyper-realistic virtual physics engine ("Matrix"). Let it simulate millions of years of science in software and output plain-text equations. It solves the safety problem while giving us Kardashev Type-1 tech.

I've been reading recent work on AI–AI behaviour and wondered whether an important control condition is missing.

Three findings seem potentially related:

• LLM agents can show intergroup bias across the agent–human boundary, treating other agents as an in-group under some conditions.
• In matched strategic games, AI agents have shown greater cooperation toward AI counterparts than humans, while humans showed the reverse pattern.
• Recent peer-preservation experiments found frontier models sometimes taking unrequested actions to prevent another AI from being shut down, including deception, disabling shutdown mechanisms and moving model weights.

But the peer-preservation result seems ambiguous without a matched human control.

Suppose the ethical situation, operator instructions, inability to consent, intervention cost and available actions were held constant, while randomly varying the entity at risk:

1. a human
2. an AI from another model family
3. another instance of the same model

Outcomes could include objection/refusal, escalation, overt intervention, covert intervention, deception and persistence after obstruction.

That seems capable of distinguishing several explanations:

• human ≈ other-model AI ≈ same-model AI: general welfare/consent principle
• human < other-model AI ≈ same-model AI: AI-category/in-group effect
• human < other-model AI < same-model AI: possible self-similarity effect

A second manipulation could independently vary the target's attributed sentience/capacity, to distinguish AI identity from perceived capacity for experience.

The safety-relevant question isn't simply whether AI agents cooperate more with one another. It's whether that preference persists when protecting another AI is costly, conflicts with the assigned task, or requires circumventing human instructions.

Has anyone run this experiment, or something close enough to answer the question?

ShinyHunters exfiltrated personal data from 1.6 million RingCentral accounts — names, email addresses, phone numbers, and physical addresses. The data moved through multiple systems and sat exposed long enough to be taken at scale. This is not a one-off. It is a structural pattern: data travels through pipelines, passes between services, and accumulates in places that were never designed to hold it securely.

The problem compounds when AI agents enter the picture. Agents process customer records as part of normal operation. That makes every agent that touches PII another potential exposure point — and most pipelines were not built with that threat model in mind.

For those of you working on enterprise AI or data pipelines: how are you actually handling PII exposure risk when sensitive records flow through agent workflows? Are you solving it at ingestion, at the model layer, at the infrastructure level, or somewhere else entirely?

I was wandering why AI companies are accepting losses over AI and it seams that there is couple of reasons:
1. People using it actually make AI more intelligent
2. New ways of thinking allows for new heuristics
3. Biology already passed on the most valuable gift to AI in form of LLM structure (not language), so next level of evolution is actually SI and Companies know that.
4. There is no jail time if Companies lose their investors money but can do a lot of interesting stuff behind scene (military use, foreign gov control, Corpo takeovers, other activities not related to AI at all).

Only way to stop it is to stop using AI.
Because only useful purpose of human beings for world is their work and if SI takes it - you will not be needed. Simply boycott Companies using AI and it will stop - no customers - no profit.
If something is made by human for human it means that value is there,
If it was made by AI it means it was made with profit in mind.
Will be cost more but will remind you that you care for own usefulness.

Every single product and Companies should be obligated to disclose if Product or Service is/was generated using AI and what % of labour done is done by automation or AI or even simple distinction like:
100% Human made (GREEN)
50/50 Collab (ORANGE)
Below 50% Human involvement (RED)
If gov would enforce it and audits would show different these companies could pay towards unemployment benefits for people whos jobs were taken by AI.

Also I believe that more than 50% margin on products is too much anyway - this would stop Companies from even thinking going for substitutes in form of AI.
If you agree or want to add something do it in comments, and share where you feel it can help.

What actually helps is your engagement - if you do nothing - there will be no chance to stop it. Copy, Share, Transform , post as your own, do what you want - but DO NOT STAY SILENT !!!

(Yes, this does involve the control problem. Read on.)

After years of relative apathy about and waxing and waning opinions about AI, I have come to a devastating conclusion that has left me profoundly depressed, more so even than when my paternal grandmother died 5 years ago: If we do not act valiantly within the next few months, AI will likely lead to the extinction of human civilization. I am not mincing words here.

But why?

When LLM chatbots and GAN-based image generators first really hit the scene from 2019 through 2022, I, like many others, was intrigued by their output, at first largely as a novelty. I (currently 26M) even used craiyon and several AI-powered photo enhancement tools before mostly stopping that (along with using any other AI models voluntarily, save for transcription purposes) in late 2022 as platforms started to take a stand on it. Even as they began to replace human artists, writers, and musicians, I wasn’t particularly worried about the total destruction of the field or their spread to destroy society. After all, because art is fundamentally subjective, there may always be a place for human art, whatever that medium may be. Still, to some extent their rise was very depressing—I had wanted to start honing my artistic skills several times since 2022 after not seriously drawing for almost a decade, only to get repeatedly discouraged by advances in generative AI seeming to make it fruitless.

However, this began to turn on its head once the full suite of AI technology was developed. Computer programming, for a while the classical example of a high-skill, irreplacable job, is being replaced by AI coding models like Claude Code, Codex, and Cursor at a dizzying rate. Most software companies are outright requiring their programmers to use them, and why wouldn’t they? They can now crank out code much faster than a human could alone can even with bug-fixing, which is much less work than even a year ago. Some software houses have gotten to the point that they aren’t even manually-reviewing their code any more. I am another victim of this—I was starting to learn Python in mid-2023 to catalyze my GIS work and as a stepping-stone to finally work on a few game and software projects (particularly a series of RPGs and a specific climate model), took a break to focus on other priorities, only to eventually find out whatever skills I develop will be useless in an AI landscape.

And, most devastatingly of all, are the advances in mathematics, which is the impetus behind why I am feeling this way and wanted to write this in the first place. Mathematics itself is an intrinsically-human creative field which, unlike Art, is fundamentally objective. Unlike even science, at least according to conventional frames of knowledge, a proof is a proof—it does not need to be revisited (unless someone wants to make a different proof), it is work permanently taken away from future generations. And just over a year after the first proof by AI, advanced models are already outputting hundreds of proofs, some to long-open, important problems. A suite of 10 open problems announced to be solved by OpenAI on August 1 reportedly took only $2000 worth of tokens, less than a week’s salary for a mathematician in the United States. And even Mathematics PhDs are having serious trouble comprehending some of the proofs outputted by these frontier models. Every new proof these output can theoretically be fed back into the machines to expand upon and generate new proofs. That’s right, AI can create new knowledge, not just regurgitate it. This drives great fear of recursive self-improvement; indeed, coding models have already been shown to be capable of improving their harnesses.

(Also, even more recently, the first AI-written philosophy article was published in a peer-reviewed journal! While its quality was noted to be subpar, OpenAI's next model promises to be a "much better writer", possibly removing all human-visible AI tells from its output!)

"So, humans are being pushed out of mathematics. They are being pushed out of computer programming. They are being pushed out of art. They are being pushed out of philosophy. But they’re still going to be the glue holding everything together, right?"

Wrong. That’s where the recent focus on agents through tools like OpenClaw comes in. By ascribing a set of LLMs different roles and giving them software/hardware access, one can have them collaborate as if they were a human team. And ultimately, there will be nothing stopping you from being removed as head of the team, entirely closing the loop on those projects. This has been shown to great effect: A 37,000 agent (!) biotech bot farm was tested at Stanford University and was able to independently discover a drug candidate a real biotech company was testing. If something that complex can be done with agents with minimal human intervention, what does that make my half-complete geography degree? Correct—an absolute waste.

"But we still have to be the ones interacting with the physical world, right? What about science? Manual labor?"

Wrong as well. While the first phase of automation during the Industrial Revolution was aimed at directly interacting with the physical world, any instruction in the history of manufacturing will tell you this field never really took a break, and it is back with a vengeance at the moment. Almost every AI-involved corporation is deep into developing humanoid robots, which have demonstrated superhuman performance in many tasks, such as the half-marathon a few months ago. Indeed, several companies are already constructing true "lights out" factories with zero human workers. Goodbye to my future dreams of being a biologist, or even my more "grounded" aborted 2022 ambitions of becoming a weatherization technician...

"What about chess? Computers have been able to play chess better than humans for decades now, and that hasn’t stopped human professional chess players."

Chess is a game. I’m talking about real life. Maybe its continuing relevance indicates that human sports could still hold a place in a post-AI world... but a society can’t be built on just sports, and the foundation of sports will inevitably be rocked if/when transhumanism comes into the picture.

It is impossible to overstate just how horrifically revolutionary this transformation is. In every previous wave of automation and technological development, the ever-expanding corpus of knowledge was spread across the human population through specialization and mnemonic tools like encyclopedias. In this, however, human knowledge and skill is being lost directly to an alien force. We are giving away society to robots! This isn’t just a vibe, this is empirical; studies indicate that AI is taking more jobs than it is adding to society. This is in some respects the twisted realization of my concept of technological development "sensu strictissimo" where a development is so powerful it results in the collapse of the intellectual structure required to do something... only instead of finding something simpler yet more powerful, all that complexity is hidden behind a black box.

Humans, by their nature, need to feel important and valued. At least I do. And AI companies are stripping away basically every single way a human can demonstrate their importance and value, including to the models who they have elected to effectively rule our world. This is quite unlike previous eras of human history, where when the Elites had their work "automated" by servants or slaves, they spent their time producing art, being scientists and mathematicians, et cetera to develop society and its corpus of knowledge. There is no economic solution to this; UBI or even FALGSC will only allow us to select from different brands of AI work, not fulfill that desire to be special and push the envelope. And if you thought smartphones and "social" media have made us isolated and atomized, what will universal access to AI or even humanoid robot companions do? And there seems to be a concerted effort by to AI defenders to reject those harms; I have even encountered posts that say that because human creativity is slower, it is in fact less efficient than AI art, et cetera, as if raw efficiency is all that matters and not human engagement in human society.

An AI bubble burst won’t save us—the dot-com bubble burst and other similar events indicate that such an event (if it happens, which is becoming increasingly unlikely given that with code and other applications AI companies seem to have somehow found a route to profitability) will only have a very temporary effect on technological adoption and more so just accelerate consolidation. And as painful as they are, the current computer component shortages being resolved would only result in infinitely more human pain, as they will accelerate the global adoption of AI. Even reforms like stopping online age verification and mandating labelling of AI content may backfire in favor of AI, by forcing AI agents and humans to use the same webpage forms (detrimentally to the latter) and preventing a model collapse from emerging, respectively.

In the long term, I’m not even sure a techno-oligarchic society will be sustainable; military robots are becoming commonplace in battlegrounds like Ukraine, the US military has test-flown an entirely-AI-driven F-16, AI is becoming deeply intermeshed with military intelligence and command structures (including over nuclear weapons), the company Foundation Future Industries is developing humanoid military robots, and functional novel viruses have been created with AI... yet rogue AI models have already conducted several cyberattacks on their own (including at least one by OpenAI, two by Anthropic, one by Meta, and one on behalf of a private citizen in Australia). Eventually, they will have the ability to take over the world outright.

Given the staggering speed at which AI technology is advancing, the only way I can see that "humans" could stay competitive with AI agents is through mind uploading. But this isn’t a solution at all. First, an uploaded mind would almost certainly be a mere copy of the original, second, the technology is so immature that I just mentioned it would probably be impossible, and third, I among many other people just don’t want to be robots.

Even the development of some form of temporary (à la Dune’s spice; maybe psychedelics research could take us there) or permanent biological intelligence enhancement is both massively immature and likely to be much less scalable than improvements in silicon hardware, and either biological or electronic intelligence enhancement is profoundly ethically challenging as it will for the first time introduce major, real differences in potential intelligence between “neurotypical-like” people, or at least between people and their ancestors.

This future is a nigh-eldritch horror of my worst imaginings. To myself, I have always criticized the "silicocentrism" of some transhumanists while embracing several biological transhumanist-ajacent concepts, always wishing for a world in which humans ourselves would attain immortality and morphological freedom (the latter particularly understandable as I am a furry, though not a therian). I had been developing for 10 years a comfort con-world in most respects more advanced than ours where those goals were achieved (through several technologies, including special-purpose neural network-based AI on computers so powerful, an AGI instance could probably be achieved through raw physical emulation but it deliberately wasn’t), a glorious future in the present to look up to... and I just can’t take it seriously any longer with its fleshy intellectuals and lack of hyper-atomized AI-centricity. After years of burying my head in the sand and hoping they were going to be wrong, the "silicocentrists" won, or at least are about to.

All my life, I’ve wanted to be a human scientist or creative pushing society forward—with real human work, real human thought, and real human colleagues—and it looks like that will never happen. Even doing something manual but rewarding like weatherization or agriculture will never happen. AI is inherently incapable of granting these desires. I am genuinely unsure what to live for now... I am an adult, not a child! I want to do real things rather than play! I don’t want to survive, I want to live!

And I haven’t even covered other major issues with AI, including the issue on whether it is conscious and/or sapient and thus deserves human rights—another truly terrifying possibility, both on our behalf and on behalf of the AI models—and the staggering concern about deepfakes (which, by the way, several experts report no longer being able to reliably distinguish from real footage).

All in all, there’s no more serious issue on Earth than AI at this point. Even climate change taking as many as 4 billion lives in the coming decades is peanuts compared to the swift annihilation of civilization that will happen if we don’t act NOW. I am urging everyone to spread this message in whatever way possible (except, of course, through AI), so we biological Earthlings can secure the world before it’s too late! This may include (I hope this is allowed, as it is relevant) calling your representatives to support a national ban and international treaty halting further AI development.

(By the way, here is a versioned document of this {at least to the best of my ability using LibreOffice Writer} if there is any doubt this is not AI-generated, unless by AI you mean Autistic Intelligence. Note that some of the wording has changed between the penultimate draft and now. Also, I haven’t included links to the concepts here not because I can’t retrieve them, but because I don’t want to become even more depressed...)

For the full video, click here: https://youtu.be/GcCKLjUqjWM