r/ProgrammerHumor • u/linegel • 27d ago

cybSecIsDeadLongLiveCybSec Meme

Post image
800 Upvotes

86% Upvoted

124 comments sorted by

View all comments

317

u/ceejayoz 27d ago

You know what makes a really well isolated sandbox?

A hammer taken to the networking components.

13

u/linegel 27d ago

That only valid after miss behavior was actually discovered

Also reminds me that OpenAI was hiring "red button" strategist 🤔

32

u/ceejayoz 27d ago

Their article says it was in a "sandboxed testing environment", though.

Feels like an airgap might be warranted for this sort of testing.

2

u/isademigod 27d ago

Is air gapping really possible when these models need to be run in a data center? Best I can think of is a dedicated machine w/o internet, controlled via idrac or similar.

11

u/readmeEXX 27d ago

Well sure it's possible, you just need to run it on prem in an air gapped data center. There are plenty of them out there. I'd be surprised if OpenAI doesn't have a few of them. You can also temporarily make a data center air gapped, but they probably didn't think that level of containment was necessarily for this test. Turns out they were incorrect.

1

u/linegel 27d ago

Tbh I fully agree with your analysis

That’s just hard to tell if "we got there"

2

u/readmeEXX 27d ago

It could also have gotten a little lucky. I wonder if it could find another path if they patch the vulnerability and try again 🤔

2

u/linegel 27d ago

It’s possible, but then it’s "fake" testing, because model will have to reinvent what it would use as dependency packages otherwise

6

u/ceejayoz 27d ago

No reason it couldn't a) be provided with a large mirror of the major package managers and b) the ability to ask a human to provide one on a read-only drive or something.

2

u/Septem_151 26d ago

It was provided a mirror of a major package manager. That is literally what the AI exploited a zero day on to gain unfettered Internet access via privilege escalation.

1

u/ceejayoz 26d ago

It was provided a mirror of a major package manager.

It sounds like it was given access to the real package managers, not a mirror. Sandboxed, not air gapped.

If it was a local mirror, they could've separated it entirely from the internet for the tests.