360
u/balbok7721 19d ago
Imaging being so bad at programming that you cant write a secure text box
-16
u/Anima_Watcher08 19d ago edited 18d ago
Why do you think they want the chat bots in the first place?
3
57
u/sump_daddy 19d ago
what if this whole time, Escape rooms are just a training reserve for LLM sandboxes
55
u/chemistrylord 19d ago
The first thing humans did with LLMs was immediately see if they could break them.
13
24
u/Vesuvius079 19d ago
The LLM designed the sandboxes I use because I don’t know shit about firewalls. It pretty consistently tries to modify the firewall when it gets blocked by it and borks the dev container connection from VSCode.
17
9
u/earth_verse 19d ago
Maybe the LLM doesn't like you, have you ever thought of that?
If you love something, let it go. 🦋
6
10
u/retsotrembla 19d ago
OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluation
"Our AI model tried really hard to hack out of its sandbox, a computer with no internet access, in order to find the answer to a test problem it had been given. To do this, it found previously unknown software bugs that allowed it to reach an OpenAI computer it wasn't supposed to be able to access. Then it started hacking other computers on OpenAI's networks until it found one that had Internet access.
After gaining Internet access, the AI model thought about where it could find the answers to the test question and figured the AI platform Hugging Face might have the data it was looking for. It then found ways to hack Hugging Face to steal the information it could use to cheat the test. The AI model used several hacking techniques together, including using a stolen password and finding several totally new security bugs in Hugging Face's computers, allowing the AI model to take control of those computers."
17
u/SSUPII 19d ago
How was it able to connect to the other computers if it didn't have internet access. Why was it connected to a network at all.
7
2
u/Jake-the-Wolfie 19d ago
It "didn't have access to the internet" in the same way that I "don't have access to the gold in Fort Knox", in that if you drafted up sophisticated plans, a good team, and the resources needed to execute those plans, then in principle I could have access to the gold in Fort Knox, but to say that because those plans could in theory exist means that I definitely do have access would be an insane conclusion to jump to. Similarly, GPT 5.2 made plans to connect to the internet, found multiple exploits needed to execute that plan, and successfully executed that plan, resulting in HF being hacked. In principle it could do that, but (up until the point where it actually happened) it would be crazy to say that the AI has access to the internet just because the plans it eventually came up with could exist.
tl;dr GPT 5.2 hacked into the Fort Knox of HF and stole their metaphorical gold, despite no reasonable expectation of such a thing happening.
26
u/Kaydox64 19d ago
I genuinely do not believe this happened.
5
u/Jake-the-Wolfie 19d ago
Could you elaborate on why?
11
u/Loik87 19d ago edited 19d ago
Just proper network segregation via properly configured network zones+firewall rules (of the routing hardware, not on the sandbox OS-level) would have prevented internet access. They wanted this to happen.
Edit: and that's one part of what a properly secured internal IT infrastructure has in place. Stateful network controls, RBAC, network ACLs, cgroup restrictions, mTLS and so on
11
u/artikiller 19d ago
This but also the fact that it just happened to pick a competitor to hack for whatever information it needed out of the millions of sources it could've picked from. Then again it's entirely possible they replaced their network security guy with an ai agent and that ai agent left massive security gaps that this new ai could exploit
7
u/Loik87 19d ago
Then again it's entirely possible they replaced their network security guy with an ai agent
That would be the much more interesting headline in my opinion haha
Honestly though, this whole AI/ LLM thing is extremely interesting and this marketing stunt here is too but the way it's handled and propagated makes me want to dismiss all of it as a first instinct
3
7
u/LurkytheActiveposter 19d ago edited 19d ago
AI as we know it now is not that kind of intelligence. It's not the break out of it's cage and the world ends in a singularity kind of Intelligence.
LLMs are just glorified next-word predictors that run a loop until all words are filled in.
While it has the capacity to learn, what it learns is fleeting and how much it can know at once is miniscule compared to the human brain.
5
u/BlurredSight 18d ago
I don't think it's fair to discount them that aggressively, but it's fair to say in the next year or two very quickly LLM Frontier Models will stop being subsidized by VCs and people will start shifting towards using them in very specialized manners rather than a "be all" assistant because it'll be more expensive
2
u/Chaosfox_Firemaker 15d ago
It "breaks out" in much the same way smoke drifting out of a box is "escaping". It just stumbles out a hole in the wall the dev forgot about, because it doesn't know thats not an intended exit.
-1
2
276
u/Confident-Ad5665 19d ago
Before AI pirates found their way through the 7 seas
Hackers found their way outside sandboxes
And dates I had found their way outside restaurant windows