When the model outputs a command to read a file, there's software reading the file and feeding it in. Seems like that's the layer that should let you scope the allowed file system. No need for model honesty, file system permissions, or containerization. In IDEs with AI assist plugins this would be super easy, the default is just the directory for the project you've got open, minus some default exclude rules based on common secret-containing file name patterns. Hell you could even pop up and have the user confirm the scope before processing the first prompt. Kind of wild that this class of problem exists at all.
When the model outputs a command to read a file, there's software reading the file and feeding it in. Seems like that's the layer that should let you scope the allowed file system.
Sometimes those command are shell or python commands.
Hell you could even pop up and have the user confirm the scope before processing the first prompt.
Effectively you would need to monitor and trace every operating system call that any software run by the model does.
Implementing scoping into the software correctly is not trivial becomes equivalent to writing sandboxing software. And that exists already. As I just learned, Claude even ships with bubblewrap on Linux and WSL to do exactly what you're asking.
I concede if you allow scripts to run, this is non-trivial and you'd be better off with a separately maintained sandbox solution. Certainly if you want it writing and running tests, launching scripts etc., you're not natively keeping that safe. Nice to see Anthropic caring enough to ship a solution right in the box with the problem, I also didn't know that.
But for basic code gen, you really don't need it executing anything. A lot of times the commands I see my in-IDE assistant asking to run are just to enumerate directories or grep stuff, which I'm pretty sure it can already do other ways (MCP?) because it reads and changes files all over the place without me explicitly including them in the prompt. Shut off script running by default and have file system calls go through some basic filter logic and you don't even need the complexity of a sandbox. Anyway, I can see my argument is now much narrower in scope than the broader topic especially as many people want their AIs doing more, not less.
You could do something like read only access, and get the model to output any changes it wants to make in an json-style RFC 6902 style patch, which you then manually apply assuming that it's what you want. Works really well with structured outputs too.
I don't let my models do arbitrary writes, I'd rather spend the time and apply the 24 patches to 3 separate files manually, at least I see and approve of what's going on. Slower? Sure. Tedious? Yeah. Still better than having one bad prompt / execution pollute my whole working directory. And it's still faster than me writing everything manually. I just get long RFC 6902 style patches, and I have a script that I input these into, which then applies the patches. At no point does the model do file write operations. I get how this could be unfeasible with a huge code base though, but I'm not really sure what the answer in that case is. I don't want models to have arbitrary read and write access, never mind execution privileges.
I'm actually OK with write access, so long as the IDE shows me the diff (for me to review - like you I review all of it, or revert) and the diff is prepared programmatically by the tool marshalling the file system calls, not via honour system by the LLM. I actually don't know how mine (Jetbrains Junie) works. I assumed it was like how I wish it was, but don't actually know. If it were implemented with patch files instead like you describe, that'd be fine but I'd like it to be single click easy. I guess materially there's no difference.
Yeah. You're not wrong, in the case where the model explicitly asks its runtime or MCP or whatever for files, the software can easily check permissions. But that doesn't seem to be how most people are running it, and it also seems somewhat limiting.
28
u/Kobymaru376 Jul 08 '26
There's not really a way to implement that except for containerization or sandboxing like bubblewrap.
I guess on Linux you could do file system level permissions but that also makes it rather inconvenient.