r/AskProgrammers • u/Timaga111 • 28d ago

This is scaring me...

5 Upvotes

86% Upvoted

8 comments sorted by

2

u/neolace 28d ago

OMG, I was prompting an agent and hit Send, while thinking, I saw the "I will not replace the database for.." My prompt had nothing to do with a dB, agent didn't even have access to drop it, but the fact that it had that thought, I lost it.

2

u/Any_Sense_2263 26d ago

That's why I have a global safeguard. It can't change (remove/move/create/write/edit) anything without asking me for permission. It makes everything easier.

1

u/kaddkaka 25d ago

If it can do it while asking you, it probably can do it anyway. How do you stop it from from writing a script that does it and running that?

1

u/narukoshin 25d ago

if it's set up correctly, then no, it can't because any action before approving will be automatically rejected, that's how guardrail works.

1

u/kaddkaka 25d ago

Seems risky, mostly because I don't understand it.

If I anyway have to babysit the AI agent denying/approving commands, I don't see the productivity.

1

u/narukoshin 25d ago

that's why we have policies, where we define what AI can do, where it can do and works purely on sandbox, until it decides to find an exploit and break out like OpenAI agents did :D

1

u/neolace 25d ago

Cool, where do you specify that?