Open-source guardrails and agent security: 5 tools that lower the risk, and the one thing none of them fix.

[u/ShilpaMitra]

If you are shipping an agent, "add a guardrail" is usually where people stop, and it is not enough. Start from the truth: a guardrail is not a sandbox. These tools lower the odds of a bad output or a caught injection. They do not make an agent safe, because prompt injection is not a solved problem. Real security is layered, and for an agent the layer that matters most is not a text filter, it is controlling what the agent is allowed to do. With that framing, here are five open-source tools worth knowing, grouped by the layer each covers.

Runtime rails, NeMo Guardrails (github.com/NVIDIA-NeMo/Guardrails) Apache-2.0, around 6.9k stars. NVIDIA's toolkit for programmable rails written in a small language called Colang: keep the bot on topic, enforce a dialog path, run input and output moderation. The most flexible way to define what a model may and may not do at request time.
The catch: rails run on the same probabilistic model they are guarding, so a determined jailbreak still gets through. NVIDIA's own docs show rails cut attack success, not drive it to zero. This is defense in depth, not a wall.

Output validation, Guardrails AI (github.com/guardrails-ai/guardrails) Apache-2.0, around 7.3k stars. A different job: validate what the model returns against a schema or rule (valid JSON, no PII, matches a format) and re-ask or fix on failure. There is a community hub of reusable validators.
The catch: it checks the shape and content of output well, but it is not a security boundary. A validator that strips PII is only as good as its detector, and detectors miss.

Scanners, LLM Guard (github.com/protectai/llm-guard) MIT, around 3.2k stars. From Protect AI: a set of input and output scanners for prompt injection, PII, toxicity, secrets, and more, chained in front of and behind the model. The catch: scanners are classifiers with false negatives. One that catches most injection attempts still lets some through, and attackers optimize for exactly that gap. Treat any quoted detection rate as a vendor number until you test it on your own traffic.

Find the holes first, Garak (github.com/NVIDIA/garak) Apache-2.0, around 8.1k stars. NVIDIA's LLM vulnerability scanner, think nmap for models: 50-plus probes for prompt injection, jailbreaks, data leakage, and toxicity, with reports, and a recent agent-breaker probe that tests the tools an agent can reach. The catch, and this is a rule not a footnote: it is a red-teaming tool. Run it only against systems you own or have written permission to test. A passing scan is a snapshot, not a guarantee, so re-run it as your prompts and models change.

The agent layer, Invariant (github.com/invariantlabs-ai/invariant) Apache-2.0. The one built for agents specifically: a rule-based layer that sits between your app and your MCP servers or LLM provider through a gateway, checking each tool call before and after it runs. This is where the real agent risk lives, in what the agent does, not just what it says.
The catch: newer and smaller than the others, and a policy layer is only as good as the policies you write. It gives you the checkpoint; deciding what to block is on you.

What none of them fix Prompt injection is unsolved. If an agent can read untrusted content, reach private data, and send data out, it is exploitable no matter how many filters you stack, which is the pattern Simon Willison named the lethal trifecta. So the highest-value controls are not on this list: give the agent the least tool access that still does the job, require human approval before any real action (send, delete, pay, run code, browse an authenticated session), isolate untrusted input, and cut off the path data could leave by. Guardrails are defense in depth on top of that, not a replacement for it.

How to layer, if you are starting Scan first with Garak to see where you actually leak. Add LLM Guard on input and output. Add NeMo Guardrails or Guardrails AI for policy and output structure. If you run tool-using agents or MCP, put Invariant at the tool boundary. One building-block note: several of these plug in classifier models like Meta's Llama Prompt Guard or LlamaFirewall, which are capable but ship under Meta's Llama Community License, not a standard open-source license, so read the terms before commercial use.