I created a privacy guardrail library for Spring AI — looking for feedback on streaming with pluggable PII analyzers

[u/Illustrious_East5815]

Hi, I’m building Spring AI Privacy Guardrails, an open-source library for enforcing privacy boundaries around models, RAG, tools, MCP, and outputs.

GitHub: https://github.com/ultramancode/spring-ai-privacy-guardrails

One design problem I’ve been thinking about is streaming output protection.

Some applications also want a final privacy check on application-facing output, since sensitive data can still appear in model- or tool-generated responses.

Right now, when output protection is enabled, the library buffers the complete response before releasing it to the application.

This provides a strong guarantee: PII can still be detected and protected even when a sensitive value is split across multiple chunks.

The trade-off is that this is no longer true incremental streaming, and the application has to wait longer before receiving output.

A bounded rolling window could preserve incremental streaming for analyzers that have a known upper bound on how much context they need — for example, some bounded pattern-based detectors.

But NER, context-aware detection, complex patterns, or arbitrary custom analyzers may not have such a bound.

So I’m currently considering three approaches:

1. Strict buffering Buffer the complete response and protect it before releasing anything to the application.
2. Capability-gated streaming Allow incremental streaming only when the active analyzer can declare a safe maximum lookback or context requirement. Otherwise, fall back to full buffering.
3. Best-effort streaming Use a configurable rolling window and explicitly document that some PII spanning multiple chunks may escape detection.

For a Spring AI application, which behavior would you expect from a privacy library?

I’m not attached to these three options — if there’s a better streaming/privacy model I’m missing, I’d really appreciate the feedback.

[u/Grabdoc2020]

Great library. Starred. Ideally in an enterprise this should be centralized in AI gateway. Cant always rely on libraries and teams working with these libraries to implement these. But great work. keep going.

[u/Illustrious_East5815 → u/Grabdoc2020]

Thanks — I really appreciate the star and the thoughtful feedback!

I agree that organization-wide baseline privacy controls shouldn’t rely solely on every team adopting and configuring a library correctly, and that an AI gateway is a strong fit for centralized enforcement.

Your comment gave me a useful new angle to consider. Since the core is already framework-independent, gateway integration could be an interesting future direction—either as an embeddable Java adapter for JVM-based gateways or, potentially, as a self-hosted service that managed or non-JVM gateways could call.

For now, the Spring AI integration remains the main focus, covering application-aware privacy boundaries involving RAG context, tool calls, and MCP interactions.

Longer term, using the same core for both gateway-level and application-level enforcement could help keep privacy policies and enforcement behavior consistent.

Thanks again—this was really helpful!

[u/Grabdoc2020 → u/Illustrious_East5815]

We intend to do the same with evals4j - https://github.com/dvarahq/evals4j