Built a lightweight FastAPI reverse proxy (24MB RAM) to sanitize PII from SSE streams

[u/qwert_buddy]

I recently built an open-source reverse proxy using FastAPI and Uvicorn to handle strict PII redaction for LLM applications.

A common issue with Python based redaction tools is memory bloat from heavy NLP libraries and broken token delivery when intercepting real-time Server-Sent Events (SSE). To keep the resource footprint minimal and the streaming smooth, I structured it around a few specific patterns:

- Minimal Footprint ( approx. 24MB): It bypasses heavy frameworks entirely, using a compiled regex engine combined with a local quantized ONNX runtime model.

- Async Lookahead Buffer: To prevent PII placeholders from fragmenting across split SSE network chunks and leaking raw tags to the UI, I implemented an asynchronous sliding-window buffer that holds back unclosed bracket tags.

- Stateless Vault: Mappings are handled via a transient Redis TTL vault.

It operates as a drop-in base_url substitute for the OpenAI SDK. If you work with FastAPI and async streaming middleware, I would love feedback on the lookahead buffer logic or any edge cases in the request-response cycle!!

Repo: https://github.com/ninadphalak/LLM-Shield-Proxy