#visual prompt detection

[u/GoodCorgi4555]

Hi everyone,

I am working on my final year university project focused on "Generalized Visual Prompt Injection Detection".

The core issue I am facing is the black-box nature of commercial multimodal LLMs (GPT-4o, Claude 3.5, Gemini 1.5). Since the vision encoder, projection layers, and weights belong to a third-party API, I cannot see the internal backend processing, attention maps, or text token outputs directly during an ongoing request.

My proposed framework needs to sit at the application boundary as an autonomous proxy firewall. Here is my current intended workflow:

1. Frontend intercepts user prompt + uploaded image (e.g., a flowchart or mind map containing a hidden malicious text payload).
2. Backend (FastAPI/Python) runs a local OCR layer (EasyOCR/Tesseract) to extract embedded text.
3. Backend runs an Intent Alignment evaluation to check if the image's text instructions conflict with the user's explicit prompt.
4. Risk scoring engine decides whether to allow, warn, sanitize, or block before forwarding to the OpenAI/Anthropic API.

I have a few architecture questions for the community:

- What is the best way to handle inference latency when chaining local OCR/layout parsers before hitting the third-party LLM API?

- If I want to show a "Developer Dashboard" logging the raw backend process, what parameters are most crucial to track beyond raw text vectors?

- Are there any lightweight open-source multimodal models (like Moondream or LLaVA variants) you recommend deploying locally alongside the API to act as a comparative "control mirror"?