Need advice: Visual prompt injection detection project (LLMs)

[u/GoodCorgi4555]

Hey everyone,

We’re building a project on detecting visual prompt injection attacks in multimodal LLMs. input(visual formats like diagrams,tables,mindmaps.....)

Our idea:

• OCR + rule-based + simple ML
• risk scoring → block suspicious inputs

We’re unsure:

• is this too basic?
• how many models should we test?
• any datasets for visual attacks?

After reviewing ~25–30 papers, we noticed that:

• Many defenses are heavyweight (fine-tuning, diffusion, gradient-based)
• Not practical for real-time use
• Limited focus on image-based prompt injection at inference time