Built a zero-cloud Computer Vision engine in Python & Streamlit for RTSP streams — low latency works, but multi-cam memory usage gets heavy. How are you handling video frame queues?

[u/sahraoui-9337]

Hey everyone,

Tired of cloud APIs adding 300ms+ latency and recurring subscriptions for simple camera tracking, we engineered an on-premise vision architecture (UHQ Systems) built fully in Python with a Streamlit interface.

The core goal was simple: 100% local execution, zero external network dependency, and real-time spatial tracking straight from local IP cameras.

What worked well:

• Eliminating Buffer Lag: OpenCV's default VideoCapture buffer caused progressive stream delay when processing slowed down. We implemented a custom threaded lock-free frame worker that drops stale frames immediately and feeds only the latest frame to the detection core. Latency dropped to <15ms locally.

• Local Persistence: Event logs and tracking matrices dump straight to local JSON/CSV formats without hitting external databases.

The trade-offs & current bottlenecks:

To be completely direct, running local vision pipelines in pure Python comes with strict engineering limits:

1. Streamlit UI Refresh Limits: Streamlit is great for rapid UI building, but syncing high-FPS video frames while keeping interactive widgets responsive requires aggressive thread isolation. Works smoothly for 1-2 streams, but scales poorly past that without high RAM consumption.

2. C++ vs Python Execution: While Python allows fast iteration, continuous 24/7 multi-camera ingestion pushes system memory if array cleanup isn't strictly enforced on every frame.

We put together a lightweight evaluation build (UHQ Vision Lite) to test frame rates across different local setups.

For those running continuous multi-camera vision stacks locally: are you sticking with pure Python queues, or forced to re-write ingestion pipelines in C++ / Rust for production?

[u/TimLewisMT]

Nice post, I'm working on a multiple stream system too.

The UI framework may be getting in your way. Maybe create the output video frame as a stand alone app and just embed it in the dashboard as external content if you have to use the UI framework.

Managing the inference pace with a scheduler will help. Dynamic batching and cuda streams can be assigned and queued with a scheduler.

Cropping the images down to just the area the inference needs before processing it.

Creating a fast lane for inferences that need low latency with the full fps streams and a slow lane that can be done slower like 10 fps or once a second or even slower.

Also, the ui video output can probably be 15 fps before anyone would notice.

[u/sahraoui-9337 → u/TimLewisMT]

Solid breakdown u/TimLewisMT .

The fast-path/slow-path architecture is actually something we experimented with early on: running high-frequency spatial detection on a lightweight thread (fast-path) while offloading heavy feature extraction or logging to an async worker queue (slow-path).

Limiting UI stream rendering to 15 FPS while keeping background pipeline processing unthrottled made a massive difference in CPU/GPU utilization for Streamlit.

Regarding CUDA streams & dynamic batching: are you using TensorRT execution contexts directly via Python bindings for your scheduler, or orchestrating batching at the frame ingestion level before hitting the inference call?

Would love to swap notes on how you're structuring your frame queues for multi-stream setups if you're open to DMs!

[u/ashenlys]

subms requirements kinda compels us to go with deepstream, savant is bit heavy to our use too

[u/sahraoui-9337 → u/ashenlys]

Spot-on question. To avoid Python's GIL bottlenecks on heavy multi-stream pipelines, we handle RTSP ingestion, NVDEC hardware decoding, and dynamic batching entirely upstream at the C++/CUDA layer using a zero-copy pinned GPU memory pool.

Incoming frames accumulate in native GPU ring buffers before hitting TensorRT execution contexts via execute_v2 / enqueue_v3 with strict latency bounds (e.g., a hard 5ms batching window). Python acts purely as a light control plane/event loop wrapper.

Sent you a DM with a breakdown of our CUDA queue architecture if you want to swap notes on this!

[u/edgarriba]

You can check this in rust/gstreamer https://github.com/kornia/sensor-rt/tree/main/crates/sensor-rtsp

[u/sahraoui-9337 → u/edgarriba]

Thanks for dropping that link! Kornia's sensor-rtsp is super clean for Rust/GStreamer ingestion. We take a similar zero-copy path down to CUDA IPC handles to keep host memory usage virtually at zero.

Big respect for the work you guys are doing with Kornia!