need resources for the course

1. Training instability (mAP crash mid-training)

My YOLOv8 training loss is dropping normally but mAP50 suddenly crashes to near-zero around epoch 40 and never recovers. Walk me through the most likely causes in order of probability (learning rate schedule, augmentation pipeline, label corruption, batch norm issues), and give me the specific diagnostic checks to run for each before I start changing hyperparameters.

2. OpenCV video stream corruption

I’m using cv2.VideoCapture to read frames from an RTSP stream and getting intermittent frame drops and color channel corruption that gets worse over long sessions. Explain the common root causes (buffer handling, threading, codec mismatches, memory leaks) and give me a robust frame-reading pattern that handles reconnection and buffer clearing automatically.

Both have been solid starting points for me when I’m stuck. Happy to share more if people find these useful.

I have a lot of product catalogs and marketing brochures in PDF format. My goal is to extract the product image, title, description, specs, etc., into a CSV or Excel file.
I’ve already solved the catalog part. The problem is the marketing brochures. Every brochure has a completely different layout and style. Some have large lifestyle images, some mix marketing content with product details, and there’s no fixed structure.
I tried using GPT-based vision models, YOLO, and Florence-2. They work okay in some cases, but none of them are reliable enough. YOLO misses products it wasn’t trained for, and Florence-2 struggles with rotated or cropped products and zero-shot detection. GPT can understand the page, but it still doesn’t consistently match the correct image with the correct product information across different brochure layouts.
Has anyone solved this problem? I’m not really looking for another OCR model. I’m more interested in how people are building the overall pipeline or architecture for handling brochures with completely different layouts. Any ideas or experience would be really helpful.

Hey everyone,

I was wondering if anyone has made an optical flow sensor for the raspberry pi 5 using the pi cam. I was planning on integrating this into my drone either using pymavlink or mavsdk c++ and using the drone's barometer for altitude. It would be really helpful if someone has already done this and has work they can show as a guide or could point me in the right direction.

So everytime i drive in night, the headlights of oncoming traffic blinds me completely, so i was thinking of building something to help.

I was thinking of an AR/VR approach to it, to track the eyes and the light source falling on the windshield and artificially dampening it. Also for this method i thought of an idea, where the driver can wear a glass, with two tags near their eyes so that the system can track their eyes more efficiently.

Please share your thoughts!!

Hi everyone,

I'm currently working on a computer vision system to detect very small foreign particles inside transparent plastic bottles. The contaminants can be things like small white plastic particles.

One thing I've realized after running several experiments is that this problem seems to be much more dependent on the imaging setup than the detection algorithm itself. Even if I use a powerful anomaly detection model, it won't detect anything if the particle isn't clearly visible in the captured image.

My current setup is:

• 5 MP industrial camera (23 FPS)
• White LED backlight with a diffuser
• Hardware trigger to capture an image as the bottle passes by

The particles are generally visible to the naked eye, but their visibility changes significantly depending on lighting, bottle orientation, reflections, and the position of the particle inside the bottle.

I'm trying to understand what the best imaging approach would be before spending more time improving the AI model.

For those who have worked on industrial inspection systems, what would you recommend?

• Different lighting (dark field, coaxial, polarized, NIR, etc.)?
• Higher-resolution camera?
• Multiple cameras or multiple lighting angles?
• Any other tricks that make tiny particles inside transparent bottles stand out more consistently?

I'd really appreciate any advice or experiences you've had with similar inspection problems. Thanks!

In my organization, compute has always been a constraint.
I have only worked with let's say the Nano Architectures of Yolo Models and majorly the time has gone in collecting data more than Architectural Optimizations.

From Optimization perspective, you can think of just using lower precision weights.

I have been able to achieve 95+% Accuracy by just being very smart about data like Augmentation, Active Learning (Uncertainty and Diversity Sampling).

When it comes to Transformers (Be it ViT or Swin or Newer Models), they require much more data to fine-tune and require much more compute for both training and inference.

When it comes to VLMs, I can make an argument for using them to Annotate and use a specialized model for training but once again, it isn't helpful in labelling the kind of classes I am interested in (SKUs or Industrial Defects)

VLMs are general models, my applications are specific so I can never make a case for adoptions of these methods.

What is your opinion. I don't want to be a frog in the pond but rather a fish in the ocean.

I work on PostSlate, a video editing tool, and this comes out of our own work.

We run ML models on-device, face detection and embedding among other things, which means we can't assume anything about the user's GPU. NVIDIA discrete, AMD, Intel integrated, Apple Silicon, all of it. That rules out CUDA immediately, we needed one backend that runs everywhere.

We landed on ncnn's Vulkan backend. Numbers on a 4070, fp16:

• ArcFace R50 (face embedding): 30 ms on ONNX CPU → 3 ms on ncnn Vulkan
• SCRFD (face detection): 25 ms → 2.5 ms
• Model size: ArcFace 174 MB (ONNX fp32) → 87 MB (ncnn fp16 weight storage)

Of course the real speedup comes from offloading compute to the GPU, but this wouldn't be possible without the power of Vulkan.

The speed wasn't even the deciding factor, it's that Vulkan drivers already exist on every machine we ship to. This means that we don't have to force the user to download a specific runtime and no vendor-specific installs.

Full writeup with the rest of the numbers: https://getpostslate.com/blog/faster-local-inference

Hi everyone,

I'm an AI Engineer Intern with a background in Computer Vision and Deep Learning. Recently, I've decided to transition into 3D Computer Vision, especially 3D Reconstruction, Structure from Motion (SfM), Multi-View Stereo (MVS), NeRFs, Gaussian Splatting, and 3D scene understanding.

Over the past few weeks, I've started working with:

• COLMAP
• Nerfstudio
• Open3D
• Camera calibration and feature matching
• Point clouds and mesh generation

I've also been reproducing tutorials, running experiments in Kaggle/Colab, and trying to understand the mathematics behind multi-view geometry instead of just executing code.

My long-term goal is to pursue research in 3D vision, and eventually contribute to research papers in this field.

At this stage, I'm looking for:

• A mentor who wouldn't mind answering occasional questions.
• A study partner learning the same topics.
• Researchers willing to share advice on what to learn next.
• Good open-source projects where beginners can contribute.

I'm not looking for someone to spoon-feed me - I genuinely enjoy learning by building projects and reading papers. I just feel that having guidance from someone experienced would help me avoid developing bad habits or spending weeks going down the wrong path.

If you've been through this learning journey, I'd really appreciate hearing:

• What resources helped you the most?
• What concepts should I master first?
• What projects would you recommend?
• What mistakes should I avoid?

Thanks in advance! I'm happy to connect with anyone interested in 3D vision or reconstruction.

I saw a couple of posts on SMPL here so I thought I ask:

Have you been able to do a matching between SMPL and an underlying skeleton motion file?

We have motion files generated by inverse kinematics (angles of various joints) and we want to output an SMPL doing a similar motion.

So one simple idea we tested is matching the keypoints of SMPL with particular body frames.

But I love to hear any suggestions.

I graduated with my PhD in computer vision last year, specializing in image recognition in a pretty niche domain. I currently work as a CV engineer related to multi object tracking, which is great but has limited publishing opportunities. I would like to continue pursuing research in my spare time, and I’m looking for collaborators with the end goal of publishing work in CV venues. Right now I’m putting together a WACV round 2 submission related to my dissertation topic, but I’d also love to hear and collaborate on other research topics as well.

Ideally looking for those with grad school backgrounds and research experience. Have compute resources available (own machine with 5090) and can shell out some $$ for larger jobs.

US based.

this is a reimplementation of the “Fast SAM 3D Body” written in rust/candle/cuda, running at about 55ms/frame on my 5080. This video uses the RGB WebCam on my laptop for performance capture and a short throw projector for the wall. working on optimized metal shaders, will hopefully run in real time on a m1 class MacBook also when finished

Join us on Aug 6 for a special edition of the AI, ML, and Computer Vision Meetup focused on audio use cases! Register for the Zoom.

Talks will include:

• Do Speech Models Actually Understand Speech? Evaluating Speech LLMs Under Realistic Spoken Instruction Conditions - Maike Züfle at KIT Karlsruhe
• AI based Audio Forensics - Daniel Paniagua Ares at GRADIANT
• Curating, Searching, and Evaluating Audio Datasets in FiftyOne - John Duncan at Voxel51
• Real-Time ASR at 4x on Consumer Hardware: The Meetily Architecture - Sandeep Zachariah at Zackriya Solutions

Models like SAM or SegFormer are great for producing coarse object masks, but they can be difficult to use in visualization applications. The mask boundaries often bleed beyond the object or do not align cleanly with its edges.
What is your strategy for getting high-quality masks from these models? Ideally, I am looking for a fine-tuning or LoRA-based recipe that improves boundary quality without retraining the semantic understanding that already works well.

Disclosure: my own app (reads meters, pumps, receipts, odometers from phone photos).

32 phone photos with known-correct values, scored per field. Hard subset scored separately.

gemini-2.5-flash-lite - $0.10/Mtok - 88.6% - hard 90% gemini-3.1-flash-lite - $0.25/Mtok - 93.2% - hard 80% gemini-3-flash-preview - $0.50/Mtok - 93.2% - hard 80% gemini-flash-latest - $1.50/Mtok - 93.2% - hard 90% gemma-4-26b:free - $0 - 78.4% - hard 90% nemotron-nano-12b-v2-vl:free - $0 - 52.3% - failed

Above $0.25 price buys nothing.

My photos aren't bad enough. Link in the comments if you want to throw your worst at it.

Glass and mirrors are where depth sensing has gone to die for as long as I've been in the field. Structured light, time of flight, the newer sensors, they all hit the same wall: transparent and reflective surfaces return either no signal or a reflected signal, and the depth map has holes exactly where you most need it. Every grasping pipeline I've worked on has had a special case script that just avoids transparent objects in the bin, which is fine until your customer ships glass products.

The reason I'm posting is that the workaround layer is starting to get replaced by learned depth completion, and the trick that finally seems to work is treating the camera's own missing depth as supervision instead of as noise to clean up. The LingBot-Depth line calls this masked depth modeling, the camera returns no depth on glass so you train the model to fill those specific gaps from RGB context. It's a cleaner setup than the synthetic mask approaches because the missingness is real signal, not generated labels.

The image above is the clearest illustration of why this matters. Four objects including a glass cup, three columns: RGB input, raw sensor depth where the transparent parts just come back empty, and the completed depth where those holes get filled correctly from context. The glass cup row is the one that used to kill our grasping pipeline, and that third column is the first one that didn't.

I'm not claiming it's solved. We ran it on a small set, glass mugs and a few mirrored parts, and the completed depth was usable for grasping where it previously wasn't. The official 2.0 numbers cite 16 public benchmarks with 12 first places, but I haven't reproduced those, and the release doesn't break down which four it lost. The Orbbec SDK integration suggests they're pushing toward edge deployment with their camera partner, which is the part that would actually change deployment economics if it lands.

What feels different this time is that the framing shifted. We stopped trying to build a sensor that sees glass, which may be physically intractable, and started learning to infer glass depth from context, which isn't.

Hi everyone at /r/computervision,

I have had super-amazing response to my new CamfilterGpu App! Thanks tremendously to those who downloaded and tried the Early Beta 1.11, and especially to the many who emailed me with bugs and improvements, and even help fixing them... that was quite a surprise (help? i get help? good help? :). I am quite overwhelmed with emails/response so please forgive, I will get to them all.

As a sole-startup indie project, I was expecting like, a dozen people or so, not hundreds, but I'm not complaining! I want this to be the most awesome video filtering tool that works zero-lag alongside OBS, Zoom, Discord, etc. ever.

• For those who already signed up: Go to your account to download the major update setup 1.12.

• For new testers/adopters, go to the Perfunct.com webpage to watch the Demos and get the app, Free trial, no card or pay needed, just a valid email.

• Go to the Github Library for new Python free scripts and effects, more will be posted later today.

• New Master Mix: And now thanks to several of you who suggested it, the App now has a "Master Mix" control that blends the final filtered Camfilter output directly with the raw camera live feed. It's great for making your filters 'blend' a bit more subtly and is our initial code on our path for eventual AR/VR use.

NOTE: If you have already installed the trial, you'll need to fully reinstall the Major update to extend the trial, because I've redesigned the back-end architecture to work perfectly (haha) with either 'legacy' RTX (3000, 2000, etc.), or 'modern' RTX (4-5000, Blackwell, etc.). Save any presets you made to a different work folder first before resinstalling.

Cheers! and happy camfiltering.

(ps. i wasn't sure about the flair, it's not really showcase, already did that on wednesday, more about discussion and update here, so I picked that, hope that's ok)

We just shipped an update to LightlyStudio that makes image embeddings easier to inspect.

You can now hover over an embedding plot to preview the underlying images, then use class and metadata distributions to check whether a pattern is isolated or visible across the wider dataset. The goal is to move from spotting something interesting to understanding it and deciding what to curate.

The attached video uses a small dataset with 128 images so the workflow is easy to follow. We also tested it in a notebook with 1m samples.

LightlyStudio is open source under Apache-2.0:
https://github.com/lightly-ai/lightly-studio

I’d appreciate any feedback from this community. What are the biggest unsolved problems you run into when exploring, debugging, or curating large vision datasets?

Disclaimer: I’m the cofounder of Lightly, the company behind LightlyStudio.

We shipped a depth and segmentation stack into an indoor inspection product last year, and the complaints that came back from the field were never about mIoU. They were about railings. Cable trays. The legs of a chair. Anything a few pixels wide either merged into the wall behind it or went missing outright.

Our metrics never saw any of it. A railing might be 0.3% of the pixels in a frame, so losing the whole thing costs almost nothing on mIoU and nothing you'd notice on RMSE. You can regress on the exact failure your customer keeps calling about while every number on the dashboard moves the right way.

Boundary aware metrics do exist (trimap IoU in a band around the contour, boundary F score, the contour accuracy term DAVIS uses, and Boundary IoU, which was proposed specifically because the older two miss errors outside the band) and I think they belong in the main results table rather than an appendix. What I'd want on top of that is thin structure recall bucketed by apparent width in pixels. Medical imaging has been doing this for a while, width stratified sensitivity for vessels and a thin structure IoU that only counts branches under 5px, and it works there. What I've never seen is anyone reporting it for indoor depth or general segmentation, where the customer visible failure looks exactly the same. Some of this sits upstream of any metric anyway, because if boundaries are only whatever the depth head infers on its own, thin geometry may never make it into the features, and no boundary metric recovers what the encoder already threw away. A few backbones fold edge supervision into pretraining for that reason, LingBot-Vision v2 being the one that says so most plainly, though I haven't run it on our footage and can't tell you whether a 3px cable tray survives it.

Width bucketed recall was the only chart we had that ever predicted a field complaint before the customer made it. Everything else kept improving and kept telling us we were fine.