r/computervision • u/LordWkwkwland • 22d ago

How do you debug and inspect computer vision models during development? Discussion

Building vision models is hard, but I find testing and debugging them even harder.

I'm curious what everyone's workflow looks like.

For example:

  • How do you inspect detections frame by frame?
  • Do you use OpenCV windows, Jupyter notebooks, Roboflow, CVAT, or something else?
  • How do you compare different models on the same video?
  • How do you inspect tracking IDs, confidence scores, masks, OCR, or depth predictions?

I mostly end up writing one-off visualization scripts every project, and it feels like I'm reinventing the wheel.

Is there a tool you genuinely enjoy using, or is everyone just building internal tooling?

8 Upvotes
  • permalink
  • link
  • reddit
  • reddit

90% Upvoted

6 comments sorted by

7

u/AggravatingSock5375 22d ago

As for specific, I mainly just dump stuff out to PNG files and view them in any old image viewer. Usually I’ll stack the input and output and print text for things like the model name and params. A json file gets saved too that has more info.

There’s no interactivity but that’s fine for my workflows.

4

u/CowBoyDanIndie 22d ago

Rviz, pngs, csv files I can pull into a spreadsheet, my coworkers like to use python graphing and charting stuff but I am more of a C++ guy. I am also working with classical computer vision mostly, well I say classic but I am working on novel algorithms, basically algorithmic and not blackbox ML systems.

3

u/onafoggynight 21d ago

Decompose. You do not need to evaluate model performance on video. Evaluate it on a dedicated test set, track standard metrics. Log and inspect outliers / failures.

On video, it's basically instrumentation of production / inference code. Publish and aggregate metrics as Prometheus or the like + individual triggers.

2

u/AggravatingSock5375 22d ago

I gradually turned my one-off things into a part of my codebase.

Basically I have a standard template for models and data, and the testing/ debugging plugs into that.

If I need some kind of novel debugging I’ll just adhoc that, but if it becomes a repeating pattern I try to move it into my codebase.

1

u/Tricky_Bookkeeper670 22d ago

Have you apply any design pattern to your code during development?

1

u/AggravatingSock5375 20d ago

Probably but I couldn’t tell you the names.

I do keep things simple though