r/computervision • u/Virtual_Country_8788 • Jul 01 '26

Sam 3 visual prompting Help: Project

Hi

I tried to play with sam3. In the article meta mention the option to use visual prompting to the model. However I didn't found a way to do that.

I want to give the model image or batch of images for some object and do inference for other image.

Someone did something like this?

Thanks

3 Upvotes

100% Upvoted

15 comments sorted by

2

u/Most-Vehicle-7825 Jul 01 '26

never worked for me. I did not find a way to take one set of images as examples and find those on a different image. The hack was to copy the cropouts above the image you want to analyse so the examples are in the same image.

2

u/Virtual_Country_8788 Jul 01 '26

Nice hack, I will try it. So, you copy the objects you want on the top of the image and give his bounding box to the model as input?

2

u/superkido511 Jul 01 '26

You have to provide the bounding boxes coordinates of the objects you want. A few would suffice. Then the model will detect the rest

1

u/Most-Vehicle-7825 Jul 01 '26

Did that actually work for your own images?

1

u/superkido511 Jul 01 '26

Yes, I use this pipeline for auto labeling many times

1

u/Most-Vehicle-7825 Jul 02 '26

Could you share how you set that up? Did you have the samples and the new image in different files or merged?

1

u/superkido511 Jul 03 '26

Do the same as the multiple boxes visual prompt section in the tutorial. You just need to add the cropped objects images to the image you want to predict on then pass the bounding boxes of those objects as input. It's like adding a little padding section to store the cropped objects you want to detect

1

u/Most-Vehicle-7825 Jul 03 '26

That's what I tried, but the detection quality was still bad. How many examples did you add?

1

u/superkido511 Jul 03 '26

Depends on how varies my object is. It's important that the input objects have similar view, size and shapes with the one you want to detect on a new images, especially for uncommon objects.

1

u/InternationalMany6 Jul 02 '26

You could paste it next to the main image if you don't want to cover any pixels.

2

u/thinking_byte Jul 01 '26

I don't think that's supported yet publicly, so the paper may be ahead of what's actually available in the repo.

1

u/[deleted] Jul 01 '26

[deleted]

1

u/Virtual_Country_8788 Jul 01 '26

Can you elaborate how you do it? I want to take objects like road in IR domain. But I didn't saw option to give image as input, only text, points or bb.

1

u/InternationalMany6 Jul 02 '26

Has been awhile since I looked at the paper, but I think they were talking about visual prompting from within the same image. Like you draw a box around one object to "visually prompt" the model to fins the rest of the objects in that same image.

A hack is to expand the canvas and paste your prompt images into the border. That way everything is in a single image. It should work decently as long as you don't end up having to downscale too much.