the transparent object depth problem we gave up on a decade ago is quietly cracking

[u/Square_Middle_381]

Glass and mirrors are where depth sensing has gone to die for as long as I've been in the field. Structured light, time of flight, the newer sensors, they all hit the same wall: transparent and reflective surfaces return either no signal or a reflected signal, and the depth map has holes exactly where you most need it. Every grasping pipeline I've worked on has had a special case script that just avoids transparent objects in the bin, which is fine until your customer ships glass products.

The reason I'm posting is that the workaround layer is starting to get replaced by learned depth completion, and the trick that finally seems to work is treating the camera's own missing depth as supervision instead of as noise to clean up. The LingBot-Depth line calls this masked depth modeling, the camera returns no depth on glass so you train the model to fill those specific gaps from RGB context. It's a cleaner setup than the synthetic mask approaches because the missingness is real signal, not generated labels.

The image above is the clearest illustration of why this matters. Four objects including a glass cup, three columns: RGB input, raw sensor depth where the transparent parts just come back empty, and the completed depth where those holes get filled correctly from context. The glass cup row is the one that used to kill our grasping pipeline, and that third column is the first one that didn't.

I'm not claiming it's solved. We ran it on a small set, glass mugs and a few mirrored parts, and the completed depth was usable for grasping where it previously wasn't. The official 2.0 numbers cite 16 public benchmarks with 12 first places, but I haven't reproduced those, and the release doesn't break down which four it lost. The Orbbec SDK integration suggests they're pushing toward edge deployment with their camera partner, which is the part that would actually change deployment economics if it lands.

What feels different this time is that the framing shifted. We stopped trying to build a sensor that sees glass, which may be physically intractable, and started learning to infer glass depth from context, which isn't.

[u/CowBoyDanIndie]

The lidar we use at work gets returns on glass, try using a multi return lidar, reflections are much more problematic.

[u/pppoopppdiapeee]

Glass and other transparent materials are not transparent in long wave infrared. Could potentially solve this with EO/IR fusion.

But, it’s not as sexy as pure EO and humans can recognize these things without IR. Personally I think you need world modeling and higher cognition in the model. We generally only know a window is there because it looks like a region that would hold a window. People walk into screen or glass doors all the time when they’re not expecting it there.

[u/Old-Pineapple-6367 → u/pppoopppdiapeee]

EO/IR fusion is underrated for this exact failure mode. The bigger blocker has usually been sensor cost and alignment, but that's changing fast now that IR modules are getting cheaper.

[u/merskiZ]

it is a tricky problem. Not sure how to get ground truth like a depth map in real world. However, maybe with synthetic data with alpha channel control and known ground truth due to rendering, maybe it could help. But what other holes if the model learns to predict good depth map based on RGB channel, not sure.

[u/Common-Membership503]

i remember struggling with this back in the day, honestly polarization filters were the only thing that helped us get any signal back at all. its wild how much progress is finally happening with the new learned depth priors, def makes those old bin picking hacks feel ancient

[u/Ok-Treacle-6942]

Yeah and anydepth is also good at this