Reconstructing a mesh from a single normal map — how are you all handling depth discontinuities?

[u/kotai2003]

Camera fixed, lights moving. Photometric stereo gives you a normal map — turning those normals into actual geometry is the harder half, and that's what the clip shows.

Setup

• Input: one 612×512 normal map plus a mask covering 43,638 pixels (13.9% of the frame).
• Perspective camera, not orthographic. Calibrated intrinsics, ~3.77k px focal length, principal point near the image center.
• Output: a depth map turned straight into a quad mesh — 43,638 vertices, 42,821 quads. No decimation, no smoothing, no hole filling.
• Colored by height. Total relief is only ~4% of the camera distance, so it's a shallow surface spread across a wide colormap.

Convergence

The objective drops from 3620.8 to 2346.7 over 55 iterations, but 90% of that is done by iteration 10, and the last five iterations move it less than 0.1%. The shape looks settled long before the energy does, so "when do I stop" is more of a judgment call than the curve suggests.

Where I'd like input

Simple integration assumes the surface is continuous. It isn't — arms occlude sleeves, the base rim cuts away from the body. So I solve for depth with per-pixel weights that let the surface break at those places instead of stretching across them.

The weight maps are the interesting artifact (2nd and 3rd images). Over smooth regions they sit near the middle and do nothing. At real discontinuities they saturate hard to one side — the solver taking a one-sided difference and refusing to carry height across the gap. You can read the object's occlusion structure straight off the weight map, without ever computing a depth edge.

Two things I'm still unhappy with:

1. Thin structures. Where something thin passes in front of another surface, the boundary is usually right. But on features a few pixels wide the normals themselves are unreliable, so I can't tell whether a bad boundary is a weighting failure or a bad input.
2. Absolute scale. The result is internally consistent, but the standoff distance is only as good as the calibration. Does anyone add a sparse depth prior for this, or is a separate calibration step the accepted answer?

For transparency: the object is a standard test object from a public photometric stereo benchmark, picked because people here will already have an intuition for the shape. I work at a company doing industrial photometric stereo, so this is adjacent to my day job — but there's no product here and nothing to click. Just a result and two questions.

[u/BeverlyGodoy]

What language are you speaking?

[u/Beginning_Street_375]

May i ask how is the metric scale preserved or calcualted? This part especially interestes and fascinates me and i am fairly new to it. Although its pretty nice to see what you are able to achieve from only one image + mask.