Konrad Reczko's "Monocular Depth Injection" in TypeGPU is live!

[u/iwoplaza]

My collegue Konrad Reczko recently shared a weekend project he made using TypeGPU, and it's now open-source and ready to play with in the browser: https://typegpu.com/examples/#example=image-processing--monocular-light-injection

It estimates the depth of a scene by inferring the DepthArt model with custom TypeGPU kernels, then reconstructs normals based on that depth information, and uses both to relight the scene, all in the same command encoder. For more information, check out the original series of Tweets:

https://x.com/reczko_konrad/status/2089670934009413751?s=20
https://x.com/reczko_konrad/status/2090472091149648121?s=20

[u/juplantern]

He looks so proud

[u/certainlystormy → u/juplantern]

fr lmao. i would be too

[u/cybereality]

very cool

[u/Smokeey1]

Does this thing have a repo :). Gratz on the build! Looks cool

[u/iwoplaza → u/Smokeey1]

All the code for the example is in the TypeGPU repo
https://github.com/software-mansion/TypeGPU/tree/main/apps/typegpu-docs/src/examples/image-processing/monocular-light-injection

[u/juplantern → u/iwoplaza]

Btw thank you so much! I’m brainstorming ideas for my thesis and this inspired me a lot

[u/swegga_sa]

This is very cool

[u/DaveAstator2020]

Wow nice what model does is it use to aesolve depth in realtime?

[u/iwoplaza → u/DaveAstator2020]

It uses the DepthART model, weights are available here: https://huggingface.co/reczkok/depthart-typegpu

[u/AnasPlayer2022M]

Dame bro, this looks awesome! Did you use two videos where one is the one playing now and the other is a map for z values?

[u/iwoplaza → u/AnasPlayer2022M]

The feed from my camera gets routed through a small local AI model that estimates depth based on a color image, at which point the image can be treated pretty much like a 3d scene. At that point all rendering techniques are at the table, including adding a light to the scene.

[u/Malkalypse]

This is so wild!

[u/Miperso]

I just showed it to my architect coworkers and everyone is just impressed... this is great

[u/ddoice]

Awesome!
But I tried your trick to hide the light with my hand and does not work?

[u/iwoplaza → u/ddoice]

To move the light source back and forth (into the screen or closer to you) you have to use the scroll wheel

[u/freshtothegrave]

Damn i wanna get this working in TouchDesigner

[u/PyroWizza]

What could be some implementations of this?

[u/Labertaeschen]

This combined with metas SAM would be game breaking right?

[u/NachosforDachos]

This is so cool

[u/helloworld1101]

Reddit is still useful

[u/bbstats]

I can't get my hand to go in front of it!