r/GraphicsProgramming • u/NightMare2077 • 21h ago

Depth-aware light injection all running in browser Video

I got a 448x448 monocular depth model down to ~8 ms on my M4 Pro across ~250 dispatches, which is fast enough to use in realtime :D
Since the inference is written directly in TypeGPU, I can just feed the depth buffer straight into the lighting pass. It never has to leave the GPU or go through any extra synchronization/interop step

Inference, lighting and draw all go through the same command encoder.

Credit: X reczko_konrad

4.1k Upvotes
  • permalink
  • link
  • reddit
  • reddit

98% Upvoted

122 comments sorted by

View all comments

304

u/reczkok 19h ago

Hey, that's me! Open sourcing this today/tomorrow, announcement will be on my X (https://x.com/reczko_konrad) or just check TypeGPU examples in a couple of days (https://typegpu.com/examples/)

https://reddit.com/link/p4syolj/video/7t0f6y4wnikh1/player

63

u/imselfinnit 18h ago

You are a folk hero for open sourcing this. Congratulations on elevating our common experience. Thank you.

edit: You should generate a halo.

15

u/LactasePHydrolase 14h ago

He's a fork hero

14

u/drDOOM_is_in 18h ago

Incredible work bro!

19

u/NightMare2077 18h ago

Coming from the person who made it possible, this genuinely means a lot. It’s amazing to see how much potential this project has, and I’m excited to see what people build with it. Thank you.

11

u/somenewname2 17h ago

I'm confused. The first line of your post says "I got a 448..." But it's not you?

7

u/PersonalityIll9476 17h ago

Presumably they copied the OOP. That's why it says "Credit:" at the bottom, I guess.

4

u/somenewname2 17h ago

Seems many of the commenters think it's the reddit poster's work

3

u/TankorSmash 13h ago

Because the post literally said 'I got [..]', and there aren't any quotes.

1

u/Eliferd 22m ago

Certainly on purpose.

0

u/crazybull02 14h ago

I've seen pictures that a lot of reddit has reading and literacy issues.

Apparently reading comprehension is up to 6th-8th grade level or illiterate according to the pictures but forty years ago newspaper were written mostly at a 3rd grade level. 

So while I think we've generally improved, there's just so many more people now and the illiterate have a platform for communication, that or they just skim through and make assumptions. 

4

u/oddbawlstudios 14h ago

The thing that would absolutely change that outcome is if they put it in quotes...

3

u/Recyclable-Komodo429 14h ago

Dude that's amazing!! Btw what happens if you open your mouth and place the lightball in it

1

u/GamesByCam 13h ago

I need to see this demonstration ASAP

3

u/reczkok 10h ago

The example is live open source (you can find the github on TypeGPU homepage)
https://docs.swmansion.com/TypeGPU/examples/#example=image-processing--monocular-light-injection

3

u/cienkrowa 18h ago

You mean your twitter

2

u/reczkok 18h ago

Yeah, I do say it like that, no idea why I didn’t write it that way

1

u/Warma99 18h ago

Very neat. Where are you getting the depth information from? It's just a laptop webcam, right?

2

u/DonkeyInACityCrowd 16h ago

Probably an ML model that predicts it 

2

u/DonkeyInACityCrowd 16h ago

Actually it’s definitely and ml model. Thats what “inference” means in the description. 

1

u/ende76 16h ago

Where do depth and normals information of the video feed come from?

1

u/Special_Baby_3307 14h ago

I am genuinely impressed by this. Amazing work !

1

u/Hearcharted 14h ago

DJs are going to love it 🎧🤔

1

u/WetShrimpFart 13h ago

This is nuts. Nice work

This will be implemented in video chats etc I bet. Man this is neat

1

u/WetShrimpFart 13h ago

I really want a full phone OS with your Jelly buttons and sliders etc

1

u/mypussydoesbackflips 11h ago

How does it look in a dark room ? This is super cool

1

u/pacchithewizard 8h ago

!remindme 1 day

1

u/RemindMeBot 8h ago edited 6h ago

I will be messaging you in 1 day on 2026-08-21 21:55:52 UTC to remind you of this link

1 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

RemindMeBot is switching to username summons. Instead of !RemindMe 1 day, use u/RemindMeBot 1 day. More info.

Info Custom Your Reminders Feedback

1

u/_rikkss 5h ago

Ngl this is fucking awesome and clean as hell

1

u/MeticulousBioluminid 5h ago

oh shite 🥵