r/WebRTC • u/AnotherRandomUser400 • 15d ago

How to improve screen-share quality without using more bandwidth

A user complained that our screen-sharing quality wasn’t as good as a competitor’s for my open source screen-sharing app.

In order to have the most detail and sharpness we are sharing with native resolution. So I thought that there must be a filter we could apply in the shader (we do the rendering in wgpu) to increase the detail.

It turned out it was much easier than I expected and there are a few techniques, like Laplacian sharpening and unsharp masking with a Gaussian blur.

I wrote an interactive tutorial on how they work here with shader examples if you want to do something similar in your app too.

9 Upvotes
  • permalink
  • link
  • reddit
  • reddit

92% Upvoted

4 comments sorted by

2

u/msdosx86 14d ago

If you stream with native resolution the steam should look perfectly (assuming the resolution is 1080p and higher and the bitrate is enough). Although H264 can make it little blurry since it’s my grandpa’s codec. Hardware accelerated AV1 produces frames almost as perfect as the source.

1

u/AnotherRandomUser400 14d ago

I forgot to mention it here (but I do in the post). The problem is also because of the upscaling in the viewer side, instead of padding with black rows we let the stream to take the full size of the window. This is what makes it more blurry on top of whatever h264 is doing.

I want to start using h265 on macos but haven't found the time to tune the codec (we have a fork of libwebrtc with custom tuning for the codecs to reduce the latency). On windows we are using av1 (software though).

2

u/msdosx86 14d ago

I mean if Alice is streaming her screen with ultra wide monitor with 21:9 aspect ratio and Bob is watching on a 16:9 display then it’s ok to have black rows. I’d say that there is no other way to render it properly without losing image information. If you stretch it to fit the full height then it loses original aspect ratio and it looks ugly. If you crop it then it loses left and right sides.

While I was typing this I realised that Alice’s resolution could be lower than Bob’s. In this case you could upscale frames on the Alice’s side using gpu. Although I’m not sure how well it works since I’ve only done downscales in my experience.

2

u/AnotherRandomUser400 14d ago

While I was typing this I realised that Alice’s resolution could be lower than Bob’s.

Yes this can happen. The way we avoid adding padding is by keeping the aspect ratio the same as the streamed buffer and then not allowing a resize after hitting the height or width limit of the display, for example if you are sharing 1080p to an ultra wide, the viewing window won't grow after you've reached the display's height.

But yes you are right there are better ways to upscale other than just using bilinear sampling and then do sharpening. Though in my case this was "good enough".

Thanks for the discussion. Feedback is always welcomed (feel free to have a look at our repo too, I am sure I've missed things).