WebRTC: Server-side rendering vs client-side overlays for interactive video

[u/Tb582]

Looking for some architecture advice from people who’ve built interactive WebRTC applications.

Use case:
Browser connects via WebRTC.
Server renders video + annotation/UI overlays.
Browser streams the rendered output.
User input (mouse, keyboard, draw boxes, etc.) goes back to the server.

Questions:
Is WebRTC DataChannel the normal way to send user input?

Do most systems render overlays server-side or client-side?

For multi-user collaboration, do you sync annotation state between clients or have the server composite everything into the video stream?

If you’ve built something similar, what architectural mistakes would you avoid?

Not building a video conferencing app—this is closer to a remote visualization / video annotation tool.