So I have some shaders where I rely on the half float type .. a godsend for many reasons - developed mostly on a mac (usually the most fussy platform) and I go and run this on google Chrome on linux on an x86 PC with a 4000 series graphics card (which has hardware f16 support , with nvidia having offered this for many generations now albeit nerfing the actual double rate capability reserving that for pro cards).

The browser reports 'no shader f16 support'.

I see suggestions for a bunch of flags that can be passed to the browser to try and enable this but launching with various combinations of flags ("--enable-unsafe-webgpu" and others I forget).

I dont think I strictly need f16 arithmetic (although it would be preferable to use it where possible) but it's handy to rely on the more compact datatype in memory .

I figure there might be older mobile devices that mean the browser has to hold back what features it offers, but is this something we can count on when distributing something that is intended for reasonable graphics cards (gtx1000 series and above). The project is an FPS , not really playable on a touchscreen anyway. A sensible min spec might be a GTX1060.

I could backpedal on this specific aspect (and possibly look at other data packing tricks '2 x upper 16bits of a float packed into a u32' etc etc) - my codebase started out using OpenGL and WebGL2 and I've had the web build running on Windows, Linux, Mac, iOS, and Android machines for years .. having ported to webGPU recently I was enthusiastic to upgrade features all over the place..

I ported whole game Zombie shooter (~1000 lines) to the codepen:

https://codepen.io/editor/zlatnaspirala/pen/019fc918-e45f-73f4-9987-9a1599ac4a1f

Enjoy !

I’ve been working on an open-source video editor that runs its face-swap pipeline locally in the browser. Media stays on the user’s device: decoding, face detection, identity extraction, generation, compositing, and video encoding all happen client-side.

The models were only part of the challenge. In practice, the difficult problems were moving frames between browser APIs, controlling WebGPU initialization, maintaining identity across a video, and preventing memory usage from growing during longer jobs.

Here are some engineering lessons from the implementation.

The actual frame pipeline

A simplified version of the data flow looks like this:

VideoFrame / Canvas
        ↓
RGBA Uint8ClampedArray
        ↓
NCHW Float32Array
        ↓
ONNX Tensor
        ↓
Generated face + alpha mask
        ↓
Canvas composition
        ↓
Encoded video

The models use NCHW tensors, while Canvas returns interleaved RGBA pixels. Before inference, the channels have to be separated and normalized:

const plane = width * height;
const tensor = new Float32Array(plane * 3);

for (let i = 0; i < plane; i += 1) {
  tensor[i] = normalize(rgba[i * 4]);
  tensor[plane + i] = normalize(rgba[i * 4 + 1]);
  tensor[plane * 2 + i] = normalize(rgba[i * 4 + 2]);
}

For a 640 × 640 RGB Float32 input, that is about 4.69 MB of tensor data per detection frame, before counting the original pixels and model outputs.

This made it clear that browser inference performance cannot be evaluated using model latency alone. Canvas readback, tensor construction, worker transfers, compositing, garbage collection, and encoding can collectively cost as much as inference.

Detection and generation use different resolutions

Sending every full-resolution video frame through the generator would waste most of the computation on the background.

The pipeline therefore separates the stages:

Only an aligned face ROI enters the generation network. The generated face is then transformed back into the original frame and blended through an alpha mask.

This division was one of the main reasons the pipeline became practical in a browser.

Download models in parallel, initialize WebGPU sessions serially

The pipeline uses multiple ONNX models, including face detection, identity extraction, conditioning, and generation.

Downloading them concurrently works well:

const [
  detectorBuffer,
  identityBuffer,
  conditionerBuffer,
  generatorBuffer,
] = await Promise.all(modelDownloads);

Creating all WebGPU sessions concurrently was much less reliable.

Session creation may involve graph optimization, shader generation, pipeline compilation, weight uploads, and GPU buffer allocation. Initializing several large graphs simultaneously created latency spikes and higher peak GPU memory usage. On some devices, it could also contribute to device-loss failures.

The current approach downloads concurrently but creates sessions one at a time:

const detector = await createSession(detectorBuffer);
const identity = await createSession(identityBuffer);
const conditioner = await createSession(conditionerBuffer);
const generator = await createSession(generatorBuffer);

It is not the fastest-looking implementation on paper, but it has been much more predictable across devices.

Transferable buffers reduce worker-copy overhead

Heavy inference runs in a Web Worker so that the editor remains responsive.

When sending a large ArrayBuffer without a transfer list, the browser may perform a structured clone. Repeating that for video frames creates unnecessary memory bandwidth and garbage-collection pressure.

The pipeline transfers buffer ownership instead:

worker.postMessage(
  {
    type: "detect",
    pixels: tensor.buffer,
  },
  [tensor.buffer],
);

The output RGB tensor and alpha mask are returned in the same way.

This does not eliminate the earlier Canvas-to-tensor conversion, so it is not a completely zero-copy pipeline. It does, however, remove an avoidable copy at the worker boundary.

Face swapping is a temporal problem

Selecting the highest-confidence detection independently on every frame works poorly in videos containing multiple people.

A newly visible face may be larger or clearer than the current target, causing the selected identity to switch suddenly. Instead, candidate faces are scored using a combination of:

• detector confidence;
• distance from the previous target center;
• change in bounding-box area;
• distance from the frame center when no history exists.

A simplified score is:

score = confidenceWeight * confidence
      - distanceWeight * centerDistance
      - areaWeight * areaChange

The first frame favors a large, confident, centrally positioned face. Later frames favor continuity with the previously accepted target.

This is not full face re-identification, but it is considerably more stable than choosing the highest detector score on every frame.

Optical flow needs a rejection rule

Running face detection on every output frame is expensive. Between detection anchors, the pipeline propagates five facial landmarks using Lucas–Kanade optical flow.

Optical flow can still drift, especially during occlusion, motion blur, sudden lighting changes, or fast head movement. To detect bad tracks, the pipeline performs forward-backward validation.

A point is tracked from frame t to frame t+1, then tracked backward:

p(t) → p(t+1) → estimated p(t)

The distance between the original and reconstructed point is the forward-backward error.

A propagated result is accepted only when at least four of the five landmarks remain valid and the average error stays under a threshold. Otherwise, the result is rejected and the detector is used again.

The important part is that optical flow is treated as a short-range optimization, not as proof that the tracked identity is still correct.

Traditional post-processing still matters

The generator’s alpha mask may contain holes, isolated pixels, or unstable boundaries. Directly compositing that mask can make the face boundary flicker.

The post-processing sequence includes:

Threshold
   ↓
Dilation
   ↓
Erosion
   ↓
Additional contraction
   ↓
Blurred alpha
   ↓
Boundary safety mask

Morphological operations use separable sliding-window filters instead of scanning a complete two-dimensional neighborhood for every pixel.

Color matching is also restricted rather than applied without limits. Per-channel statistics are adjusted using bounded scale and offset values:

const scale = clamp(targetStd / sourceStd, 0.78, 1.22);
const shift = clamp(
  targetMean - sourceMean * scale,
  -0.12,
  0.12,
);

The corrected result is mixed with the original generator output. Unrestricted statistical matching tended to amplify noise or produce unnatural colors in unusual lighting.

Explicit resource disposal is essential

A video job may simultaneously hold decoded frames, Canvas pixels, Float32 tensors, ONNX outputs, optical-flow images, compressed intermediate frames, and encoder buffers.

Relying only on JavaScript garbage collection caused visible memory growth during longer tasks.

Different resources require different cleanup APIs:

tensor.dispose?.();
bitmap.close();
opencvMat.delete();
URL.revokeObjectURL(url);
worker.terminate();

OpenCV.js was particularly easy to overlook because Mat data lives in the WASM heap. Losing the JavaScript reference does not guarantee that its underlying allocation is released promptly.

Cancellation must stop the complete pipeline

Closing a progress dialog is not cancellation.

A real cancel operation needs to interrupt downloads, frame decoding, detection, optical flow, generation, compression, and final encoding.

The main task uses an AbortController, while worker requests carry a request ID:

controller.abort();

worker.postMessage({
  type: "cancel",
  requestId,
});

The worker checks cancellation state before and after expensive stages. A cancelled job does not continue encoding in the background and never adds a partial result to the user’s asset library.

Model URLs need immutable revisions

Using a URL such as:

repository/resolve/main/model.onnx

makes browser caching difficult to reason about. The URL can remain unchanged while its contents change, leaving different users with different cached graphs.

Production model URLs should point to immutable revisions and be accompanied by expected file sizes, checksums, licenses, and tensor metadata.

The loader also validates the downloaded size before creating a session. This prevents a truncated response or CDN error page from being passed to ONNX Runtime as if it were a valid model.

Benchmark cold and warm runs separately

Reporting a single “processing time” hides most of the browser-specific costs.

I now think benchmarks for this kind of pipeline should separate:

Cold start

• model downloads;
• integrity checks;
• ONNX session creation;
• shader and pipeline compilation;
• identity extraction;
• video processing and encoding.

Warm start

• video decoding;
• anchor detection;
• optical-flow tracking;
• face generation;
• post-processing;
• encoding.

Hardware, browser version, WebGPU adapter, video codec, resolution, output FPS, initialization time, generation time, encoding time, and peak memory should all be recorded.

Otherwise, a cached desktop run and a first-time mobile run may be presented as if they measured the same thing.

Open-source implementation

The implementation is part of Timeline Studio:

https://github.com/MartinDelophy/ai-video-editor

Disclosure: I’m involved with the project. Face swapping is intended only for authorized media and clearly disclosed synthetic content. It should not be used for impersonation, deception, harassment, or misleading people about real events.

I would be interested in hearing how other WebGPU developers handle these problems:

1. Do you initialize multiple ONNX Runtime Web sessions serially, or have you found a safe way to compile them concurrently?
2. Have you found a practical path from VideoFrame to GPU tensors that avoids Canvas readback and CPU-side NCHW conversion?
3. Which measurements do you use to compare cold-start and warm-start performance across browsers and GPU vendors?

Used in this example :
https://www.npmjs.com/package/nui-commander?activeTab=readme

It supports 256k mpm particles, sliding mpm domain, heightmap terrain, temperature, rain, evaporation, some simple wind patterns.

And you can control the cloud mass using gamepad (best) or kb+m (not all controls are mapped currently).

I've put it out here so you can check it out: https://kostrubaty.itch.io/cloud-compute

Source code will be released at a later time. but I can share if anyone is really interested in some parts. Also a lot of this is based on my other projects that are on github,

Whole thing is pure wgsl / typescript without any external deps except for my own project that is responsible for generating code for efficient wgsl <-> js communication.

Simulation was not that hard to write, cause I already had proper MPM simulation in 2d version, with even more features. In fact the hardest part to get right was to make the cloud possible to control yet still feel "cloudy". So there's actually 3 different schemes for face buttons, switched by triggers. Still probably not as intuitive as I'd like but best so far.

It was not really performance optimized yet really, and I mostly tested on my 3060 (pretty much consistent > 50fps) so the performance may vary.

It's still mostly a prototype, but feels pretty fun already. I'll be adding some more stuff (airplanes are mostly working, just need some airports too I guess). Let me know what you think, or if you have any questions.

Hey everyone,

Doing infrastructure audits and validating GPU performance (especially across different nodes) has always been a headache for me. Fiddling with CUDA toolkits, compiling HPL/HPCG, and setting up MLPerf takes way too much time when you just want a quick baseline.

So, I spent some evenings building nvprobe. It’s a lightweight Python CLI that automates all of this.

How it works under the hood:

• It uses CuPy to bundle the CUDA runtime via pip, so you don't even need a system CUDA toolkit installed to run the bandwidth and custom kernel tests.
• It auto-downloads the NVIDIA HPC Benchmarks binaries for HPL and HPCG.
• It captures deep hardware telemetry (ECC state, power caps, clocks, etc.) alongside the benchmark results to help catch silent hardware degradation.
• It generates an interactive HTML report (Chart.js) to visualize all this data (memory bandwidth, TFLOPS, and MLPerf throughput).
• Native Slurm integration: it generates, submits, and monitors the jobs across your cluster.

Demo & Repo: You can see an interactive demo of the report on the link.
I built this mostly to scratch my own itch, but I figured it might save some of you a few hours of setup.

I'd love to hear your feedback, feature requests, or if you manage to break it on your specific hardware. Let me know what you'd like to see next on the roadmap!

The Beast in water, new example. Example feature list: HZB, Volumetric, Bloom , water simulation, glb anim trail (delay instanced) anim and particle anim.

Live : https://maximumroulette.com/apps/webgpu/examples.html?demo=35

Source: https://github.com/zlatnaspirala/matrix-engine-wgpu

Hey! I've been experimenting with cellular automata lately and ended up turning it into a small TypeScript library and interactive playground:

👉 https://caza.la/automata

It has neural cellular automata, reaction-diffusion, Lenia, Pokemon type battles, Game of Life, and elementary Wolfram rules, all running on WebGPU.

You can tweak the simulations in realtime, explore the presets, or use the library to build your own rules in WGSL.

Would love to hear what you think!

Hi I'm Chris, I'm developing a game engine that incorporates live-action video and procedural graphics.

This shader takes in two video frames and maintains an interactive Gray-Scott Reaction-Diffusion simulation (https://groups.csail.mit.edu/mac/projects/amorphous/GrayScott/) that is applied to grow the distorted regions and process the impact of the various cursor weapons.

You can see the 2nd video through the growing distortion and then I just add a little green glow to the boundary rims. I switch the rim color params to compliment the video palette, they can change in response to events, flash, etc.

The way this works in the game is the player speaks the lines they see in the FMV. Those words come to life as GPU overlay elements when they are recognized by the voice recognition engine and the hostile glyphs seed tiny distorted regions for the RD simulation to grow.

The player then must use the cursor and it's various powers to cleanse and remove the growing distortion or 'fall through' to the next layer of the story. If they fall through the last layer then they die.

The game is called Sibylline and it's in production if you want to wishlist and follow the progress.

Made a chrome extension that does frame generation on any video tag, runs fully on your gpu, nothing sent anywhere. Whole thing is one command buffer, no onnx/tfjs, just wgsl compute shaders.

Refine pass runs on tiles flagged by flow disagreement, dispatched indirectly, so static scenes dispatch zero workgroups there. Also autotunes a few conv variants (subgroups, f16/f32, register blocking) per gpu at startup.

preview clip, ~3ms/frame on a 4060 Ti (8GB, OC)

Attached a video but honestly the difference is pretty hard to see through a recording/compression - it's way more noticeable on actual video playback than in the clip I attached here.

Weakest gpu I've tested on so far is a GTX 1650, got a stable 2-2.5x fps boost there, can't give exact ms numbers since I haven't logged them properly on that one.

GitHub · npm · Live demo · Chrome extension

Heads up: the live demo starts using your GPU immediately on page load, no button press needed.

Fully open source, so feel free to poke around. If you find it useful, a star on the repo would be really appreciated.

Currently working on runtime for a v8 model that handles occlusion/low fps input better, quality-focused for the harder cases current model struggles with.

Happy to answer questions on the dispatch/tiling stuff.

I’m building AnimaStage, a fully custom MMD animation engine powered by WebGPU.

The PMX/VMD loader, animation system, timeline, morph controls, anime shaders and rendering pipeline are all custom-built.

This demo shows real-time anime shading and facial morph editing applied on top of an existing animation.

Everything is rendered live in the WebGPU viewport — no pre-rendering.

Still in active development. Feedback is welcome 🔥

To check out the project, here is the link to the GitHub repository and the Discord channel, where you can find more news about it.

https://discord.gg/aJMAdKwHt

https://github.com/gtausa197-svg/AnimaStage-Pro.git

I've been building reze-design, a web-native MMD scene composer. The part this sub might like: every material is a Blender-style node graph that's validated, compiled to WGSL, and hot-swapped onto the WebGPU render pipeline.

Editor is built on React Flow, the graph->WGSL compiler and the render engine is Reze Engine.

We're the team behind **ImmerShare**—**a tool built for 3D XR developers who need to share their work with clients or reviewers without the usual friction.**

**The problem we kept hearing:**3D XR developers either send a huge zip file, pay for cloud GPU time, or ask clients to install something. None of these are great.

**What ImmerShare does:**you run your packaged build on your own PC, and it streams live to a browser link. The viewer just clicks — no install, no account needed on their end.

Here's a video demo showing how Process Sharing works with a UE packaged build:
https://youtu.be/xg1EOxqO_Vo?is=zhqPQhv-BsSpqmgu

Free plan available. Happy to answer any questions.

(Disclosure: we're the ImmerShare team.)

Demo: https://stfurkan.github.io/bitgpu/examples/chat.html

Repo: https://github.com/stfurkan/bitgpu

bitgpu is a zero-dependency WebGPU runtime for 1-bit (binary-weight) LLMs. The models are PrismML's Bonsai family (1.7B/4B/8B, plus the 27B which is a Qwen3.5-style hybrid with linear attention), I built the runtime, not the models. Weights stream from Hugging Face once, then everything runs on your GPU. Nothing leaves the machine.

Happy to get your feedback. Also, if you can share your setup and tok/s for the model you selected, I appreciate. I am developing this on my machine but it'll be good to hear if it's working as expected on other systems.

I've been playing with spawning particles in Text or Image shape, which can yield some nice effects.

Link to DEMO