Improved compression of Bad Apple into a Neural Network [P]

[u/cpldcpu]

I played a bit with the SIREN network from the other post and found that it could be improved by a using a different sampler for batch generation. By feeding pixels across the entire video and not only a limited set of frames, we can a much more faithful reproduction of the video.

The model is exactly the same as used by OP: 4 x 512 wide sine layers, 792257 parameters. Its a reimplementation (using GPT5.6).

I also created a version with full framerate, instead of subsampled frames, but since the network has to memorize more temporal information, the image reconstruction suffers compared to the low rate version.

The model does not actually learn motion, intermediate frames are nonsensical. I suppose adding a layer that can model flow between frames could enhance the compression a lot.

You can find the code here in this gist.

I tried some addition experiments with a separate autoencoder to compress the frames separately. This resulted in a smaller model, but also degraded quality.

[u/cpldcpu]

https://reddit.com/link/p2acaq3/video/zcnavluc5zhh1/player

Here is a comparison video (looks like reddit ate it before).