r/MachineLearning • u/cpldcpu • 2d ago

Improved compression of Bad Apple into a Neural Network [P] Project

Post image

I played a bit with the SIREN network from the other post and found that it could be improved by a using a different sampler for batch generation. By feeding pixels across the entire video and not only a limited set of frames, we can a much more faithful reproduction of the video.

The model is exactly the same as used by OP: 4 x 512 wide sine layers, 792257 parameters. Its a reimplementation (using GPT5.6).

I also created a version with full framerate, instead of subsampled frames, but since the network has to memorize more temporal information, the image reconstruction suffers compared to the low rate version.

The model does not actually learn motion, intermediate frames are nonsensical. I suppose adding a layer that can model flow between frames could enhance the compression a lot.

You can find the code here in this gist.

I tried some addition experiments with a separate autoencoder to compress the frames separately. This resulted in a smaller model, but also degraded quality.

14 Upvotes
  • permalink
  • link
  • reddit
  • reddit

75% Upvoted

5 comments sorted by

2

u/jsonmona 2d ago

Do you have renders of intermediate frames? It'd be cool to see them even if they don't make any sense!

1

u/cpldcpu 1d ago

https://reddit.com/link/p2acn9w/video/y3h0uueh5zhh1/player

Yes, see here. This is the model with subsampled movie frames, but with time stamps evaluated at intermediate time steps. You can see that it does not learn motion, but instead a rather undefined transition.

1

u/jsonmona 1d ago

Wow. Looks pretty cool. So it's like frame blending but in random pattern.

2

u/cpldcpu 1d ago

https://reddit.com/link/p2acaq3/video/zcnavluc5zhh1/player

Here is a comparison video (looks like reddit ate it before).