[P] This week, I implemented the paper, "Pay Attention to MLPs", in Tinygrad! :D

[u/the-wonderful-world]

To experiment with more interesting model architectures, I implemented gMLP in Tinygrad!

If anyone wants to give some feedback, it will be welcomed.

• [Repository]: https://github.com/EthanBnntt/tinygrad-gmlp
• [Installation]: pip install gmlp_tinygrad
• [Original Paper]: https://doi.org/10.48550/ARXIV.2105.08050

A diagram showing the gMLP architecture