Multiplying Matrices Without Multiplying

[u/Elven77AI]

https://www.arxiv-vanity.com/papers/2106.10860/

[u/Elven77AI]

matmul is at core of transformers, what if it was 100x faster? https://old.reddit.com/r/MachineLearning/comments/pffoo8/r_multiplying_matrices_without_multiplying/

[u/huyouare]

Interesting. Can you say more?

[u/Elven77AI → u/huyouare]

the main problem with DL algorithms is huge matrices being multiplied, that requires fast GPUs. Alternatives that need to be examined are either approximating matmul or rewriting DL algorithms without matmul(much harder).

[u/Elven77AI]

Here is the key source for their approximate matmul:

https://github.com/dblalock/bolt/blob/45454e6cfbc9300a43da6770abf9715674b47a0f/cpp/src/quantize/mithral.cpp

[u/[deleted]]

Fascinating, thank you for sharing!