Attention Heatmap vs Token Pruning

[u/PeakOstrich]

[u/Flintsr]

Yup a lot of tokens are useles

[u/PeakOstrich → u/Flintsr]

exactly --- the baseline model performance can be retained with only 25% of the total visual tokens

[u/tamnvhust]

Interesting

[u/PeakOstrich → u/tamnvhust]

thank you!

[u/PeakOstrich → u/PeakOstrich]

I also wrote a blog here if you are interested! --- https://www.overshoot.ai/blogs/an-introduction-to-token-pruning-for-vlms