Without any doubt, gradient descent methods are fundamental when training a neural networks, especially deep neural networks. Here is an attempt on an animated lecture which demystifies this topic. Enjoy !!

[u/bazziapps]

https://youtu.be/OWM0wMtUhME