- We will consider additional strategies that are helpful for optimizing gradient descent.
- You need to be a member of Data Science Central to add comments!
- We will also take a short look at algorithms and architectures to optimize gradient descent in a parallel and distributed setting.
- Every state-of-the-art Deep Learning library contains implementations of various algorithms to optimize gradient descent (e.g. lasagne’s , caffe’s , and keras’ documentation).
- If you are unfamiliar with gradient descent, you can find a good introduction on optimizing neural networks .

This article was written by Sebastian Ruder. Sebastian is a PhD student in Natural Language Processing and a research scientist at AYLIEN. He blogs about Machi…

Continue reading “An overview of gradient descent optimization algorithms”