Optimization

This article will first introduce gradient descent, and then go through most of popular optimization methods, such as:

• SGD
• RMSprop
• Adam

Gradient Descent