Optimization

AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

Projection of the radial component in momentum optimizers to slow down the decay of effective step sizes for scale-invariant weights - ___[ICLR 2021](https://iclr.cc/Conferences/2021)___