High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes

arXiv:2511.03952v2 Announce Type: replace-cross Abstract: We develop a high-dimensional scaling limit for Stochastic Gradient Descent with Polyak Momentum (SGD-M) and adaptive step-sizes. This provides a framework to rigourously compare online SGD with some of its popular vari...

🔗 Read more: https://arxiv.org/abs/2511.03952

#News #Software #Math #Policy #AI #Academic