Unifying Low Dimensional Observations in Deep Learning Through the Deep Linear Unconstrained Feature Model
arXiv:2404.06106v2 Announce Type: replace Abstract: Empirical studies have revealed low dimensional structures in the eigenspectra of weights, Hessians, gradients, and feature vectors of deep networks,...