|
| Publications [#361802] of Jianfeng Lu
Papers Published
- Lu, J; Steinerberger, S, Neural collapse under cross-entropy loss,
Applied and Computational Harmonic Analysis, vol. 59
(July, 2022),
pp. 224-241 [doi]
(last updated on 2026/01/15)
Abstract: We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in Rd. We prove that when d≥n−1, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that, as n→∞ with fixed d, the minimizing points will distribute uniformly on the hypersphere and show a connection with the frame potential of Benedetto & Fickus.
|