Publications [#373539] of Jianfeng Lu

Papers Published

1. Agazzi, A; Lu, J; Mukherjee, S, Global optimality of Elman-type RNNs in the mean-field regime, Proceedings of Machine Learning Research, vol. 202 (January, 2023), pp. 196-227
   (last updated on 2024/11/20)

   Abstract:
   We analyze Elman-type Recurrent Reural Networks (RNNs) and their training in the mean-field regime. Specifically, we show convergence of gradient descent training dynamics of the RNN to the corresponding mean-field formulation in the large width limit. We also show that the fixed points of the limiting infinite-width dynamics are globally optimal, under some assumptions on the initialization of the weights. Our results establish optimality for feature-learning with wide RNNs in the mean-field regime.