Publications [#349557] of Rong Ge

Papers Published

1. Ge, R; Li, Z; Kuditipudi, R; Wang, X, Learning two-layer neural networks with symmetric inputs, 7th International Conference on Learning Representations Iclr 2019 (January, 2019)
   (last updated on 2026/01/16)

   Abstract:
   We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network y = Aσ(Wx) + ξ, where A, W are weight matrices, ξ represents noise, and the number of neurons in the hidden layer is no larger than the input or output, our algorithm is guaranteed to recover the parameters A, W of the ground-truth network. The only requirement on the input x is that it is symmetric, which still allows highly complicated and structured input. Our algorithm is based on the method-of-moments framework and extends several results in tensor decompositions. We use spectral algorithms to avoid the complicated non-convex optimization in learning neural networks. Experiments show that our algorithm can robustly learn the ground-truth neural network with a small number of samples for many symmetric input distributions.