Publications [#371241] of Rong Ge

Papers Published

1. Ge, R; Li, Z; Wang, W; Wang, X, Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization, Proceedings of Machine Learning Research, vol. 99 (January, 2019), pp. 1394-1448
   (last updated on 2026/01/16)

   Abstract:
   Variance reduction techniques like SVRG (Johnson and Zhang, 2013) provide simple and fast algorithms for optimizing a convex finite-sum objective. For nonconvex objectives, these techniques can also find a first-order stationary point (with small gradient). However, in nonconvex optimization it is often crucial to find a second-order stationary point (with small gradient and almost PSD hessian). In this paper, we show that Stabilized SVRG – a simple variant of SVRG – can find an ε-second-order stationary point using only O^e(n^2/3/ε² + n/ε^1.5) stochastic gradients. To our best knowledge, this is the first second-order guarantee for a simple variant of SVRG. The running time almost matches the known guarantees for finding ε-first-order stationary points.