Publications [#374520] of Hongkai Zhao

Papers Published

1. Zhang, S; Lu, J; Zhao, H, On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network, Proceedings of Machine Learning Research, vol. 202 (January, 2023), pp. 41452-41487
   (last updated on 2026/01/15)

   Abstract:
   This paper explores the expressive power of deep neural networks through the framework of function compositions. We demonstrate that the repeated compositions of a single fixed-size ReLU network exhibit surprising expressive power, despite the limited expressive capabilities of the individual network itself. Specifically, we prove by construction that L2◦g^◦r◦L1 can approximate 1-Lipschitz continuous functions on [0, 1]^d with an error O(r^−1/d), where g is realized by a fixed-size ReLU network, L1 and L2 are two affine linear maps matching the dimensions, and g^◦r denotes the r-times composition of g. Furthermore, we extend such a result to generic continuous functions on [0, 1]^d with the approximation error characterized by the modulus of continuity. Our results reveal that a continuous-depth network generated via a dynamical system has immense approximation power even if its dynamics function is time-independent and realized by a fixed-size ReLU network.