Publications [#363966] of Lawrence Carin

Papers Published

1. Cong, Y; Zhao, M; Li, J; Chen, J; Carin, L, GO Hessian for Expectation-Based Objectives, 35th AAAI Conference on Artificial Intelligence, AAAI 2021, vol. 13B (January, 2021), pp. 12060-12068, ISBN 9781713835974
   (last updated on 2024/12/31)

   Abstract:
   An unbiased low-variance gradient estimator, termed GO gradient, was proposed recently for expectation-based objectives Eqγ (y)[f(y)], where the random variable (RV) y may be drawn from a stochastic computation graph (SCG) with continuous (non-reparameterizable) internal nodes and continuous/ discrete leaves. Based on the GO gradient, we present for Eqγ (y)[f(y)] an unbiased low-variance Hessian estimator, named GO Hessian, which contains the deterministic Hessian as a special case. Considering practical implementation, we reveal that the GO Hessian in expectation obeys the chain rule and is therefore easy-to-use with auto-differentiation and Hessian-vector products, enabling efficient cheap exploitation of curvature information over deep SCGs. As representative examples, we present the GO Hessian for non-reparameterizable gamma and negative binomial RVs/nodes. Leveraging the GO Hessian, we develop a new second-order method for Eqγ (y)[f(y)], with challenging experiments conducted to verify its effectiveness and efficiency.