Publications [#356943] of Lawrence Carin

Papers Published

1. Chen, L; Bai, K; Tao, C; Zhang, Y; Wang, G; Wang, W; Henao, R; Carin, L, Sequence generation with optimal-transport-enhanced reinforcement learning, AAAI 2020 - 34th AAAI Conference on Artificial Intelligence (January, 2020), pp. 7512-7520, ISBN 9781577358350
   (last updated on 2024/12/31)

   Abstract:
   Reinforcement learning (RL) has been widely used to aid training in language generation. This is achieved by enhancing standard maximum likelihood objectives with user-specified reward functions that encourage global semantic consistency. We propose a principled approach to address the difficulties associated with RL-based solutions, namely, high-variance gradients, uninformative rewards and brittle training. By leveraging the optimal transport distance, we introduce a regularizer that significantly alleviates the above issues. Our formulation emphasizes the preservation of semantic features, enabling end-to-end training instead of ad-hoc fine-tuning, and when combined with RL, it controls the exploration space for more efficient model updates. To validate the effectiveness of the proposed solution, we perform a comprehensive evaluation covering a wide variety of NLP tasks: machine translation, abstractive text summarization and image caption, with consistent improvements over competing solutions.