Math @ Duke

Publications [#338599] of Vahid Tarokh
Papers Published
 Ding, J; Diao, E; Zhou, J; Tarokh, V, A Penalized Method for the Predictive Limit of Learning,
2015 Ieee International Conference on Acoustics, Speech, and Signal Processing (Icassp), vol. 2018April
(September, 2018),
pp. 44144418, IEEE, ISBN 9781538646588 [doi]
(last updated on 2019/04/24)
Abstract: © 2018 IEEE. Machine learning systems learn from and make predictions by building models from observed data. Because large models tend to overfit while small models tend to underfit for a given fixed dataset, a critical challenge is to select an appropriate model (e.g. set of variables/features). Model selection aims to strike a balance between the goodness of fit and model complexity, and thus to gain reliable predictive power. In this paper, we study a penalized model selection technique that asymptotically achieves the optimal expected prediction loss (referred to as the limit of learning) offered by a set of candidate models. We prove that the proposed procedure is both statistically efficient in the sense that it asymptotically approaches the limit of learning, and computationally efficient in the sense that it can be much faster than cross validation methods. Our theory applies for a wide variety of model classes, loss functions, and high dimensions (in the sense that the models' complexity can grow with data size). We released a python package with our proposed method for general usage like logistic regression and neural networks.


dept@math.duke.edu
ph: 919.660.2800
fax: 919.660.2821
 
Mathematics Department
Duke University, Box 90320
Durham, NC 277080320

