Papers Published
Abstract:
In this paper we present a comprehensive framework for learning robust
low-rank representations by combining and extending recent ideas for learning
fast sparse coding regressors with structured non-convex optimization
techniques. This approach connects robust principal component analysis (RPCA)
with dictionary learning techniques and allows its approximation via trainable
encoders. We propose an efficient feed-forward architecture derived from an
optimization algorithm designed to exactly solve robust low dimensional
projections. This architecture, in combination with different training
objective functions, allows the regressors to be used as online approximants of
the exact offline RPCA problem or as RPCA-based neural networks. Simple
modifications of these encoders can handle challenging extensions, such as the
inclusion of geometric data transformations. We present several examples with
real data from image, audio, and video processing. When used to approximate
RPCA, our basic implementation shows several orders of magnitude speedup
compared to the exact solvers with almost no performance degradation. We show
the strength of the inclusion of learning to the RPCA approach on a music
source separation application, where the encoders outperform the exact RPCA
algorithms, which are already reported to produce state-of-the-art results on a
benchmark database. Our preliminary implementation on an iPad shows
faster-than-real-time performance with minimal latency.