TY - GEN
T1 - Stochastic Frank-Wolfe methods for nonconvex optimization
AU - Reddi, Sashank J.
AU - Sra, Suvrit
AU - Poczos, Barnabas
AU - Smola, Alex
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2017/2/10
Y1 - 2017/2/10
N2 - We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization due to their projection-free property and their ability to exploit structured constraints. However, our understanding of these algorithms in the nonconvex setting is fairly limited. In this paper, we propose nonconvex stochastic Frank-Wolfe methods and analyze their convergence properties. Furthermore, for objective functions that decompose into a finite-sum, we leverage ideas from variance reduction for convex optimization to obtain new variance reduced nonconvex Frank-Wolfe methods that have provably faster convergence than the classical Frank-Wolfe method.
AB - We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization due to their projection-free property and their ability to exploit structured constraints. However, our understanding of these algorithms in the nonconvex setting is fairly limited. In this paper, we propose nonconvex stochastic Frank-Wolfe methods and analyze their convergence properties. Furthermore, for objective functions that decompose into a finite-sum, we leverage ideas from variance reduction for convex optimization to obtain new variance reduced nonconvex Frank-Wolfe methods that have provably faster convergence than the classical Frank-Wolfe method.
UR - http://www.scopus.com/inward/record.url?scp=85015187744&partnerID=8YFLogxK
U2 - 10.1109/ALLERTON.2016.7852377
DO - 10.1109/ALLERTON.2016.7852377
M3 - Conference contribution
AN - SCOPUS:85015187744
T3 - 54th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2016
SP - 1244
EP - 1251
BT - 54th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2016
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 54th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2016
Y2 - 27 September 2016 through 30 September 2016
ER -