TY - GEN
T1 - Parallel and distributed block-coordinate frank-wolfe algorithms
AU - Wang, Yu Xiang
AU - Sadhanala, Veeranjaneyulu
AU - Dai, Wei
AU - Neiswanger, Willie
AU - Sra, Suvrit
AU - Xing, Eric P.
N1 - Publisher Copyright:
© 2016 by the author(s).
PY - 2016
Y1 - 2016
N2 - We study parallel and distributed Frank-Wolfe algorithms; the former on shared memory ma-chines with mini-batching, and the latter in a delayed update framework. In both cases, we perform computations asynchronously whenever possible. We assume block-separable constraints as in Block-Coordinate Frank-Wolfe (BCFW) method (Lacoste-Julien et al., 2013), but our analysis subsumes BCFW and reveals problem- dependent quantities that govern the speedups of our methods over BCFW. A notable feature of our algorithms is that they do not de-pend on worst-case bounded delays, but only (mildly) on expected delays, making them robust to stragglers and faulty worker threads. We present experiments on structural SVM and Group Fused Lasso, and observe significant speedups over competing state-of-the-art (and synchronous) methods.
AB - We study parallel and distributed Frank-Wolfe algorithms; the former on shared memory ma-chines with mini-batching, and the latter in a delayed update framework. In both cases, we perform computations asynchronously whenever possible. We assume block-separable constraints as in Block-Coordinate Frank-Wolfe (BCFW) method (Lacoste-Julien et al., 2013), but our analysis subsumes BCFW and reveals problem- dependent quantities that govern the speedups of our methods over BCFW. A notable feature of our algorithms is that they do not de-pend on worst-case bounded delays, but only (mildly) on expected delays, making them robust to stragglers and faulty worker threads. We present experiments on structural SVM and Group Fused Lasso, and observe significant speedups over competing state-of-the-art (and synchronous) methods.
UR - http://www.scopus.com/inward/record.url?scp=84998881973&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84998881973
T3 - 33rd International Conference on Machine Learning, ICML 2016
SP - 2317
EP - 2340
BT - 33rd International Conference on Machine Learning, ICML 2016
A2 - Weinberger, Kilian Q.
A2 - Balcan, Maria Florina
PB - International Machine Learning Society (IMLS)
T2 - 33rd International Conference on Machine Learning, ICML 2016
Y2 - 19 June 2016 through 24 June 2016
ER -