Download A Casual Chat on Convex Optimization in Machine Learning

A Casual Chat on Convex Optimization in
Machine Learning
Data Mining at Iowa Group
Qihang Lin
02/09/2014
Optimization Models in Machine
Learning
•
•
•
•
•
•
LASSO
Graphical LASSO
Matrix Completion
Robust PCA
SVM
……
LASSO
•
•
•
•
•
•
•
Linear Regression
Ridge Regression
L1-Regularized Regression
Elastic Net Regression
Fused LASSO
Group LASSO
Multi-Task LASSO
Graphical LASSO
Matrix Completion
Robust PCA
𝐶 =𝐵+𝐴
𝐶 = 𝑈Λ𝑉 𝑇
http://perception.csl.illinois.edu/matrix-rank/introduction.html#PCA
SVM
Convex Optimization Algorithms
•
•
•
•
Proximal Gradient
Frank-Wolfe (Conditional Gradient)
Alternating Direction Method of Multipliers
Coordinate Descent