Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Matrix calculus wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Gaussian elimination wikipedia , lookup
Four-vector wikipedia , lookup
Singular-value decomposition wikipedia , lookup
System of linear equations wikipedia , lookup
Matrix multiplication wikipedia , lookup
Principal component analysis wikipedia , lookup
Jordan normal form wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Distance Metric Learning with Spectral Clustering By Sheil Kumar Spectral Clustering Based on the MinCut Problem Cuts deal with pairwise similarity measures, and thus can capture non-linear relationships. Spectral Clustering Cont. sigma = .1 Spectral Clustering Cont. sigma = .1 Spectral Clustering Cont. sigma = 3 Motivation Finding the sigma parameter automatically and optimally will give us better clustering of the data. It is hard to formulate a RBF distance metric such that the sigma is easily isolatable. Mahalanobis Distance? Defining a Distance Metric 1) The distance metric must represent similarities between data points. 2) Commonly RBF Kernels are used as distance metrics in SC. 3) The Mahalanobis Distance must form positive distance values representing similarity, NOT dissimilarity. The MinCut Problem The about eqn is subject to constraints yi must take on discrete binary values, and yTD1 = 0. If y is relaxed to take on real values, this minimization is equivalent to the eigenvalue system This eqn is easily shown by substituting z = D1/2y. z0 = D1/21 is an eigenvector, with eigenvalue = 0. Minimizing Eigenvectors D-1/2(D-W)D-1/2 is a symmetric semi-positive definite matrix because (D-W) (also known as the Laplacian Matrix) is known to be symmetric semi-positive definite. z0 is the smallest eigenvector of D-1/2(D-W)D-1/2, and all other eigenvectors are perpendicular to it. z1, the second smallest eigenvector has the property z1Tz0 = 0 = y1TD1 Minimizing Eigenvectors Cont. Thus we obtain arg.min zTz0 = 0 : zTD-1/2(D-W)D-1/2z zTz and equivalently: arg.min yTD1 = 0 : yT(D-W)y yTDy Minimizing the second smallest eigenvector solution of this equation is guaranteed to give us the normalized cut solution with the second constraint satisfied. Trace SDP Given that the lambdas are eigenvector solutions to matrix K, we see that minimizing over the lambdas is equivalent to minimizing over the tr(KB). Minimizing our second eigenvector can be rewritten as a Procrustes Problem. B = a weighted outer product of the eigenvectors such that the eigenvectors are normalized, and the weights are in strictly increasing order. Trace SDP Cont. Because we know that we want to only minimize our second smallest and smallest eigenvector, we can set alpha n and n-1 to 1, and the rest to 0. Solving the SDP The K Matrix Solving the SDP Cont. Solving the SDP Some Results (more coming) More results Conclusions Unclear as of right now, whether linear transformations *help* clustering. More interesting Distance Metrics