Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Determinant wikipedia , lookup
Four-vector wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Gaussian elimination wikipedia , lookup
Matrix calculus wikipedia , lookup
Matrix multiplication wikipedia , lookup
Jordan normal form wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Spectral Graph Theory and Applications WS 2011/2012 Lecture 2: Spectra of Graphs Lecturer: Thomas Sauerwald & He Sun Our goal is to use the properties of the adjacency/Laplacian matrix of graphs to first understand the structure of the graph and, based on these insights, to design efficient algorithms. The study of algebraic properties of graphs is called algebraic graph theory. One of the most useful algebraic properties of graphs are the eigenvalues (and eigenvectors) of the adjacency/Laplacian matrix. 1 Definitions Definition 2.1. Let G = (V, E) be an undirected graph with vertex set [n] := {1, . . . , n}. The adjacency matrix of G is an n by n matrix A given by ( 1 if i and j are adjacent Ai,j = 0 otherwise If G is a multi-graph, then Ai,j is the number of edges between vertex i and vertex j. • The sum of elements in every row/column equals the degree of the corresponding vertex. • If G is undirected, then A is symmetric. Example. The adjacency matrix of a triangle is 0 1 1 1 0 1 1 1 0 Definition 2.2. We define the Laplacian matrix L of graph G as follows: if i = j deg(i) −1 if i and j are adjacent Li,j = 0 otherwise where deg(i) is the degree of vertex i. Let D be an n by n diagonal matrix with deg(1), . . . , deg(n) as diagonal elements. We can rewrite L as L = D − A. In particular, if G is d-regular, then L = d · I − A. Given a matrix A, a vector x 6= 0 is defined to be an eigenvector of A if and only if there is a λ ∈ C such that Ax = λx. In this case, λ is called an eigenvalue of A. 1 Definition 2.3 (graph spectrum). Let A be the adjacency matrix of an undirected graph G with n vertices. Then A has n real eigenvalues, denoted by λ1 ≥ · · · ≥ λn . These eigenvalues associated with their multiplicities compose the spectrum of G. Here are some basic facts about the graph spectrum. Lemma 2.4. Let G be any undirected simple graph with n vertices. Then Pn 1. i=1 λi = 0. Pn Pn 2 2. i=1 λi = i=1 deg(i). 3. If λ1 = · · · = λn , then E[G] = ∅. 4. degavg ≤ λ1 ≤ degmax . p 5. degmax ≤ λ1 ≤ degmax . Proof. We only prove the first three items. (1) Since G does have self-loops, the diagonal elements of A are zero. By the definition Pall Pnot n n of trace, we have ı=1 λi = tr(A) = i=1 Ai,i = 0. P P (2) By the properties of matrix trace, we have nı=1 λ2i = tr A2 = ni=1 A2i,i . Since A2i,i is 2 the degree of vertex Pn i, tr(A ) equals the sum of every vertex’s degree in G. (3) Combing i=1 λi = 0 with λ1 = · · · = λn , we have λi = 0 for every vertex i. By item (2), we have deg(i) = 0 for any vertex i. Therefore E[G] = ∅. For a graph G with adjacency matrix A and integer k ≥ 1, Aku,v is the number of walks of length k from u to v . P Let kxkp := ( ni=1 |xi |p )1/p . Then for any 1 ≤ p ≤ q < ∞, it holds that kxkq ≤ kxkp ≤ n1/p−1/q · kxkq . Lemma 2.5. For any graph G with m edges, the number of cycles of length k in G is bounded by O mk/2 . Proof. Let A be the adjacency matrix of G with eigenvaluesPλ1 , . . . ,λn . Thus the number of n k Ck (cycles of length k) in G is bounded by tr Ak /(2k) = i=1 λi /(2k). For any k ≥ 3, it holds that !1/k !1/k !1/2 n n n X X X k k 2 |λi | |λi | λi ≤ ≤ = (2 · m)1/2 . i=1 Hence tr Ak ≤ (2m)k/2 i=1 i=1 and the number of Ck is at most O mk/2 . Example. Some examples for different spectra of graphs: • For the complete graph Kn , the eigenvalues are n − 1 with multiplicity 1 and −1 with multiplicity n − 1. √ √ • For the complete bipartite graph Km,n , the eigenvalues are + mn, − mn and 0 with multiplicity m + n − 2. • For the cycle Cn , the spectrum is 2 cos(2πj/n) (j = 0, 1, . . . , n − 1). Two assumptions that we make throughout the course are as follows: 2 1. We only consider undirected graphs. Note that if G is not undirected, then A and L is not symmetric any more and the eigenvalues of A and L could be complex numbers. A matrix A = (ai,j )n×n is called a Hermitian matrix if ai,j = aj,i for any element ai,j . Hermitian matrices always have real eigenvalues. 2. Unless mentioned otherwise, we consider regular graphs. Lemma 2.6. Consider any undirected graph G with adjacency matrix A. 1. If G is d-regular, then λ1 = d and |λi | ≤ d for i = 2, . . . , n. 2. G is connected iff λ2 < d, i.e., the eigenvalue d has multiplicity 1. Moreover, the number of connected components of G equals the multiplicity of eigenvalue d. 3. If G is connected, then G is bipartite iff λn = −d. Lemma 2.7. All eigenvalues of L are non-negative. Proof. Follows from Lemma 2.6 and the definition of L = D − A. For studying regular graphs, it is convenient to work with the normalized adjacency matrix M of graph G. For any d-regular graph with adjacency matrix A, define M := 1 · A. d Throughout this course, we use λ1 ≥ · · · ≥ λn to denote the eigenvalues of matrix M of graph G. For regular graphs, λ1 = 1 and we mainly consider the second largest eigenvalue in absolute value. The formal definition is as follows. Definition 2.8 (spectral expansion). The spectral expansion of graph G is defined by λ := max {|λ2 |, |λn |}, i.e. λ = max kAxk kxk=1,x⊥u Courant-Fischer Formula. Let B be an n by n symmetric matrix with eigenvalues λ1 ≤ · · · ≤ λn and corresponding eigenvectors v1 , . . . , vn . Then λ1 = min xT Bx = min xT Bx , xT · x λ2 = min xT Bx = min xT Bx , xT · x λn = max xT Bx = max xT Bx . xT x kxk=1 x6=0 kxk=1 x⊥v1 x6=0 x⊥v1 kxk=1 x6=0 It is well known that λ relates to various graph properties. In particular, we shall see that there is a close connection between λ and the expansion of the graph. Lemma 2.9. s λ≥ n−d d(n − 1) 3 . P Proof. Follows from tr M2 = n/d = ni=1 λ2i ≤ 1 + (n − 1)λ2 . Theorem 2.10. [Alo86] √ Any infinite family of d-regular graphs {Gn }n∈N has spectral expansion (as n → ∞) at least 2 d − 1/d − o(1). Definition 2.11 (Ramanujan graphs). A family of d-regular graphs with spectral expansion at √ most 2 d − 1/d is called Ramanujan graphs. Although Friedman [Fri91] showed that random d-regular graphs are close to being Ramanujan in the sense that λ satisfies √ λ ≤ 2 d − 1/d + 2 log(d)/d + o(1), constructing families of Ramanujan graphs with arbitrary degree is one of the biggest open problems in this area. So far, we only know the construction of Ramanujan graphs with certain degrees and these constructions are based on deep algebraic knowledge. See [LPS88] for example. Another quite important problem is to find a combinatorial construction of Ramanujan graphs. At the end of this section, we list some more interesting facts on eigenvalues of graphs: 1. If graphs G and H are isomorphic, then there is a permutation matrix P such that P · A(G) · PT = A(H) and hence the matrices A(G) and A(H) are similar. 2. There are nonisomorphic graphs with the same spectrum. See Figure 1. Figure 1: An example for two graphs which are not isomorphic but have the same spectrum. Their common graph spectrum is 2, 0, 0, 0, −2. 2 Combinatorial Expansion of Graphs For any d-regular graph G = (V, E), let Γ(v) be the set of neighbors of v, i.e., Γ(v) = {u | (u, v) ∈ E }. For any subset S ⊆ V , let Γ(S) = ∪v∈S Γ(v) and Γ0 (S) = Γ(S) ∪ S. Moreoever, for any set S ⊆ V we define ∂S := E(S, S). Definition 2.12 (vertex expansion). A graph G with n vertices is said to have vertex expansion (K, A) if |Γ(S)| min ≥ A. S : |S|≤K |S| If K = n/2, then for simplicity we call G an A-expander. Informally expanders are graphs with the property that every subset (under some constraint on their size) has many neighbors outside the set. Moreover, we can use different ways to study expanders: (1) Combinatorically, expanders are highly connected graphs, and to disconnect a large part of the graph, one has to remove many edges; (2) Geometrically, every vertex set has a relatively very large boundary; (3) From the Probabilistic view, expanders are graphs whose behavior is “like” random graphs. (4) Algebraically, expanders are the real-symmetric matrix whose first positive eigenvalue of the Laplace operator is bounded away from zero. 4 Figure 2: Comparison of the vertex expansion and the edge expansion of a set of vertices of size 5. Definition 2.13 (edge expansion). The edge expansion of a graph G = (V, E) is defined by h(G) := |∂S| . S : |S|≤|V |/2 |S| min To explain edge expansion, let us see two examples. (1) If G is not connected, we choose one connected component as S so that |E(S, S)| = 0. Therefore h(G) = 0. (2) If G is a complete graph Kn , then |E(S, S)| = |S| · (n − |S|) and h(G) = dn/2e. Definition 2.14 (expanders). Let d ∈ N. A sequence of d-regular graphs {Gi }i∈N of size increasing with i is a family of expanders if there is a constant > 0 such that h(Gi ) ≥ for all i. Usually, when speaking of an expander Gi , we actually mean a family of graphs {Gi }i∈N , where each graph in {Gi }i∈N is d-regular and its expansion is lower bounded by > 0. Observation 2.15. Any expander graph is a connected graph. 3 Spectral Expansion vs. Combinatorial Expansion The next result shows that small spectral expansion implies large vertex expansion. Theorem 2.16 (spectral expansion ⇒ vertex expansion). If G has spectral expansion λ, then 1 for all 0 < α < 1, G has vertex expansion αn, (1−α)λ2 +α . Before showing the proof, we introduce some notations at first. For any probability distribution π, the support of π is defined by support(π) = {x : πx > 0}. Definition 2.17. Given a probability distribution π, the collision probability ofP π is defined to be the probability that two independent samples from π are equal, i.e. CP(π) = x πx2 . Lemma 2.18. Let u = (1/n, . . . , 1/n) be the uniform distribution. Then for every probability distribution π ∈ [0, 1]n , we have 1. CP(π) = ||π||2 = ||π − u||2 + 1/n. 2. CP(π) ≥ 1/|support(π)|. Proof. (1) We write π as π = u + (π − u) where u⊥(π − u). By Pythagorean theorem CP(π) = ||π||2 = ||π − u||2 + ||u||2 = ||π − u||2 + 1/n. (2) By Cauchy-Schwarz inequality, we get 2 1= X πx ≤ |support(π)| · X x x∈support(π) 5 πx2 and hence CP(π) = X πx2 ≥ x 1 . |support(π)| Let us turn to the proof of Theorem 2.16. Proof. Let |S| ≤ αn. Choose a probability distribution π that is uniform on S and 0 on the S, i.e. 1 1 1 π= , ,..., , 0, . . . , 0 . |S| |S| |S| Note that M is a real symmetric matrix, then M has n orthonormal eigenvectors v1 , . . . , vn , then Pn we can decompose π as i=1 πi where πi is a constant multiplicity of vi . Then CP(π) = 1/|S| and by Lemma 2.18 (2), CP(Mπ) ≥ 1 1 = . |support(Mπ)| |Γ(S)| On the other hand, by item (1) of Lemma 2.18 we have CP(Mπ) − 1 = kMπ − uk2 n = kMu + Mπ2 + · · · + Mπn − uk2 = kλ2 π2 + · · · + λn πn k2 1 1 1 2 2 2 2 ≤ λ kπ − uk = λ CP(π) − =λ − . n |S| n Hence 1 1 1 − ≤ CP(Mπ) − ≤ λ2 |Γ(S)| n n 1 1 − |S| n , and 1 |Γ(S)| ≥ λ2 ≥ λ2 1 |S| − 1 n + 1 n = |S| λ2 1 − |S| n + |S| n = |S| λ2 + (1 − λ2 ) · |S|/n |S| |S| = . 2 + (1 − λ )α α + (1 − α)λ2 Theorem 2.19 (vertex expansion ⇒ spectral expansion). Let G be a d-regular graph. For every δ > 0 and d > 0, there exists γ > 0 such that if G is a d-regular (1 + δ)-expander according to Definition 2.12, then it G has spectral expansion (1 − γ). Specifically, we can take γ = Ω(δ 2 /d). When talking about expanders, we often mean a family of d-regular graphs satisfying one of the following two equivalent properties: • Every graph in the family has spectral expansion λ. • Every graph in the family is a (1 + δ)-expander for some constant δ. 6 References [Alo86] N. Alon. Eigenvalues and expanders. Combinatorica, 6(2):83–96, 1986. [Fri91] Joel Friedman. On the second eigenvalue and random walks in random d-regular graphs. Combinatorica, 11:331–362, 1991. [LPS88] A. Lubotzky, R. Phillips, and P. Sarnak. Ramanujan graphs. Combinatorica, 8:261– 277, 1988. 7