* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Vector Spaces
Cross product wikipedia , lookup
Hilbert space wikipedia , lookup
Determinant wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Laplace–Runge–Lenz vector wikipedia , lookup
Jordan normal form wikipedia , lookup
Geometric algebra wikipedia , lookup
Tensor operator wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Fundamental theorem of algebra wikipedia , lookup
Matrix multiplication wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Euclidean vector wikipedia , lookup
System of linear equations wikipedia , lookup
Vector space wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
Cartesian tensor wikipedia , lookup
Four-vector wikipedia , lookup
Matrix calculus wikipedia , lookup
Bra–ket notation wikipedia , lookup
Vector Spaces Dr. Doreen De Leon Math 152, Fall 2015 1 Vector Spaces Vector Spaces Definition. A vector space is a nonempty set V of objects called vectors on which are defined two operations: vector addition (which combines two elements of V and is denoted by “+”) and scalar multiplication (which combines a complex number with an element of V ), that satisfy the below properties for all u, v, w ∈ V and all scalars α and β. (1) u + v ∈ V (closure under addition) (2) αu ∈ V (closure under scalar multiplication) (3) u + v = v + u (commutativity) (4) (u + v) + w = u + (v + w) (associativity) (5) There is a zero vector 0 ∈ V such that u + 0 = u. (6) For each u ∈ V , there is a vector −u ∈ V such that u + (−u) = 0. (7) α(u + v) = αu + αv (8) (α + β)u = αu + βu (9) α(βu) = (αβ)u (10) 1u = u Recall: We discussed previously that Cn is a vector space. Notes: (1) The scalars do not need to be complex numbers. Scalars are defined depending on the field over which our set is defined. Examples of other fields are R, the set of rational numbers, etc. In that case, we say V is a vector space over the field F . (2) The objects in V can be anything. They do not necessarily need to be column vectors. 1 Examples of Vector Spaces There are many other examples of vector spaces. Here are a few. (1) Pn = the set of all polynomials of degree at most n (i.e., all polynomials of the form p(t) = a0 + a1 t + a2 t2 + · · · + an tn , where ai ∈ C and t ∈ R) is a vector space. • Addition: Let p(t) = a0 + a1 t + a2 t2 + · · · + an tn and q(t) = b0 + b1 t + b2 t2 + · · · + bn tn . Then, (p + q)(t) = p(t) + q(t) = (a0 + b0 ) + (a1 + b1 )t + (a2 + b2 )t2 + · · · (an + bn )tn . • Scalar multiplication: Let p(t) ∈ Pn and let c be a scalar. Then, (cp)(t) = cp(t) = (ca0 ) + (ca1 )t + (ca2 )t2 + · · · + (can )tn . • The zero vector is the zero polynomial, 0 = 0 + 0t + 0t2 + · · · + 0tn . The ten axioms can be “easily” verified. (2) Let F be the set of all real-valued functions defined on R. • Addition: Let f ∈ F and g ∈ F. Then, (f + g)(t) = f (t) + g(t). • Scalar multiplication: Let f ∈ F and let c be a scalar. Then, (cf )(t) = cf (t). • The zero vector for F is the zero function, f (t) = 0 for all t. Properties (1) - (10) can be verified to show that F is a vector space. Example: Property (2). Let f, g ∈ F. Then, (f + g)(t) = f (t) + g(t) = g(t) + f (t) = (g + f )(t). (3) Mmn = the set of all m × n matrices is a vector space. • Addition: matrix addition • Scalar multiplication: matrix scalar multiplication • The zero vector is the zero matrix. 2 Example: Let V be the set of positive real numbers with addition defined by x + y = xy and scalar multiplication defined by cx = xc . Is V a vector space? Solution: Properties (1) - (4) are straightforward to verify, so we will try to verify properties (5)-(10). (5) The zero vector is 1, since for all x ∈ V , x + 1 = x(1) = x. (6) The additive inverse is 0, since for all x ∈ V , x + 0 = x(0) = 0. Problem: 0 ∈ / V. (7) c(x + y) = (x + y)c = (xy)c = xc y c = (cx)(cy) = cx + cy. (8) (c + d)x = xc+d = xc xd = cx + dx. (9) c(dx) = cxd = (xd )c = xdc = xcd = (cd)x. (10) 1x = x1 = x. Since V does not satisfy Property (6), V is not a vector space. Theorem 1. If V is a vector space, then the following properties hold for all vectors v ∈ V and all scalars α. • The zero vector 0 is unique. • The additive inverse −v is unique. • −v = (−1)v. • 0u = 0. • α0 = 0. 3 2 Subspaces If a vector space H is a subset of vectors from a larger vector space V , then H is a subset of V (i.e., H ⊆ V ). Definition. A nonempty subset H of a vector space V is a subspace of V provided that H is istelf a vector space with the operations of vector addition and scalar multiplication as defined in V . Testing Subspaces Theorem 2. A subset H of a vector space V is a subspace of V if H satisfies the following. (a) The zero vector of V is in H. (b) For each u, v ∈ H, u + v ∈ H (closure under addition). (c) For each u ∈ H and each scalar α, αu ∈ H (closure under scalar multiplication). NOTES: Assume that property (a) above is satisfied. * To prove that a subset H of a vector space V is a subspace of V , you need to show that properties (b) and (c) hold for arbitrary vectors in H. * If you believe that a subset H of a vector space V is not a subspace of V , it suffices to find one example showing that either (b) or (c) fails. Examples: (1) Let H = {(x, y) ∈ C2 | x ≥ 0, y ≥ 0}. Is H a subspace of C2 ? Why or why not? (a) 0 = (0, 0) ∈ H. Let u ∈ H, so u = (u1 , u2 ), where u1 ≥ 0 and u2 ≥ 0; let v ∈ H, so v = (v1 , v2 ), where v1 ≥ 0 and v2 ≥ 0; and let α be a scalar. (b) Then, u + v = (u1 + v1 , u2 + v2 ) ∈ H since u1 + v1 ≥ 0 and u2 + v2 ≥ 0. (c) Let u = (1, 1). Then u ∈ H. Let α = −1. Then, αu = (−1, −1) ∈ / H, so H is not closed under scalar multiplication. Therefore, H is not a subspace of C2 . x 3 (2) Let H be the set of all vectors in C of the form y . Is H a subspace of C3 ? Why or why 0 not? 4 0 (a) 0 = 0 ∈ H. 0 u1 v1 Let u ∈ H, so u = u2 ; let v ∈ H, so v = v2 ; and let α be a scalar. 0 0 (b) Then, u1 + v1 u + v = u2 + v2 ∈ H, 0 so H is closed under addition. (c) We have αu1 αu = αu2 ∈ H, 0 so H is closed under scalar multiplication. Therefore, H is a subspace of C3 . (3) Let H = {p ∈ P2 | p(t) = at2 }. Is H a subspace of P2 ? Why or why not? (a) The zero polynomial satisfies 0 = 0t2 , so 0 ∈ H. (b) Let p ∈ H =⇒ p(t) = at2 and let q ∈ H =⇒ q(t) = bt2 . Then, (p + q)(t) = p(t) + q(t) = (a + b)t2 ∈ H, so H is closed under addition. (c) Let p ∈ H =⇒ p(t) = at2 and let α be a scalar. Then, (αp)(t) = α · p(t) = (αa)t2 ∈ H, so H is closed under scalar multiplication. Therefore, H is a subspace of P2 . (4) Let H = {p ∈ P3 | p(0) = 8}. is H a subspace of P3 ? Why or why not? (a) The zero polynomial in P3 is 0 = 0t3 + 0t2 + 0t + 0, which is 0 for all t. So, 0 ∈ / H. Therefore H is not a subspace of P3 . (5) Let H be the set of all upper triangular 2 × 2 matrices. Is H a subspace of M22 ? Why or why not? (a) 0 ∈ H since the zero matrix is upper triangular. 5 a b d e (b) Let A ∈ H =⇒ A = and let B ∈ H =⇒ B = . Then 0 c 0 f a+d b+e A+B = ∈ H, 0 c+f so H is closed under addition. a b (c) Let A ∈ H =⇒ A = and let α be a scalar. Then 0 c αa αb αA = ∈ H, 0 αc so H is closed under scalar addition. Therefore, H is a subspace of M22 . (6) C 1 (R) is the set of all real-valued continuous functions with one continuous derivative defined on R. Is C 1 (R) a subspace of F ? Why or why not? (a) The zero function, f (t) = 0 for all t ∈ R, is in C 1 (R), since f (t) is continuous and f 0 (t) = 0, which is also continuous. Let g, h ∈ C 1 (R), and let c be a scalar. (b) We have that (g + h)(t) = g(t) + h(t) is continous on R, and (g + h)0 (t) = g 0 (t) + h0 (t) is continous on R, so g + h ∈ C 1 (R), and C 1 (R) is closed under addition. (c) We have that (cg)(t) = cg(t) is continous on R, and (cg)0 (t) = cg 0 (t) is continous on R, so cg ∈ C 1 (R), and C 1 (R) is closed under scalar mutliplication. Therefore, C 1 (R) is a subspace of F. Notes: • C n (R) is the set of all real-valued continuous functions with n continuous derivatives defined on R. • C n [a, b] is the set of all functions that are continuous on [a, b] and have n continuous derivatives on [a, b]. (7) H is the set of all symmetric 12 × 12 matrices. Is H a subspace of M12,12 ? WHy or why not? 6 (a) O ∈ H since Ot = O. Let A, B ∈ H =⇒ At = A and B t = B, and let α be a scalar. (b) We have that (A + B)t = At + B t = A + B, so A + B ∈ H and H is closed under addition. (c) We have that (αA)t = αAt = αA, so αA ∈ H and H is closed under scalar multiplication. Therefore, H is a subspace of M12,12 . (8) H is the set of all functions on [0, 1] such that f (0)f (1) ≥ 0. Is H a subspace of F? Why or why not? (a) The zero function, f (t) = 0 for all t ∈ R, satisfies f (0)f (1) = 0, so the zero function is in H. (b) Let g ∈ H =⇒ g(0)g(1) ≥ 0 and let h ∈ H =⇒ h(0)h(1) ≥ 0. We must determine if g + h ∈ H. This does not seem likely. We need only come up with an example where g, h ∈ H, but g + h ∈ / H. 2 Let g(t) = 1 − t . Then g(0) = 1 and g(1) = 0, so g(0)g(1) = 0 and g ∈ H. Let h(t) = t2 − 2t. Then h(0) = 0 and h(1) = −1, so h(0)h(1) = 0 and h ∈ H. But, (g + h)(t) = g(t) + h(t) = 1 − 2t. Since (g + h)(0) = 1 and (g + h)(1) = −1, [(g + h)(0)][(g + h)(1)] = −1 < 0 and g + h ∈ / H. So, H is not closed under addition. Therefore, H is not a subspace of F. Definition. Given the vector space V , the subspaces V and {0} are each called a trivial subspace. Recall: Definition. The null space of an m × n matrix, denoted N (A) (or Nul(A)), is the set of all solutions to the homogeneous system of equations, Ax = 0. We can write N (A) = {x ∈ Cn | Ax = 0}. Theorem 3. Let A be an m × n matrix. Then the null space of A, N (A), is a subspace of Cn . Proof. (a) 0 ∈ N (A) since A0 = 0. (b) Let u, v ∈ N (A). Then Au = 0 and Av = 0. So, A(u + v) = Au + Av = 0 + 0 = 0. Therefore, u + v ∈ N (A) and N (A) is closed under addition. 7 (c) Let u ∈ N (A) and let α be a scalar. Then A(αu) = α(Au) = α0 = 0, so, αu ∈ N (A) and N (A) is closed under scalar multiplication. Therefore, N (A) ⊆ Cn is a subspace of Cn . The Span of a Set Definition. Let v1 , v2 , . . . , vp be vectors in a vector space V and let α1 , α2 , . . . , αp be scalars. Then the linear combination of v1 , v2 , . . . , vp with weights α1 , α2 , . . . , αp is given by α1 v1 + α2 v2 + · · · + αp vp . Definition. Let V be a vector space and S = {v1 , v2 , . . . , vp } a set of vectors in V . Then, the span of S, denoted Span(S), is the set of all possible linear combinations of v1 , v2 , . . . , vp . Theorem 4. If v1 , v2 , . . . , vp are in a vector space V , then Span{v1 , v2 , . . . , vp } is a subspace of V . Proof. (a) The zero vector is in Span(S), because 0 = 0v1 + 0v2 + · · · + 0vp . (b) Let u, w ∈ Span(S). Then there are constants α1 , α2 , . . . , αp and β1 , β2 , . . . , βp such that u = α1 v1 + α2 v2 + · · · αp vp w = β1 v1 + β2 v2 + · · · βp vp Then, u + w = (α1 v1 + α2 v2 + · · · αp vp ) + (β1 v1 + β2 v2 + · · · βp vp ) = (α1 + β1 )v1 + (α2 + β2 )v2 + · · · (αp + βp )vp ∈ Span(S). Therefore, Span(S) is closed under vector addition. (c) Let u ∈ Span(S) and let α be a scalar. Since u ∈ S, we have u = α1 v1 + α2 v2 + · · · αp vp . Then αu = α(α1 v1 + α2 v2 + · · · αp vp ) = (αα1 )v1 + (αα2 )v2 + · · · + (ααp )vp ∈ Span(S). Therefore, Span(S) is closed under scalar multiplication. Since Span(S) ⊆ V contains the zero vector and is closed under vector addition and scalar multiplication, Span(S) is a subspace of V . 8 So, Span{v1 , v2 , . . . , vp } is the subspace spanned by v1 , v2 , . . . , vp . Given any subspace H of V , a spanning set for H is a set of vectors v1 , v2 , . . . , vp in H such that H = Span{v1 , v2 , . . . , vp }. Example: Let W be the set of all vectors of the form 2b + 3c −5b , 2c where b, c are arbitrary complex numbers. Show that W is a subspace of C3 . Solution: Let x ∈ W . Then 2b + 3c 2 3 −5b x= = b −5 + c 0 . 2c 0 2 3 2 −5 , 0 =⇒ W is a subspace of C3 . So, W = Span 0 2 Subspace Constructions Recall: Definition. The column space of an m×n matrix A, C(A), is the set of all linear combinations of the columns of A. If A = a1 a2 · · · an , then C(A) = Span{a1 , a2 , . . . , an }. Theorem 5. The column space of an m × n matrix is a subspace of Cm . Proof. Let A be an m×n matrix. Then C(A) is the set of all linear combinations of the columns of A, or C(A) = Span{a1 , a2 , . . . , an }, where ai ∈ Cm for i = 1, 2, . . . , n. Therefore, C(A) is a subspace of Cm . Note: If v ∈ C(A) for some m × n matrix A, then v = Ax for some x ∈ Cn (because Ax is a linear combination of the column vectors of A). So, C(A) = {b ∈ Cm | b = Ax for some x ∈ Cn }. Examples: 1 2 3 (1) Consider the matrix A = . 4 5 6 (a) If C(A) is a subspace of Ck , what is k? Why? 9 (b) If N (A) is a subspace of Ck , what is k? Why? Solution: (a) k = 2 since the columns of A are in C2 . (b) k = 3, because for Ax to be defined, x must be in R3 . 1 2 −2 (2) Let A = . Is u = ∈ C(A)? In N (A)? 4 8 1 To determine if u ∈ N (A), we must solve Ax = 0. 1 2 | 0 r2 →r2 −4r2 1 2 | 0 −−−−−−→ . 4 8 | 0 0 0 | 0 We see that x2 is free and x1 + 2x2 = 0, so x1 = −2x2 and −2x2 −2 = x2 x= . x2 1 So, u ∈ N (A). 1 We see that C(A) = Span and so u ∈ / C(A). 4 (3) Find a matrix A so that W = C(A), where 2a − b W = 3b | a, b ∈ C . a+b We have that 2a − b x ∈ W =⇒ x = 3b a+b 2 −1 3 =a 0 +b 1 1 −1 2 =⇒ W = Span 0 , 3 . 1 1 2 −1 3 is one such matrix. Therefore, A = 0 1 1 10 3 Linear Independence and Spanning Sets Linear Independence Definition. A set of vectors S = {v1 , v2 , . . . , vp } in a vector space V is linearly independent (LI) if the only linear combination of vectors in S that form the zero vector, i.e., α1 v1 + α2 v2 + · · · αp v = 0, is the combination with α1 = α2 = · · · = αp = 0. Otherwise, the set is linearly dependent (LD). (In other words, S is a linearly dependent set if there exist constants α1 , α2 , . . . , αp not all zero so that α1 v1 + α2 v2 + · · · + αp vp = 0, which is known as a linear dependence relation among v1 , v2 , . . . , vp .) Theorem 6. Let x1 , x2 , . . . , xn ∈ Cn , and let X = x1 x2 · · · xn . Then, {x1 , x2 , . . . , xp } is linearly dependent if and only if X is singular. Proof. The equation c1 x1 + c2 x2 + · · · + cn vn = 0 can be written as Xc = 0. This equation has nontrivial solution if and only if X is singular. Thus, {x1 , x2 , . . . , xp } is linearly dependent if and only if X is singular. Example: Are the vectors 4 2 2 2 , 3 , −5 3 1 2 linearly independent? Why or why not? Solution: To see if the vectors are linearly independent, we first form the matrix whose columns are the vectors and then determine if that matrix is singular. So, we need to check if 4 2 2 2 3 −5 3 1 3 is singular. Since 4 2 2 2 3 −5 = 0, 3 1 3 the matrix is singular and so the vectors are not linearly independent. 1 −2 1 −1 3 0 Example: (Review) Are 2 , 1 , 7 linearly independent? Why or why not? 3 −2 7 11 Solution: We need to determine if the vector equation 1 −2 1 −1 3 0 α1 2 + α2 1 + α3 7 = 0 3 −2 7 has only the trivial solution. We form the augmented matrix and perform Gaussian elimination. 1 −2 1 | 0 1 −2 1 | 0 1 −2 1 | 0 −1 3 0 | 0 1 1 | 0 1 1 | 0 −→ 0 −→ 0 . 2 0 0 1 7 | 0 5 5 | 0 0 0 | 0 3 −2 7 | 0 0 4 4 | 0 0 0 0 | 0 We see that α3 is a free variable, so there are infinitely many nontrivial solutions. Therefore, the vectors are linearly dependent. What about Functions? Matrices? 1 0 3 4 0 0 Example: Are , , linearly independent? 1 1 2 5 −1 6 Solution: We need to determine if the equation 1 0 3 4 0 0 0 0 c1 + c2 + c3 = 2 5 −1 6 1 1 0 0 has only the trivial solution. We have c1 + 3c2 4c2 0 0 = . 2c1 − c2 + c3 5c1 + 6c2 + c3 0 0 Applying the property of matrix equality gives the system c1 + 3c2 4c2 2c1 − c2 + c3 5c1 + 6c2 + c3 =0 = 0 =⇒ c2 = 0 =⇒ c1 = 0 = 0 =⇒ c3 = 0 =0 So, c1 = c2 = c3 = 0 is the only solution. Therefore, the matrices are linearly independent. Example: Are p1 (t) = t2 − 2t + 3, p2 (t) = 2t2 + t + 8, p3 (t) = t2 + 8t + 7 linearly independent? Solution: We need to determine if the equation c1 p1 (t) + c2 p2 (t) + c3 p3 (t) = 0 has only the trivial solution. So, we will solve c1 (t2 − 2t + 3) + c2 (2t2 + t + 8) + c3 (t2 + 8t + 7) = 0 for c1 , c2 , and c3 . Collect terms to obtain (c1 + 2c2 + c3 )t2 + (−2c1 + c2 + 8c3 )t + 3c1 + 8c2 + 7c3 = 0. 12 Equate coefficients of like terms to obtain the system of equations c1 + 2c2 + c3 = 0 −2c1 + c2 + 8c3 = 0 3c1 + 8c2 + 7c3 = 0. This is equivalent to the equation 1 2 1 −2 1 8 c = 0, 3 8 7 which has only the trivial solution if the coefficient matrix is nonsingular. Since 1 2 1 −2 1 8 = 0, 3 8 7 p1 , p2 , and p3 are not linearly independent. Functions in C n−1 [a, b] Let f1 , f2 , . . . , fn ∈ C n−1 [a, b]. We start by searching for a direct way to determine if the functions are linearly dependent. If the functions are linearly dependent, then there exist weights c1 , c2 , . . . , cn , not all zero such that c1 f1 (t) + c2 f2 (t) + · · · cn fn (t) = 0 (1) for each t ∈ [a, b]. Differentiating both sides of (1) with respect to t gives c1 f10 (t) + c2 f20 (t) + · · · cn fn0 (t) = 0. We may continue differentiating to obtain a system of n equations in n unknowns (the ci , i = 1, 2, . . . , n). c1 f1 (t) c1 f10 (t) (n−1) c1 f 1 + ··· + ··· .. . + c2 f2 (t) + c2 f20 (t) (n−1) (t) + c2 f2 (t) + · · · + cn fn (t) + cn fn0 (t) (n−1) + cn f n = 0 = 0 .. . (2) (t) = 0. Writing (2) as a matrix equation which for each t ∈ [a, b] has the same solution set as (2): f1 (t) f2 (t) ··· fn (t) f10 (t) f20 (t) ··· fn0 (t) (3) c = 0. .. .. .. . . . (n−1) f1 (n−1) (t) f2 (t) · · · 13 (n−1) fn (t) So, if the functions are linearly dependent in C n−1 [a, b], for each t ∈ [a, b], the coefficient matrix in (3) is singular. In other words, the determinant of the coefficient matix in (3) is 0 for each t ∈ [a, b]. This leads to the following: Definition. Let f1 , f2 , . . . , fn ∈ C n−1 [a, b]. The Wronskian of f1 , f2 , . . . , fn on [a, b], denoted W [f1 , f2 , . . . , fn ](t), is the determinant of the coefficient matrix in (3). Theorem 7. Let f1 , f2 , . . . , fn ∈ C n−1 [a, b]. If there exists a point t0 ∈ [a, b] such that W [f1 , f2 , . . . , fn ](t0 ) 6= 0 then f1 , f2 , . . . , fn are linearly independent. Examples: (1) Determine if the functions f (x) = cos(πx) and g(x) = sin(πx) are linearly independent in C 1 [0, 1]. f (x) g(x) cos(πx) sin(πx) W [f, g](x) = 0 = f (x) g 0 (x) −π sin(πx) π cos(πx) = π cos2 (πx) + π sin2 (πx) = π 6= 0. Therefore, f and g are linearly independent in C 1 [0, 1]. (2) Determine if f (t) = 1, g(t) = et + e−t , h(t) = et − e−t are linearly independent. f (t) g(t) h(t) 1 et + e−t et − e−t W [f, g, h](t) = f 0 (t) g 0 (t) h0 (t) = 0 et − e−t et + e−t f 00 (t) g 00 (t) h00 (t) 0 et + e−t et − e−t t −t et + e−t 1+1 e − e = 1(−1) t e + e−t et − e−t = (et − e−t )2 − (et + e−t )2 = e2t − 2 + e−2t − (e2t + 2 + e−2t ) = −4 6= 0. Therefore, the functions are linearly independent. Spanning Sets Definition. A subset S of a vector space V is a spanning set of V if V = Span(S). In this case, we typically say that S spans V . Examples: (1) Does the set {x, 1 − x2 , 1 + x2 } span P2 ? Why or why not? 14 1 0 0 1 (2) Does the set , span M22 ? Why or why not? 0 1 1 0 Solution: (1) We need to determine if we can write an arbitrary element in P2 as a linear combination of x, 1 − x2 , and 1 + x2 . So, let p(x) = a + bx + cx2 . we need to determine if there are constants c1 , c2 , and c3 such that c1 x + c2 (1 − x2 ) + c3 (1 + x2 ) = a + bx + cx2 . We have (c2 + c3 ) + c1 x + (c2 − c3 )x2 = a + bx + cx2 . Equating coefficients of like terms gives the system c1 c2 + c3 = a =b c2 − c3 = c. 1 1 We can solve this system directly to obtain c1 = b, c2 = (a + c), c3 = (a − c). Therefore, 2 2 the set spans P2 . (2) We need to determine if we can write any matrix in M22 as a linear combination of the matrices in the set. We suspect not, so we will try to find a matrix that won’t work. Let 1 0 A= . 2 2 Then we have c1 1 0 0 1 1 0 + c2 = , 0 1 1 0 2 2 which gives c1 c2 1 0 = . c2 c1 2 2 By the definition of matrix equality, we obtain c1 c2 c2 c1 =1 =0 =2 = 2, a system with no solution. Therefore, A is not in the span of the set, so the set does not span P2 . 15 Vector Representation We will just introduce this topic now, although we will come back to it later this semester. Theorem 8 (Vector Representation Relative to a Basis). Suppose that V is a vector space and B = {v1 , v2 , . . . , vn } is a linearly independent set of vectors that spans V . Let w be any vector in V . Then there exist unique scalars a1 , a2 , . . . , an such that a1 v1 + a2 v2 + · · · + an vn = w. We will use these constants to define a column vector in Cn with which we can represent w. 4 Bases Definition. Let V be a vector space. A set S = {b1 , b2 , . . . , bp } in V is a basis of V if (i) S is linearly independent, and (ii) S spans V (i.e., V = Span{b1 , b2 , . . . , bp }). Theorem 9 (The Spanning Set Theorem). Let S = {v1 , v2 , . . . , vp } in vector space V and let H = Span{v1 , v2 , . . . , vp }. (a) If vk is a linear combination of the other vectors in S for some k, then the set formed by removing vk from S spans H. (b) If H 6= {0}, then some subset of S is a basis of H. Proof. (a) Without loss of generality, assume that vp is a linear combination of v1 , v2 , . . . , vp−1 . So, vp = a1 v1 + a2 v2 + · · · ap−1 vp−1 for some constants a1 , a2 , . . . , ap−1 . Let x ∈ H. Then, x = c1 v1 + c2 v2 + · · · cp−1 vp−1 + cp vp for some scalars c1 , c2 , . . . , cp−1 , cp . Then, x = c1 v1 + c2 v2 + · · · cp−1 vp−1 + cp (a1 v1 + a2 v2 + · · · ap−1 vp−1 ) = (c1 + a1 )v1 + (c2 + a2 )v2 + · · · + (cp−1 + ap−1 )vp−1 . Therefore, {v1 , v2 , . . . , vp−1 } spans H. (b) If S is linearly independent, we are done. If not, one vector can be written as a linear combination of the others and can be deleted. Repeat this procedure until the set is linearly independent. If at least one vector v 6= 0 remains, then the set is linearly independent and is, thus, a basis of H. 16 Theorem 10. The set of standard vectors for Cn , B = {e1 , e2 , . . . , en }, is a basis for the vector space Cn . We know that B is linearly independent and it is straightforward to show that it is a spanning set for Cn . 1 0 0 1 0 0 0 0 Example: Show that the set B = , , , is a basis for M22 . 0 0 0 0 1 0 0 1 Solution: We need to show that B is linearly independent and spans M22 . To verify that B is linearly independent, we need to show that the vector equation 1 0 0 1 0 0 0 0 0 0 c1 + c2 + c3 + c4 = 0 1 0 0 1 0 0 1 0 0 has only the trivial solution. But this is equivalent to c1 c2 0 0 = , c3 c4 0 0 which has as its unique solution c1 = c2 = c3 = c4 = 0. So, B is linearly independent. a b We next show that B spans M22 . Let A = ∈ M22 . Then, c d a b 1 0 0 1 0 0 0 0 =a +b +c +d . c d 0 0 0 0 1 0 0 1 =⇒ B spans M22 . Therefore, B is a basis for M22 . The set B is known as the standard basis for M22 . Example: Find a basis for the subspace S of P2 , where S = {p ∈ P2 | p(0) = 0}. Solution: An arbitrary element in S is p(x) = a1 x+a2 x2 . This is a linear combination of x and x2 . Therefore, the set {x, x2 } spans S. It is also linearly independent (c1 x+c2 x2 = 0 = 0x+0x2 only if c1 = c2 = 0). Therefore {x, x2 } is a basis for S. Bases and Nonsingular Matrices We have the following awesome theorem. Theorem 11. Suppose that A is an n × n matrix. Then the columns of A form a basis for Cn if and only if A is nonsingular. Proof. First, we assume that the columns of A form a basis for Cn . Then by definition the set of columns is linearly independent. Therefore, A is nonsingular. 17 Next, we assume that A is nonsingular. Then, the set of the columns of A is linearly independent. In addition, we know that C(A) = Cn . Therefore, the columns of A span Cn . Since the columns of A are linearly independent and span Cn , they form a basis for Cn . We can then add this to the theorem of Nonsingular Matrix Equivalences, obtaining. Theorem 12 (Nonsingular Matrix Equivalences, Round 6). Suppose that A is a square matrix. Then the following are equivalent. (1) A is nonsingular. (2) A row reduces to the identity matrix. (3) The null space of A contains only the zero vector (i.e., N (A) = {0}). (4) The linear system LS(A, b) has a unique solution for every b. (5) The columns of A are linearly independent. (6) A is invertible. (7) The column space of A is Cn (C(A) = Cn ). (8) The determinant of A is nonzero, i.e., det(A) 6= 0. (9) The columns of A form a basis for Cn . Orthonormal Bases and Coordinates We have seen that orthogonal sets are automatically linearly independent. We get even nicer properties if the set is orthonormal. Again, we will ignore how we obtain an orthonormal basis for now. One good source of an orthonormal basis is a unitary matrix, since the columns of a unitary matrix are orthonormal. Why? Suppose Q is an n×n orthonormal matrix. Then the n columns of Q are linearly independent. Since Q is invertible, it is nonsingular. Therefore, the columns of Q form a basis for Cn . Check out the following theorem. Theorem 13 (Coordinates and Orthonormal Bases). Suppose that B = {v1 , v2 , . . . , vp } is an orthonormal basis of the subspace W of Cn . For any w ∈ W , we have w = hv1 , wiw + hv2 , wiw + · · · + hvp , wiw. Proof. (Idea) Since B is a basis for W , we can write any vector w ∈ W as a linear combination of v1 , v2 , . . . , vp . So, w = α1 v1 + α2 v2 + · · · + αp vp . We simply compute hvi , wi for i = 1, 2, . . . , p. to see that ai = hv1 , wi for i = 1, 2, . . . , p. 18 Another interesting thing is the following. Theorem 14. Let A be an n × n matrix and B = {v1 , v2 , . . . , vp } be an orthonormal basis of Cn . Define C = {Av1 , Av2 , . . . , Avp }. Then A is a unitary matrix if and only if C is an orthonormal basis of Cn . 5 Dimension Almost every vector space we have discussed has contained infinitely many vectors. We seek some way to measure the “size” of a vector space, which we can use to study the properties of vector spaces. Dimension Before we can make the definition of dimension (at least one that makes sense), we will discuss the following two theorems. Theorem 15 (Spanning Sets and Linear Dependence). Suppose that S = {v1 , v2 , . . . , vp } is a finite set of vectors which spans the vector space V . Then any set of more than p vectors is linearly dependent. The idea of the proof is that we take a set R of m > p vectors and show that R is linearly dependent by writing each vector in the set as a linear combination of the vectors in S, thereby allowing us to find a nontrivial linear combination of the vectors in R that equal the zero vector. Theorem 16. Any two bases of vector space V contain the same number of vectors. Proof. Suppose that we have two bases of V : S, which contains n vectors, and T , which contains m vectors. We know that m ≤ n, since if m > n, then T is linearly dependent. Similarly, we have n ≤ m; otherwise, S is linearly dependent. Since m ≥ n and n ≥ m, we have m = n. Definition. Suppose that V is a vector space and {v1 , v2 , . . . , vn } is a basis of V . Then the dimension of V is defined by dim(V ) = n, and V is said to be finite-dimensional. If V has no finite basis, we say that V has infinite dimension or that V is infinite-dimensional. The dimension of the zero vector space, {0}, is defined to be zero. Examples: 1. dim(R2 ) = 2, dim(R3 ) = 3, . . . , dim(Rn ) = n 2. dim(Cn ) = n 19 3. dim(P1 ) = 2, dim(P2 ) = 3, . . . , dim(Pn ) = n + 1 4. The set C n (R) is infinite-dimensional. 5. The set Mmn has dimension m · n. Examples: (1) Find the dimension of the subspace a + 2b b − c a, b, c ∈ R . H= 5c Solution: a + 2b 1 2 0 v ∈ H =⇒ v = b − c = a 0 + b 1 + c −1 = av1 + bv2 + cv3 . 5c 0 0 5 Therefore, a basis of H is {v1 , v2 , v3 } =⇒ dim(H) = 3. 1 −2 −3 (2) Find dim(H), where H = Span , , . −5 10 15 1 −2 −3 Solution: First, we need to find a basis of C(A), where A = . −5 10 15 1 −2 −3 r2 →r2 +5r1 1 −2 −3 −−−−−−→ . −5 10 15 0 0 0 1 The first column is the only pivot column, so is a basis of C(A), so H = −5 1 Span . Therefore, dim(H) = 1. −5 (3) Find dim(S), where S is the set of 2 × 2 symmetric matrices. Solution: a b 1 0 0 1 0 0 A ∈ S =⇒ A = =a +b +c . b c 0 0 1 0 0 1 Therefore, a basis of S is 1 0 0 1 0 0 , , , 0 0 1 0 0 1 and dim(S) = 3. (4) Find dim(H), where H = {p ∈ P2 | a1 = a2 }. Solution: p ∈ H =⇒ p(t) = a0 + a1 t + a1 t2 = a0 + a1 (t + t2 ). Therefore, a basis of H is {1, t + t2 }, and dim(H) = 2. 20 Rank and Nullity of a Matrix Definition. Suppose that A is an m × n matrix. • The rank of A, rank(A) , is the dimension of the column space of A (so rank(A) = dim(C(A))). • The nullity of A, nullity(A), is the dimension of the null space of A (so, nullity(A) = dim(N (A))). Theorem 17 (Rank-Nullity Theorem). The dimensions of the column space and the row space of an m × n matrix A are equal. This common dimension, rank(A), also equals the number of pivot positions in A and satisfies rank(A) + nullity(A) = n. (4) Proof. rank(A) = dim(C(A)) = number of pivot columns in A = number of pivot columns in an echelon form B of A. Since B has a nonzero row for each pivot, and these rows form a basis of R(A), rank(A) = dim(R(A)). Now, nullity(A) = dim(N (A)) = number of free variables in LS(A, 0) = number of columns of A that are not pivot columns. But, (number of pivot columns) + (number of non-pivot columns) = (number of columns). So, (4) is true. Example: Suppose a 4 × 7 matrix A has rank 2. Find nullity(A), dim(R(A)), and rank(At ). Is C(A) = C2 ? Solution: dim(C(A)) = 2 =⇒ dim(R(A)) = 2. rank(A) + nullity(A) = 7 =⇒ nullity(A) = 5. rank(At ) = dim(C(At )) = dim(R(A)) = 2. C(A) 6= C2 because each column of A is in C4 . 21 Example: Suppose the solutions of a homogeneous system of five equations in six unknowns are all multiples of one nonzero solution. Will the system necessarily have a solution for every possible right-hand side? Why or why not? Solution: Let A be the 5×6 coefficient matrix of the system. Then nullity(A) = dim(N (A)) = 1. So, dim(C(A)) = 6 − 1 = 5. Since C5 is the only subspace of C5 whose dimension is 5, C(A) = C5 . Therefore, every nonhomogeneous system of equations has a solution. What if A were 6 × 5? Then dim(C(A)) = 5 − 1 = 4, so the columns of A do not span C6 . Rank and Nullity of a Nonsingular Matrix Theorem 18. Suppose A is an n × n matrix. Then the following are equivalent. (1) Matrix A is nonsingular. (2) The rank of A is n (i.e., rank(A) = n). (3) The nullity of A is zero (i.e., nullity(A) = 0). Proof. (1) implies (2): If A is nonsingular, then C(A) = Cn . Therefore, dim(C(A)) = n, or rank(A) = n. (2) implies (3): Suppose rank(A) = n. By the Rank-Nullity Theorem, rank(A) + nullity(A) = n. Therefore, nullity(A) = n − rank(A) = 0. (3) implies (1): Suppose nullity(A) = 0. Therefore, a basis for nullity(A) is the empty set. Therefore, N (A) = {0} and A is nonsingular. This gives us the new Nonsingular Matrix Equivalences theorem. Theorem 19 (Nonsingular Matrix Equivalences, Round 7). Suppose that A is a square matrix. Then the following are equivalent. (1) A is nonsingular. (2) A row reduces to the identity matrix. (3) The null space of A contains only the zero vector (i.e., N (A) = {0}). (4) The linear system LS(A, b) has a unique solution for every b. (5) The columns of A are linearly independent. 22 (6) A is invertible. (7) The column space of A is Cn (C(A) = Cn ). (8) The determinant of A is nonzero, i.e., det(A) 6= 0. (9) The columns of A form a basis for Cn . (10) The rank of A is n, rank(A) = n. (11) The nullity of A is 0, nullity(A) = 0. 6 Properties of Dimension Goldilock’s Theorem Theorem 20. Suppose that V is a vector space and S is a linearly independent set of vectors S 0 in V . If w is a vector with w ∈ / Span(S), then the set S = S {w} is linearly independent. Proof. Let S = {v1 , v2 , . . . , vp }. Consider the equation c1 v1 + c2 v2 + · · · + cp vp + cp+1 w = 0. There are two possibilities. If cp+1 = 0, then we are left with c1 v1 + c2 v2 + · · · + cp vp = 0, and since S is linearly independent, we have ci = 0 for i = 1, 2, . . . , p. If cp+1 6= 0, then we may write cp+1 w = −c1 v1 − c2 v2 − · · · cp vp c2 cp c1 v1 + − v2 + · · · + − vp , =⇒ w = − cp+1 cp+1 cp+1 a linear combination of the vectors in S.S This contradicts our assumption that w ∈ / Span(S). 0 Therefore, cp+1 = 0, and the set S = S {w} is linearly independent. We need this a bit later. Another useful theorem is the following. Theorem 21. Suppose V is a vector space with dim(V ) = n. Let S = {v1 , v2 , . . . , vm } be a set of m vectors from V . Then 1. If m > n, then S is linearly dependent. 2. If m < n, then S does not span V . 3. If m = n and S is linearly independent, then S spans V . 23 4. If m = n and S spans V , then S is linearly independent. Proof of 1 and 2. Let B be a basis for V . Since dim(V ) = n, B is a linearly independent set of n vectors that spans V . 1. If S were linearly independent, then B is a smaller set of vectors that spans V , which contradicts the theorem on Spanning Sets and Linear Dependence. Therefore, S must be linearly dependent. 2. If S spanned V , then B is a larger set of vectors that is linearly independent. This also contradicts the theorem on Spanning Sets and Linear Dependence. Therefore, S does not span V . Theorem 22 (The Basis Theorem). Let V be a p-dimensional vector space with p ≥ 1. Any linearly independent set of p vectors in V is a basis of V . Any set of p vectors in V that spans V is a basis of V . The Basis Theorem follows directly by applying Theorem 21. Example: Show that {1 + 2t, t2 − t, t − 1} is a basis of P2 . Solution: Since dim(P2 ) = 3 and the set contains three vectors, it is a basis of P2 if it is linearly independent. c1 (1 + 2t) + c2 (t2 − t) + c3 (t − 1) = 0 c2 t2 + (2c1 − c2 + c3 )t + c1 − c3 = 0. Equate coefficients of like terms. c2 =0 2c1 − c2 + c3 = 0 c1 − c3 = 0. Solve using Gaussian elimination. 0 1 0 | 0 2 −1 1 | r1 ↔r2 2 −1 1 | 0 −− 1 0 | −→ 0 1 0 −1 | 0 1 0 −1 | 1 −1 2 r3 →r3 −r1 1 0 −−−−−−→ 0 0 1 −3 0 1 −1 2 | r1 →r1 −r3 0 −− 1 0 | −−−−→ 0 0 1 0 −1 | | 0 1 −1 2 r3 →r3 −2r2 | 0 −−−−−−→ 0 1 0 | 0 0 0 −3 Back substitution gives us −3c3 = 0 =⇒ c3 = 0 c2 = 0 c2 − c2 + 2c3 = 0 =⇒ c1 = 0. 24 0 0 0 | 0 | 0 . | 0 Since c1 = c2 = c3 = 0 is the only solution, the set is linearly independent. Therefore, it is a basis of P2 . Note: We could also have used the Wronskian to determine if the set is linearly independent. Theorem 23. Let H be a subspace of a finite-dimensional vector space V . Then H is finitedimensional and dim(H) ≤ dim(V ). Proof. Let S be a linearly independent set of vectors in H such that H = Span(S). Then the number of vectors in S cannot exceed dim(V ) or the vectors will be linearly dependent. Therefore, dim(H) ≤ dim(V ). So, you may ask, how do we know when dim(H) < dim(V ) and not dim(H) = dim(V )? Theorem 24. If U and V are subspaces of vector space W such that U ⊆ V (but U 6= V ), then dim(U ) < dim(V ). Proof. Suppose dim(U ) = m and dim(V ) = n. Then U has a basis B of size m. If m > n, then since U ⊆ V , B is linearly dependent, which contradicts the fact that it is a basis for U . If m = n, then U = Span(B) = V , which contradicts the fact that U 6= V . Therefore, m < n is the only possibility. Theorem 25. If U and V are subspaces of vector space W such that U ⊆ V and dim(U ) = dim(V ), then U = V . Ranks and Transposes Theorem 26. If A is an m × n matrix, then rank(A) = rank(At ). Proof. Suppose A is row-equivalent to matrix B in reduced row echelon form, and B has r nonzero rows. Then B has r pivot columns, and so A has r pivot columns. This means that a basis for C(A) consists of the r pivot columns of A and a basis for R(A) consists of the r nonzero rows of B. Using this, we have rank(A) = dim(C(A)) =r = dim(R(A)) = dim(C(At )) = rank(C(At )). 25