* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Linear Algebra Basics A vector space (or, linear space) is an
Survey
Document related concepts
Hilbert space wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Fundamental theorem of algebra wikipedia , lookup
Cross product wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Tensor operator wikipedia , lookup
System of linear equations wikipedia , lookup
Exterior algebra wikipedia , lookup
Laplace–Runge–Lenz vector wikipedia , lookup
Geometric algebra wikipedia , lookup
Matrix calculus wikipedia , lookup
Euclidean vector wikipedia , lookup
Four-vector wikipedia , lookup
Vector space wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
Cartesian tensor wikipedia , lookup
Bra–ket notation wikipedia , lookup
Transcript
Linear Algebra Basics A vector space (or, linear space) is an algebraic structure that often provides a home for solutions of mathematical models. Linear algebra is a study of vector spaces and a class of special functions (called “linear transformations”) between them. The terms vector space and linear space are interchangeable. We’ll use them both. 1. Linear Spaces: The elements of a linear space are called vectors. We can scale and add vectors. By scaling and adding vectors we can build new vectors. This simple construction paradigm is remarkably useful. Definition: A linear space (or vector space) is a set L together with field of scalars F and two functions (operations); addition + : L × L → L and (scalar) multiplication · : F × L → L satisfying conditions (a)-(h) below. For us, the scalar field F will almost always be the real numbers R (rarely, for us, it may be the complex numbers C). Also, for brevity (clarity?), if α ∈ F and x ∈ L and we want to scale x using α, then we’ll write αx rather than α · x. (a) x + y = y + x for x, y in L (b) x + (y + z) = (x + y) + z for x, y, z ∈ L (c) there is an element O ∈ L such that for each x ∈ L, x + O = x (d) for each x ∈ L there corresponds y ∈ L such that x + y = O (e) for each α, β ∈ F and x ∈ L, α(βx) = (αβ)x (f) for each α, β ∈ F and x ∈ L, (α + β)x = αx + βx (g) for each α ∈ F and x, y ∈ L, α(x + y) = αx + αy, and (h) 1x = x for each x ∈ L. Remarks: • If x + y = x + z then y = z. This cancellation law holds because y = O + y = (−x + x) + y = −x + (x + y) = −x + (x + z) = (−x + x) + z = O + z = z. • 0x + x = (0 + 1)x = x so 0x = O. • The zero element O of (b) and y, the additive inverse of x, in (c) are unique; we usually write −x for the additive inverse of x. Since −1x + x = −1x + 1x = (−1 + 1)x = 0x = O then −x = −1x. • For us, the scalar field F will always be either the real numbers R or complex numbers C. There are other fields however, for example the rational numbers Q. There are finite fields, e.g., Zp (the integers mod p) where p is a prime number. Vector spaces over finite fields find applications in, for example, crytography. We’ll not consider them in this course. Examples of linear spaces: (a) Rn : A vector x ∈ Rn is an n-tuple of real numbers, x = (x1 , x2 , . . . , xn ). The set of real numbers R is the scalar field. Vector additon and scalar multiplication are defined component-wise: if x and y are vectors and α ∈ R then x + y = (x1 , x2 , . . . , xn ) + (y1 , y2 , . . . , yn ) := (x1 + y1 , x2 + y2 , . . . , xn + yn ) and αx = α(x1 , x2 , . . . , xn ) := (αx1 , αx2 , . . . , αxn ). (b) Cn : A vector z ∈ Cn is an n-tuple of complex numbers, z = (z1 , z2 , . . . , zn ). The set of complex numbers C is the scalar field. Vector addition and scalar multiplication are defined component-wise: if w and z are vectors and α ∈ C then w + z = (w1 , w2 , . . . , wn ) + (z1 , z2 , . . . , zn ) := (w1 + z1 , w2 + z2 , . . . , wn + zn ) and αz = α(z1 , z2 , . . . , zn ) := (αz1 , αz2 , . . . , αzn ). (c) Rnp : If p ∈ Rn then define Rnp := {(p, x) | x ∈ Rn }. Rnp (vectors based at p) becomes a real vector space if addition and scalar multiplication are defined by (p, x) + (p, y) = (p, x + y) and α(p, x) = (p, αx) for each scalar α ∈ R and pair of vectors x, y ∈ Rn . (d) Mm×n : A vector is a rectangular m × n array of real numbers. Vector addition and scalar multiplication are defined entry-wise: If A and B are in Mm×n and α ∈ R then A + B and αA are defined to be the m × n matrices whose entry in the ith row and j th column are (A + B)(i, j) := A(i, j) + B(i, j) and (αA)(i, j) := αA(i, j). For example, M2×2 : a b M2×2 = | a, b, c, d ∈ R c d a b e f a+e b+f a b αa αb + = and α = c d g h c+g d+h c d αc αd Note: allowing entries and scalars to be complex numbers turns Mm×n into a complex vector space. (e) C(I; R), I an interval of real numbers: A vector is any continuous real valued function f : I → R. The scalars are real numbers. Functions f and g are added and scaled “component-wise” or pointwise, i.e., f + g and αf are functions whose value at x ∈ I is (f + g)(x) := f (x) + g(x) and (αf )(x) := αf (x). (f) Π: A vector is any polynomial on R. Vector addition and scalar multiplicaton are defined pointwise, as in C(I; R). (g) Πn : A vector is any polynomial of degree n on R. Vector addition and scalar multiplicaton are defined pointwise, as in C(I; R). (h) BC(R; R): A vector is any bounded continuous function f : R → R. Addition and scalar multiplication are defined pointwise. (i) l2 : AX vector x is a sequence of real numbers that is sqaure sum-able: so x ∈ l2 if and only if x = (x1 , x2 , . . . , xk , . . . ) and x2i < ∞. Addition and scalar multiplication are, as usual, defined component-wise. i≥1 p (j) l (for p ≥ 1): A vector x is a sequence x = (x1 , x2 , . . . , xk , . . . ) of real numbers such that and X |xi |p < ∞. i≥1 Addition and scalar multiplication are, as usual, defined component-wise. (k) l∞ : A vector x is a bounded sequence of real numbers. More precisely, x = (x1 , x2 , . . . , xk , . . . ) ∈ l∞ if and only if there is an M > 0 (a bound) such that |xi | < M for each i = 1, 2, . . . . Addition and scalar multiplication are, as usual, defined component-wise. (l) c: A vector x is a convergent sequence of real numbers. Addition and scalar multiplication are defined componentwise. (m) c0 : A vector x is a convergent sequence of real numbers with limit 0. Addition and scalar multiplication are defined component-wise. By scaling and adding vectors in L we obtain other vectors in L. For example, in R2 we can generate (2, 2) by combining (1, 0) and (0, 1) as follows: 2(1, 0) + 2(0, 1). In fact, given any vector (a, b) in R2 we can write (a, b) = a(1, 0) + b(0, 1). The right hand side of this last equation is called a linear combination of the vectors (0, 1) and (0, 1) and we’ve just shown that any vector from R2 is a linear combination of these two vectors: {(0, 1), (1, 0)}. By contrast, (2, 2) is not a linear combination of (1, 3) and (−2, −6) (try to do it). It turns out that the set {(1, 3), (−2, −6)} can only generate vectors along the line described by y = 3x (by plotting the two vectors you should be able to see this). Definition(s): Assume that L is a linear space. 0) Subpaces: A subset M of L is called a subspace of L provided M, together with (vector) addition and scalar multiplication it inherits from L, is itself a vector space. Several examples of subspaces are listed in section 2. 1) Linear combinations: If v1 , v2 ,. . . ,vn are vectors from L and α1 , α2 , . . . , αn are scalars then the vector α1 v1 + α2 v2 + · · · + αn vn is called a linear combination of v1 ,. . . ,vn . 2) The span of a set of vectors: If S ⊂ L then span(S) is defined to be the set of all linear combinations of vectors in S. That is, k X span(S) = αj vj α1 , . . . , αk ∈ F and v1 , . . . , vk ∈ S . j=1 span(S) is a subspace of L. It’s called the span of S. 3) Linear independence: If v1 , v2 ,. . . ,vn are vectors in L and there are scalars α1 , α2 , . . . , αn , not all zero, such that α1 v1 + · · · + αn vn = 0 then the vectors v1 , v2 ,. . . ,vn are said to be linearly dependent. On the other hand, if the only way to make the linear combination α1 v1 + · · · + αn vn vanish is by choosing α1 = α2 = · · · = αn = 0 then the vectors v1 , v2 ,. . . ,vn are said to be linearly independent. 2 4) Finite dimensional vector spaces: If S ⊂ L is a finite set and span(S) = L then L is said to be finite dimensional. Otherwise, the dimension of L is said to be infinite. 5) A basis for a finite dimensional vector space: If B ⊂ L is a finite set of linearly independent vectors and span(B) = L then B is called a basis set (or just a basis) for L. It turns out (see below) that the number of vectors in any basis for L is the same. This number is called the dimension of L. Remark: If S = {v1 , . . . , vn } is a linearly independent set of vectors in L and v ∈ span(S) then v = α1 v1 +· · ·+αn vn for some scalars α1 , . . . , αn . This linear combination is unique; that is, suppose v = β1 v1 +· · ·+βn vn then α1 v1 +· · ·+αn vn = β1 v1 +· · ·+βn vn or, equivalently, (α1 −β1 )v1 +· · ·+(αn −βn )vn = 0. Since the vectors v1 , . . . , vn are linearly independent then αk − βk = 0 for each k, that is, β1 = α1 , β2 = α2 , etc. Exercise 1: (a) If S = {(1, 1)} then argue that span(S) 6= R2 , i.e., S is not a basis for R2 . (Find a vector not in span(S)). (b) Describe all vectors in the span of {(1, 0, 1), (0, 0, 1)}. (Draw xyz-axes and try to visualize span{(1, 0, 1), (0, 0, 1)}.) (c) If {v1 , . . . , vn } is a linearly dependent set of vectors then show that, for some index k, vk must be a linear combination of v1 , . . . , vk−1 , vk+1 , . . . , vn . That is, in a linearly dependent set, at least one of the vectors can be built up from the others. (d) Show that {(1, 0, 0), (1, 1, 0), (1, 1, 1)} is a linearly independent subset of vectors in R3 . (e) Argue that if vk = O for some k then {v1 , . . . , vn } is a linearly dependent set of vectors. (f) Set q0 (x) ≡ 1 and, for each k ∈ N, let qk be the polynomial qk (x) = xk . Show that {q0 , q1 , . . . , qn } is a linearly independent subset of Π for each n ∈ N. Hint: Remember, the zero element O of Π is the the zero function O(x) ≡ 0. Use either calculus or the fundamental theorem of algebra to show that α0 = α1 = · · · = αn = 0 if 0 = α0 + α1 x + α2 x2 · · · + αn xn for all x ∈ R. (g) If S is a linearly independent set of vectors then any subset of S is also linearly independent. (h) If B = {v1 , . . . , vn } is a basis for L and w ∈ L then {w, v1 , . . . , vn } is a linearly dependent set of vectors. Theorem 1. If B = {v1 , . . . , vn } is a basis for L and S ⊂ L contains m vectors with m > n then S is a linearly dependent set. Corollary 1. Any two bases of L contain the same number of vectors. Here is an outline for a proof of Theorem 1: (a) It’s enough to prove this result for m = n + 1. So let S = {w1 , . . . , wn+1 } be a subset of L. (b) If any wj is the zero vector then S is linearly dependent (why?). So assume wj 6= 0 for all j. X (c) w1 = αj vj for (unique) scalars α1 , . . . αn (why?). j≤n (d) For some index k1 , αk1 6= 0 (why?). So, vk1 = X αj 1 w1 − vj (why?). αk1 αk1 j6=k1 (e) {w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } is a basis for L. This means two things: i. span{w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } = L: if u ∈ L then there are scalars β̂1 , β1 , . . . , βk−1 , βk+1 , . . . , βn such that u = β̂1 w1 + β1 v1 + · · · + βk−1 vk−1 + βk+1 vk+1 + · · · + βn vn . ii. The set of vectors {w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } is linearly independent: the condition β̂1 w1 + β1 v1 + · · · + βk−1 vk−1 + βk+1 vk+1 + · · · + βn vn = 0 (A) implies β̂1 = β1 = · · · = βk−1 = βk+1 = · · · = βn = 0. (Observe that if (A) holds then we must have β̂1 = 0 [why?] and, hence, the rest of the βj ’s vanish [why?]). (f) Now, by modifying the steps appropriately, “repeat” (c), (d) and (e) using w2 in place of w1 and the “new” basis {w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } in place of {v1 , . . . , vn }. P For example, (c) would become w2 = γ1 w1 + j6=k1 αj vj for some scalars γ1 and αj , j 6= k1 . And then in (d): if all coefficients αj of vj (j 6= k1 ) are 0 then w2 will be a multiple (linear combination) of w1 and hence, S is linearly dependent and you can stop. If at least one of the αj ’s , say αk2 , is nonzero and vk2 can be “solved” for in terms of the other vectors, that is, vk2 depends on {w1 , w2 , v1 , . . . , vn } \ {vk1 , vk2 }, a basis for L. (g) Continue in this fashion until either one of two things happen: (1) there will be an index p ≤ n such that wp is a linear combination of the basis vectors w1 , . . . , wp−1 , in which case S is linearly dependent, or (2) {w1 , . . . , wn } is a basis, in which case S is linearly dependent. 3 2. Subspaces of Linear Spaces: If L is a vector space then, as it turns out, to check that a subset M ⊂ L is actually a subspace of L we need only determine whether or not M is “closed” under linear combinations. More precisly, M will be a subspace if αx + βy ∈ M whenever α and β are scalars and x, y ∈ M. See Theorem 2 and Exercise 2 below. Here are some examples of subspaces: (a) Any plane in R3 containing the origin is a subspace of R3 . (b) c0 is a subspace of c. (c) BC(R; R) is a subspace of C(R; R). (d) Πn is a subspace of Π which, in turn, is a subspace of C(R; R). (e) Define the set D := {f ∈ C([a, b]; R) | f 0 exists and is continuous on [a, b]}. D is a subspace of C([a, b]; R). (f) The set {0} consisting of just the zero vector of L is always a subspace of L. (g) If S and T are two subspaces of L then so is S ∩ T, the set of all vectors in L common to both S and T. (h) If S ⊂ L is not empty then span(S) is a subspace of L. This is a common (and useful) way to generate a variety of subspaces of L. Theorem 2. A subset M of a linear space L is a subspace of L if and only if αx + βy ∈ M whenever x, y ∈ N and α, β are scalars. Exercise 2: Verify Theorem 2 by directly checking that all properties (a)-(h) of the vector space definition hold. Additional Exercises: 4