Download Linear Algebra Basics A vector space (or, linear space) is an

Linear Algebra Basics
A vector space (or, linear space) is an algebraic structure that often provides a home for solutions of mathematical models.
Linear algebra is a study of vector spaces and a class of special functions (called “linear transformations”) between them.
The terms vector space and linear space are interchangeable. We’ll use them both.
1. Linear Spaces: The elements of a linear space are called vectors. We can scale and add vectors. By scaling and
adding vectors we can build new vectors. This simple construction paradigm is remarkably useful.
Definition: A linear space (or vector space) is a set L together with field of scalars F and two functions (operations);
addition + : L × L → L and (scalar) multiplication · : F × L → L satisfying conditions (a)-(h) below.
For us, the scalar field F will almost always be the real numbers R (rarely, for us, it may be the complex numbers C).
Also, for brevity (clarity?), if α ∈ F and x ∈ L and we want to scale x using α, then we’ll write αx rather than α · x.
(a) x + y = y + x for x, y in L
(b) x + (y + z) = (x + y) + z for x, y, z ∈ L
(c) there is an element O ∈ L such that for each x ∈ L, x + O = x
(d) for each x ∈ L there corresponds y ∈ L such that x + y = O
(e) for each α, β ∈ F and x ∈ L, α(βx) = (αβ)x
(f) for each α, β ∈ F and x ∈ L, (α + β)x = αx + βx
(g) for each α ∈ F and x, y ∈ L, α(x + y) = αx + αy, and
(h) 1x = x for each x ∈ L.
Remarks:
• If x + y = x + z then y = z. This cancellation law holds because y = O + y = (−x + x) + y = −x + (x + y) =
−x + (x + z) = (−x + x) + z = O + z = z.
• 0x + x = (0 + 1)x = x so 0x = O.
• The zero element O of (b) and y, the additive inverse of x, in (c) are unique; we usually write −x for the additive
inverse of x. Since −1x + x = −1x + 1x = (−1 + 1)x = 0x = O then −x = −1x.
• For us, the scalar field F will always be either the real numbers R or complex numbers C. There are other fields
however, for example the rational numbers Q. There are finite fields, e.g., Zp (the integers mod p) where p is a
prime number. Vector spaces over finite fields find applications in, for example, crytography. We’ll not consider
them in this course.
Examples of linear spaces:
(a) Rn : A vector x ∈ Rn is an n-tuple of real numbers, x = (x1 , x2 , . . . , xn ). The set of real numbers R is the scalar
field. Vector additon and scalar multiplication are defined component-wise: if x and y are vectors and α ∈ R
then x + y = (x1 , x2 , . . . , xn ) + (y1 , y2 , . . . , yn ) := (x1 + y1 , x2 + y2 , . . . , xn + yn ) and αx = α(x1 , x2 , . . . , xn ) :=
(αx1 , αx2 , . . . , αxn ).
(b) Cn : A vector z ∈ Cn is an n-tuple of complex numbers, z = (z1 , z2 , . . . , zn ). The set of complex numbers
C is the scalar field. Vector addition and scalar multiplication are defined component-wise: if w and z are
vectors and α ∈ C then w + z = (w1 , w2 , . . . , wn ) + (z1 , z2 , . . . , zn ) := (w1 + z1 , w2 + z2 , . . . , wn + zn ) and
αz = α(z1 , z2 , . . . , zn ) := (αz1 , αz2 , . . . , αzn ).
(c) Rnp : If p ∈ Rn then define Rnp := {(p, x) | x ∈ Rn }. Rnp (vectors based at p) becomes a real vector space if addition
and scalar multiplication are defined by (p, x) + (p, y) = (p, x + y) and α(p, x) = (p, αx) for each scalar α ∈ R and
pair of vectors x, y ∈ Rn .
(d) Mm×n : A vector is a rectangular m × n array of real numbers. Vector addition and scalar multiplication are
defined entry-wise: If A and B are in Mm×n and α ∈ R then A + B and αA are defined to be the m × n matrices
whose entry in the ith row and j th column are (A + B)(i, j) := A(i, j) + B(i, j) and (αA)(i, j) := αA(i, j).
For example, M2×2 :
a b
M2×2 =
| a, b, c, d ∈ R
c d
a b
e f
a+e b+f
a b
αa αb
+
=
and α
=
c d
g h
c+g d+h
c d
αc αd
Note: allowing entries and scalars to be complex numbers turns Mm×n into a complex vector space.
(e) C(I; R), I an interval of real numbers: A vector is any continuous real valued function f : I → R. The scalars
are real numbers. Functions f and g are added and scaled “component-wise” or pointwise, i.e., f + g and αf are
functions whose value at x ∈ I is (f + g)(x) := f (x) + g(x) and (αf )(x) := αf (x).
(f) Π: A vector is any polynomial on R. Vector addition and scalar multiplicaton are defined pointwise, as in C(I; R).
(g) Πn : A vector is any polynomial of degree n on R. Vector addition and scalar multiplicaton are defined pointwise,
as in C(I; R).
(h) BC(R; R): A vector is any bounded continuous function f : R → R. Addition and scalar multiplication are defined
pointwise.
(i) l2 : AX
vector x is a sequence of real numbers that is sqaure sum-able: so x ∈ l2 if and only if x = (x1 , x2 , . . . , xk , . . . )
and
x2i < ∞. Addition and scalar multiplication are, as usual, defined component-wise.
i≥1
p
(j) l (for p ≥ 1): A vector x is a sequence x = (x1 , x2 , . . . , xk , . . . ) of real numbers such that and
X
|xi |p < ∞.
i≥1
Addition and scalar multiplication are, as usual, defined component-wise.
(k) l∞ : A vector x is a bounded sequence of real numbers. More precisely, x = (x1 , x2 , . . . , xk , . . . ) ∈ l∞ if and only
if there is an M > 0 (a bound) such that |xi | < M for each i = 1, 2, . . . . Addition and scalar multiplication are,
as usual, defined component-wise.
(l) c: A vector x is a convergent sequence of real numbers. Addition and scalar multiplication are defined componentwise.
(m) c0 : A vector x is a convergent sequence of real numbers with limit 0. Addition and scalar multiplication are
defined component-wise.
By scaling and adding vectors in L we obtain other vectors in L. For example, in R2 we can generate (2, 2) by combining
(1, 0) and (0, 1) as follows: 2(1, 0) + 2(0, 1). In fact, given any vector (a, b) in R2 we can write (a, b) = a(1, 0) + b(0, 1).
The right hand side of this last equation is called a linear combination of the vectors (0, 1) and (0, 1) and we’ve just
shown that any vector from R2 is a linear combination of these two vectors: {(0, 1), (1, 0)}.
By contrast, (2, 2) is not a linear combination of (1, 3) and (−2, −6) (try to do it). It turns out that the set
{(1, 3), (−2, −6)} can only generate vectors along the line described by y = 3x (by plotting the two vectors you
should be able to see this).
Definition(s): Assume that L is a linear space.
0) Subpaces: A subset M of L is called a subspace of L provided M, together with (vector) addition and scalar
multiplication it inherits from L, is itself a vector space. Several examples of subspaces are listed in section 2.
1) Linear combinations: If v1 , v2 ,. . . ,vn are vectors from L and α1 , α2 , . . . , αn are scalars then the vector α1 v1 +
α2 v2 + · · · + αn vn is called a linear combination of v1 ,. . . ,vn .
2) The span of a set of vectors: If S ⊂ L then span(S) is defined to be the set of all linear combinations of vectors
in S. That is,


k
X

span(S) =
αj vj α1 , . . . , αk ∈ F and v1 , . . . , vk ∈ S .


j=1
span(S) is a subspace of L. It’s called the span of S.
3) Linear independence: If v1 , v2 ,. . . ,vn are vectors in L and there are scalars α1 , α2 , . . . , αn , not all zero, such that
α1 v1 + · · · + αn vn = 0 then the vectors v1 , v2 ,. . . ,vn are said to be linearly dependent. On the other hand, if the
only way to make the linear combination α1 v1 + · · · + αn vn vanish is by choosing α1 = α2 = · · · = αn = 0 then
the vectors v1 , v2 ,. . . ,vn are said to be linearly independent.
2
4) Finite dimensional vector spaces: If S ⊂ L is a finite set and span(S) = L then L is said to be finite dimensional.
Otherwise, the dimension of L is said to be infinite.
5) A basis for a finite dimensional vector space: If B ⊂ L is a finite set of linearly independent vectors and span(B) =
L then B is called a basis set (or just a basis) for L. It turns out (see below) that the number of vectors in any
basis for L is the same. This number is called the dimension of L.
Remark: If S = {v1 , . . . , vn } is a linearly independent set of vectors in L and v ∈ span(S) then v = α1 v1 +· · ·+αn vn for
some scalars α1 , . . . , αn . This linear combination is unique; that is, suppose v = β1 v1 +· · ·+βn vn then α1 v1 +· · ·+αn vn =
β1 v1 +· · ·+βn vn or, equivalently, (α1 −β1 )v1 +· · ·+(αn −βn )vn = 0. Since the vectors v1 , . . . , vn are linearly independent
then αk − βk = 0 for each k, that is, β1 = α1 , β2 = α2 , etc.
Exercise 1:
(a) If S = {(1, 1)} then argue that span(S) 6= R2 , i.e., S is not a basis for R2 . (Find a vector not in span(S)).
(b) Describe all vectors in the span of {(1, 0, 1), (0, 0, 1)}. (Draw xyz-axes and try to visualize span{(1, 0, 1), (0, 0, 1)}.)
(c) If {v1 , . . . , vn } is a linearly dependent set of vectors then show that, for some index k, vk must be a linear
combination of v1 , . . . , vk−1 , vk+1 , . . . , vn . That is, in a linearly dependent set, at least one of the vectors can be
built up from the others.
(d) Show that {(1, 0, 0), (1, 1, 0), (1, 1, 1)} is a linearly independent subset of vectors in R3 .
(e) Argue that if vk = O for some k then {v1 , . . . , vn } is a linearly dependent set of vectors.
(f) Set q0 (x) ≡ 1 and, for each k ∈ N, let qk be the polynomial qk (x) = xk . Show that {q0 , q1 , . . . , qn } is a linearly
independent subset of Π for each n ∈ N.
Hint: Remember, the zero element O of Π is the the zero function O(x) ≡ 0. Use either calculus or the fundamental
theorem of algebra to show that α0 = α1 = · · · = αn = 0 if 0 = α0 + α1 x + α2 x2 · · · + αn xn for all x ∈ R.
(g) If S is a linearly independent set of vectors then any subset of S is also linearly independent.
(h) If B = {v1 , . . . , vn } is a basis for L and w ∈ L then {w, v1 , . . . , vn } is a linearly dependent set of vectors.
Theorem 1. If B = {v1 , . . . , vn } is a basis for L and S ⊂ L contains m vectors with m > n then S is a linearly
dependent set.
Corollary 1. Any two bases of L contain the same number of vectors.
Here is an outline for a proof of Theorem 1:
(a) It’s enough to prove this result for m = n + 1. So let S = {w1 , . . . , wn+1 } be a subset of L.
(b) If any wj is the zero vector then S is linearly dependent (why?). So assume wj 6= 0 for all j.
X
(c) w1 =
αj vj for (unique) scalars α1 , . . . αn (why?).
j≤n
(d) For some index k1 , αk1 6= 0 (why?). So, vk1 =
X αj
1
w1 −
vj (why?).
αk1
αk1
j6=k1
(e) {w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } is a basis for L. This means two things:
i. span{w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } = L: if u ∈ L then there are scalars β̂1 , β1 , . . . , βk−1 , βk+1 , . . . , βn such
that u = β̂1 w1 + β1 v1 + · · · + βk−1 vk−1 + βk+1 vk+1 + · · · + βn vn .
ii. The set of vectors {w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } is linearly independent: the condition
β̂1 w1 + β1 v1 + · · · + βk−1 vk−1 + βk+1 vk+1 + · · · + βn vn = 0
(A)
implies β̂1 = β1 = · · · = βk−1 = βk+1 = · · · = βn = 0. (Observe that if (A) holds then we must have β̂1 = 0
[why?] and, hence, the rest of the βj ’s vanish [why?]).
(f) Now, by modifying the steps appropriately, “repeat” (c), (d) and (e) using w2 in place of w1 and the “new” basis
{w1 , v1 , . . . , vk−1 , vk+1 , . . . , vn } in place of {v1 , . . . , vn }.
P
For example, (c) would become w2 = γ1 w1 + j6=k1 αj vj for some scalars γ1 and αj , j 6= k1 . And then in (d): if
all coefficients αj of vj (j 6= k1 ) are 0 then w2 will be a multiple (linear combination) of w1 and hence, S is linearly
dependent and you can stop. If at least one of the αj ’s , say αk2 , is nonzero and vk2 can be “solved” for in terms
of the other vectors, that is, vk2 depends on {w1 , w2 , v1 , . . . , vn } \ {vk1 , vk2 }, a basis for L.
(g) Continue in this fashion until either one of two things happen: (1) there will be an index p ≤ n such that wp is a
linear combination of the basis vectors w1 , . . . , wp−1 , in which case S is linearly dependent, or (2) {w1 , . . . , wn } is
a basis, in which case S is linearly dependent.
3
2. Subspaces of Linear Spaces: If L is a vector space then, as it turns out, to check that a subset M ⊂ L is actually a
subspace of L we need only determine whether or not M is “closed” under linear combinations. More precisly, M will
be a subspace if αx + βy ∈ M whenever α and β are scalars and x, y ∈ M. See Theorem 2 and Exercise 2 below.
Here are some examples of subspaces:
(a) Any plane in R3 containing the origin is a subspace of R3 .
(b) c0 is a subspace of c.
(c) BC(R; R) is a subspace of C(R; R).
(d) Πn is a subspace of Π which, in turn, is a subspace of C(R; R).
(e) Define the set D := {f ∈ C([a, b]; R) | f 0 exists and is continuous on [a, b]}. D is a subspace of C([a, b]; R).
(f) The set {0} consisting of just the zero vector of L is always a subspace of L.
(g) If S and T are two subspaces of L then so is S ∩ T, the set of all vectors in L common to both S and T.
(h) If S ⊂ L is not empty then span(S) is a subspace of L. This is a common (and useful) way to generate a variety
of subspaces of L.
Theorem 2. A subset M of a linear space L is a subspace of L if and only if αx + βy ∈ M whenever x, y ∈ N and α,
β are scalars.
Exercise 2: Verify Theorem 2 by directly checking that all properties (a)-(h) of the vector space definition hold.
Additional Exercises:
4