Download Norm and inner products in Rn Math 130 Linear Algebra

In many ways, norms act like absolute values.
For instance, the norm of any vector is nonnegative,
and the only vector with norm 0 is the 0 vector.
Like absolute values, norms are multiplicative in
the sense that
Norm and inner products in Rn
Math 130 Linear Algebra
kcvk = |c| kvk
D Joyce, Fall 2015
when c is a real number and v is a real vector.
So far we’ve concentrated on the operations of
There’s also a triangle inequality for norms
addition and scalar multiplication in Rn and, more
generally, in abstract vector spaces.
kw − vk ≤ kwk + kvk.
There are two other algebraic operations on Rn
H
HH
we mentioned early in the course, and now it’s time
w−v
H
to look at them in more detail.
HH
H
One of them is the length of a vector, more comj
H
1
v
monly called the norm of a vector. The other is a
w
kind of multiplication of two vectors called the in ner product or dot product of two vectors. There’s a
connection between norms and inner products, and
we’ll look at that connection.
Here’s a geometric argument for the triangle inToday we’ll restrict our discussion of these conequality.
If you draw a triangle with one side being
cepts to Rn , but later we’ll abstract these concepts
the vector v, and another side the vector w, then
to define inner product spaces in general.
the third side is w−v. Then the triangle inequality
just says that one side of a triangle is less than or
The norm, or length, kvk of a vector v. Con- equal to the sum of the other two sides. Equality
sider a vector v = (v1 , v2 ) in the plane R2 . By the holds when the vectors v and w point in the same
Pythagorean theorem of plane geometry, the dis- direction.
tance k(v1 , v2 )k between the point (v1 , v2 ) and the
Later, we’ll prove the triangle inequality algeorigin (0, 0) is
braically.
q
k(v1 , v2 )k = v12 + v22 .
The inner product hv|wi of two vectors. This
Thus, we define the length or norm of a vector v =
are also commonly called a dot product and denoted
(v1 , v2 ) as being
with the alternate notation v · w.
q
We’ll start by defining inner products algekvk = k(v1 , v2 )k = v12 + v22 .
braically, then see what they mean geometrically.
The norm of a vector is sometimes denoted |v|
The inner product hv|wi of two vectors v and w
rather than kvk.
in Rn is the sum of the products of corresponding
n
Norms are defined for R as well
coordinates, that is,
kvk = k(v1 , v2 , . . . , vn )k
v
u n
q
uX
=
v12 + v22 + · · · + vn2 = t
vk2 .
hv|wi = h(v1 , v2 , . . . , vn )|(w1 , w2 , . . . , wn )i
n
X
= v1 w1 + v2 w2 + · · · vn wn =
vk wk
k=1
k=1
1
Notice right away that we can interpret the The inner product of two vectors and the
square of the length of the vector as an inner prod- cosine of the angle between them. For this
discussion, we’ll restrict our attention to dimension
uct. Since
2 since we know a lot of plane geometry.
kvk2 = v12 + v22 + · · · + vn2 ,
The law of cosines for oblique triangles says that
given a triangle with sides a, b, and c, and angle θ
therefore
between sides a and b,
kvk2 = hv|vi.
Because of this connection between norm and inner
product, we can often reduce computations involving length to simpler computations involving inner
products.
The inner product acts like multiplication in a
lot of ways, but not in all ways. First of all, the inner product of two vectors is a scalar, not another
vector. That means you can’t even ask if it’s associative because the expression hhu|vi|wi doesn’t
even make sense; hu|vi is a scalar, so you can’t take
its inner product with the vector w.
But aside from associativity, inner products act
a lot like ordinary products. For instance, inner
products are commutative:
HH
HH
c
H
HH
H
a
θ b
c2 = a2 + b2 − 2ab cos θ.
Now, start with two vectors v and w, and place
them in the plane with their tails at the same point.
Let θ be the angle between these two vectors. The
vector that joins the head of v to the head of w is
w − v. Now we can use the law of cosines to see
that
hu|vi = hv|ui.
Also, inner products distribute over addition,
hu|v + wi = hu|vi + hu|wi,
H
HH
Hw − v
HH
H
v
and over subtraction,
θ hu|v − wi = hu|vi − hu|wi,
w
j
H
1
kw − vk2 = kvk2 + kwk2 − 2kvk kwk cos θ.
and the inner product of any vector and the 0 vector is 0
We can convert the distances to inner products to
hv|0i = 0.
simplify this equation.
Furthermore, inner products and scalar products
kw − vk2 = hw − v|w − vi
have a kind of associativity, namely, if c is a scalar,
= hw|wi − 2hw|vi + hv|vi
then
= kwk2 − 2hw|vi + kvk2
hcu|vi = chu|vi = hu|cvi.
These last few statements can be summarized by Now, if we subtract kvk2 + kwk2 from both sides
saying that inner products are linear in each coor- of our equation, and then divide by −2, we get
dinate, or that inner products are bilinear operations.
hv|wi = kvk kwk cos θ.
2
That gives us a way of geometrically interpreting u =
the inner product. We can also solve the last equa3
4
tion for cos θ,
>> norm(u)
hv|wi
cos θ =
,
kvk kwk
ans =
5
which will allow us to do trigonometry by means of
linear algebra. Note that
>> v = [5 12]
hv|wi
.
θ = arccos
v =
kvk kwk
5
12
Orthogonal vectors. The word “orthogonal” is
synonymous with the word “perpendicular,” but for >> norm (v)
some reason is preferred in many branches of mathematics. We’ll write w ⊥ v if the vectors w and v ans =
13
are orthogonal, or perpendicular.
>> dot(u,v)
ans =
63
MB
B
vB
B
B
B
w
1
>> costheta = dot(u,v)/(norm(u)*norm(v))
costheta =
0.9692
Two vectors are orthogonal if the angle between
them is 90◦ . Since the cosine of 90◦ is 0, that means
>> acos(costheta)
w ⊥ v if and only if hw|vi = 0
ans =
0.2487
Vectors in Matlab. You can easily find the
length of a vector in Matlab; where the length
Thus, the angle between the vectors (3, 4) and
of a vector is called its norm. Let’s find the length
(5, 12) is 0.2487 radians.
of two vectors and the angle between them using
the formula
Math 130 Home Page at
http://math.clarku.edu/~ma130/
hv|wi
θ = arccos
.
kvk kwk
Note that arccosines are computed with the acos
function, and inner products with the dot function
>> u = [3 4]
3