Download Week_2_LinearAlgebra..

GX01 – Robotic Systems Engineering
Dan Stoyanov
George Dwyer, Krittin Pachtrachai
2016
Vectors and Vector Spaces
•
•
•
•
Introduction
Vectors and Vector Spaces
Affine Spaces
Euclidean Spaces
2
Vectors
• The most fundamental element in linear algebra is a vector
• Vectors are special types of tuples which satisfy various types
of scaling and addition operations
• We shall meet several kinds in this course:
– Coordinate vector spaces
– Functional vector spaces (perhaps)
• Vectors actually “live” in a vector space, but we can’t talk about
this until we’ve looked a bit more at the properties of vectors
3
Vectors as Geometric Entities
• Vectors are geometric entities which encode relative
displacement (usually from an origin)
• They do not encode an absolute position
• They are not special cases of a matrix
• They have their own algebra and rules of combination
4
4
Equivalency of Vectors
Equivalent vectors are parallel and of the same length
• Two vectors are the same iff (if and only if):
– They have the same direction
– They have the same length
5
5
Basic Operations on Vectors
• Vectors support addition and scaling:
– Multiplication by a scalar,
– Addition of one vector to another,
6
Scaling Vectors by a Positive Constant
• Length changed
• Direction unchanged
7
7
Scaling Vectors by a Negative Constant
• Length changed
• Direction reversed
8
8
Addition and Subtraction of Vectors
• Summation vectors “closes the triangle”
• This can be used to change direction and length
9
9
Other Properties of Vectors
• Commutativity
• Associativity
• Distributivity of addition over multiplication
• Distributivity of multiplication over addition
10
Vector Spaces
• A vector space
has the following properties:
– Addition and subtraction is defined, and the result is another vector
– The set is closed under linear combinations:
• Given
and
– There is a zero vector
the combination
such that
11
Illustrating Closure and the Zero Vector
12
Examples of Vector and Vector Spaces
Which of the following are vector spaces?
1. A tuple of n real numbers:
2. The zero-vector:
3. The angle displayed on a compass:
13
Answers
• All of the previous examples are vector spaces apart from
an angle on a compass. The reason is that it usually
“wraps around” – e.g., 180 degrees negative becomes 180
degrees positive
14
Yaw Angles Measured by 3 iPhones Over
Time
Angle discontinuities caused by “wrap around”
Summary
• Vectors encode relative displacement, not absolute
position
• Scaling a vector changes its length, but not its direction
• All vectors “live” in a vector space
• We can add scaled versions of vectors together
• The vector space is closed under linear combinations and
possesses a zero vector
• We can analyse the structure of vector spaces looking,
oddly enough, at the problem of specifying the coordinates
of a vector
16
16
Spanning Sets and Vector Spaces
• I am given a vector
• I am given a spanning set of vectors,
• I form the vector w as a linear combination of S,
17
Coordinate Representation of a Vector
• I want to find the set of values l1, ln such that
• What properties must S obey if:
– For any value of x at least one solution exists?
– For any value of x the solution must be unique?
• It actually turns out that it’s easier to answer the second
question first
18
Uniqueness of the Solution
• We want to pick a set S so that the values of l1, ln
are unique for each value of x
• Because everything is linear, it turns out that guaranteeing
linear independence is sufficient
19
Linear Independence
• The spanning set S is linearly independent if
only for the special case
• In other words, we can’t write any member of the set as a
linear combination of other members of the set
20
Dependent Spanning Set Example
• For example suppose that,
where
and a, b are real scalar constants
• This set is not linearly independent and so will yield an
ambiguous solution
21
Linearly Dependent Example
• To see this, compute w from the linear combination of the
vectors of S,
22
Linearly Dependent Example
• Suppose we want to match a vector w which can be
written as
• Matching coefficients, we have three unknowns but just
two equations
23
Linear Independence and Dimensionality
• Therefore, if our basis set
solution is not unique
is linearly dependent, the
• Conversely, the number of linearly independent vectors in
a spanning set defines its dimension
24
Linear Independence in the Dimension of 2
25
Existence of a Solution
• The condition we need to satisfy is that x and w must lie in
the same vector space
• Now,
• Because this is closed under the set of linear
combinations, it must always be the case that
26
Linear Combinations and Vector Spaces
27
Existence of a Solution
• We can go a bit further
• Because we know that
it must be the case that
28
Existence of a Solution
• We have shown that
• Can’t we automatically say that
• No, because what we’ve shown so far is that
29
Arbitrary S Might Not “Fill the Space”
• Therefore, in this case
30
Existence of a Solution
• Therefore, it turns out that a solution is guaranteed to exist
only if
• This means that we must have a “sufficient number” of
vectors in S to point “in all the different directions” in V
• This means that the dimensions of both vector spaces
have to be equal
31
Vector Subspaces
• Consider the set of linearly independent vectors
• Are these spanning sets dependent or independent?
What’s the dimension and basis of the subspace?
32
The Question…
• Recall the question – for any vector
and
where
can we find a unique set of coefficients such that
33
The Answer…
• For S a basis of , we can always represent the vector
uniquely and exactly
• For S a basis of a subspace of , we can only represent
some of the vectors exactly
• Expressing the “closest” approximation of the vector in the
subspace is a kind of projection operation, which we’ll get
back to later
34
The Answer…
35
Summary
• Vector spaces can be decomposed into a set of
subspaces
• Each subspace is a vector space in its own right (closure;
zero vector)
• The dimension of a subspace is the maximum number of
linearly independent vectors which can be constructed
within that subspace
36
Summary
• A spanning set is an arbitrary set of vectors which
comprise a subspace
• If the spanning set is linearly independent, it’s also known
as a basis for that subspace
• The coordinate representation of a vector in a subspace is
unique with respect to a basis for that subspace
37
37
Changing Basis
• In the last few slides we said we could write the
coordinates of a vector uniquely given a basis set
• However, for a given subspace the choice of a basis is not
unique
• For some classes of problems, we can make the problem
significantly easier by changing the basis to
reparameterise the problem
38
Example of Changing Basis
• One way to represent
position on the Earth is to
use Earth Centered Earth
Fixed (ECEF) Coordinates
• Locally at each point on the
globe, however, it’s more
convenient to use EastNorth-Up (ENU) coordinates
• If we move to a new location,
the ENU basis has to
change in the ECEF frame
Local Tangent Plane
39
Example of an ENU Coordinate System
40
Changing Basis
• Suppose we would like to change our basis set from
to
where both basis span the same vector space
41
Changing Basis
42
Changing Basis
• Since the subspaces are the same, each vector in the
original basis can be written as a linear combination of the
vectors from the new basis,
43
Changing Basis
• Clanking through the algebra, we can repeat this for all the
other vectors in the original basis,
44
Changing Basis
• Now consider the representation of our vector in the
original basis,
• Substituting for just the first coefficient, we get
45
45
Changing Basis
• Substituting for all the other coefficients gives
46
Summary of Changing Basis
• Sometimes changing a basis can make a problem easier
• We can carry this out if our original basis is a subspace of
our new vector space
• The mapping is unique, and corresponds to writing the old
basis in terms of the new basis
• (It’s much neater to do it with matrix multiplication)
47
So What’s the Problem with Vector Spaces?
• We have talked about vector spaces
–
–
–
–
They encode displacement
There are rules for adding vectors together and scaling them
We can define subspaces, dimensions and sets of basis vectors
We can even change our basis
• However vector spaces leave a lot out!
48
The Bits Which Are Missing
• There are no points
– There is no way to represent actual geometric objects
• There is no formal definition of what things like angles, or
lengths mean
– Therefore, we can’t consider issues like orthogonality
• We haven’t discussed the idea of an origin
– Everything floats in “free space”
• Affine spaces start to redress this by throwing points into
the mix
49
More Missing Bits
• We still don’t have a notion of distances
– Just ratios on lines between points
• We still don’t have a notion of angles
• We still don’t have an absolute origin
• These are all introduced in Euclidean geometry
50
Euclidean Spaces
•
•
•
•
•
Introduction
Tuples
Vectors and Vector Spaces
Affine Spaces
Euclidean Spaces
51
Euclidean Spaces
• Euclidean spaces extend affine spaces by adding notions
of length and angle
• The Euclidean structure is determined by the forms of the
equations used to calculate these quantities
• We are going to just “give” these without proof
• However, we can motivate the problem by considering the
problem of orthogonally projecting one vector onto another
vector
• First, though, we need some additional vector notation
52
Some More Vector Notation
• Since we are going to define lengths and directions, we
can now decompose a vector into
– A scalar which specifies its length
– An orthonormal vector which defines its direction
Length
(+ve)
Orthonormal vector
(length=1)
53
Projecting Vectors onto Vectors
• Consider two vectors
space
and
that occupy the same affine
• What’s the orthogonal projection of
onto
?
54
The Answer…
Projected vector
55
The Answer…
Projected vector
Projected vector
56
Computing the Answer
• We need to compute both the direction and length of the
projection vector
• The direction of the vector must be parallel to
• Therefore, we are going to define the orthogonal projection
as
Scale
factor
Orthonormal vector
parallel to the “right
direction
57
General Case of the Scalar Product
• If we now let both of our vectors be non-normalised, then
• The scalar product is a special case of an inner product
58
Lengths, Angles and Distances
• Lengths and angles are (circularly) defined as
• The distance function (or metric) between two points is the
length of the vector between them,
59
Properties of Scalar Products
• Bilinearity:
• Positive definiteness:
60
Properties of Scalar Products
• Commutativity:
• Distributivity of the dot product over vector addition,
• Distributivity of vector addition over the dot product,
61
Scalar Product and Projection Mini-Quiz
• What’s the value of
62
Summary
• The Euclidean structure is defined by the scalar product
• The scalar product is used to compute the orthogonal
projection of one vector onto another vector
• The scalar product works in any number of dimensions
• It has many useful properties including bilinearity
• However, it only defines a one dimensional quantity
(length)
• Vector products generalise this
63
Vector Products
• The vector product is the familiar cross product and is
defined to be
where is orthogonal to and
• The vector product is a special case of an exterior product
and is a pseudovector (we won’t meet these again)
• The vector product is only defined in 3D (and 7D!)
64
Magnitude of the Vector Product
• The magnitude of the vector product
is the area of the parallelogram having the vectors as sides
65
Direction of the Vector Product
• In general, in 3D we have 3 axes of rotation
• However, if we rotate and such that they still lie in the
same plane, still points in the same direction
• Therefore, encodes information about two axes of
rotation
66
Sign of the Vector Product
• Given the plane defined by the vectors, there are two
possible choices of the normal vector – out of or into the
plane
• The choice is determined by the sign of the angle between
the vectors
– Anticlockwise is positive
67
Right Hand Rule
68
Properties of Vector Products
• Anticommutativity:
• Distributivity:
• Distributivity:
• Parallelism:
69
Scalar Triple Product and Volumes
• We have shown that:
– Scalar products define lengths and angles
– Vector products define areas and a sense of perpendicularity
• Therefore, is it possible to extend this, at least in 3D, to the
notion of a volume?
• The scalar triple product computes the volume of the
parallelepiped defined by three linearly independent
vectors
70
Scalar Triple Product and Volumes
71
Volume of the Parallelepiped
• The volume is computed as follows
• This is the absolute value of the scalar triple product
72
Cartesian Frames
• So far we’ve said an affine frame is simply an origin point
and a set of linearly independent vectors
• Now that we can talk about angles and lengths, we can
define a Cartesian coordinate frame
• A Cartesian Frame
has the property that its basis vectors are orthonormal
73
Orthonormal Basis Set
• An orthonormal basis set has the property that its vectors:
– Are orthogonal to one another
– Are of unit length
• More compactly,
74
Illustration of the Cartesian Basis Set
The basis is orthonormal
75
Cartesian Frames in 3D (Finally!)
• The basis set in 3D is written using the familiar vectors,
• Any vector can be written as
76
Scalar and Vector Product in the Cartesian
Frame
• The scalar product is given by
• The vector product is given by
77
Euclidean Space
• Euclidean spaces add notions of length and angles
through the scalar product
• These can be extended by the vector and vector triple
products to give areas and volumes in 3D
• The Cartesian Coordinate frame is a special case with a
dedicated origin and an orthonormal basis set
78
Matrices and Their Inverses
•
•
•
•
•
•
Introduction
Matrices and Their Inverses
Geometric Interpretation of SLEs
The Eigendecomposition
Kernel Spaces and Pseudoinverses
The Singular Value Decomposition
79
Isomorphic Transformations
a
b
c
d
e
g
h
i
j
k
a
b
c
d
e
g
h
i
j
k
• Recall that an isomorphic transform is one-to-one
• It also possesses an inverse
80
Isomorphism and SLEs
• For the isomorphic SLE
the solution is
• Therefore, all we need to do is:
– Work out if the matrix inverse exists (=transformation is isomorphic)
– And, if so, compute it
81
Properties of Isomorphism
• A transformation is isomorphic iff:
– The dimensions of
and
are the same
82
Assuming the Dimensions are the Same
• For the moment, we will assume the dimensions are the
same value n
• Therefore, for SLE is written as
83
Properties of Isomorphism
• A transformation is isomorphic iff:
– The dimensions of and are the same
– The transformation is one-to-one
– The transformation is onto
• We can analyse these by looking at basis and subspaces
• However, we begin by looking at these in terms of the
determinant or the metric scale factor
84
One-to-One Transformations
• Consider a 2D transformation
• If the transformation is one-to-one then any vector x maps to a
unique vector b
85
One-to-One Transformations
• If the transformation is not one-to-one, multiple values of x
map to the same value of b
• We can analyse this by considering projected areas
86
Transforms and Volumes
One-to-One Transformation
Not a One-to-One Transformation
87
Transformation Types and Areas
• For an n dimensional space, suppose I choose a set of n
orthonormal vectors
• These define the edges of an n-dimensional hypercube
with volume 1
• Then compute the set
• The determinant then is given by
88
Determinants as a Signed Volume
• It turns out that determinants are signed volumes and can
have positive or negative values
• A negative value implies that there is a reflection
• This causes the rotation order of the vectors to change
89
Using the Area to Characterise the Transform
• Looking at the area of the transformed set of points will tell
us if the transformation is one-to-one or not
– If the transformed area is zero, the transformed vectors fall on a
line
• It’s not one-to-one, so the transformation is not invertible
• The scale factor is zero
– If the transformed area is non-zero, the transformed vectors fall on
a parallelogram
• This is one to one-to-one, and the transformation is invertible
• Right now we don’t care what the value for the scale factor is
• It can be positive or negative
90
Computing the Scale Factor for a 2D Case
• For the matrix
it can be shown (coursework question) that
91
Example 2D Transformation
• Consider the transformation
• This is a pure rotation of q degrees
– The volume does not change
– The handedness does not change
92
Example 2D Transformation
• The determinant is
93
Another 2D Example
• Now consider
• This is a scale along the x-axis only
– The volume changes
– The handedness does not change
94
One More 2D Transformation
• Now consider the transformation
• This is a rotation and a reflection
– The volume does not change
– The handedness does change
95
One More 2D Transformation
• The determinant in this case is
96
Computing Determinants and Solving SLEs
• We are going to describe a brute-force method for
computing determinants
• It’s only really used for 2x2 and 3x3 matrices
– Widely useful in many image processing and geometric
applications
• However, it’s structure – large number of potentially
parallelisable calculations – makes it suitable for
implementation on an FPGA
• For big matrices on normal computers, determinants and
inverses are computed using other decompositions
97
Determinant of a 2x2 Matrix
• The basic building block for this method is the
determinant of the 2x2 matrix,
98
Determinant of a 3x3 Matrix
• For a 3-by-3 matrix,
Minors
99
Determinant of a 3x3 Matrix
• Writing this out,
100
3x3 Matrix Determinant Mini-Quiz
• Compute the determinant of
101
Answer
• Expanding the terms,
102
Determinant of an nxn Matrix
• For an nxn matrix,
the expression for the determinant is
103
Cofactors and Minors
• The cofactor is
where Mjk is the determinant of an (n-1)th dimensional
matrix formed by deleting the jth row and kth column
• In other words, we can recursively decompose our original
matrix into smaller and smaller matrices and compute the
determinant on those
104
Cofactors and Minors
+
C11
105
Cofactors and Minors
+
+
C22
C11
106
Cofactors and Minors
+
-
C23
C11
107
Cofactors and Minors
+
+
C24
C11
108
Cofactors and Minors
-
C12
109
Cofactors and Minors
-
-
C34
C12
110
Cofactors and Minors
-
-
C34
C12
111
Cofactors and Minors
-
+
C12
112
Determinants
• Show the determinant of
is -3
113
Compute the Determinants in the Easiest
Way
• We can also compute the determinant by “pivoting” along
any row
• For example, we can also compute the determinant of
by pivoting “along the bottom” to give
114
Useful Property of Determinants
• Let A and B be square matrices of the same dimensions
• Let
• It can be shown that
115
More Useful Properties of Determinants
• The determinant is zero if:
– Any of the rows or columns are 0
– Any of the rows or columns are linearly dependent upon the other
rows or columns
• The determinant is unchanged if:
– If row (or its multiple) is added to another row
– The matrix is transposed
• The determinant is negated if:
– Two rows are swapped
• The determinant is scaled by a factor c:
– If any row or column is multiplied by a factor c
116
Proving Some Properties by Products
• Consider again that
and
• It turns out that some of the properties of determinants can
be explained by this
117
Example
• From the properties of determinants, we know that adding
scaled values of rows to other rows does not change the
determinant,
118
Example
• Now, we can decompose this into the product of two
matrices
119
Example
• The determinant of the transformation which adds the rows
together is
120
Summary of Determinants
• Determinants let us describe some properties of the SLE
in terms of its volume scale factor
• They have:
– A magnitude which determines how much scaling happens
– A sign, which indicates whether there is a reflection
• We can use the determinant (metric scale factor) to show
whether a transformation is isomorphic or not
• We’ll now show that determinants can be used to directly
solve the SLE
121
Structure
•
•
•
•
•
•
Introduction
Matrices and Their Inverses
Geometric Interpretation of SLEs
The Eigendecomposition
Kernel Spaces and Pseudoinverses
The Singular Value Decomposition
122
Systems of Linear Equations (SLEs)
• One problem class which turns up very frequently is
finding the solution to a linear system of equations,
123
Systems of Linear Equations (SLEs)
• This can be more compactly expressed using vectors,
linear transformations and matrices as
• Despite their somewhat simplistic appearance, SLEs are
extremely important and near-ubiquitous
124
SLEs and Linear Spaces
• Recall that SLEs are normally expressed using vectors
and matrices as
• SLEs can be used for much more than affine, geometric
concepts we’ve considered so far
• The usual (implicit) assumption made with SLEs is that the
space is Euclidean
• Therefore, we’ll assume that things like distances and
angles have been defined in the normal manner
125
Matrices and Their Inverses
•
•
•
•
•
•
Introduction
Matrices and Their Inverses
Geometric Interpretation of SLEs
The Eigendecomposition
Kernel Spaces and Pseudoinverses
The Singular Value Decomposition
126
Summary of Determinants(from last lecture)
• Determinants let us describe some properties of the SLE
in terms of its volume scale factor
• They have:
– A magnitude which determines how much scaling happens
– A sign, which indicates whether there is a reflection
• We can use the determinant (metric scale factor) to show
whether a transformation is isomorphic or not
• We’ll now show that determinants can be used to directly
solve the SLE
127
Cramer’s Rule for Solving SLEs
• Cramer’s rule says that we can compute the ith
component of the solution to the SLE by substituting the ith
column of A with b and working out the ratio of the
determinants of the resulting matrices
• This sounds quite a mouthful, but it’s very straightforward
and a few examples show what it means
128
Cramer’s Rule for SLEs
• Recall that our SLE is
where A can be written as
and
129
Cramer’s Rule for a 2 by 2 Matrix
• For the two dimensional case, Cramer’s Rule is:
Column vector
inserted into the matrix
130
2x2 Case
• Use Cramer’s Rule to Solve for x1
131
Example of a 2x2 Case
• Using Cramer’s Rule, the solution is
132
Cramer’s Rule for a 3x3 Matrix
Column vector
inserted into the matrix
133
3x3 Matrix Example
• Consider the 3x3 system,
• Let’s solve this using Cramer’s Rule
• This is exactly the same as before, but it gets somewhat
messier
134
3x3 Example
• First off, the determinant of the matrix is
135
Solving for the First Coordinate
• Substituting for the first column,
• The solution is
136
Solving for the First Coordinate
• Solving the determinant,
• Therefore, the solution is
137
Cramer’s Rule
• The solution for the ith coefficient is the rather eye
watering
138
Computational Issues
• For an n dimensional matrix, Cramer’s rule requires n!
operations
• For example a 100 state vector would require roughly
9.3x10159 calculations
• However, MATLAB does a much better job:
139
Computational Issues
• The way in which MATLAB does this is far too complicated to
discuss in this course
– See LINPACK and LAPACK libraries if you really want to see the gory
details (or CLAPACK for the C orientated) and of course Netlib
• However, the basis of most of these approaches is to do some
clever rearranging of the matrices
• We’ll consider Gaussian elimination
• This is a direct solution method which simplifies the structure of
the SLE
140
Simplifying the Structure of the SLE
• Recall that
• The difficulty lies in the arbitrary structure of A
• Gaussian Elimination seeks to solve a simpler problem
with the same solution
141
The Structure of the Simplified SLE
• A is square and so
142
The Structure of the Simplified SLE
• Suppose we can rearrange problem so that the matrix
becomes upper triangular,
143
Solving a Triangular Matrix
• The reason is that, if we expand the SLE, it’s now
144
Solving a Triangular Equation
• We can solve by back substitution,
145
Solving a Triangular Equation
• For example, consider the equation
• We can directly solve to find that
146
Arranging the SLE in Triangular Form
• The first thing we do is stack the matrix and the solution
vector to produce the augmented matrix
• We now want to make the A part of this triangular
147
Simplifying the SLE
• To simplify the SLE, recall that the determinant of a matrix
does not change if we add a row (or its multiple) to another
row
• It can be shown that if we apply these operations to the
augmented vector, the solution does not change
148
A Simple Example
• Suppose our SLE is
• The augmented matrix is
149
A Simple Example
• Subtracting half of the first row from the second gives
• Therefore,
150
A Simple Example
• Since we can add and subtract any multiple of a row to
another row (including itself) we can also rescale the rows
to make the maths simpler
• Doubling the second row,
• Subtracting the first row from the second,
151
A Simple Example
• Substituting, we get
152
3x3 Matrix Example
• Consider the 3x3 system,
• The actual solution to this equation is
153
3x3 Matrix Example
• The augmented matrix is
154
3x3 Matrix Example
• We can use Gaussian Elimination again
• Adding the first row with the second row to the third row,
we get
155
3x3 Matrix Example
• If we now add the first row to the second, we have
• Solving from the bottom up gives the same answer as
before
156
Inconsistent Solutions
• However, it is possible to have equations which are
inconsistent
• These cannot be solved
• For example, consider the SLE
157
Inconsistent Solutions
• The augmented matrix is
• Adding two times the first row to the third leads to
158
Infinite Number of Solutions
• It is also possible to have an infinite number of solutions to
a problem
• For example, consider the SLE
159
Infinite Number of Solutions
• The augmented matrix is
• Adding two times the first row to the third leads to
160
Infinite Number of Solutions
• Subtracting seven times the first row from the second
gives
• We have three unknowns but only two equations, hence
the system is under constrained
161
We Can Simplify Rectangular Matrices as
Well
• Consider the case that A is rectangular, and the number of
rows do not equal the number of columns
162
Row Echelon Form
• When a matrix isn’t square, we can produce matrices
which are “as triangular as possible”
• If n > m, we get a matrix in row echelon form,
163
Triangular Rectangular Matrices
• If n < m, the row echelon form is
164
Examples of Row Echelon Matrices
165
Summary of Gaussian Elimination
• Cramer’s Rule is very inefficient
• Gaussian Elimination exploits the invariance of the
determinant to various row operations
• Matrices are rearranged into triangular forms
• The solution is then linear in the dimension of the state
• Solutions are either unique, inconsistent or contain an
infinite number of solutions
167
Matrix Inversion Mini-Quiz
• How can we use Cramer’s Rule or Gaussian Elimination
to compute the inverse of A?
168
Answer
• Recall the matrix inverse has the property that
• Now, writing out in terms of column vectors,
• Therefore, this can be written as n SLEs of the form
169
Limitations of the Inversion Methods
• There are two problems with these approaches:
– There might be no unique solution (=non-isomorphic)
– The SLE might be invertible, but very sensitive to noise
170
Non-Invertible Matrices
• A can be non-invertible for two main reasons:
– The dimensions of
– The dimensions of
and
and
are the same, but A is singular
are different
• Furthermore, there are two possible outcomes:
– There is no solution
– There are an infinite number of solutions
171
Solution Sensitivity
• Sometimes A can be inverted but the solution itself is
extremely sensitive to errors in the SLE
• Errors in the SLE can arise in two ways:
– Real-world parameter errors
• Both A and b might be estimated from parameters of some real world
problem and these could contain errors (e.g., due to measurement
noise or the wrong model being used)
– Implementation errors
• Many real implementations of numerical algorithms introduce subtle
errors (rounding, iterative solutions with finite termination criteria, etc.)
172
Solution Sensitivity
• The actual SLE we want to solve is
• In the presence of noise, we are computing the solution to
the approximate SLE
• We would hope that the two solutions are (roughly) the
same
173
Sensitivity and SLEs
•
Consider the harmless-looking SLE,
•
Computing the inverse, the solution is,
174
Sensitivity and SLEs
•
Now suppose we have a slightly incorrect value for ,
•
The computed solution is
175
Sensitivity and SLEs
•
Now consider the equally harmless-looking SLE,
•
Computing the inverse, the solution is,
176
Example of a Very Sensitive SLE
•
Now suppose we have a slightly incorrect value for ,
•
The computed solution is
177
Use of Matrix Decompositions
• Matrix decompositions analyse the structure of the SLE in
terms of properties such as:
– What parts of the SLE are invertible?
– What parts of it aren’t?
– How sensitive is the solution?
• The decompositions can be used to create new solution
algorithms
178