Download ENGR-25_Lec-21_Linear_Equations-2

Engr/Math/Physics 25
Chp8 Linear
Algebraic Eqns-2
Bruce Mayer, PE
Licensed Electrical & Mechanical Engineer
[email protected]
Engineering/Math/Physics 25: Computational Methods
1
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Learning Goals
 Define Linear Algebraic Equations
 Solve Systems of Linear Equation by
Hand using
• Gaussian Elimination
• Cramer’s Method
 Distinguish between Equation System
Conditions: Exactly Determined,
Overdetermined, Underdetermined
 Use MATLAB to Solve Systems of Eqns
Engineering/Math/Physics 25: Computational Methods
2
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method for Eqn Sys
 Solves equations using determinants.
 Gives insight into the
• existence and uniqueness of solutions
– Identifies SINGULAR (a.k.a. Divide by Zero)
Systems
• effects of numerical inaccuracy
– Identifies ILL-CONDITIONED
(a.k.a. Stiff) Systems
Engineering/Math/Physics 25: Computational Methods
3
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-1
 Cramer’s Method
can Solve “Square”
Systems;
• i.e., [No. Eqns] =
[No. Unknowns]
 Consider Sq Sys
 Calc Cramer’s
Determinant, Dc
• Also Called the
“Characteristic” or
“Denominator”
Determinant
 Dc  Determinant of
21x  9 y  12 z   33
the Coefficients
 3x  6 y  2 z 
 8 x  4 y  22 z 
Engineering/Math/Physics 25: Computational Methods
4
3
50
21  9  12
Dc   3 6  2
 8  4 22
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-2
 Now, to Find The
Individual Solns,
Sub The Constraint
Vector for the
Variable Coefficients
and Compute the
Determinant for
Each unknown, Dk
 In this Example Find
Dx, Dy, Dz as
Engineering/Math/Physics 25: Computational Methods
5
21x  9 y  12 z   33
 3x  6 y  2 z 
3
 8 x  4 y  22 z 
50
 33  9  12
Dx 3
6 2
50  4 22
21  33  12
Dy 3 3
2
 8 50
22
21  9 33
Dz  3 6
3
 8  4 50
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-3
 Once We’ve
Calculated all these
Determinants, The
Rest is Easy
Dx
x
Dc
y
Dy
Dc
 These Eqns Ilustrate
the most UseFul
Feature of Cramer’s
Method
Dc appears in all
THREE
Denominators
Dz
z
Dc
Engineering/Math/Physics 25: Computational Methods
6
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-4
 Since
Dk
k
Dc
 Can ID “Condition”
by Calculating Dc
– However, We can
ANTICIPATE
Problems if |Dc| <<
than the SMALLEST
Coefficient
 Completing the
Example
• SINGULAR Systems
→ Dc = 0
21x  9 y  12 z   33
• ILL-CONDITIONED
Systems →
 3x  6 y  2 z 
3
Dc = “Small”
 8 x  4 y  22 z 
50
– Small is technically
relative to the Dk
Engineering/Math/Physics 25: Computational Methods
7
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-6
 Calc the
Determinants
6 2
3 2
3 6
D c  211
 9 1
 121
 4 22
 8 22
8  4
 First Recall The
Dc 
216 * 22   2*  4
SIGN pattern for
  9 3* 22   8*  2
Determinants
  
  
  
 Find Dc
Engineering/Math/Physics 25: Computational Methods
8
  12 3*  4   8* 6
D c  2604  738  720
D c  1146
 Dc is LARGE → WELL
Conditioned
System
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-7
 Solve using MATLAB’s det Function
>> EqnSys = [21 -9 -12 -33; -3 6 -2 3; -8 -4 22 50];
All Row Elements of Cols 1-3
>> Dc = det(EqnSys(:,1:3))
Dc =
1146
All Row Elements of Cols 4, 2-3
>> Dx = det([EqnSys(:,4),EqnSys(:,2:3)])
Dx =
1146
All Row Elements of Cols 1, 4, 3
>> Dy = det([EqnSys(:,1),EqnSys(:,4), EqnSys(:,3)])
Dy =
2292
Engineering/Math/Physics 25: Computational Methods
9
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Method – Illustrated-8
 Solve using MATLAB’s det Function
>> Dz = det([EqnSys(:,1:2),EqnSys(:,4)])
Dz =
3438
All Row Elements of Cols 1-2, 4
>> x = Dx/Dc
x =
1
>> y = Dy/Dc
y =
2
>> z = Dz/Dc
z =
3
Engineering/Math/Physics 25: Computational Methods
10
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer vs Homogenous: Ax  b  0
 In general, for a set of
HOMOGENEOUS linear algebraic
equations that contains the same
number of equations as unknowns
• a nonzero solution exists only if the
set is singular; that is, if Cramer’s
determinant is zero
• furthermore, the solution is not unique.
• If Cramer’s determinant is not zero, the
homogeneous set has a zero solution; that
is, all the unknowns are zero
Engineering/Math/Physics 25: Computational Methods
11
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Cramer’s Rule Summary
 Cramer’s determinant gives some
insight into ill-conditioned problems,
which are close to being singular.
 A Cramer’s determinant
close to zero indicates an
ill-conditioned problem.
Engineering/Math/Physics 25: Computational Methods
12
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
UnderDetermined Systems
 An UNDERdetermined system does not
contain enough information to solve for
ALL of the unknown variables
• Usually because it has fewer equations
than unknowns.
 In this case an INFINITE number of
solutions can exist, with one or more of
the unknowns dependent on the
remaining unknowns.
• For such systems the Matrix-Inverse
and Cramer’s methods will NOT work
Engineering/Math/Physics 25: Computational Methods
13
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
UnderDetermined Example-1
 A simple UnderDetermined
system is the equation
x  3y  6
 All we can do is solve for one of the
unknowns in terms of the other; for
example, x = 6 – 3y OR y = −x/3 + 2
• An INFINITE number of (x,y) solutions
satisfy this equation
Engineering/Math/Physics 25: Computational Methods
14
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
UnderDetermined Example-2
 When there are more Unknowns than
Equations, the LEFT-DIVISION method
will give a solution with some of the
unknowns set equal to ZERO
• For Example
 x
x  3 y  6  1 3   6
 y
>>A = [1, 3];
b = 6;
>>solution = A\b
solution =
0
2
Engineering/Math/Physics 25: Computational Methods
15
 which
corresponds to
• x=0
• y=2
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
More UnderDetermined Systems
 An infinite number of solutions might exist
EVEN when the number of equations
EQUALS the number of knowns
• Predict by Cramer as:
det A   D c  0
 For such systems the Matrix Inverse
method and Cramer’s method will also
NOT work
• MATLAB’s left-division method generates
an error message warning us that the
matrix A is singular
Engineering/Math/Physics 25: Computational Methods
16
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Minimum Norm Solution
 When det(A) = 0, We can use the
PSEUDOINVERSE method to find ONE
Solution, x, such that the Euclidean (or
Pythagorean) Length of x is MINIMIZED
 In MATLAB: x = pinv(A)*b
 MATLAB will return the MINIMUM
NORM SOLUTION →
x  min
 x  x  x  x 
Engineering/Math/Physics 25: Computational Methods
17
2
1
2
2
2
3
2
n
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Existence and Uniqueness
 The set Ax = b with m equations and n
unknowns has solutions if and only if
rank A  rank Ab 
1
 Rank[A] is the maximum number of
LINEARLY INDEPENDENT rows of A
• Linear Independence → No Row of A
is a SCALAR multiple of ANY OTHER
Combinations of Rows
Engineering/Math/Physics 25: Computational Methods
18
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
An INconsistent Example
 Consider Ax = b
1 2  x1  4
 2 4  x    5 

 2   
 ERO: Multiply the 1st
row by −2 and add
to the 2nd row
1 2
0 0 


Rank[A]=1
1 2 4 
0 0  3


Rank[Ab]=2
Engineering/Math/Physics 25: Computational Methods
19
 Since The Ranks
are Unequal → this
system of equations
is NOT solvable
 Graphically
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Existence and Uniqueness
 Recall Rank for m-Eqns & n-Unknwns
rank A  rank Ab 
1
 Now Let r = rank[A]
• If condition (1) is satisfied and if r = n, then
the solution is unique
• If condition (1) is satisfied but r < n, an
infinite number of solutions exists and
– r unknown variables can be expressed as linear
combinations of the other n−r unknown variables,
whose values are ARBITRARY
Engineering/Math/Physics 25: Computational Methods
20
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Homogenous Case
 The homogeneous set Ax = 0 is a
special case in which b = 0
 For this case rank[A] = rank[Ab] always,
and thus the system in all cases has the
trivial solution x = 0
 A nonzero solution, in which at least
one unknown is nonzero, exists if and
only if rank[A] < n (n  No. Unknowns)
 If m < n, the homogeneous set always
has a nonzero solution
Engineering/Math/Physics 25: Computational Methods
21
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
ILLconditioned Systems
 An ill-conditioned set of equations is a
set that is CLOSE to being SINGULAR
 The ill-conditioned status depends on
the ACCURACY with which the solution
calculations are made
• When the internal numerical accuracy used
by MATLAB is INsufficient to obtain a
solution, then MATLAB prints a message
to warn you that the matrix is close to
singular and that the results might be
INaccurate
Engineering/Math/Physics 25: Computational Methods
22
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
OVERdetermined Systems - 1
 By Contrast an OVERdetermined
system is a set of equations that has
MORE independent equations than
unknowns
 For such a system the Matrix Inverse
method and Cramer’s method will NOT
work because the A matrix is not square
 However, SOME overdetermined
systems have exact solutions, and they
can be obtained with the left division
method x = A\b
Engineering/Math/Physics 25: Computational Methods
23
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
OVERdetermined Systems - 2
 For OTHER OverDetermined systems,
NO exact solution exists.
• In some of these cases, the left-division
method does not yield an answer, while in
other cases the left-division method gives
an answer that satisfies the equation set
only in a “LEAST SQUARES” sense
– When MATLAB gives an answer to an
overdetermined set, it does NOT tell us whether
the answer is Exact or Least-Squares in Nature
 We need to check the ranks of A and [Ab] to
know whether the answer is the exact solution.
Engineering/Math/Physics 25: Computational Methods
24
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Example  Under/Over Determined
% Bruce Mayer, PE * 27Feb08
% ENGR25 * Under/OverDetermined Linear Systems
% file = Over_Under_Lin_Sys_Example_0904.m
% Solve Under & Over Determined Systems for [x1; x2; x3; x4]
%
% An UNDERdetermined case => 3-Eqns in 4-Unknwns
Au = [2 -3 7 4; -9 2 -2 -7; 7 6 -5 -4]
bu = [5; -9; 7]
disp('Showing UNDERdetermined solution & check - Hit Any Key to
continue')
xu = Au\bu
bu_chk = Au*xu
pause
%
% An OVERdetermined case => 5-Eqns in 4-Unknwns
Ao = [2 -3 7 4; -9 2 -2 -7; 7 6 -5 -4; 1 9 0 -6; 2 -5 -8 1]
bo = [5; -9; 7; 2; -6]
disp('Showing OVERdetermined solution & check')
xo = Ao\bo
bo_chk= Ao*xo
Engineering/Math/Physics 25: Computational Methods
25
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Uniqueness of solutions
 A system has a unique solution If, and
Only If (IFF): Rank[A]=Rank[Ab] = n
• Where n is the ORDER of the system;
i.e., the Number of UNKNOWNS
 Such systems are called full-rank
systems
Engineering/Math/Physics 25: Computational Methods
26
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
OverDetermined Rank
 To interpret MATLAB answers correctly
for an OVERdetermined system, first
check the ranks of [A] and [Ab] to see
whether an exact solution exists
 r[Ab] = r[A] = n (the no. of Unknowns)
 If r[Ab] = r[A] > n then a unique soln does
not exist, then you know that the leftdivision answer is a least squares solution
• Can easily Chk: Axsoln = b (exactly)?
– This is at least as easy as the rank calc
Engineering/Math/Physics 25: Computational Methods
27
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 1

If the number of equations (m) in the
set equals the number of unknown
variables (n), the matrix A is square
and MATLAB provides two ways of
solving the equation set Ax = b:
1. The matrix inverse method; solve for
x by typing x = inv(A)*b.
2. The matrix left-division method; solve for
x by typing x = A\b.
Engineering/Math/Physics 25: Computational Methods
28
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 2

If A is square and if MATLAB does not
generate an error message when you
use one of these methods, then the
set has a unique solution, which is
given by the left-division method.
You can always CHECK the solution
for x by typing A*x to see if the result
is the same as b.

•
Use this to avoid the rank calcs
Engineering/Math/Physics 25: Computational Methods
29
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 3


If when you type A*x you receive an
error message, the set is
UNDERdetermined, and either it does
not have a solution or it has more than
one solution
In such a case, if you need more
information, you must use the
following procedures.
Engineering/Math/Physics 25: Computational Methods
30
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 4

For UNDERdetermined and
OVERdetermined sets, MATLAB provides
three ways of dealing with the equation set
Ax = b. (Note that the matrix inverse
method will NEVER work with such sets.)
1. The matrix left-division method; solve for
x by typing x = A\b
2. The pseudoinverse method; solve for
x by typing x = pinv(A)*b
3. the Reduced Row Echelon Form (RREF)
method. This method uses the MATLAB
function rref to obtain a solution.
Engineering/Math/Physics 25: Computational Methods
31
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 5

UNDERdetermined Systems
•
In an underdetermined system NOT
enough information is given to determine
the values of all the unknown variables.
– An INFINITE number of solutions might exist
in which one or more of the unknowns are
dependent on the remaining unknowns
– For such systems Cramer’s method and the
matrix inverse method will not work because
either A is not square or because |A| = 0.
Engineering/Math/Physics 25: Computational Methods
32
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 6
•
Underdetermined Systems cont
–
–
–
The left-division method will give a solution with some
of the unknowns arbitrarily set equal to zero, but
this solution is NOT the general solution; it’s just ONE
solution (of Trillions)
An infinite number of solutions might exist even when
the number of equations equals the number of
unknowns. The left-division method fails to give a
solution in such cases
In cases that have an infinite number of solutions, some
of the unknowns can be expressed in terms of the
remaining unknowns, whose values are arbitrary. The
rref function can be used to find these relations.
Engineering/Math/Physics 25: Computational Methods
33
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 7

OVERdetermined Systems
•
An overdetermined system is a set of
equations that has more independent
equations than unknowns
– For such a system Cramer’s method and the
matrix inverse method will not work because
the A matrix is NOT SQUARE
– Some overdetermined systems have exact
solutions, which can be obtained with the leftdivision method A\b.
Engineering/Math/Physics 25: Computational Methods
34
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Solving Linear Eqns Summary - 8
•
OVERdetermined systems cont.
– For overdetermined systems that have no
exact solution, the estimated-answer given by
the left-division method satisfies the equation
set only in a LEAST SQUARES sense
– When we use MATLAB to solve an
overdetermined set, the program does not tell
us whether the solution is exact. We must
determine this information ourselves. The first
step is to check the ranks of A and [A b] to
see whether a solution exists; if no solution
exists, then we know that the left-division
solution is a least squares answer.
Engineering/Math/Physics 25: Computational Methods
35
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
FlowChart for LinSys Solution
Can Construct
Least-Squares
Approximation
Engineering/Math/Physics 25: Computational Methods
36
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Descison Tree for LinSys Solns
if Rank(A)==Rank(Ab) & Rank(A)==NoUnknowns
disp('EXACT Solution')
elseif Rank(A)==Rank(Ab) & Rank(A)<NoUnknowns
disp('INFINITE Solutions')
elseif Rank(Ab) > NoUnknowns
disp('LEAST SQS Approximation')
else
disp('NO solution; can least-sqs est.')
end
Engineering/Math/Physics 25: Computational Methods
37
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
All Done for Today
Gabriel
Cramer
Born: 31 July 1704 in Geneva, Switzerland
Died: 4 Jan 1752 in Bagnols-sur-Cèze, France
Engineering/Math/Physics 25: Computational Methods
38
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
Engr/Math/Physics 25
Appendix
f x   2 x  7 x  9 x  6
3
2
Bruce Mayer, PE
Licensed Electrical & Mechanical Engineer
[email protected]
Engineering/Math/Physics 25: Computational Methods
39
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
MATLAB LinSys Solver Code - 1
% Script file lineq.m
% Solves the set Ax = b, given A and b.
% Check the ranks of A and [A b].
if rank(A) == rank([A b])
% Then The ranks are equal.
Size_A = size(A);
% Does the rank of A equal the number of
unknowns?
if rank(A) == size_A(2)
% Then the Rank of A equals the number
of unknowns.
disp(’There is a unique solution,
which is:’)
x = A\b % Solve using left division.
Engineering/Math/Physics 25: Computational Methods
40
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
MATLAB LinSys Solver Code - 2
else
% Rank of A does not equal the number
of unknowns.
disp(’There is an infinite number of
solutions.’)
disp(’The augmented matrix of the
reduced system is:’)
rref([A b]) % Compute the augmented
matrix.
end
else
% The ranks of A and [A b] are not equal.
disp(’There are no solutions.’)
end
Engineering/Math/Physics 25: Computational Methods
41
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt
MATLAB rank
 rank
• Rank of matrix
 Syntax
• k = rank(A)
 Description
• The rank function provides an estimate of
the number of linearly independent rows or
columns of a full matrix.
Engineering/Math/Physics 25: Computational Methods
42
Bruce Mayer, PE
[email protected] • ENGR-25_Linear_Equations-1.ppt