Download G-inverse and Solution of System of Equations and their

G-inverse and Solution of System of Equations
and their Applications in Statistics
PROFESSOR DR. S. K. BHATTACHARJEE
DEPARTMENT OF STATISTICS
UNIVERSITY OF RAJSHAHI, BANGLADESH
System of Linear Equations
Linear Combination of Vectors
One extremely helpful view is that each unknown is a weight
for a column vector in a linear combination.
Vector Form
The vector equation is equivalent to a matrix equation of the
form
where A is an m×n matrix, x is a column vector with n
entries, and b is a column vector with m entries.
Solution of Linear System
 A solution of a linear system is an assignment of




values to the variables x1, x2, ..., xn such that each of
the equations is satisfied. The set of all possible
solutions is called the solution set.
A linear system may behave in any one of three
possible ways:
The system has infinitely many solutions.
The system has a single unique solution.
The system has no solution.
Geometric Interpretation
 For a system involving two variables (x and y), each
linear equation determines a line on the xy-plane.
Because a solution to a linear system must satisfy all of
the equations, the solution set is the intersection of
these lines, and is hence either a line, a single point, or
the empty set.
 For three variables, each linear equation determines a
plane in three-dimensional space, and the solution set
is the intersection of these planes. Thus the solution set
may be a plane, a line, a single point, or the empty set.
 For n variables, each linear equations determines a
hyperplane in n-dimensional space. The solution set is
the intersection of these hyperplanes, which may be a
flat of any dimension
Solution of Two Vectors
Solution of Three Equations
Below is a picture of three planes that have no
solution. There is no single point at which all
three planes intersect, therefore this system has
no solution.
Solution of Three Equations

Each plane intersects the other two planes. However, there is no
single point at which all three planes meet. Therefore, the
system of 3 variable equations below has no solution.
Solution of Three Equations
One Solution of three variable systems
If the three planes intersect as pictured below then the three
variable system has 1 point in common, and a single solution
represented by the black point below.
Solution of Three Equations
Infinite Solutions of three variable systems
If the three planes intersect as pictured below then the three
variable system has a line of intersection and therefore an infinite
number of solutions.

Solution of Two Equations in Three Variables
The solution set for two equations in three variables is usually
a line.
Solution of Linear System
 In
general, the behavior of a linear system is
determined by the relationship between the number of
equations and the number of unknowns:
 Usually, a system with fewer equations than unknowns
has infinitely many solutions. Such a system is also
known as an underdetermined system.
 Usually, a system with the same number of equations
and unknowns has a single unique solution.
 Usually, a system with more equations than unknowns
has no solution. Such a system is also known as an
overdetermined system.
Examples

The system has exactly 1 solution.
 Systems have 1 and only 1 solution when the two lines have different slope. Think
about it, if the two lines have different slopes then eventually at some point they must
meet. After all the lines are not parallel.

system has no solutions
 Systems have no solution when the lines are parallel (ie have the same slope) and the
lines have different y-intercepts.
 As an example look at the following two lines
 Line 1: y = 5x +13
 Line 2: y = 5x + 12

The system has infinite solutions
 Systems have infinite solutions when the lines are parallel and the lines have the same
y-intercept. If two lines have the same slope (ie are parallel) and the same y-intercept,
they are actually the same exact line. In other words, systems have infinite solutions
when the two lines are the same line!
 As an example consider the following two lines
 Line 1: y = x +3
 Line 2: 2y = 2x +6
 These two lines are exactly the same line. If you multiply line 1 by two you get line 2.
Solution of Linear System
The following pictures illustrate this in the case of two
variables:
One Equation Two Equations Three Equations
Two Variables Three Eq.
The equations x − 2y = −1, 3x + 5y = 8, and 4x + 3y = 7 are
not linearly independent.
Example
The equations 3x + 2y = 6 and 3x + 2y =
12 are inconsistent.
Methods of Solution
 The Methods of finding the solution to systems




of linear equations:
graph : by looking at where lines intersect (meet) on a
graph
algebraic equation : by setting the equations of the
system equal to each other then solving this equation.
substitution : by solving for one of the variables and
substituting its value in to the other equation.
Elimination : Elimination involves algebraic
manipulations of two or more equations. The end goal
is to eliminate a variable by creating opposite
coefficients (The examples below should clarify this
straightforward approach).
Graphical Method
The Graph Method
On the left, the system of
linear equations is the
following two lines:
 y=x+1
 y=2x
What is the solution?
answer:
The point (1,2) is where
the two lines intersect.
Algebraic Equation Method
The Algebraic Equation Method
Let's take another look at the system of
equations from above:
y=2x+1
y=4x-1
By examining the graph we can see that the point of
intersection, or the solution, is the point (1,3) where
the lines intersected.


Steps for the algebraic method:
 make sure that each linear equation is
reduced to slope intercept form
o




(ie y=3x+2 is good but 2y=6x+4 is NOT)
set the two equations equal to each
other
o 2x+1=4x-1
Solve for X
o 2x+1=4x-1
o 2=2x
o x= 1
insert x value into either equation to
determine y coordinate of solution
o 4(1)-1=3
The solution is the ordered pair you've
just calculated
o (1,3)
Substitution Method
 The Substitution Method

 The substitution method involves algebraic
substitution of one equation into a variable of the
other.
 A quick refresher on algebraic substitution:
Refresher:Substitution

 Equation 1 : x = 5
 Equation 2: y = x +2
 How to Substitute
 1) Use equation 1( x= 5) to substitute 5 for x in second equation


y = (5) + 2
2) So love for Y

y = 5+ 2 = 7
Example
 Substitution Example Two


Line 1 : y=2x+1
Line 2 : 2y=3x-2
 Step 1: Substitute one equation into the other

2(2x+1)=3x-2
 Step 2: Now that you have a single variable equation,
solve for that variable's equation



4x+2 = 3x-2
x+2= -2
x= –4
 Step 3 : Once you have solved for the one variable
insert that variable back into either equation to obtain
the value of y at the solution.

Insert x= –4 to find y value y = 2(–4)+1= –7
 This example's solution is ( –4, –7).
Elimination Method
 Elimination method is an algebraic method
for solving systems. To use elimination you
perform an operation on 1 equation then add
the two equations so that one of the variables
cancels.
 Example of Elimination
 Line 1: y = x + 1
 Line 2: y = –x
Elimination Method
Matrix Methods of Solution
Cramer's rule is an explicit formula for the solution of a system of linear equations, with
each variable given by a quotient of two determinants. For example, the solution to the
system
is given by
Inverse Method
 The Matrix form of a linear system of
equations is
Ax= b
If the coefficient matrix is non-singular
then the solution is:
x = A-1b
Elementary Row Transformation
Solving a system of linear equations by reducing the
augmented matrix of the system to row canonical form
Example
Example. Solve the system
The augmented matrix is
Example
Homogeneous System
Homogeneous System
 Homogeneous Systems are always Consistent
 Every homogeneous system has at least one solution,
known as the zero solution (or trivial solution),
which is obtained by assigning the value of zero to each
of the variables. The solution set has the following
additional properties:
 If u and v are two vectors representing solutions to a
homogeneous system, then the vector sum u + v is also a
solution to the system.
 If u is a vector representing a solution to a homogeneous
system, and r is any scalar, then ru is also a solution to
the system.
Homogeneous System
 Suppose that a homogeneous system of linear
equations has m equations and n variables with n>m.
Then the system has infinitely many solutions.
Example
Example
Example
Generalized Inverse
Generalized Inverse
Generalized Inverse
Method of Obtaining g-Inverse
Example
Method of Obtaining g- Inverse
Example
Properties of g Inverse of X’X
The matrix X’X has an important role in statistics
where it arises in least square equations X’Xb =
X’y. When G is a generalized inverse of X’X:
1. G’ is also a generalized inverse of X’X
2. XGX’X=X, i.e., GX’ is a generalized inverse of X
3. XGX’ is invariant to G
4. XGX’ is symmetric whether G is or not.
Let X+ be the Moore – Penrose inverse of X. Then
XX+ = XGX’ but X+ may not be equal to GX’.
g-Inverse
g-Inverse
Properties
 Every singular symmetric matrix (of order two or




more) has both symmetric and non-symmetric
generalized inverses.
Let A represent a matrix of full column rank
and B a matrix of
full row rank. Then, (1) a matrix G is a
generalized inverse of A if and only if G
is a left inverse of A. And, (2) a matrix G is a
generalized inverse of B if and only
if G is a right inverse of B.
Properties
 For any matrix A and any nonzero scalar k,
(1/k)A− is a generalized inverse of kA.
 For any matrix A,−A− is a generalized inverse
of −A.
 For any matrix A, (A−)’ is a generalized
inverse of A’ .
 For any symmetric matrix A, (A−)’ is a
generalized inverse of A’ .
Properties
 A linear system Ax = b is consistent if and only
if
 AA- b = b
 or, equivalently, if and only if
 (I − AA-)B = 0
The Moore-Penrose Inverse
Given any matrix A, there is a unique matrix M such
that
(i) AMA=A
(ii) MAM=M
(iii) AM is symmetric
(iv) MA is symmetric
Then matrix M is called the Moore-Penrose inverse of
A.
M = L’(K’AL’)-1 K’
The Moore-Penrose Inverse
Example
Example