Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Backgrounds-Convexity Def: line segment joining two points x1 , x 2 R n is the collection of points x x1 (1 ) x 2 , 0 1 ( x 1 x1 2 x 2 , 1 2 1, 1 , 2 0) (Generally, x im1 i x i , im1 i 1, i 0 i, called convex combination) x2 x 2 ( x1 x 2 ) x2 x1 x1 ( x1 x 2 ) OR-1 2009 1 Def: C R n is called convex set iff x1 (1 ) x 2 C whenever x1 C , x 2 C , and 0 1. Convex sets OR-1 2009 Nonconvex set 2 Def: The convex hull of a set S is the set of all points that are convex combinations of points in S, i.e. conv(S)={x: x = i = 1k i xi, k 1, x1,…, xkS, 1, ..., k 0, i = 1k i = 1} Picture: 1x + 2y + 3z, i 0, i = 13 i = 1 1x + 2y + 3z = (1+ 2){ 1 /(1+ 2)x + 2 /(1+ 2)y} + 3z (assuming 1+ 2 0) z x y OR-1 2009 3 Proposition: Let C R n be a convex set and for k R , define kC { y R n | y kx for some x C} Then kC is a convex set. Pf) If k = 0, kC is convex. Suppose k 0. For any x, y kC, x' , y' C such that x kx' , y ky' Then x (1 ) y kx'(1 )ky' k (x'(1 ) y' ) kC Hence the property of convexity of a set is preserved under scalar multiplication. Consider other operations. OR-1 2009 4 Convex function Def: Function f : R n R is called a convex function iff f (x1 (1 ) x 2 ) f ( x1 ) (1 ) f ( x 2 ), 0 1, Also called strictly convex function iff f (x1 (1 ) x 2 ) f ( x1 ) (1 ) f ( x 2 ), 0 1, f (x) ( x1, f ( x1)) R n 1 ( x 2 , f ( x 2 )) f ( x1) (1 ) f ( x 2 ) f (x1 (1 ) x 2 ) 1 x OR-1 2009 x x1 (1 ) x 2 x2 5 Equivalent definition: f is called convex if the points above or on the graph of f(x) (points in R epigraph of f) is a convex set Def: f is a concave function if –f is a convex function. Def: xC is an extreme point of a convex set C if x cannot be expressed as y (1 ) z, 0 1 for distinct y, z C ( x y, z ) : extreme points OR-1 2009 6 Review-Linear Algebra 2 x1 x2 5x3 x4 20 x1 5x2 4 x3 5x4 30 3x1 x2 6 x3 2 x4 20 2 1 5 1 A 1 5 4 5 3 1 6 2 OR-1 2009 x1 x x 2 x3 x4 20 b 30 20 Ax b in matrix, vector notation 7 Submatrices multiplication a11 a12 A a21 a22 a31 a32 a13 a23 a33 a14 a a24 11 A21 a34 A12 A22 b1 b 2 b1 B b3 B2 b4 a11b1 A12 B2 AB A21b1 A22 B2 OR-1 2009 8 submatrices multiplication which will be frequently used. a11 a12 a 21 a22 A am1 am 2 Ax b A1 y' A y'A1 y ' A y1 OR-1 2009 a1n a1' a ' a2n A1 A2 An 2 amn am ' x1 x A2 An 2 A1x1 A2 x2 An xn b x3 x4 A2 An y' A1 y' A2 y' An a1' a ' y2 ym 2 y1a1' y2a2 ' ym am ' am ' 9 B 1 Ax B 1b B 1 A B 1 A1 Suppose A B A2 An B 1 A1 B 1 A2 B 1 An N where B : m m, (nonsingul ar) N : m (n m) Then B 1 A B 1 B N B 1B B 1 N I OR-1 2009 B 1 N 10 1 2 m 1 2 m Def: {x , x , , x } is said to be linearly dependent if c1 , c2 ,, cm 1 2 m , not all equal to 0, such that c1 x c2 x cm x 0 ( i.e., there exists a vector in {x1 , x 2 , , x m } which can be expressed as a linear combination of the other vectors. ) Def: {x , x , , x } linearly independent if not linearly dependent. i In other words, im1 ci x 0 implies ci 0 for all i 1 2 m (i.e., none of the vectors in {x , x , , x } can be expressed as a linear combination of the remaining vectors.) Def: Rank of a set of vectors : maximum number of linearly independent vectors in the set. Def: Basis for a set of vectors : collection of linearly independent vectors from the set such that every vector in the set can be expressed as a linear combination of them. (maximal linearly independent subset, minimal generator of the set) OR-1 2009 11 Thm) r linearly independent vectors form a basis iff the set has rank r. Def: row rank of a matrix : rank of its set of row vectors column rank of a matrix : rank of its set of column vectors Thm) row rank = column rank Def : nonsingular matrix : rank = number of rows = number of columns. Otherwise, called singular Thm) If A is nonsingular, then unique inverse exists. ( AA1 I A1 A) OR-1 2009 12 Simutaneous Linear Equations Thm: Ax = b has at least one solution iff rank(A) = rank( [A, b] ) Pf) ) rank( [A, b] ) rank(A). Suppose rank( [A, b] ) > rank(A). Then b is lin. ind. of the column vectors of A, i,e., b can’t be expressed as a linear combination of columns of A. Hence Ax = b does not have a solution. ) There exists a basis in columns of A which generates b. So Ax = b has a solution. Suppose A: mn, rank(A) = rank [A, b] = r. Then Ax = b has a unique solution if r = n. has infinitely many solutions if r < n. (m-r equations are redundant) OR-1 2009 13 Operations that do not change the solution set of the linear equations (Elementary row operations) Change the position of the equations Multiply a nonzero scalar k to both sides of an equation Multiply a scalar k to an equation and add it to another equation X {x | a1 ' x b1 , a2 ' x b2 ,, am ' x bm } Y {x | (a1 'ka2 ' ) x (b1 kb2 ), a2 ' x b2 ,, am ' x bm } Show that x* X implies x* Y ( X Y ) and x * Y implies x* X (Y X ) Hence X = Y The operations can be performed on the coefficient matrix A, b. OR-1 2009 14 Solving systems of linear equations (Gauss-Jordan Elimination, 변수의 치환) (will be used in the simplex method to solve LP problems) x1 x2 x1 3x2 2 x2 4 x3 10 10 5 x3 22 6 x3 20 2 x3 10 x3 2 x1 x2 x1 x2 2 x2 2 x2 4 x3 10 4 x3 20 5 x3 22 x1 x2 x2 2 x2 4 x3 10 2 x3 10 5 x3 22 OR-1 2009 x1 x2 8 6 x3 2 15 Infinitely many solutions x1 x2 x1 3x2 2 x2 x1 x2 2 x2 2 x2 x1 x2 x2 2 x2 4 x3 x4 x4 5 x3 x4 10 10 22 4 x3 x4 4 x3 2 x4 5 x3 x4 10 20 22 4 x3 x4 2 x3 x4 5 x3 x4 10 10 22 x1 x2 6 x3 2 x4 2 x3 x4 x3 x4 20 10 2 x2 8 x4 3x4 x3 x4 x2 8 8 x4 6 3x4 x3 2 x4 x1 x1 8 6 2 Assign x4 t for arbitrary t and get x1 8 8t , x2 6 3t , x3 2 t OR-1 2009 16 x1 x2 x3 8 8 x4 6 3x4 2 x4 x4 is independent variable and x1 , x2 , x3 are dependent variables. Particularly, the solution obtained by setting the indepent variables to 0 and solving for the dependent variables is called a basic solution. Here x1 8, x2 6, x3 2, x4 0 (will be used in the simplex method) OR-1 2009 17 x1 x2 x1 x2 8 x4 3x4 x3 x4 6 x3 2 x4 2 x3 x4 x3 x4 8 6 2 x1 8 8x4 , x2 6 3x4 , x3 2 x4 20 10 2 x1 x2 8 x3 3x3 x3 x4 24 12 2 x1 24 8x3, x2 12 3x3, x4 2 x3 Both systems have the same set of solutions, but representation is different e.g.) x1 8, x2 6, x3 2, x4 0 OR-1 2009 18 Elementary row operations are equivalent to premultiplying a nonsingular square matrix to both sides of the equations x1 x2 x1 3x2 2 x2 x1 x2 2 x2 2 x2 4 x3 10 10 5 x3 22 4 x3 10 4 x3 20 5 x3 22 1 1 4 x1 10 1 3 0 x 10 2 0 2 5 x3 22 1 1 1 4 x1 1 10 1 1 1 3 0 x 1 1 10 2 1 0 2 5 x3 1 22 1 1 4 x1 10 0 2 4 x2 20 0 2 5 x3 22 OR-1 2009 19 x1 x2 2 x2 2 x2 4 x3 10 4 x3 20 5 x3 22 x1 x2 x2 2 x2 4 x3 10 2 x3 10 5 x3 22 1 1 1 4 x1 1 10 1 1 1 3 0 x 1 1 10 2 1 0 2 5 x3 1 22 1 1 1 1 4 x1 1 1 10 1 / 2 1 1 1 3 0 x 1 / 2 1 1 10 2 1 1 0 2 5 x3 1 1 22 OR-1 2009 20 1 1 1 1 4 x1 1 1 10 1 / 2 1 1 1 3 0 x 1 / 2 1 1 10 2 1 1 0 2 5 x3 1 1 22 1 1 1 4 x1 1 10 1 / 2 0 2 4 x2 1 / 2 20 1 0 2 5 x3 1 22 1 1 1 4 x1 1 10 1 / 2 1 / 2 1 3 0 x2 1 / 2 1 / 2 10 1 0 2 5 x3 1 22 OR-1 2009 21 So if we multiply all elementary row operation matrices, we get the matrix having the information about the elementary row operations we performed x1 x2 x1 3x2 2 x2 4 x3 10 10 5 x3 22 x1 x2 8 6 x3 2 7.5 6.5 6 1 1 4 x1 7.5 6.5 6 10 2.5 2.5 2 1 3 0 x 2.5 2.5 2 10 2 1 0 2 5 x3 1 1 1 22 1 1 1 0 0 x1 8 0 1 0 x2 6 0 0 1 x3 2 OR-1 2009 7.5 6.5 6 A1 2.5 2.5 2 1 1 1 22 Finding inverse of a nonsingular matrix A. Perform elementary row operations (premultiply elementary row operation matrices) to make [A : I ] to [ I : B ] Let the product of the elementary row operations matrices be C. Then C [ A : I ] = [ CA : C ] = [ I : B] Hence CA = I C = A-1 and B = C OR-1 2009 23