Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
主讲教师:张德富 Tel: 18959217108 Email: [email protected] 课件下载地址: http://algorithm.xmu.edu.cn/pages/course_download/cours e_algorithm.php 1 教材及参考书 教材 张德富.算法设计与分析(高级教程),国防工业出版社, 2007 参考书 T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, Introduction to Algorithms (the second edition),The MIT Press,2001,中文名《算法导论(第二版)》(影印 版),高等教育出版社,2003 Brassard, G., Bratley, P., Fundamentals of algorithmics, Prentice hall, 2003 D. S. Hochbaum, Approximation Algorithms for NP-hard Problems, PWS Pulishing Company, 1995 Rajeev Motwani and Prabhakar Raghavan, Randomized Algorithms, Cambridge University Press, 1995 2 要求: 50% 考试70% 15%(8个大实验任选4个做,并报告一个) 提交一篇小论文(15%) 额外加分5%:对书中问题或者课堂讨论,回答问题,有新的 想法或者建议的 3 Introduction Why do you study advanced algorithm? The world is of random and approximation Summary Introduction Mathematics Fundaments Probability Fundaments Optimization problem Algorithm Linear Programming vs Integer programming Computational complexity Dealing with NP problems Performance Measurements 5 1.1 Mathematics Fundaments Floors and ceiling x 1 x x x x 1. Modular arithmetic a mod n a a / nn. Exponential and Logarithm functions 1 x e x 1 x x 2 for | x | 1. x n lim (1 ) e x for all x. n n x ln( 1 x) x for x 1. 1 x Harmonic number n 1 ln( n 1) H (n) ln n 1. k 1 k 6 Probability fundaments Discrete random variable A (discrete) random variable X is a function from a finite or countably infinite sample space S to the real numbers. For a random variable X and a real number x, we define the event X = x to be {s∈S : X(s) =x}; thus, Pr{ X x} Pr{s} { sS : X ( s ) x} Especially, suppose we are given a sample space S and an event s. Then the random variable I{s} associated with event s is defined as 1 if s occurs I {s} 0 otherwise 7 Conditional probability formalizes the notion of having prior partial knowledge of the outcome of an experiment. The conditional probability of an event s1 given that another event s2 occurs is defined to be Pr(s1|s2)=Pr(s1∩s2)/Pr(s2) Expected value of a random variable The simplest and most useful summary of the distribution of a random variable is the "average" of the values it takes on. The expected value of a discrete random variable X is E[ X ] x Pr{ X x} x 8 Variance and standard deviation The notion of variance mathematically expresses how far from the mean a random variable‘s values are likely to be. The variance of a random variable X with mean E[X] is Var[X]=E[(X-E[X])2] The standard deviation δ of a random variable X is the positive square root of the variance of X. Namely,δ2=Var[X]. 9 Normal distribution Binomial distribution Central Limit Theorem It turns out that for large samples the sampling distribution of the mean is approximately Normal whatever the distribution of the observations. This result is called the Central Limit Theorem. It allows us to use the Normal distribution as a good approximation for large sample size. Confidence Intervals 10 For example Flipping a coin. 1 if Y H , X H I {Y H } { 0 if Y T . The expected number of heads obtained in one flip of the coin is simply the expected value of our random variable XH: E[XH] = E[I{Y = H}] = 1 · Pr{Y = H} + 0 · Pr{Y = T} = 1 · (1/2) + 0 · (1/2)= 1/2. 11 Lemma Given a sample space S and an event A in the sample space S, let XA = I{A}. Then E[XA] =Pr{A}. Proof :By the definition of a random variable and the definition of expected value, we have E[XA] = E[I{A}] = 1 · Pr{A} + 0 · Pr{Ā} = Pr{A}, 12 Let Xi be the indicator random variable associated with the event in which the ith flip comes up heads. Letting Yi be the random variable denoting the outcome of the ith flip,we have that Xi = I{Yi = H}. Let X be the random variable denoting the total number of heads in the n coin flips, so that n X Xi i 1 13 We wish to compute the expected number of heads, so we take the expectation of both sides of the above equation to obtain n E[ X i ] E[ X ] i 1 n E[ X ] i 1 i n 1 / 2 n / 2. i 1 14 1.2 Computational complexity Optimization problem Definition 1 An optimization problem Q is characterized by three components Instances D: a set of input instances Solutions S(I): the set of all feasible solutions for an instance I∈D Value f: a function which assigns a value to each solution ie f: S(I) →R 15 For example The maximization problem Q: max f ( s) s S (I ) namely, any given I∈D, find a solution s0∈S(I) such that s S ( I ), f (s0 ) f (s) Similarly, one can define the minimization problem. For convenience, we will also refer to the value f(s0) (maximization or minimum value) of the optimal solution as OPT(I), namely OPT(I)= f(s0). 16 Algorithm An algorithm is a sequence of computational steps that transform the input into the output to solve a given problem. Algorithm—problem (optimization problem). A mapping/relation between a set of input instances (domain) and an output set (range) 17 Uncomputable problems vs Intractable problems Genuinely uncomputable problems that just can’t be solved Intractable or impractical problems that we don’t have enough computational power to solve in general. Intractable problems: Easy and hard problems Reductions Decision problem vs. optimization problem 18 P,NP NP-hard, NP-complete Some NP-complete problems SAT, TSP, 0/1 knapsack problem, n queens problem, packing and covering problem, scheduling problem, timetabling problem, network routing problem et al. 19 Dealing with NP problems In practice, we quite often encounter instances of intractable problems that we actually need solutions for: The problem might be generally intractable, but we have a specific instance. For example, scheduling and networking problems often fit into this class. Even when finding optimal solutions is intractable, finding near-optimal solutions might be possible. 20 Dealing with NP problems Your boss asks you to implement an algorithm for an NP-hard optimization problem. What do you do? I couldn’t find a polynomial-time algorithm; I guess I’m too dumb. 21 Dealing with NP problems I couldn’t find a polynomial-time algorithm, but neither could all these other smart people. 22 Dealing with NP problems I couldn’t find a polynomial-time algorithm, because no such algorithm exists! 23 Coping with NP problems Your boss asks you to implement an algorithm for an NP-hard optimization problem. What do you do? -Exact algorithms / Global optimization Dynamic Programming, Backtracking, Branch-andbound and variants (either hope that the input is very small or that worst case manifests itself very rarely) (Special cases maybe solvable in polynomial time) -Linear programming algorithms 24 -Heuristic algorithms / Local optimization Greedy, Local Search, Meta-heuristics (Tabu search, GA, SA, Ant Algorithms, NN) etc Approximation algorithms Polynomial-time algorithms with provable performance bounds Randomized algorithms DNA and quantum computation 25 Other problems except NP problems The world behaves randomly and fuzzily, so randomization and approximation are themes that cut across many areas of science and engineering. Other problems except NP problems can be solved by making use of randomized and approximation algorithms, for example Deadlock and symmetry problems Computational number theory( cryptography) Random sampling pattern matching …… 26 Performance Measurements Heuristic algorithm What do we expect from a “good” heuristic: 1. It should do so quickly 2. It should find a solution of good quality Tools to measure heuristic performances: 1. Run time analysis 2. Analysis on solution quality (experiment) 27 Performance Measurements Approximation algorithms What do we expect from a “good” approximation: 1. It runs quickly 2. It should find a solution of good quality Tools to measure approximation performances: 1. Run time analysis 2. Solution quality with performance guarantee( approximation ratio) 28 Performance Measurements Randomized algorithms What do we expect from a “good” randomized algorithm: 1. It runs quickly 2. It should make a mistake as small as possible Tools to measure randomized performances: 1. Run time analysis 2. Analysis on solution quality ( The error probability) 29 1.3 Linear programming problem(LP) Specify the objective of the optimization problem as a linear function of certain variables, and specify the constraints on resources as equalities or inequalities on those variables. 30 General linear programs Given a set of real numbers a1, a2, ..., an and a set of variables x1, x2,..., xn, a linear function f on those variables is defined by n f ( x1 ,, xn ) a1 x1 an xn a j x j linear equality: f ( x1 ,, xn ) b linear inequalities: f ( x1 ,, xn ) b and f ( x1 ,, xn ) b j 1 31 Linear constraints equalities or linear equalities. Formally a linear-programming problem is the problem of either minimizing or maximizing a linear function subject to a finite set of linear constraints. Minimization linear program Maximization linear program 32 Example min 2 x1 3 x2 s.t. x1 x2 7 x1 2 x2 4 x1 0 33 Standard forms Given n real numbers c1, c2, ..., cn; m real numbers b1, b2, ..., bm; and mn real numbers aij for i = 1, 2, ..., m and j = 1, 2, ..., n. We wish to find n real numbers x1, x2,..., xn that n max c j x j j 1 Subject to n a x j 1 ij xj 0 j bj for i 1,, m for j 1,, n. 34 Feasible solution Infeasible solution Objective value Optimal solution Optimal objective value Unbounded 35 Example max 2 x1 3x2 3x3 s.t. x1 x2 x3 7 x1 x2 x3 7 x1 2 x2 2 x3 4 x1 , x2 , x3 0 36 Slack forms z v cjxj jN xi bi aij x j , jN xi 0. 37 Example Standard forms max 2 x1 3x2 3x3 s.t. x1 x2 x3 7 x1 x2 x3 7 x1 2 x2 2 x3 4 x1 , x2 , x3 0 Slack forms z 2 x1 3x2 3x3 x4 7 x1 x2 x3 x5 7 x1 x2 x3 x6 4 x1 2 x2 2 x3 x1 , x2 , x3 , x4 , x5 , x6 0 38 The simplex algorithm The simplex algorithm takes as input a linear program and returns an optimal solution. It starts at some vertex of the simplex and performs a sequence of iterations. In each iteration, it moves along an edge of the simplex from a current vertex to a neighboring vertex whose objective value is no smaller than that of the current vertex (and usually is larger.) 39 The simplex algorithm terminates when it reaches a local maximum, which is a vertex from which all neighboring vertices have a smaller objective value. Because the feasible region is convex and the objective function is linear, this local optimum is actually a global optimum. 40 Integer Programming (IP) Integer Programming is simply Linear Programming with an added condition: All variables must be integers Many problems can be stated as Integer Programs. For example, the Vertex Cover problem can be stated as an integer programming problem. Given an undirected graph G = (V, E), find a minimum subset V' of V such that if (u, v)∈E, then u∈V' or v ∈V' (or both). 41 Suppose that we associate a variable x(v) with each vertex v∈V , and let us require that x(v)∈{0, 1} for each v∈V. 0-1 integer program for finding a minimum vertex cover(VC): min s.t. x (v ) vV ' x(u ) x(v) 1 for any edge (u, v)∈ E x (v ) {0,1} for each v∈ V 42 0-1 linear programming relaxation : min s.t. x (v ) vV ' x(u ) x(v) 1 x (v ) 1 x (v ) 0 for any edge (u, v)∈ E for each v∈ V for each v∈ V 43 Homework Exercise 1.3 2 实现微信红包程序或者随机点名程序 44