Download Document


主讲教师：张德富
Tel: 18959217108
Email: [email protected]

课件下载地址：


http://algorithm.xmu.edu.cn/pages/course_download/cours
e_algorithm.php
1
教材及参考书

教材
张德富.算法设计与分析（高级教程）,国防工业出版社, 2007

参考书




T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein,
Introduction to Algorithms (the second edition)，The MIT
Press，2001，中文名《算法导论（第二版）》（影印
版），高等教育出版社，2003
Brassard, G., Bratley, P., Fundamentals of algorithmics,
Prentice hall, 2003
D. S. Hochbaum, Approximation Algorithms for NP-hard
Problems, PWS Pulishing Company, 1995
Rajeev Motwani and Prabhakar Raghavan, Randomized
Algorithms, Cambridge University Press, 1995
2
要求:

50%




考试70%
15%(8个大实验任选4个做，并报告一个）
提交一篇小论文（15%）
额外加分5%:对书中问题或者课堂讨论,回答问题,有新的
想法或者建议的
3
Introduction
Why do you study advanced algorithm?
The world is of random and approximation

Summary









Introduction
Mathematics Fundaments
Probability Fundaments
Optimization problem
Algorithm
Linear Programming vs Integer programming
Computational complexity
Dealing with NP problems
Performance Measurements
5
1.1 Mathematics Fundaments

Floors and ceiling
x  1  x  x  x  x  1.

Modular arithmetic
a mod n  a  a / nn.

Exponential and Logarithm functions
1  x  e x  1  x  x 2 for | x | 1.
x n
lim (1  )  e x for all x.
n 
n
x
 ln( 1  x)  x for x  1.
1 x

Harmonic number
n
1
ln( n  1)  H (n)    ln n  1.
k 1 k
6
Probability fundaments


Discrete random variable
A (discrete) random variable X is a function from a finite or
countably infinite sample space S to the real numbers.
For a random variable X and a real number x, we define the
event X = x to be {s∈S : X(s) =x}; thus,
Pr{ X  x} 
 Pr{s}
{ sS : X ( s )  x}

Especially, suppose we are given a sample space S and an event
s. Then the random variable I{s} associated with event s is
defined as
1 if s occurs
I {s}  
0 otherwise
7


Conditional probability formalizes the notion of having prior
partial knowledge of the outcome of an experiment. The
conditional probability of an event s1 given that another event s2
occurs is defined to be
Pr(s1|s2)=Pr(s1∩s2)/Pr(s2)
Expected value of a random variable
The simplest and most useful summary of the distribution of a
random variable is the "average" of the values it takes on. The
expected value of a discrete random variable X is
E[ X ]   x Pr{ X  x}
x
8



Variance and standard deviation
The notion of variance mathematically expresses how
far from the mean a random variable‘s values are
likely to be. The variance of a random variable X
with mean E[X] is
Var[X]=E[(X-E[X])2]
The standard deviation δ of a random variable X is
the positive square root of the variance of X.
Namely,δ2=Var[X].
9




Normal distribution
Binomial distribution
Central Limit Theorem
It turns out that for large samples the sampling
distribution of the mean is approximately Normal
whatever the distribution of the observations. This
result is called the Central Limit Theorem. It allows
us to use the Normal distribution as a good
approximation for large sample size.
Confidence Intervals
10
For example

Flipping a coin.
1 if Y  H ,
X H  I {Y  H }  {
0 if Y  T .
 The expected number of heads obtained in one
flip of the coin is simply the expected value of our
random variable XH:
E[XH] = E[I{Y = H}]
= 1 · Pr{Y = H} + 0 · Pr{Y = T}
= 1 · (1/2) + 0 · (1/2)= 1/2.
11

Lemma
Given a sample space S and an event A in the sample
space S, let XA = I{A}. Then E[XA] =Pr{A}.
Proof :By the definition of a random variable and the
definition of expected value, we have
E[XA] = E[I{A}]
= 1 · Pr{A} + 0 · Pr{Ā}
= Pr{A},
12

Let Xi be the indicator random variable
associated with the event in which the ith flip
comes up heads. Letting Yi be the random
variable denoting the outcome of the ith flip,we
have that Xi = I{Yi = H}. Let X be the random
variable denoting the total number of heads in
the n coin flips, so that
n
X   Xi
i 1
13

We wish to compute the expected number of
heads, so we take the expectation of both sides
of the above equation to obtain
n
E[ X i ]
E[ X ] 
i 1
n
 E[ X ]

i 1

i
n
1 / 2  n / 2.
i 1
14
1.2 Computational complexity

Optimization problem
Definition 1 An optimization problem Q is characterized
by three components
Instances D: a set of input instances
Solutions S(I): the set of all feasible solutions for an
instance I∈D
Value f: a function which assigns a value to each solution
ie f: S(I) →R
15
For example

The maximization problem Q:
max f ( s)
s  S (I )
namely, any given I∈D, find a solution s0∈S(I) such
that
s  S ( I ), f (s0 )  f (s)


Similarly, one can define the minimization problem.
For convenience, we will also refer to the value f(s0)
(maximization or minimum value) of the optimal
solution as OPT(I), namely OPT(I)= f(s0).
16
Algorithm


An algorithm is a sequence of computational
steps that transform the input into the output to
solve a given problem.
Algorithm—problem (optimization problem).
A mapping/relation between a set of input instances
(domain) and an output set (range)
17

Uncomputable problems vs Intractable problems





Genuinely uncomputable problems that just can’t be
solved
Intractable or impractical problems that we don’t
have enough computational power to solve in general.
Intractable problems: Easy and hard problems
Reductions
Decision problem vs. optimization problem
18



P,NP
NP-hard, NP-complete
Some NP-complete problems
SAT, TSP, 0/1 knapsack problem, n queens problem,
packing and covering problem, scheduling problem,
timetabling problem, network routing problem et al.
19
Dealing with NP problems


In practice, we quite often encounter instances of
intractable problems that we actually need solutions for:
 The problem might be generally intractable, but we
have a specific instance.
 For example, scheduling and networking problems
often fit into this class.
Even when finding optimal solutions is intractable,
finding near-optimal solutions might be possible.
20
Dealing with NP problems

Your boss asks you to implement an algorithm for an
NP-hard optimization problem. What do you do?
I couldn’t find a polynomial-time algorithm;
I guess I’m too dumb.
21
Dealing with NP problems
I couldn’t find a polynomial-time algorithm, but neither
could all these other smart people.
22
Dealing with NP problems
I couldn’t find a polynomial-time algorithm, because
no such algorithm exists!
23
Coping with NP problems

Your boss asks you to implement an algorithm for an
NP-hard optimization problem. What do you do?
-Exact algorithms / Global optimization
Dynamic Programming, Backtracking, Branch-andbound and variants
(either hope that the input is very small or that worst
case manifests itself very rarely)
(Special cases maybe solvable in polynomial time)
-Linear programming algorithms
24
-Heuristic algorithms / Local optimization
Greedy, Local Search, Meta-heuristics (Tabu
search, GA, SA, Ant Algorithms, NN) etc
Approximation algorithms
Polynomial-time algorithms with provable
performance bounds
Randomized algorithms
DNA and quantum computation
25
Other problems except NP problems


The world behaves randomly and fuzzily, so
randomization and approximation are themes that cut
across many areas of science and engineering.
Other problems except NP problems can be solved by
making use of randomized and approximation
algorithms, for example
Deadlock and symmetry problems
Computational number theory( cryptography)
Random sampling
pattern matching
……
26
Performance Measurements
Heuristic algorithm
What do we expect from a “good” heuristic:
1. It should do so quickly
2. It should find a solution of good quality
 Tools to measure heuristic performances:
1. Run time analysis
2. Analysis on solution quality (experiment)

27
Performance Measurements
Approximation algorithms
What do we expect from a “good” approximation:
1. It runs quickly
2. It should find a solution of good quality
 Tools to measure approximation performances:
1. Run time analysis
2. Solution quality with performance
guarantee( approximation ratio)

28
Performance Measurements
Randomized algorithms
What do we expect from a “good” randomized
algorithm:
1. It runs quickly
2. It should make a mistake as small as possible
 Tools to measure randomized performances:
1. Run time analysis
2. Analysis on solution quality ( The error
probability)

29
1.3 Linear programming problem(LP)

Specify the objective of the optimization
problem as a linear function of
certain variables, and specify the constraints on
resources as equalities or inequalities on those
variables.
30
General linear programs

Given a set of real numbers a1, a2, ..., an and a set of
variables x1, x2,..., xn, a linear function f on those
variables is defined by
n
f ( x1 ,, xn )  a1 x1    an xn   a j x j
linear equality:
f ( x1 ,, xn )  b
linear inequalities:
f ( x1 ,, xn )  b and f ( x1 ,, xn )  b
j 1
31
Linear constraints
equalities or linear equalities.
 Formally a linear-programming problem is the
problem of either minimizing or maximizing a
linear function subject to a finite set of linear
constraints.
 Minimization linear program
 Maximization linear program

32
Example
min
 2 x1  3 x2
s.t.
x1  x2
 7
x1  2 x2
 4
x1  0
33
Standard forms

Given n real numbers c1, c2, ..., cn; m real numbers b1,
b2, ..., bm; and mn real numbers aij for i = 1, 2, ..., m and
j = 1, 2, ..., n. We wish to find n real numbers x1, x2,...,
xn that
n
max  c j x j
j 1
Subject to
n
a x
j 1
ij
xj  0
j
 bj
for
i  1,, m
for
j  1,, n.
34






Feasible solution
Infeasible solution
Objective value
Optimal solution
Optimal objective value
Unbounded
35
Example
max
2 x1  3x2  3x3
s.t.
x1  x2  x3  7
 x1  x2  x3   7
x1  2 x2  2 x3  4
x1 , x2 , x3  0
36
Slack forms
z  v  cjxj
jN
xi  bi   aij x j ,
jN
xi  0.
37
Example
Standard forms
max
2 x1  3x2  3x3
s.t.
x1  x2  x3  7
 x1  x2  x3   7
x1  2 x2  2 x3  4
x1 , x2 , x3  0
Slack forms
z  2 x1  3x2  3x3
x4  7  x1  x2  x3
x5   7  x1  x2  x3
x6  4  x1  2 x2  2 x3
x1 , x2 , x3 , x4 , x5 , x6  0
38
The simplex algorithm

The simplex algorithm takes as input a linear
program and returns an optimal solution. It starts
at some vertex of the simplex and performs a
sequence of iterations. In each iteration, it moves
along an edge of the simplex from a current
vertex to a neighboring vertex whose objective
value is no smaller than that of the current vertex
(and usually is larger.)
39

The simplex algorithm terminates when it reaches a
local maximum, which is a vertex from which all
neighboring vertices have a smaller objective value.
Because the feasible region is convex and the objective
function is linear, this local optimum is actually a global
optimum.
40
Integer Programming (IP)



Integer Programming is simply Linear Programming
with an added condition:
All variables must be integers
Many problems can be stated as Integer Programs.
For example, the Vertex Cover problem can be stated
as an integer programming problem.
Given an undirected graph G = (V, E), find a minimum
subset V' of V such that if (u, v)∈E,
then u∈V' or v ∈V' (or both).
41

Suppose that we associate a variable x(v) with each
vertex v∈V , and let us require that x(v)∈{0, 1} for
each v∈V. 0-1 integer program for finding a minimum
vertex cover(VC):
min
s.t.
 x (v )
vV '
x(u )  x(v)  1 for any edge (u, v)∈ E
x (v )  {0,1} for each v∈ V
42

0-1 linear programming relaxation :
min
s.t.
 x (v )
vV '
x(u )  x(v)  1
x (v )  1
x (v )  0
for any edge (u, v)∈ E
for each v∈ V
for each v∈ V
43
Homework
Exercise 1.3
2
实现微信红包程序或者随机点名程序
44