Download Presentation Slides

Randomized Algorithms
Morteza ZadiMoghaddam
Amin Sayedi
Types of Randomized algorithms


Las Vegas
Monte Carlo
Las Vegas




Always gives the true answer.
Running time is random.
Running time is bounded.
Quick sort is a Las Vegas algorithm.
Monte Carlo



It may produce incorrect answer!
We are able to bound its probability.
By running it many times on independent
random variables, we can make the failure
probability arbitrarily small at the expense
of running time.
Monte Carlo Example

Suppose we want to find a number among
n given numbers which is larger than or
equal to the median.
Monte Carlo Example
Suppose A1 < … < An.
We want Ai, such that i ≥ n/2.
It’s obvious that the best deterministic
algorithm needs O(n) time to produce the
answer.
n may be very large!
Suppose n is 100,000,000,000 !
Monte Carlo Example



Choose 100 of the numbers with equal
probability.
find the maximum among these numbers.
Return the maximum.
Monte Carlo Example



The running time of the given algorithm is
O(1).
The probability of Failure is 1/(2100).
Consider that the algorithm may return a
wrong answer but the probability is very
smaller than the hardware failure or even
an earthquake!
Monte Carlo



Suppose the output is Yes or No.
One sided error.
Two sided error.
RP Class
( randomized polynomial )




Bounded polynomial time in the worst
case.
If the answer is Yes; Pr[ return Yes] > ½.
If the answer is No; Pr[ return Yes] = 0.
½ is not actually important.
PP Class
( probabilistic polynomial )




Bounded polynomial time in worst case.
If the answer is Yes; Pr[ return Yes] > ½.
If the answer is No; Pr[ return Yes] < ½.
Unfortunately the definition is weak
because the distance to ½ is important
but is not considered.
Routing Problem





There are n computers.
Each computer has a packet.
Each packet has a destination D(i).
Packets can not follow the same edge
simultaneously.
An oblivious algorithm is required.
Routing Problem

For any deterministic oblivious algorithm
on a network of N nodes each of
outdegree d, there is an instance of
permutation routing requiring (N/d) ½.
Routing Problem




Pick random intermediate destination.
Packet i first travels to the intermediate
destination and then to the final
destination.
With probability at least 1-(1/N), every
packet reaches its destination in 14n of
fewer steps in Qn.
The expected number of steps is 15n.
Maximum Satisfiability




You have m clauses and n boolean
variables.
Each clause contains some of variables or
some of complements.
A clause is satisfied if at least one of it’s
variables are satisfied.
We want to set the variables such that the
number of satisfied clauses is maximized.
Example for Maximum Sat






There are 3 variables A, B and C.
M1 = (A) or (B)
M2 = (A) or (not B) or (not C)
M3 = (C)
M4 = (B) or (not C)
M5 = (not C)
Example of Maximum Sat

Set A = True
Set B = True
Set C = False

Four of the clauses are satisfied.


Maximum Sat

This problem is a famous problem which
has no polynomial time algorithm yet. It’s
NP-hard.
Maximum Sat

For any set of m clauses, there is truth
assignment for the variables that satisfies
at least m/2 clauses.
Maximum Sat





Let Zi =1 if the i-th clause is satisfied and
0 otherwise.
Set the variables in a random way.
The probability of a clause with k variables
to be true is 1- (1/(2k)) >= ½.
So E[Z1]+…+E[Zm] >= ½.
Thus there exist at least one assignment
such that Z1+…+Zm >= ½.
Maximum Sat algorithm



This problem is NP-hard so we seek for
approximation algorithms.
We have an algorithm that produces an answer
which is at least ½ of the best answer.
If all clauses have at least 2 literals then we
have an algorithm that produces an answer
which is at least ¾ of the best one.
Maximum Sat algorithm








We want to maximize Z1+…+Zm.
We have some inequalities:
∑ yi (if Xi is in Zj and is uncomplemented)
∑ (1-yi) (if Xi is in Zj and is complemented).
This inequality must be hold:
∑ yi + ∑ (1-yi) >= Zj
This problem could be solved using integer
linear programming.
We have to use linear programming.
Maximum Sat




Solve the problem using linear
programming.
You get a real number for each yi or zi.
Assign Xi true with the probability yi.
The expected number of clauses that are
satisfied is (1- 1/e) of the best answer.
Maximum Sat algorithm

Using both algorithms and choosing the
better answer gives us an answer which is
at least ¾ of the best answer!!! Which is
better than ½ and 1- 1/e.
2-Sat







Every clause has at most 2 literals.
We want to check if all clauses can be satisfied.
It has polynomial algorithm.
Assign random values to the variables.
If all clauses are satisfied we are finished.
If there is an unsatisfied clause, the value of one
of it’s literals is different from the best answer.
Change the value of one of the variables in this
clause. You may make a good change or bad
one.
2-Sat





You are walking on a path.
If you are on 0 you go to 1.
If you are on i you go to i+1 or i-1 with
equal probability.
The expected number of steps to reach
the end of the path is O(n2).
So the given algorithm is O(n3).
Graph Connectivity





You want to check if two vertices u and v
are in the same connected component.
Start a random walk from v.
Have a random walk of length 2n3.
If you haven’t visited u, the probability of
u to be in this component is less than ½.
By repeating this algorithm, you can make
the probability of failure arbitrarily small.
Graph Connectivity


Running time of algorithm is O(n3).
Required space is O(logn).
Diameter of a Point Set



You want to find the diameter of set of n
points in the space.
Suppose I(x) is the convex body formed
by the intersection of n sphere centered at
n points with radius x.
F(p) is distance between p and the point
in the set that is farthest from p.
Diameter of a Point Set



Consider I(x) when x=F(p).
For any q in S, if q is in I(x) then
F(q)<F(p).
And if q is not in I(x) then F(p)<F(q).
Diameter of a Point Set




Pick a point p in s at random. Computer
F(p). [O(n)]
Set x=F(p). Compute I(x). [O(n logn)]
Find the points outside I(x). Call this set T.
[O(n logn)]
If T is empty return x as the answer, else
continue on T.
Diameter of a Point Set


The running time of algorithm above is
O(n log n).
In each step, all points that have smaller
F(x) than the chosen point are removed.
All-Pairs Shortest Paths



Let G(V,E) be an undirected, connected
graph with V={1,…,n} and |E|=m.
The adjacency matrix A is an n  n 0-1
matrix with Aij=Aji=1 if and only if the
edge (i,j) is present in E.
We are going to compute matrix D which
Dij equals the length of a shortest path
from vertex i to vertex j.
All-Pairs Distances






Z A2
Compute matrix A’ such that A’ij=1 if and only if
i ≠ j and (Aij=1 or Zij>0)
If A’ij=1 for all i ≠ j then return D = 2A’-A.
Recursively compute the APD matrix D’ for the
graph G’ with adjacency matrix A’.
S AD’
Return matrix D with Dij=2D’ij if Sij≥D’ijZii ,
otherwise Dij=2D’ij-1.
APSP


The APD algorithm computes the distance
matrix for an n-vertex graph in time
O(MM(n)log(n)) using integer matrix
multiplication algorithm.
Matrix multiplication algorithm running in
time O(n2.376).
Boolean Product Witness Matrix


Suppose A and B are nn boolean
matrices and P=AB is their product under
Boolean matrix multiplication.
A witness for Pij is an index k  {1,…,n}
such that Aik=Akj=1. Observe that Pij=1 if
and only if it has some witness k.
BPWM




W -AB
2. for t=0,…, log(n) do
2.1. r 2t
2.2. Repeat 3.77log(n) times




2.2.1. Choose random R  {1,…,n} with |R|=r .
2.2.2. Compute AR and BR .
2.2.3. Z ARBR .
2.2.4. for all (i,j) do


if Wij < 0 and Zij is witness then Wij Zij
3. for all (i,j) do

if Wij < 0 then find witness Wij by brute force.
BPWM



The BPWM algorithm is a Las Vegas
algorithm for the BPWM problem with
expected running time O(MM(n)log2(n)).
The probability that no witness is found
for Pij before the end of Step 2 is at most
(1-1/2e)3.77log(n)  1/n.
Determining Shortest Path

A successor matrix S for an n-vertex graph
G is an n  n matrix such that Sij is the
index of a neighbor of vertex i that lies on
a shortest path from i to j.
APSP


Compute the distance matrix D=APD(A).
for s={0,1,2} do



Compute 0-1 matrix Dkj(s)=1 if and only if
Dkj+1 = s (mod 3)
Compute the witness matrix
W(s)=BPWM(A,D(s)).
Compute successor matrix S for G.
APSP

Algorithm APSP computes the successor
matrix for an n-vertex graph G in expected
time O(MM(n)log2(n)).
Algorithm contract


H G
While H has more than 2 vertices do




Choose an edge (x,y) uniformly at random
from the edges in H.
F F  {(x,y)}.
H H / (x,y).
(C,V/C) the sets of vertices corresponding
to the two meta-vertices in H=G/F.
FastCut


n |v|
if n  6 then compute min-cut of G by bruteforce enumeration else




t 1+n/2
Using Algorithm Contract, perform two independent
contraction sequences to obtain graphs H1 and H2
each with t vertices.
Recursively compute min-cuts in each of H1 and H2.
Return the smaller of the two min-cuts.
Fastcut


Algorithm Fastcut succeeds in finding a
min-cut with probability (1/log(n)).
Algorithm Fastcut runs in O(n2log(n)) time
and uses O(n2) space.
MST


Finding MST in a graph with n vertices and
m edges has a Las Vegas algorithm which
has the expected running time O(n+m).
But We don’t have enough time to Explain
it !!!
Research problems


Devise an algorithm for the all-pairs shortest
paths problem that does not use matrix
multiplication and runs in time O(n3-) for a
positive constant .
Devise an algorithm for computing the diameter
of an unweighted graph that does not use
matrix multiplication and runs in time O(n3-) for
a positive constant .
Research problems


Devise a Las Vegas or a deterministic
algorithm for min-cuts with running time
close to O(n2).
Is there a randomized algorithm for mincuts with expected running time close to
O(m)?