Download Theory of Computational Complexity_sakaidani

1
PROBABILITY AND COMPUTING
RANDOMIZED ALGORITHMS AND
PROBABILISTIC ANALYSIS
CHAPTER 1
IWAMA and ITO Lab. M1 Sakaidani Hikaru
About the Book
2







Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
 Chapter 9
 Chapter 10
Chapter 11
Corematerial
 Chapter 12
 Chapter 13
 Chapter 14

About the Book
3
Chapter 1
 Chapter 2
 Chapter 3
Additional
 Chapteradvanced
4
material
 Chapter
5
 Chapter 6
 Chapter 7








Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
Chapter 14
About the Book
4







Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Review elementary
 Chapter 9
probability theory
 Chapter 10
 Chapter 11
 Chapter 12
 Chapter 13
 Chapter 14

About the Book
5







Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
 Chapter 9
 Chapter 10
 Chapter 11
 Chapter 12
More advanced topics
 Chapter 13
 Chapter 14

About the Book
6







Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7







Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
Chapter 14
CHAPTER 1
Events and probability
7
1.1 Verifying Polynomial Identities
1.2 Axioms of Probability
1.3 Verifying Matrix Multiplication
1.4 A Randomized Min-Cut Algorithm
CHAPTER 1
Events and probability
8
1.1 Verifying Polynomial Identities
1.2 Axioms of Probability
1.3 Verifying Matrix Multiplication
1.4 A Randomized Min-Cut Algorithm
Verifying Polynomial Identities
9

The problem of verifying the following identity
?

More generally…
?
: canonical form
Verifying Polynomial Identities
10

First method (deterministic algorithm)
?

It takes
multiplications
Verifying Polynomial Identities
11

There are at least two problems
This approach is to write another program that does
the same thing the first program to do.
 1.
If there is a bug in the first program, the same bug
may occur in the checking program.
 2.
we would like to check the answer
in less time than it takes to solve.
Verifying Polynomial Identities
12

Second method (randomized algorithm)
 First,
the algorithm chooses an integer uniformly at
random in the range of {1, … , 100d} (d : degree of F(x))
 If
the algorithm decides
 If
the algorithm decides

But this algorithm may give a wrong answer.
→ How can the algorithm give the wrong answer?
Verifying Polynomial Identities
13


If
If
, the algorithm gives the correct answer.
,
 if
, the algorithm gives the correct answer,
 if
, the algorithm gives the wrong answer.
For this,
must be a root of
has no more than
.
roots.
is chosen in the range of {1, … , 100d} .
→ the chance that the algorithm returns a wrong answer
is no more than 1/100.
CHAPTER 1
Events and probability
14
1.1 Verifying Polynomial Identities
1.2 Axioms of Probability
1.3 Verifying Matrix Multiplication
1.4 A Randomized Min-Cut Algorithm
Axioms of Probability
15

Definition 1.1
A probability space has three components:
1. a sample space : the set of all possible outcomes;
2. a family of sets
representing the allowable events
: each set in
is a subset of ;
3. a probability function
satisfying
Definition 1.2.
Axioms of Probability
16

Definition 1.2
A probability function is any function
that satisfies the following conditions:
1. for any event
;
2.
;
3. for any finite or countably infinite sequence of
pairwise mutually disjoint events
Axioms of Probability
17

Because events are sets,
: the occurrence of both and
: the occurrence of either
or
: the occurrence of an event that is in
but not in
Axioms of Probability
18
Lemma 1.1
For any two events


Proof
and
Axioms of Probability
19
Lemma 1.2
For any finite or countably infinite sequence of
events

Axioms of Probability
20
Lemma 1.3 : inclusion-exclusion principle
Let
be any events. Then

Axioms of Probability
21

The randomized algorithm for verifying polynomial
identities
 Let
represent the event that the algorithm failed to
give the correct answer.
 It
may seem unusual to have an algorithm that can
return the wrong answer.
speed
trade-off
correctness
Axioms of Probability
22

The randomized algorithm for verifying polynomial
identities
 Let
represent the event that the algorithm failed to
give the correct answer.
 The
algorithm give the correct answer 99% of the time
when the polynomials are not equivalent.
 Can we improve this probability?
Axioms of Probability
23

One way to improve the probability
 the
algorithm chooses an integer uniformly at
random in the range of {1, … , 1000d}
→ then the probability of a wrong answer is at most 1/1000
 But
the range of values we can use is limited by the
precision available on the machine.
Axioms of Probability
24

Another approach
 Repeat
the algorithm multiple times
 The
algorithm has one-sided error.
The algorithm may be wrong only when it outputs
 If
the algorithm finds
in at least one
round of the algorithm, we know
.
 Only if there is equality for all runs, the algorithm
outputs
.
Axioms of Probability
25


Repeatedly choosing random numbers according
to a given distribution is generally referred to
as sampling.
In this case, we can choose random numbers in
the range of {1, … ,100} in two ways:
1. with replacement
2. without replacement
Axioms of Probability
26

The case where sampling is done with replacement
 We
repeat the algorithm k times.
 The input polynomials are not equivalent.

What is the probability that the algorithm give a
wrong answer?
Axioms of Probability
27

Definition 1.3
Two events
and
are independent if and only if
More generally, events
are mutually
independent if and only if, for any subset
,
Axioms of Probability
28


Let
be the event that, on the ith run of algorithm,
we choose a root such that
.
The probability that the algorithm returns the wrong
answer is given by
Axioms of Probability
29

The case where sampling is done without replacement
 We
repeat the algorithm k times.
 The input polynomials are not equivalent.

What is the probability that the algorithm give a
wrong answer?
Axioms of Probability
30

Definition 1.4
The conditional probability that event
that event occur is
When
and
are independent and
occurs given
,
Axioms of Probability
31


Let
be the event that the random number
chosen in the ith iteration of the algorithm is a root
of
.
The probability that the algorithm returns the wrong
answer is given by
Axioms of Probability
32
In the th iteration of the algorithm, there are only
values out of the
remaining choices
Axioms of Probability
33

The probability that the algorithm returns the wrong
answer is given by
CHAPTER 1
Events and probability
34
1.1 Verifying Polynomial Identities
1.2 Axioms of Probability
1.3 Verifying Matrix Multiplication
1.4 A Randomized Min-Cut Algorithm
Verifying Matrix Multiplication
35




We want to verify whether
(
are
matrices)
(we are working over the integers modulo 2.)
One way is to multiply
and and compare the
result to .
The simple matrix multiplication algorithm takes
operations.
More sophisticated algorithm takes
operations.
Verifying Matrix Multiplication
36

We use a randomized algorithm.
 The
 If
algorithm chooses a random vector
, then
 Otherwise , it returns that

.
The algorithm can be done in time
obvious way.
.
in the
Verifying Matrix Multiplication
37
Theorem 1.4
If
and if is chosen uniformly at random
from
, then

Verifying Matrix Multiplication
38
Lemma 1.5
Choosing
uniformly at
random is equivalent to choosing each
independently and uniformly {0, 1}.

Proof
If each is chosen independently and uniformly at
random, then each of the possible vector is
chosen with probability
, giving the lemma.

Verifying Matrix Multiplication
39

Let
.
 Since
it must have some nonzero entry.
Let that entry be
.

If
For
and
, then
.
, it must be the case that
Verifying Matrix Multiplication
40





Instead of reasoning about the vector , we choose
the
in order, from down to .
Consider the situation just before
is chosen.
The equality holds with probability at most ½.
The probability that
is at most ½.
This idea is called the principle of deferred decisions.
Verifying Matrix Multiplication
41
Theorem 1.6 [Law of Total Probability]
Let
be mutually disjoint events in the
sample space , and let
. Then

Verifying Matrix Multiplication
42

Proof of Theorem 1.4
Verifying Matrix Multiplication
43
Theorem 1.4
If
and if is chosen uniformly at random
from
, then

Verifying Matrix Multiplication
44




This algorithm has a one-sided error.
Run the algorithm multiple times.
The running time is
The probability of error is at most
Verifying Matrix Multiplication
45
Theorem 1.7 [Bay’s Law]
Assume that
are mutually disjoint sets
such that
. Then

Verifying Matrix Multiplication
46

Problem
We are given three coins. Two of the coins are fair.
One of the coins is biased, landing heads with probability 2/3.
Flip each the coins. The first and second coins come up heads,
and the third comes up tails.
What is the probability that the first coin is the biased one?
A
B
C
Pr(heads)
1/2
1/2
2/3
Pr(tails)
1/2
1/2
1/3
1st coin
2nd coin
3rd coin
heads
heads
tails
Verifying Matrix Multiplication
47



Let
be the event that the ith coin flipped is the
biased one.
Let
be the event that the three coin flips come up
heads, heads, and tails.
Before flips,
Verifying Matrix Multiplication
48

We can compute the probability of the event
conditioned on
Applying Bay’s law,
Verifying Matrix Multiplication
49



In the matrix multiplication case,
if we have no information, a reasonable prior assumption
is that the identity is correct with probability ½.
Let be the event that the identity is correct.
Let be the event that the test returns the identity is
correct.
Verifying Matrix Multiplication
50

If we run the randomized test ones and it returns
that the matrix identity is correct,
∵
Verifying Matrix Multiplication
51

We run the randomized test again and it returns
that the identity is correct
Verifying Matrix Multiplication
52


In general : If our prior model is that
and if the test returns that identity is correct, then
Thus, if all 100 calls return that the identity is
correct, our confidence is at least
CHAPTER 1
Events and probability
53
1.1 Verifying Polynomial Identities
1.2 Axioms of Probability
1.3 Verifying Matrix Multiplication
1.4 A Randomized Min-Cut Algorithm