Download hidh-dimesion

S.Safra
some slides borrowed from Dana Moshkovits
1
The Crazy Tea Party
Problem To seat all guests at a round table, so people
who sit an adjacent seats like each other.
John
John
Mary
Alice
Bob


Bob
Jane
Mary
Jane






Alice





2
Solution for the Example
Problem To seat all guests at a round table, so people
who sit an adjacent seats like each other.
John
Jane
Mary
Alice
Bob
3
Naive Algorithm
• For each ordering of the guests around the
table
– Verify each guest likes the guest sitting in
the next seat.
4
How Much Time Should This
Take? (worse case)
guests
n
5
15
100
steps
(n1)!
24
87178291200
9·10155
say our computer is
capable of 1010
instructions per
second, this will still
take  3·10138 years!
5
Tours
Problem Plan a trip that visits every site exactly once.
6
Solution for the Example
Problem Plan a trip that visits every site exactly once.
7
Is a Problem Tractable?
• YES! And here’s an efficient
algorithm for it
• NO! and I can prove it
and what if neither is the case?
10
Growth Rate: Sketch
time
n! =2O(n lg n)
2n
n2
10n
input length
12
The World According to
Complexity
reasonable
polynomial  nO(1)
unreasonable
exponential  2nO(1)
13
Could one be Fundamentally
Harder than the Other?
?
Seating
Tour
14
Relations Between Problems
Assuming an efficient procedure for problem A,
there is an efficient procedure for problem B
B cannot be radically harder than A
15
Reductions
B
p
A
B cannot be radically harder than A
In other words: A is at least as hard as B
16
Which One is Harder?
?
Seating
Tour
17
Reduce Tour to Seating
First Observation: The problems aren’t so different
site
“directly reachable from…”
guest
“liked by…”
18
Reduce Tour to Seating
Second Observation: Completing the circle
• Let’s invite to our party a very popular guest,
• i.e one who can sit next to everybody else.
19
Reduce Tour to Seating
• If there is a tour, there is also a way
to seat all the imagined guests around
the table.
popular guest
...
...
20
Reduce Tour to Seating
• If there is a seating, we can easily
find a tour path (no tour, no seating).
popular guest
...
...
21
Bottom Line
The seating problem is at least
as hard as the tour problem
22
What have we shown?
• Although we couldn’t come up with an
efficient algorithm for the problems
• Nor to prove they don’t have one,
• We managed to show a very powerful
claim regarding the relation between
their hardness
23
Furthermore
• Interestingly, we can also reduce the
seating problem to the tour problem.
• Moreover, there is a whole class of
problems, which can be pair-wise
efficiently reduced to each other.
24
NPC
NPC
Contains thousands of
distinct problem
each reducible to all
others
 exponential algorithms

? efficient algorithms

26
How can Studying P vs NP Make You a
Millionaire?
• This is the most fundamental open
question of computer science.
• Resolving it would grant the solver a
great honor
• … as well as substantial fortune…
www.claymath.org/prizeproblems/pvsnp.htm
• Huge philosophical implications:
– No need for human ingenuity!
– No need for mathematicians!!!
27
Constraints Satisfaction
Def Constraints Satisfaction Problem (CSP):
– Instance:
• Constraints: A set of constraints  = { 1, …, l } over two
sets of variables, X of range RX and Y of range RY
• Determinate: each constraint determines the value of a
variable yY according to the value of some xX
xy : RX  RY , satisfied if xy(x)=y
• Uniform: each xX appears in dX of , and each yY
appears in dY of , for some global dX and dy
– Optimize:
• Define () = maximum, over all assignments to X and Y
A: X RX; Y RY
of the fraction of  satisfied
28
Cook’s Characterization of NP
Thm: It is NP-hard to distinguish
between
– () = 1
– () < 1
For any language L in NP
can be reduced to...
testing
membership in L
CSP
29
Showing hardness
From now on, to show a problem NP-hard, we
merely need to reduce CSP to it.
any NP
problem
can be reduced to...
CSP
can be reduced to...
Cook’s Thm
will imply the new
problem is NP-hard
new, hard
problem
31
Max Independent-Set
Instance: A graph G=(V,E) and a threshold k.
Problem: To decide if there is a set of vertices
I={v1,...,vk}V, s.t. for any u,vI: (u,v)E.
33
Max I.S. is NP-hard
Proof: We’ll show CSPp Max I.S.
 x y ,  x
1
46
12 y43
,..., x783 y416
≤p
34
The reduction: Co-Partite Graph
• G comprise k=|X| cliques of size |RX| - a
vertex for each plausible assignment to x:
k
E  {(<i,j1>, <i,j2>) | iM, j1≠j2 RX}
An edge: two
assignments
that determine
a different
value to same y
35
Proof of Correctness
An I.S. of size k must
contain exactly one
vertex in every clique.
A satisfying
assignment implies
an I.S. of size k
k
An I.S. of size k
corresponds to a
consistent, satisfying
assignment
36
Generalized Tour Problem
• Add prices to the roads of the tour problem
• Ask for the least costly tour
$3
$17
$8
$19
$13
$10
$12
$13
37
Approximation
• How about approximating the optimal tour?
• I.e – finding a tour which costs, say, no more
than twice as much as the least costly.
$3
$17
$8
$19
$13
$10
$12
$13
38
Hardness of Approximation
39
Promise Problems
• Sometimes you can promise something
about the input
I know my graph has
clique of size n/4!
Does it have a clique
of size n/2?
• It doesn’t matter what you say for
unfeasible inputs
40
Promise Problems &
Approximation
• We’ll see promise problems of a
certain type, called gap problems, can
be utilized to prove hardness of
approximation.
41
Gap Problems (Max Version)
• Instance: …
• Problem: to distinguish between
the following two cases:
The maximal solution  B
The maximal solution ≤ A
46
Idea
• We’ve shown “standard” problems are
NP-hard by reductions from CSP.
• We want to prove gap-problems are NPhard
• Why won’t we prove some canonical gapproblem is NP-hard and reduce from it?
• If a reduction reduces one gap-problem
to another we refer to it as gappreserving
51
Gap-CSP[]
Instance: Same as CSP
Problem: to distinguish between the
following two cases:
There exists an assignment that
satisfies all constraints.
No assignment can satisfy more
than  of the constraints.
52
PCP (Without Proof)
Theorem [FGLSS, AS, ALMSS]:
For any >0,
Gap-CSP[] is NP-hard,
as long as |RX|,|RY| ≥ -O(1)
53
Why Is It Called PCP?
(Probabilistically Checkable Proofs)
CSP has a polynomial membership proof
checkable in polynomial time.
This
My formula is
assignment
satisfiable!
satisfies it!
Prove it!
x1 y46 , x12 y43 ,...,x783 y416
x1
x5
x2
x6
x3
x7
yn-1
x4
x8
yn
. . .
yn-3
yn-2
54
Why Is It Called PCP?
(Probabilistically Checkable Proofs)
…Now our verifier has to check the assignment
satisfies all constraints…
55
Why Is It Called PCP?
(Probabilistically Checkable InProofs)
a NO instance
While for gap-CSP the
verifier would be right
with high probability,
even by:
of gap-CSP, 1- of
the constraints
are not satisfied!
(1) pick at random a
constant number of
constraints and
(2) check only those
56
Why Is It Called PCP?
(Probabilistically Checkable Proofs)
• Since gap-CSP is NP-hard, All NP
problems have probabilistically
checkable proofs.
57
Hardness of Approximation
• Do the reductions we’ve seen also
work for the gap versions (i.e
approximation preserving)?
• We’ll revisit the Max I.S. example.
59
The same Max I.S. Reduction
An I.S. of size k must
contain exactly one
vertex in every part.
A satisfying
assignment implies
an I.S. of size k
k
An I.S. of size k
corresponds to a
consistent assignment
satisfying  of 
60
Corollary
Theorem: for any >0,
Independent-set is hard to
approximate to within any constant
factor
61
Chromatic Number
• Instance: a graph G=(V,E).
• Problem: To minimize k, so that there
exists a function f:V{1,…,k}, for
which
(u,v)E  f(u)f(v)
skip
62
Chromatic Number
Observation: Each
color class is an
independent set
63
Clique Cover Number (CCN)
• Instance: a graph G=(V,E).
• Problem: To minimize k, so that there
exists a function f:V{1,…,k}, for
which
(u,v)E  f(u)=f(v)
64
Clique Cover Number (CCN)
65
Observation
Claim: The CCN problem on graph G is
the CHROMATIC-NUMBER problem
on the complement graph Gc.
66
Reduction Idea
m
CLIQUE
CCN
.
.
.
.
G
same under
cyclic shift
clique
preserving
G’
.
.
q
67
Correctness
• Given such transformation:
– MAX-CLIQUE(G) = m  CCN(G’) = q
– MAX-CLIQUE(G) < m  CCN(G’) > q/
68
Transformation
T:V[q]
for any v1,v2,v3,v4,v5,v6,
T(v1)+T(v2)+T(v3) T(v4)+T(v5)+T(v6) (mod q)
{v1,v2,v3}={v4,v5,v6}
T is unique for triplets
69
Observations
• Such T is unique for pairs and for
single vertices as well:
• If T(x)+T(u)=T(v)+T(w) (mod q), then
{x,u}={v,w}
• If T(x)=T(y) (mod q), then x=y
70
Using the Transformation
v
vi
CLIQUE
j
T(vj)=4
T(vi)=1
CCN
0
1
2
3
4
…
(q-1)
71
Completing the CCN Graph
Construction
T(s)
(s,t)ECLIQUE 
(T(s),T(t))ECCN
T(t)
72
Completing the CCN Graph
Construction
Close the set of
edges under shift:
T(s)
For every (x,y)E,
if x’-y’=x-y (mod q),
then (x’,y’)E
T(t)
73
Edge Origin Unique
T(s)
First Observation:
This edge comes only
from (s,t)
T(t)
74
Triangle Consistency
Second Observation:
A triangle only comes
from a triangle
75
Clique Preservation
Corollary:
{c1,…,ck} is a clique in the CCN graph
iff
{T(c1),…,T(ck)} is a clique in the
CLIQUE graph.
76
What Remains?
• It remains to show how to construct
the transformation T in polynomial
time.
77
Corollaries
Theorem: CCN is NP-hard to
approximate within any constant
factor.
Theorem: CHROMATIC-NUMBER is
NP-hard to approximate within any
constant factor.
78
Max-E3-Lin-2
Def: Max-E3-Lin-2
– Instance: a system of linear equations
L = { E1, …, En } over Z2
each equation of exactly 3 variables
(whose sum is required to equal either 0 or 1)
– Problem: Compute (L)
79
Main Theorem
Thm [Hastad]: gap-Max-E3-Lin-2(1-,
½+) is NP-hard.
That is, for every constant >0 it is NPhard to distinguish between the case
1- of the equations are satisfiable
and the case ½+ are.
[ It is therefore NP-Hard to approximate
Max-E3-Lin-2 to within 2- constant >0]
81
This bound is Tight!
• A random assignment satisfies half of
the equations.
• Deciding whether a set of linear
equations have a common solution is in
P (Gaussian elimination).
82
Proof Outline
The proof proceeds with a reduction from gapCSP[], known to be NP-hard for any constant >0
Given such an instance , the proof shows a polytime construction, of an instance L of Max-E3Lin-2 s.t.
– () = 1  (L) ≥ 1 - L
– () <   (L) ≤ ½ + L
Main Idea:
Replace every x and every y with a set of variables
representing a binary-code of their assigned values.
Then, test consistency within encoding and any xy
using linear equations over 3 bits
83
Long-Code of R
• One bit for every subset of R
85
Long-Code of R
• One bit for every subset of R
to encode an element eR
0
0
1
1
1
86
The Variables of L
Consider an instance  of CSP[], for
small constant  (to be fixed later)
L has 2 types of variables:
1. a variable z[y,F] for every variable
yY and a subset F  P[Ry]
2.a variable z[x,F] for every variable
xX and a subset F  P[RX]
In fact use a “folded” long-code, s.t. f(F)=1-f([n]\F)
87
Linearity of a Legal-Encoding
An Boolean function f: P[R]  Z2, if legal longcode-word , is a linear-function, that is, for
every F, G  P[R]:
f(F) + f(G)  f(FG)
where FG  P[R] is the symmetric
difference of F and G
Unfortunately, any linear function (a sum of a
subset of variables) will pass this test
88
The Distribution 
Def: denote by  the biased,
product distribution over P[RX],
which assigns probability to a
subset H as follows:
Independently, for each aRX, let
– aH with probability 1-
– aH with probability 
One should think of  as a multiset of subsets in which
every subset H appears with the appropriate probability 89
The Linear Equations
L‘s linear-equations are the union, over
all , of the following set of
equations:
 F P[RY], G P[RX] and H 
denote F*= xy-1(F)
z[y,F] + z[x, G]  z[x, F*  G  H]
90
Correctness of Reduction
Prop: if () = 1 then (L) = 1-
Proof: let A be a satisfying assignment to .
Assign all L ‘s variables according to the legalNote: independent
encoding of A’s values.
of ! (Later we use
A linear equation of L, corresponding
that factto
to set 
xy,F,G,H, would be unsatisfied
if our
smallexactly
enough for
needs).
A(x)H, which occurs with probability
 over
the choice of H.
 = 2(L) -1
LLC-Lemma: (L) = ½+/2  () > 42
91
Denoting an Assignment to L
Given an assignment AL to L’s variables:
For any xX, denote by fx : P[RX]  {-1, 1} the
function comprising the values AL assigns to
z[x,*] (corresponding to the long-code of the
value assigned to x)
For any yY, denote by fy : P[RY]  {-1, 1} the
function comprising the values AL assigns to
z[y,*] (corresponding to the long-code of the
value assigned to y)
Replacing 1 by -1 and 0 by 1
92
Distributional Assignments
Consider a CSP instance 
Let (R) be the set of all distributions over R
Def: A distributional-assignment to  is
A: X  (RX); Y  (RX)
Denote by () the maximum over
distributional-assignments A of the average
probability for    to be satisfied, if
variables’ values are chosen according to A
Clearly ()  (). Moreover
Prop: ()  ()
93
The Distributional-Assignment A
Def: Let A be a distributional-assignment to 
according to the following random processes:
• For any variable xX
2
– Choose a subset SRX with probability f  S 
x
– Uniformly choose a random aS.
• For any variable yY
2
– Choose a subset SRY with probability f  S 
x
– Uniformly choose a random bS.
For such functions, the squares of the
coefficients constitute a distribution
94
odd(xy(S)) = {b| #{aS| xy(a) = b} is odd}
What’s to do:
Show that AL‘s expected success on
xy is > 42 in two steps:
First show that AL‘s success probability,
for any xy
 SR fy
X
2
odd 
S    fx S  S
2
xy
1
Then show that value to be  42
95
Claim 1
Claim 1: AL‘s success probability, for any
xy
 SR fy
2
X
odd 
Proof:
That success probability is
 S R ,S R fy
y
Y
x
X
2
S   f
y
x
2
S    fx S  S
2
xy
1
Sx   PraS xy  a   Sy 
x
Now, taking the sum for only the cases in
which Sy=odd(xy(Sx)), results in the
claimed inequality.
96
High Success Probability


f S   f S   f S  
EF,G,H fy F   fx  G   fx F * G H  



Sy RY ,Sx RX ,Sx' RX
y
y
x
x
x
'
x

 
 
fy Sy  fx  Sx   fx Sx'  E US
F 
 y
Sy RY ,Sx RX ,Sx' RX


Sx RX
 
 
fy odd  x y Sy
 fx
2

E USy F   USx  G   US' F * G H  
x

F,G,H 
Sx   1  2 
Sx'
F   EG US
x
Sx'
G   EH US H  
'
x
Sx
100
Related work
• Thm (Friedgut): a Boolean function f with small averagesensitivity is an [,j]-junta
• Thm (Bourgain): a Boolean function f with small highfrequency weight is an [,j]-junta
• Thm (Kindler&Safra): a Boolean function f with small highfrequency weight in a p-biased measure is an [,j]-junta
• Corollary: a Boolean function f with small noise-sensitivity is
an [,j]-junta
• [Dinur, S] Showing Vertex-Cover hard to approximate to
within 10 5 – 21
• Parameters: average-sensitivity [BL,KKL,F]; high-frequency
weight [H,B], noise-sensitivity [BKS]
102
Boolean Functions and Juntas
A Boolean function
f : P n  T,F
n
f : 1,1  1,1
Def: f is a j-Junta if there exists J[n]
where |J|≤ j, and s.t. for every x
f(x) = f(x  J)
• f is (, j)-Junta if  j-Junta f’ s.t. Pr f  x   f' x   
x
103
Motivation – Testing Long-code
• Def (a long-code test): given a codeword w, probe it in a constant number
of entries, and
– accept w.h.p if w is a monotone
dictatorship
– reject w.h.p if w is not close to any
monotone dictatorship
104
Motivation – Testing Long-code
• Def(a long-code list-test): given a codeword w, probe it in a constant number of
entries, and
– accept w.h.p if w is a monotone dictatorship,
– reject w.h.p if  a Junta J[n] s.t. f is close
to f’ and f’(F)=f’(FJ) for all F
• Note: a long-code list-test, distinguishes
between the case w is a dictatorship, to
the case w is far from a junta.
105
Motivation – Testing Long-code
• The long-code test, and the long-code listtest are essential tools in proving hardness
results.
Examples …
• Hence finding simple sufficient-conditions
for a function to be a junta is important.
106
Noise-Sensitivity
• Idea: check how the value of f changes
when the input is changed not on one, but
on several coordinates.
z
I
x
[n]
107
Noise-Sensitivity
• Def(,p,x[n] ): Let 0<<1, and xP([n]).
Then y~,p,x, if y = (x\I) z where
– I~[n] is a noise subset, and
– z~ pI is a replacement.
z
I
Def(-noise-sensitivity):
0<<1,
then
flet

ns f =
Pr
x

f
y
 

 
x~p[n] ,y~ [n],p,x
x
• Note:
p),

 
[n]
deletes a coordinate in x w.p. (1adds a coordinate to x w.p. p.
108
Noise-Sensitivity – Cont.
• Advantage: very efficiently testable (using
only two queries) by a perturbation-test.
• Def (perturbation-test): choose x~p, and
y~,p,x, check whether f(x)=f(y).
The success is proportional to the noisesensitivity of f.
2
• Prop:2 the
ns -noise-sensitivity
f =1   1    fis given
S  by
S
S
109
Related Work
• [Dinur, S] Showing Vertex-Cover hard
to approximate to within 10 5 – 21
• [Bourgain] Showing a Boolean function
with weight <1/k on characters of
size larger than k, is close to a junta
of size exponential in k ([Kindler, S]
similar for biased, product
distribution)
110