Download Decision problem

Hard Computational Problems
Some computational problems are hard
Despite a numerous attempts we do not
know any efficient algorithms for
these problems
We are also far away from the proof
that these problems are indeed hard to
solve, in other words
NP=P or NPP, this is a question …
week 10
Complexity of Algorithms
1
Decision/Optimisation Problems
A decision problem (DP) is a computational
problem for which the intended output is
either yes or no
In an optimisation problem (OP) we rather
try to maximise or minimise some value
An OP can be turned into a DP if we add a
parameter k, and then ask whether the
optimal value in OP is at most or at least k
Note that if a DP is hard, then its related
its optimisation version must be hard too
week 10
Complexity of Algorithms
2
Decision/Optimisation Problems
Example:
Optimisation problem - Given graph G
with integer weights on its edges. What
is the weight of a minimum spanning tree
(MST) in G?
Decision problem – Given graph G with
integer weights on its edges, and an
integer k. Does G have a minimum
spanning tree of weight at most k?
week 10
Complexity of Algorithms
3
Problems and Languages
We say that the algorithm A accepts an
input string x if A outputs yes on input x
A decision problem can be viewed as a set L
of (binary) strings – the strings that should
be accepted by an algorithm that correctly
solves the problem
We often refer to L as a language
We say that an algorithm A accepts a
language L if A outputs yes for each x in L
and outputs no otherwise
week 10
Complexity of Algorithms
4
The Complexity Class P
The complexity class P is the set of
all decision problems (or languages) L
that can be solved in worst-case
polynomial time
That is, there is an algorithm A that
if x  L, then on input x, A outputs
yes in time p(n), where n is the size
(length) of x and p(n) is a polynomial
week 10
Complexity of Algorithms
5
The Complement of a Language
The complement of a language L consists of
all strings that are not in L
If we have a p(n) time algorithm A that
accepts L (i.e., L is in P) we can construct a
p(n) time algorithm B (based on A) that
accepts the complement of L, i.e.,
Run algorithm A on input string x for p(n) steps
If A outputs yes, then B outputs no
If A outputs no or give no output, then B outputs yes
If L is in P, the complement of L is in P too!
week 10
Complexity of Algorithms
6
The Complexity Class NP
An algorithm that chooses (by a really good guess!)
some number of non-deterministic bits during its
execution is called a non-deterministic algorithm
We say that an algorithm A non-deterministically
accepts a string x if there exists a choice of nondeterministic bits that leads to the ultimate
answer yes
The complexity class NP is the set of decision
problems (or languages) L that can be nondeterministically accepted in polynomial time
Obviously P  NP
week 10
Complexity of Algorithms
7
The Complement of L in NP
Note that the definition of class NP does not
address the running time of rejection (which
might be very long)
And indeed even knowing that we can choose an
appropriate number of non-deterministic bits for
all strings in L in NP we cannot assure that such a
choice is feasible for the complement of L
In fact there is a class co-NP that consists of all
languages whose complements are in NP
Many researchers believe that co-NP  NP
week 10
Complexity of Algorithms
8
The P = NP Question
Computer scientists do not know for
certain whether P = NP or not
We also do not know whether
P = NP  co-NP
However there is a common believe
that P is different then both NP and
co-NP, as well as they intersection
week 10
Complexity of Algorithms
9
Hamiltonian Cycle is NP
Hamiltonian-Cycle is the problem that
takes a graph G as an input and asks
whether there is a simple (Hamiltonian)
cycle in G that visits every vertex of G
exactly once
The non-deterministic algorithm chooses a
cycle (represented by a sequence of nondeterministic bits) and then it checks
deterministically whether this cycle is
indeed a Hamiltonian cycle in G
week 10
Complexity of Algorithms
10
Boolean Circuit
A Boolean circuit is a directed graph
where each node, called a logic gate
corresponds to a simple Boolean
function AND, OR, or NOT
The incoming edges for a logic gate
correspond to inputs for its Boolean
function and the outgoing edges
correspond to the outputs
week 10
Complexity of Algorithms
11
Boolean Circuit (example)
week 10
Complexity of Algorithms
12
Circuit-SAT is in NP
Circuit-Sat is the problem that takes an
input a Boolean circuit with a single output
node, and asks whether there is an
assignment of values to the circuit’s inputs
so that its output value is 1
The non-deterministic algorithm chooses
an assignment of input bits (represented
by a sequence of non-deterministic bits)
and then it checks deterministically
whether this input generates output 1
week 10
Complexity of Algorithms
13
Vertex Cover
Given a graph G=(V,E),
a vertex cover for G is a subset CV, s.t.,
for every edge (v,w) in E, vC or wC
The optimisation problem is to find as small
a vertex cover as possible
Vertex-Cover is the decision problem that
takes a graph G and an integer k as input,
and asks whether there is a vertex cover
for G containing at most k vertices
week 10
Complexity of Algorithms
14
Vertex-Cover is in NP
Suppose we are given an integer k and a
graph G
The non-deterministic algorithm chooses a
subset of vertices C  V, s.t., ¦C¦  k,
(represented by a sequence of nondeterministic bits) and then
it checks deterministically whether this
subset C is an appropriate vertex cover
week 10
Complexity of Algorithms
15
Polynomial-Time Reducibility
We say that a language L, defining some
decision problem, is polynomial-time
reducible to a language M, if
there is a function f computable in polynomial
time, that
takes an input x to L, and transforms it to an
input f(x) of M, s.t.,
x  L if and only if f(x)  M
We use notation L poly M to signify that
language L is polynomial-time reducible to
language M
week 10
Complexity of Algorithms
16
NP-hardness
We say that a language M, defining some
decision problem, is NP-hard if every other
language L in NP is polynomial-time
reducible to M, i.e.,
M is NP-hard, if for every L NP, L poly M
If a language M is NP-hard and it belongs
to NP itself, then M is NP-complete
NP-complete problem is, in a very formal
sense, one of the hardest problems in NP,
as far as polynomial-time reducibility is
concerned
week 10
Complexity of Algorithms
17
The Cook-Levin Theorem
The Cook-Levin Theorem
Circuit-Sat is NP-complete
Proof [sketch]: A computation steps
of any (reasonable) algorithm can be
simulated by layers in appropriately
constructed (in polynomial time and
size) Boolean circuit
week 10
Complexity of Algorithms
18
The Cook-Levin Theorem
week 10
Complexity of Algorithms
19
Other NP-complete Problems
We have just noted that there is at
least one NP-complete problem
Using polynomial-time reducibility we
can show existence of other NPcomplete problems according to
Lemma:
If L1
week 10
poly
L2 and L2 poly L3 then L1
Complexity of Algorithms
poly
L3
20
Types of reduction
Let M be known NP-complete problem. The
types of reductions are:
By restriction: noting that known NP-complete
problem is a special case of our problem L
Local replacement: dividing instances of M and
L into basic units, and then showing how each
basic unit of M can be locally converted into a
basic unit of L
Component design: building components for an
instance of L that will enforce important
structural functions for instances of M
week 10
Complexity of Algorithms
21
Important NP-complete Problems
week 10
Complexity of Algorithms
22
Conjunctive Normal Form
A Boolean formula is in conjunctive
normal form (CNF) if it is formed as a
collection of clauses combined using
operator AND (·), where each clause
is formed by literals (variables or
their negations) combined using
operator OR (+), e.g.,
week 10
Complexity of Algorithms
23
CNF-SAT & 3SAT
Problem CNF-SAT takes a Boolean formula
in CNF form as input and asks if there is an
assignment of Boolean values to its variables
so that the formula evaluates to 1 (i.e.,
formula is satisfiable)
3SAT is CNF-SAT in which each clause has
exactly 3 literals
Fact: CNF-SAT and 3-SAT are NP-complete
week 10
Complexity of Algorithms
24
Vertex-Cover is NP-complete
We can show that Vertex-Cover is
NP-hard by reducing 3SAT problem
to it in polynomial time
This reduction is an example of a
reduction from a logic problem to a
graph problem
It also illustrates an application of
the component design proof technique
week 10
Complexity of Algorithms
25
Vertex-Cover is NP-complete
week 10
Complexity of Algorithms
26
Vertex-Cover is NP-complete
week 10
Complexity of Algorithms
27
Vertex-Cover is NP-complete
week 10
Complexity of Algorithms
28
Approximation Schemes
One way of dealing with NP-completeness
for optimisation problems is to use an
approximation algorithm
The goal of an approximation algorithm is
to come as close to the optimum value as
possible
Such an algorithm typically runs much
faster than an algorithm that strives for
exact solution
week 10
Complexity of Algorithms
29
Approximation Schemes
Let c(S) be the value of a solution S
delivered by an algorithm A to an
optimisation problem P and OPT will be the
optimal solution for P
We say that A is a -approximation
algorithm for a minimisation problem P if
c(S)  ·OPT
And A is is a -approximation algorithm for
a maximisation problem P if
c(S)  ·OPT
week 10
Complexity of Algorithms
30
Polynomial-Time Approximation
Scheme (PTAS)
There are some problems for which we can
construct -approximation algorithms that run in
polynomial time with =1+, for any fixed value  > 0
The running time of such collection of algorithms
depends both on n, the size of an input and also on a
fixed value 
We refer to such collection of algorithms as a
polynomial-time approximation scheme, or PTAS
If the running time is polynomial in both n and 1/
we have a fully polynomial-time approximation
scheme
week 10
Complexity of Algorithms
31