Download Quantifier-Free Linear Arithmetic

Quantifier-Free Linear Arithmetic
Nancy Estrada
Decision Procedures Seminar, Ruzica Piskac
Max Planck Institute for Software Systems, Saarbrücken Germany
January 2013
1
Outline
Outline
Introduction
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
• Introduction
• Preliminary concepts and notation
• Linear programs
• The Simplex method
• Complexity
2
Introduction
Outline
Introduction
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
Motivation
• Program verification typically requires just considering formulae form
the quantifier-free fragments of theories such as TQ.
• Deciding satisfiability in the full theory of TQ is computationally
expensive, while deciding satisfiability in just quantifier-free fragment
of TQ is fast when using for example the Simplex method.
3
Outline
Introduction
Preliminary
concepts and
notation
Preliminary concepts and notation
Basic concepts
Vector and transpose vector
Linear programs
The Simplex
method
Complexity
Matrix and transpose matrix
Vector - vector multiplication
4
Outline
Basic concepts
Introduction
Matrix - vector multiplication
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Matrix - matrix multiplication
Complexity
Important vectors and matrices
Zero vector
0
0 = 0 0 …0 0
Ones vector
1
1 = 1 1 …1 1
Unit vector
ei
Identity matrix
I
0
e2 = 1
0
1 0
I=
0 1
5
Outline
Introduction
Preliminary
concepts and
notation
Basic concepts
Important matrix concepts
Augmented matrix
Complexity
7
2
4 𝑏= 8
5
9
1 2 7
(A |B) = 3 4 8
6 5 9
Linear programs
The Simplex
method
1
𝐴= 3
6
Triangular matrix
Echelon matrix
Non singular or invertible matrix
7 8 9
U= 0 8 5
0 0 6
1
0
A=
0
0
7 0 0
L= 5 8 0
3 4 6
5 4
1 0
0 1
0 0
2
9
7
0
AA-1 = A-1A = I
6
Outline
Decision procedures for Quantifier-Free fragments
Introduction
Preliminary
concepts and
notation
Σℚ formula:
𝐹: 𝑥 ≥ 0 ⋀ y ≥ 0 ⋀ x ≥ 2 ⋀ y ≥ 2 ⋀ x+y ≤ 3
Linear programs
The Simplex
method
Complexity
Matrix form:
0
−1 0
0
0 −1 𝑥
𝐹: −1 0 𝑦 ≤ −2
−2
0 −1
1 1
3
7
Outline
Linear equations
Introduction
Vector space
If 𝑣1,⋯, 𝑣𝑘 ∈ 𝑆
then λ1 𝑣1,⋯, λ1 𝑣𝑘 ∈ 𝑆
for λ1 ,⋯ λ𝑘 ∈ 𝑄
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
Linear equation
F: ⋀𝑚
𝑖=1
F: 𝐴𝑥 = 𝑏
ai1x1 + … + ainxn = bi
Gaussian elimination to solve linear equations when A is square matrix
 Define the augmented matrix [A|𝑏] .
 Manipulate into triangular form via elementary row operations:
•
Swap two rows.
•
Multiply a row by a nonzero scalar.
•
Add one row to another.
8
Outline
Introduction
Linear equations
Gaussian elimination example
Preliminary
concepts and
notation
Linear programs
3 1 2
1 0 1
2 2 1
𝑥1
6
3 1 2 6
𝑥2 = 1 Augmented matrix = 1 0 1 1
𝑥3
2 2 1 2
2
1. Add -2 times the first row and 4 times the second row to the third row:
3 1 2
6
1 0 1
1
−4 0 −3 −10
The Simplex
method
Complexity
3 1 2
6
1 0 1 1
0 0 1 −6
2. Multiply by 3 the second row and add -1 times the first row to it:
3 1 2
6
3 0 3 3
0 0 1 −6
3 1 2
0 −1 1
0 0 1
6
−3
−6
3. Solve:
x3 = -6
-x2 -6 = -3 ∴ x2 = -3
3x1 – 3 – 12 = 6 ∴ x1 = 7
𝑥 =[7 -3 -6]T
9
Outline
Linear equations
Introduction
Gaussian elimination example, non square matrix A
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
Augmented matrix =
3 1
0 −1
0 0
2 0 6
1 −1 0
0 2 −6
Solution:
x4 = -3
-x2 +x3 –x4= 0 ∴ x2 + x3 +3= 0 ∴ x2 = 3 +x3
3x1+(3+x3) +2x3 = 6∴ x1 =1 - x3
1 − 𝑥3
3 + 𝑥3
𝑥=
𝑥3
−3
10
Linear programs
Outline
Introduction
Linear inequalities
Preliminary
concepts and
notation
Linear inequality
P:
Linear programs
The Simplex
method
⋀𝑚
𝑖=1
P: 𝐴𝑥 ≤ 𝑏
ai1x1 + … + ainxn ≤ bi
Polyhedron
𝑃 = {𝑥 ∈ ℝn : Ax ≤ 𝑏}
Complexity
Halfspace
Polyhedron
Polytope
11
Outline
Linear inequalities
Introduction
Convex set
Preliminary
concepts and
notation
A set C is convex if the line segment between any two points in C lies in C.
If for any x1,x2 ∈ 𝐶 and any 𝜃 with 0 ≤ 𝜃 ≤ 1 we have:
𝜃𝑥1 + (1 − 𝜃)x2 ∈ 𝐶
Linear programs
The Simplex
method
Complexity
Vertex
Is a corner point of a polyhedron formed by the intersection of its edges.
A point x0 ∈ P is a vertex of 𝑃 = 𝑥 ∈ 𝑅𝑛 ∶ 𝐴𝑥 ≤ 𝑏 if and only if it is an extreme point
of P.
12
Outline
Linear Programs
Introduction
Linear optimization problem
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
max 𝑐 𝑇𝑥
Subject to 𝐴𝑥 ≤ 𝑏
The problem is feasible if there exist at least one feasible point, and infeasible
otherwise.
If 𝐴𝑥 ≤ 𝑏 is unsatisfiable, then the maximum of 𝑐 𝑇𝑥 is -∞, and if the maximum is
unbounded, the maximum is ∞ by convention.
13
Outline
Introduction
Linear Programs
Duality theorem of linear programming
Preliminary
concepts and
notation
Linear programs
The Simplex
method
max {𝑐 𝑇𝑥 : 𝐴𝑥 ≤ 𝑏 } = min{𝑦𝑇𝑏 : 𝑦 ≥ 0 ⋀ 𝑦𝑇A = 𝑐 𝑇}
By convention, when 𝐴𝑥 ≤ 𝑏 is unsatisfiable, then the maximum of the primal
problem is -∞, and the minimum of the dual form is ∞.
Complexity
We seek the minimal 𝛿 such that 𝐴𝑥 ≤ 𝑏 implies 𝑐 𝑇𝑥 ≤ 𝛿 or the region defined by 𝑐 𝑇𝑥
≤ 𝛿 just covers the region defined by 𝐴𝑥 ≤ 𝑏 .
Feasibility problem
If the objective function is identically zero, the optimal value is either zero (if the
feasible set is nonempty) or ∞ (if the feasible set is empty). Sometimes write it as:
Find
x
𝑓𝑖 𝑥 ≤ 0, 𝑖 = 1, … , 𝑚
ℎ𝑖 𝑥 = 0, 𝑖 = 1, … , 𝑝
The feasibility problem is thus to determine whether the constraints are consistent,
and if so, find a point that satisfies them.
Subject to
14
The simplex method
Outline
Introduction
Introduction
Preliminary
concepts and
notation
•
•
Linear programs
•
The Simplex
method
The name of the algorithm is derived from the concept of a simplex.
Operates on linear programs in standard form:
max 𝑐 𝑇𝑥
Subject to 𝐴𝑥 = 𝑏 ; 𝑥 ≥ 0
Solves the linear program in two main steps:
1. It obtains an initial vertex 𝑣1 of 𝐴𝑥 ≤ 𝑏 .
2. It iteratively traverses the vertices of of 𝐴𝑥 ≤ 𝑏, beginning at
𝑣1, in search of the vertex that maximizes the objective funtion.
Complexity
15
Outline
Simplex method example
Introduction
Shader Electronics Company
Preliminary
concepts and
notation
Max 7x1 + 5x2
Subject to: 2x1 +x2 ≤ 100
4x1 +3x2 ≤ 240
Linear programs
The Simplex
method
Complexity
x1: Number of Walkmans produced.
x2: Number of Watch-TVs produced.
Slack variables: Constraints to equations
2x1 + x2 + 𝑆1 = 100
4x1 + 3x2 + 𝑆2 = 240
Max 7x1 + 5x2 + 0S1 + 0S2
16
Outline
Simplex method example
Introduction
Basic feasible solution
Preliminary
concepts and
notation
2x1 + x2 + 𝑆1 + 0𝑆2 = 100
Linear programs
4x1 + 3x2 + 𝑆2 + 0𝑆1 = 240
The Simplex
method
Complexity
•
•
Begin the solution at the origin, where x1 = 0 and x2 = 0 and profit 0:
S1 = 100
S2= 240
Basic feasible solution:
𝑥1
0
𝑥2
0
=
𝑆1
100
𝑆2
240
17
Outline
Simplex method example
Introduction
Preliminary
concepts and
notation
First simplex tableau
Linear programs
The Simplex
method
Complexity
Cj: Profit contribution per unit of each variable
Zj: Provides the total contribution. Is found multiplying the Cj of the row by
the number in that row an the jth column and summing.
Cj – Zj: Represents the net profit.
18
Outline
Simplex method example
Introduction
Simplex algorithm steps
Preliminary
concepts and
notation
Linear programs
1.
Choose the variable with the greatest positive Cj – Zj to enter the solution.
2.
Determine the row to be replaced by selecting the one with the smallest
(non-negative) ratio of quantity to pivot column.
3.
Calculate the new values for the pivot row.
4.
Calculate the new values for the others rows.
5.
Calculate the Cj and the Cj – Zj values for this tableau. If there are any Cj – Zj
numbers greater than zero, return to step 1.
The Simplex
method
Complexity
19
Outline
Simplex method example
Introduction
Preliminary
concepts and
notation
1. Choose the variable with the greatest positive Cj – Zj to enter the solution.
2. Determine the row to be replaced by selecting the one with the smallest (nonnegative) ratio of quantity to pivot column.
Linear programs
The Simplex
method
Complexity
20
Outline
Simplex method example
Introduction
Preliminary
concepts and
notation
Linear programs
3. Calculate the new values for the pivot row. Divide every number in the row by the
pivot number.
The Simplex
method
Complexity
4. Calculate the new values for the others rows.
New row numbers = numbers in old row - ( Number in old row above or below
pivot number * Corresponding number in the new row)
21
Outline
Simplex method example
Introduction
Preliminary
concepts and
notation
5. Calculate the Cj and the Cj – Zj values for this tableau. If there are any Cj – Zj numbers
greater than zero, return to step 1.
Linear programs
The Simplex
method
Complexity
22
Outline
Simplex method example
Introduction
Preliminary
concepts and
notation
Linear programs
The Simplex
method
1. Choose the variable with the greatest positive Cj – Zj to enter the solution.
2. Determine the row to be replaced by selecting the one with the smallest (nonnegative) ratio of quantity to pivot column.
3. Calculate the new values for the pivot row. Divide every number in the row by the
pivot number.
Complexity
23
Outline
Simplex method example
Introduction
Preliminary
concepts and
notation
Linear programs
The Simplex
method
4. Calculate the new values for the others rows.
New row numbers = numbers in old row - ( Number in old row above or below pivot
number * Corresponding number in the new row)
5. Calculate the Cj and the Cj – Zj values for this tableau. If there are any Cj – Zj numbers
greater than zero, return to step 1.
Complexity
24
Outline
Complexity
Introduction
Preliminary
concepts and
notation
Linear programs
Satisfiability in Propositional Logic is NP-Complete, any decision procedure that
considers arbitrary quantifier-free formulae must be at least NP-Hard, as it must
handle not only the theory-specific aspects of the formulae but also the
combinatorial (PL) aspects.
The Simplex
method
Complexity
LEMMA 1. Let A be an m x m integer matrix. Then the components of
the solution of Ax = b are all rationals with numerator and
denominator bounded by (ma)m where a = maxi,j {|aij|,|bj|}.
25
Outline
Introduction
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
Complexity
LEMMA 2. Let v1,…,vk be k>0 vectors in 0, ±1, ±2, … , ±𝑎 𝑚, and let
+ .
𝑀 = 𝑚𝑎 𝑚 1 Then the following are equivalent:
a) There exist k reals 𝛼1, … , 𝛼𝑘 ≥ 0, not all zero, such that
𝑘
𝑗=1 𝛼𝑗𝑣𝑗 = 0.
b) There exist k integers 𝛼1, … , 𝛼𝑘, 0 ≤ 𝛼𝑗 ≤ 𝑀 for j=1,…,k, not all
zero, such that 𝑘𝑗=1 𝛼𝑗𝑣𝑗 = 0.
c) There is no vector ℎ ∈ ℝm such that 𝑝𝑗 = ℎ𝑇𝑣𝑗 > 0 for j=1,…,k.
d) There is no vector ℎ ∈ 0, ±1, … , ±𝑀
j=1,…,k.
𝑚
such that ℎ𝑇𝑣𝑗 ≥ 1 for
26
Outline
Complexity
Introduction
THEOREM. Let A be an m x n matrix and b an m-vector, both with
Preliminary
concepts and
notation
Linear programs
The Simplex
method
Complexity
entries from {0,±1, … , ±𝑎}. Then if Ax = b has a solution 𝑥 ∈ ℕn , it
also has one in {0,1,…, n(ma)2m+1}n.
PROOF. Let 𝑀 = 𝑚𝑎 𝑚 , and consider the smallest integer solution x to Ax = b. If
all components of x are smaller than M, we are done. Otherwise assume, without
loss of generality, that xj ≥ M for j = 1,…,k. Consider the first k columns of A,
namely, v1,…,vk.
Case 1. There exist integers 𝛼1, … 𝛼𝑘 between 0 and M, not all zero, such that
𝑘
′
𝑗=1 𝛼𝑗𝑣𝑗 = 0. It follows that the vector 𝑥 = (𝑥1 − 𝛼1, … 𝑥𝑘 − 𝛼𝑘 , 𝑥𝑘 + 1, … , 𝑥𝑛)
is also a solution to Ax=b, thus contradicting the minimality of x.
Case 2. By Lemma 2 there is a vector ℎ ∈ 0, ±1, … , ±𝑀 𝑚 such that ℎ𝑇𝑣𝑗 ≥ 1 for
j=1,…,k. Let us multiply the equation Ax = b by ℎ𝑇 . We obtain
𝑘
𝑛
𝑇
𝑇
,
and
𝑗=1 ℎ 𝑣𝑗𝑥𝑗 = ℎ𝑇𝑏 − 𝑗=𝑘+1 ℎ 𝑣𝑗𝑥𝑗
+
.
therefore 𝑘𝑗=1 𝑥𝑗 ≤ 𝑛2𝑚𝑎𝑀2 = 𝑛 𝑚𝑎 2𝑚 1 .
27
Thank you

Presented by: Nancy Aracely Estrada Venegas
Decision Procedures Seminar, Ruzica Piskac
Max Planck Institute for Software Systems, Saarbrücken Germany
January 2013
28
References
•
Bradley, Aaron R., Manna, Zohar. The Calculus of Computation. Springer.
•
Christos H. Papadimitriou. 1981. On the complexity of integer programming. J.
ACM 28, 4 (October 1981), 765-768.
•
T3-2 Online Tutorial, The Simplex Method of Linear Programming.
•
Boyd Stephen, Vandenberghe
University Press. 2004.
Lieven. Convex Optimization. Cambridge
29