Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
How Good Are Sparse Cutting Planes? Santanu Dey, Marco Molinaro, Qianyi Wang Georgia Tech Cutting planes In theory β’ ππ₯ β€ π β’ Give convex hull of solutions β’ Many families of cuts, large literature, since 60βs Cutting planes In practice β’ Only want to use sparse cutting planes π .π₯ β€ π β’ Most commercial solvers use sparsity to filter cuts at most π non-zero entries β’ Very limited theoretical investigation [Andersen, Weismantel 10] β’ Do not give convex hull of solutions How good are sparse cutting planes? Geometric problem β’ π - polytope in 0,1 π β’ ππ - intersection of all π-sparse cuts β’ π π, ππ = maxπ π(π₯, π) π₯βπ β Well defined for every polytope β Upper bound on depth-of-cut β At most π Geometric problem β’ π - polytope in 0,1 π β’ ππ - intersection of all π-sparse cuts β’ π π, ππ = maxπ π(π₯, π) π₯βπ β Well defined for every polytope β Upper bound on depth-of-cut π How does π(π, π ) behave? β At most π Ex 1: π = π π₯π β€ 1, π₯ β₯ 0 ; ππ = {π₯πΌ β€ 1, k-subset πΌ of π ; π₯ β₯ 0} π π, ππ β 1 π π₯π β€ good 5 π (density) 5 π Ex 2: π = π π π ,π₯ 2 π β₯ 0 ; π = {π₯πΌ β€ π , 2 k-subset πΌ of π ; π₯ β₯ 0} π π β π π bad π/2 Ex 3: π - set of π‘ random 0/1 points β π π medium π β π Our Ourresults results β’ Upper bounds on π(π, ππ ) for polytopes in 0,1 π β’ Matching lower bound: a random 0/1 polytope, with prob ¼ β’ Hard packing IPs: sparse cuts are bad β π π, ππ π π π₯π¨π (π. #π―ππ«π π· ) π π π (density) π βπ π Our Ourresults results β’ Upper bounds on π(π, ππ ) for polytopes in 0,1 π β’ Matching lower bound: a random 0/1 polytope, with prob ¼ β’ Hard packing IPs: sparse cuts are bad π π, ππ π (density) Our results β’ Upper bounds on π(π, ππ ) for polytopes in 0,1 π β’ Matching lower bound: a random 0/1 polytope, with prob ¼ β’ Hard packing IPs: sparse cuts are bad β π π, ππ π (density) π π π₯π¨π (π. #π―ππ«π π· ) Upper bound β’ Show: π π, ππ < π π πππ π. #π£πππ‘ π = 2π π’ 2π π· π-sparse cut β’ Start with π of norm 1 st ππ β€ π β π and ππ’ β₯ π + π β’ Randomly round π to vector π·: scaling πΌ = π 2 π β’ Show: with non-zero prob. π· is π-sparse, π·π β€ π and π·π’ > π Upper bound β’ Show: π π, ππ < π π πππ π. #π£πππ‘ π = 2π π’ π· π π· β’ Start with ππ β€ π β π and ππ’ β₯ π + π; π of unit norm β’ Randomly βroundβ π to vector π·: scaling πΌ = π 2 π π1π with prob πΌπ wp πΌππ π πΌ π·π = πΌ 00 with π€π prob 1 β πΌπ 1β π πΌππ β’ Show: with non-zero prob. π· is π-sparse, π·π β€ π and π·π’ > π Upper bound β’ Show: π π, ππ < π π πππ π. #π£πππ‘ π = 2π π’ π· π π· β’ Start with ππ β€ π β π and ππ’ β₯ π + π; π of unit norm β’ Randomly βroundβ π to vector π·: scaling πΌ = π 2 π π) ππ πππ(π π withπΌπ prob πΌ ππ wp πΌ π π·π = ππ π·π = πΌ 00 π€π β πΌπ1π β πΌ ππ with1prob β’ Show: with non-zero if πΆ π β€prob. π π· is π-sparse, π·π if πΆβ€π π and > ππ·π’ π π Upper bound β’ Show: π π, ππ < π π πππ π. #π£πππ‘ π = 2π π’ π· π π· β’ Start with ππ β€ π β π and ππ’ β₯ π + π; π of unit norm β’ Randomly βroundβ π to vector π·: scaling πΌ = π 2 π ππππ with prob πΌπ wp πΌππ π πΌ π·π = πΌ 00 with π€π prob 1 β πΌπ 1β π πΌππ β’ Show: prob > 0, π· is π-sparse, π·π β€ π and π·π’ > π Upper bound Obs: For every π£, E π·π£ = ππ£; Var π·π£ = 1 πΌ 2 π£ π π |ππ | Claim 1: With high probability π· is k-sparse β’ E[#non-zeros in π·] = π πΌππ β€ π/2 β’ Stddev(#non-zeros in π·) β€ π β’ Since π·π βs are independent, #non-zeros β + π β€ π π 2 Bernsteinβs Inequality Upper bound Obs: For every π£, E π·π£ = ππ£; Var π·π£ = 1 πΌ 2 π£ π π |ππ | Claim 1: With high probability π· is k-sparse Claim 2: With high probability π·π β€ π β’ max π·π β€ πβπ max πβπ£πππ‘(π) π·π β’ For fixed vertex π, E π·π = ππ β€ π β πβ¦ β’ β¦and Stddev π·π β€ 2π π β€ π log(π.#vert π ) β’ Pr π·π > π β€ Pr π·π > πΈ π·π + π β€ β’ By union bound, Pr( max πβπ£πππ‘(π) 1 π.#π£πππ‘(π) π·π > π) < 1/n Upper bound Obs: For every π£, E π·π£ = ππ£; Var π·π£ = 1 πΌ 2 π£ π π |ππ | Claim 1: With high probability π· is k-sparse Claim 2: With high probability π·π β€ π Claim 3: With probability 1/2n, π·π’ > π β’ β’ β’ β’ π·(π’ β π£) = π·(2ππ), so π·π’ = 2ππ·π + π·π£ E π·π£ = ππ£ = π β π, so π·π’ β 2ππ·π + π β π Show: with prob 1/2π, 2ππ·π > π β‘ π«π > π/π E[π·π] = ππ = 1 β’ π·π β€ 1 2 π πΌ ππ β€π β’ π·π > 1/2 with prob 1/2π (Markovβs ineq to π β π·π) π£ π’ 2π π· π ππ₯ = π Upper bound Obs: For every π£, E π·π£ = ππ£; Var π·π£ = 1 πΌ 2 π£ π π |ππ | Claim 1: With high probability π· is k-sparse Claim 2: With high probability π·π β€ π Claim 3: With probability 1/2n, π·π’ > π Taking union bound over the claims, with prob > 0 π· is π-sparse, π·π β€ π and π·π’ > π Obs: Bottleneck of analysis is to control maxπβπ π·π in Claim 2. Can use different parameters of π (e.g. covering number, chaining) Lower bound Lemma: Let π be convex hull of π‘ random points from 0,1 π . Then with prob 1/4, π π, ππ β₯ π π log π‘ 1 β π π, ππ π (density) 1 π 3 2 β log π‘ Lower bound Lemma: Let π be convex hull of π‘ random points from 0,1 π . Then π π with prob 1/4, π π, ππ β₯ log π‘ 1 β 1 π 3 2 β log π‘ Idea: Explicitly find point in ππ far from π Step 1: Show that inequality π 2 π π₯π β€ + [πππ] is valid for π Step 2: Show that if ππ₯ β€ π is valid for ππ , then π β₯ β point β 1 2 + log π‘ π π belongs to P^k 1 2 + log π‘ π π ππ Lower bound Lemma: Let π be convex hull of π‘ random points from 0,1 π . Then π π with prob 1/4, π π, ππ β₯ log π‘ 1 β 1 π 3 2 β log π‘ Idea: Explicitly find point in ππ far from π Step 1: Show that inequality π 2 π π₯π β€ + [πππ] is valid for π Step 2: Show that if ππ₯ β€ π is valid for ππ , then π β₯ β point β 1 2 + log π‘ π π belongs to P^k Anticoncentration: Pr ππ β₯ E ππ + 1 2 + log π‘ π π ππ Hard packing IPs 0/1 with prob 1/2 π₯ β€ π΄ π₯ β 0,1 π #1β² π ππ πππ€ 2 π Used commonly as computational test-instances [Freville and Plateau 96, Chu and Beasly 98, Kaparis and Letchford 08 and 10, β¦] Theorem: With probability at least ½, π for π β₯ π/2. π, ππ β₯~ π π π Obs 1: Almost matches upper bound: as bad as it gets Obs 2: Still have distance π( π) even with sparsity Ξ©(π) β1 , Conclusion β’ Push for theoretical study of sparse cutting planes Results β’ Matching upper and lower bounds on approximation of sparse cuts: 3 phases, dependence on number of vertices β’ Analysis of hard packing IPs: sparse cuts are bad Questions β’ β’ β’ β’ Relationship with sparsity of formulation? When should we use denser cuts? Sparsify cutting planes? Reformulations that allow good sparse cuts Thank you!