Download Control of the search space size and solving linear

Control of the search space size and solving linear
systems in the FEAST algorithm
Lukas Krämer
Bergische Universität Wuppertal
—Applied Computer Science—
July 4th, 2014
Joint work with
Martin Galgon, Bruno Lang (Wuppertal),
Achim Basermann, Jonas Thies, (DLR Köln)
Andreas Alvermann, Ernst-Moritz-Arndt-Universität Greifswald
This work was partially supported by DFG within the priority
programme 1648—Software for Exascale Computing (SPPEXA)
within the project ESSEX (Equipping Sparse Solvers for Exascale).
http://www.sppexa.de
http://blogs.fau.de/essex/
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
1/25
Outline
Introduction
Introduction
Problem
The FEAST Algorithm
Linear systems
Banded systems from graphene nanoribbons
Iterative solution
Transformation method
References
Outline
Introduction
Introduction
Problem
The FEAST Algorithm
Linear systems
Banded systems from graphene nanoribbons
Iterative solution
Transformation method
References
Symmetric eigenvalue problem
We aim at solving the problem:
Given: A ∈ Rn×n , A = A? ,
an interval
Iλ = λ, λ .
Sought: X ∈ Rn×m , m ≤ n, X? X = I, Λ = diag(λ1 , . . . , λm ) such that
AX = XΛ and all λj ∈ λ, λ .
. Solve the (partial) symmetric eigenproblem. Find all
eigenvalues in a given interval.
(Everything can be generalized to complex matrices and to a pair
(A, B) with B hpd.)
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
2/25
Framework: Rayleigh-Ritz approach
I
Choose a subspace U = span(U), U ∈ Cn×m .
I
Compute the Rayleigh quotients AU := U? AU, BU := U? BU.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
3/25
Framework: Rayleigh-Ritz approach
I
Choose a subspace U = span(U), U ∈ Cn×m .
I
I
Compute the Rayleigh quotients AU := U? AU, BU := U? BU.
e W)
e of AU W = BU WΛ.
Compute eigenpairs (Λ,
I
e UW)
e of AX = BXΛ.
Form Ritz pairs (Λ,
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
3/25
Framework: Rayleigh-Ritz approach
I
Choose a subspace U = span(U), U ∈ Cn×m .
I
I
Compute the Rayleigh quotients AU := U? AU, BU := U? BU.
e W)
e of AU W = BU WΛ.
Compute eigenpairs (Λ,
I
e UW)
e of AX = BXΛ.
Form Ritz pairs (Λ,
e UW)
e are good approximations to some
. . . and hope that (Λ,
eigenpairs.
e and iterate.
Otherwise, set U := UW
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
3/25
The subspace in use
E. Polizzi, Phys. Rev. B, 2009, 79, 115112
In FEAST: The subspace
Z
1
U :=
(zB − A)−1 dzBY
2πı C
is used.
I
C = curve in complex plane surrounding Iλ .
I
Y = starting base of (possibly) random vectors.
It follows
1
2πı
Z
(zB − A)−1 Bdz = orth. projector onto span(X)
C
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
4/25
The FEAST Algorithm: Skeleton
Input: Iλ := λ, λ , an estimate m
e of the number of eigenvalues
in Iλ .
Output m̂ ≤ m
e eigenpairs with eigenvalue in Iλ .
Perform:
e of full rank and compute
1. Choose Y Z∈ Cn×m
1
(zB − A)−1 Bdz Y,
U :=
2πi C
2. Form AU := U? AU, BU := U? BU,
e = BU W
e Λ,
e
3. Solve size-m
e eigenproblem AU W
e X
e := U · W),
e
4. Compute (Λ,
e
5. If no convergence: go to Step 1 with Y := X.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
5/25
Problem: How to choose search space size m?
e
Experiment: Choose Iλ = [λ, λ] such that it contains the m = 300
lowest eigenvalues of size 1059-matrix. Let m
e vary,
m
e = 1, . . . , 450.
Residuals vs. search space size m:
e
K., Di Napoli, Galgon, Lang, Bientinesi: Dissecting the FEAST
algorithm for generalized eigenproblems, 2013
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
6/25
Problem: How to choose search space size m?
e
Do not choose too small!
Determine a good choice for m
e ⇔ Count eigenvalues in Iλ
Leads to stopping criterion:
“all” eigenpairs found ⇒ Stop Algorithm
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
7/25
Counting eigenvalues
I
rank(U ) = m = number of eigenvalues in Iλ
I
In exact arithmetic: U has singular values 0 or 1
I
numerical rank?
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
8/25
Counting eigenvalues
I
rank(U ) = m = number of eigenvalues in Iλ
I
In exact arithmetic: U has singular values 0 or 1
I
numerical rank?
I
It can be shown: If m ≤ m
e we have m = # of singular values
of U that are ≥ 1/2 (Tang & Polizzi, 2013).
I
Number 1/2 depends on implementation (integration
scheme). E.g., valid for Gauß–Legendre integration, midpoint
rule of even order.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
8/25
Counting eigenvalues: SVD of U
SVD U = WΣV? with Σ = diag(σ1 , . . . , σm , 0, . . . , 0)
⇒ span(W(:, 1 : m)) = span(U)
⇒ Found the correct subspace, but cost = O(n2 · m)
e
SVD of BU = U? BU:
e m
e , orthogonal. No order n cost.
⇒ BU = VΣ? ΣV? with V ∈ Rm×
⇒ U0 = U · V(:, 1 : m) spans the wanted space
O(n · m).
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
9/25
Counting eigenvalues: SVD of U
SVD U = WΣV? with Σ = diag(σ1 , . . . , σm , 0, . . . , 0)
⇒ span(W(:, 1 : m)) = span(U)
⇒ Found the correct subspace, but cost = O(n2 · m)
e
SVD of BU = U? BU:
e m
e , orthogonal. No order n cost.
⇒ BU = VΣ? ΣV? with V ∈ Rm×
⇒ U0 = U · V(:, 1 : m) spans the wanted space
O(n · m).
This means the dimension and a basis of the search space can be
computed.
Proceed with a subspace slightly larger than m.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
9/25
Counting eigenvalues: example
SVD count
Number of Ritz values in Iλ
30
30
Eigenvalue count
Ritz estimation
Eigenvalue count
SVD estimation
25
25
20
20
15
15
10
0
2
4
6
8
10
12
14
10
0
2
4
6
8
10
12
14
1 Point on abscissa ≡ 1 interval Iλ
◦ = count from respective method
line = exact count
. Ritz count is almost for free and not too bad. It does not
deliver the subspace. Typically overestimates.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
10/25
Counting eigenvalues: alternatives
Chebyshev polynomials
two-term recurrence
Tn (x) = cos (n arccos x)
T0 (x) = 1 ,
T1 (x) = x ,
Tn+1 (x) = 2xTn (x) − Tn−1 (x)
Tn(x)
1
0
T30(x)
-1
-1
0
for example: density of states
(DOS) =
ˆ eigenvalue count
ρ(ω) ∝ µ0 + 2
X
µn Tn (ω)
n=1
Z
µn =
with coefficients
(moments) of the matrix A
1
x
Expansion of functions
=
1
ρ(ω) Tn (ω) dω
−1
?
x Tn (A)x
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
11/25
Counting eigenvalues: alternatives
Chebyshev polynomials
two-term recurrence
Tn (x) = cos (n arccos x)
T0 (x) = 1 ,
T1 (x) = x ,
Tn+1 (x) = 2xTn (x) − Tn−1 (x)
Tn(x)
1
0
T30(x)
-1
-1
0
for example: density of states
(DOS) =
ˆ eigenvalue count
idea: use as input for FEAST
0.6
ρ(ω)
further technical details: Weiße,
Wellein, Alvermann, Fehske, Rev.
Mod. Phys. 78, 275 (2006).
cf. recent report
(Di Napoli et al., 2013)
1
x
Expansion of functions
0.4
0.2
0
-4
-3
-2
-1
0
1
2
3
ω
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
11/25
Outline
Introduction
Introduction
Problem
The FEAST Algorithm
Linear systems
Banded systems from graphene nanoribbons
Iterative solution
Transformation method
References
Linear systems
Central task: Solution of
(zB − A)V = BY.
Matrix (zB − A) is
I
large and sparse Cholesky LU
I
non-Hermitian (z complex) Cholesky CG
I
complex valued
I
often ill-conditioned GMRES GMRES??
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
12/25
Problems from Graphene nanoribbons
some lead to banded systems ⇒ special
banded direct solver.
See (Hockney & Jesshope, 1988) and (Lawrie
& Sameh, 1984).
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
13/25
Results with banded solver
I
Matrix size n ≈ 1.2M , ratio
entries per row).
I
I
nnz
≈ 0.001% (12 nonzero
n2
Graphene II with gap around zero.
Matrix
est. found iters.
Graphene I
535
535
7
Graphene II (`) 492
492
4
Graphene II (r) 546
546
8
Graphene III
574
572
25
I
Eigenpair residuals: 10−9 . . . 10−16
I
Linear system residuals: 10−10 . . . 10−16
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
14/25
General (non-banded) problems
Problem large & sparse: Iterative method needed.
Candidate 1: Special tailored method for systems
(R + iS)(x + iy) = ϕ + iψ
R = RT , S = ST , real
(Axelsson & Kucherov, 2000)
I
(zB − A)V = BY can be brought to this form.
I
Leads to real systems of size 2n.
I
Not efficient enough in practice
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
15/25
General (non-banded) problems
Problem large & sparse: Iterative method needed.
Candidate 2: CARP-CG
(cf. Talk of Jonas Thies, Code also by Jonas)
Experiment: Graphene, n = 4,200, m = 17 interior eigenpairs
Achievable residuals kAx − xλk:
−2
10
1e−6
1e−8
1e−10
1e−12
−4
Min/Max residual
10
Average number of iterations per solve
271
−6
10
−8
10
335
−10
384
−12
424
10
10
−14
10
1
2
3
4
5
6
7
FEAST iteration number
8
9
10
max κ(zj I − A) = 1097.
zj
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
16/25
Larger example
Example 1: Graphene, n = 84,000, sought m = 102 eigenpairs
from interior.
Average: 567 iterations/solve.
I
tollin = 10−10
I
kAx − xλk = 10−7 . . . 10−10 (convergence criterion: 10−7 ).
I
Orthogonality maxi6=j |x?i xj | = 10−15 .
Example 2: Graphene, n = 176,000, sought m = 201 eigenpairs
from interior.
Average: 655 iterations/solve.
I
tollin = 10−10
I
kAx − xλk = 10−8 . . . 10−10 (convergence criterion: 10−7 ).
I
Orthogonality maxi6=j |x?i xj | = 10−15 .
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
17/25
Other contours
Idea: Bring integration points away from real axis.
R
Iλ
Use U =
1
A
2πi
Iλ
Z
R
1
(zI − A)−1 dz
z
C
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
18/25
Use of ellipse: Results
Experiment: Graphene, n = 4,200, m = 17 interior eigenpairs
r = main semi axis, second semi axis b = αr, vary α
−2
10
b=0.75r
b=r
b=1.5r
−4
Min/Max residual
10
−6
10
−8
10
−10
10
−12
10
−14
10
1
2
3
FEAST iteration number
4
5
Required kAx − xλk ≤ 10−12 .
⇓ CARP-CG iterations ↔ FEAST iterations ⇑
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
19/25
Outline
Introduction
Introduction
Problem
The FEAST Algorithm
Linear systems
Banded systems from graphene nanoribbons
Iterative solution
Transformation method
References
Outlook: Transforming the integration region
• = integration point
s∈A
t
−K +
−R
r
−r
R
K + iK 0
−K
0
K
z ∈ Ωλ
u
−k −1 −1
iK 0
1
k −1
0 λ
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
λ
20/25
Outlook: Transforming the integration region
. . . for computing upper eigenpairs
Adapted from (Hale et al., 2008): results in
p
Z
0
λλ 3K+iK /2 cn(t)dn(t)
U=−
(z(t)B − A)−1 BdtY,
πik −K+iK 0 /2 (k −1 − sn(t))2
√
λ/λ−1
with k = √
< 1. Then use midpoint rule.
λ/λ+1
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
21/25
Outlook: Transforming the integration region
. . . for computing upper eigenpairs
Adapted from (Hale et al., 2008): results in
p
Z
0
λλ 3K+iK /2 cn(t)dn(t)
U=−
(z(t)B − A)−1 BdtY,
πik −K+iK 0 /2 (k −1 − sn(t))2
√
λ/λ−1
with k = √
< 1. Then use midpoint rule.
λ/λ+1
Theorem: (Hale et al., 2008)
ep
Let A, B be real symmetric and let Iλ = [λ, λ] ⊂ (0, +∞), let U
formed by midpoint rule, then for all λ, λ > 0
−π 2 p
e
kU − Up k = O exp
.
log(λ/λ) + 3
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
21/25
Outlook: Transforming the integration region
. . . for computing upper eigenpairs
Adapted from (Hale et al., 2008): results in
p
Z
0
λλ 3K+iK /2 cn(t)dn(t)
U=−
(z(t)B − A)−1 BdtY,
πik −K+iK 0 /2 (k −1 − sn(t))2
√
λ/λ−1
with k = √
< 1. Then use midpoint rule.
λ/λ+1
Theorem: (Hale et al., 2008)
ep
Let A, B be real symmetric and let Iλ = [λ, λ] ⊂ (0, +∞), let U
formed by midpoint rule, then for all λ, λ > 0
−π 2 p
e
kU − Up k = O exp
.
log(λ/λ) + 3
λ
λ
O log
instead of O
solves!
λ
λ
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
21/25
Transforming the integration region: results
(A, B) from structural engineering, n = 1473, seek 300 highest
eigenpairs.
Order (p)
4
8
16
Transformation
method
300
9
13k
300
5
14k
300
2
14k
eigenpairs
iterations
solves
eigenpairs
iterations
solves
eigenpairs
iterations
solves
Gauß–
Legendre
0
10
18k
0
10
36k
294
10
58k
!
eigenpairs
iterations
solves
eigenpairs
iterations
solves
eigenpairs
iterations
solves
Trapezoidal
rule
65
10
17k
68
10
35k
82
10
69k
eigenpairs
iterations
solves
eigenpairs
iterations
solves
eigenpairs
iterations
solves
References I
Axelsson, Owe, & Kucherov, Andrey. 2000.
Real valued iterative methods for solving complex symmetric
linear systems.
Numer. Linear Algebra Appl., 7, 197–218.
Di Napoli, E., Polizzi, E., & Saad, Y. 2013.
Efficient estimation of eigenvalue counts in an interval.
http://arxiv.org/abs/1308.4275.
Hale, Nicholas, Higham, Nicholas J., & Trefethen, Lloyd N. 2008.
Computing Aα , log(A) and Related Matrix Functions by
Contour Integrals.
SIAM J. Numer. Anal., 46(5), 2505–2523.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
23/25
References II
Hockney, R.W., & Jesshope, C.R. 1988.
Parallel Computers 2.
Hilger.
Krämer, Lukas, Di Napoli, Edoardo, Galgon, Martin, Lang, Bruno,
& Bientinesi, Paolo. 2013.
Dissecting the FEAST algorithm for generalized
eigenproblems.
J. Comput. Appl. Math., 244, 1–9.
Lawrie, D. H., & Sameh, A. H. 1984.
The Computation and Communication Complexity of a
Parallel Banded System Solver.
ACM Trans. Math. Softw., 10(2), 185–195.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
24/25
References III
Tang, Ping Tak Peter, & Polizzi, Eric. 2013.
Subspace Iteration with Approximate Spectral Projection.
http://arxiv.org/abs/1302.0432 [math.NA], version 3.
Lukas Krämer, Control of the search space size and solving linear systems in the FEAST algorithm
25/25