Download Numerical methods for physics simulations.

Numerical methods for physics simulations.
Claude Lacoursière
HPC2N/VRlab, Umeå Universiy, Umeå, Sweden,
and CMLabs Simulations, Montréal, Canada
May 8, 2006
– Typeset by FoilTEX – – May 5th 2006
Outline
– Typeset by FoilTEX – – May 5th 2006
1
Integrating Newton’s second law
Newton’s law states
dp
= f, p = M ẋ
dt
for point masses. M is the mass matrix, x is the position vector, p is the momentum
vector, and f is the force vector. Integrating these equations of motion means solving
the following system of ordinary differential equations:
ẋ = v
v̇
= M −1f (x, v)
(1)
In general, forces depend on both position and velocities but keep it simple and
assume f (x) for now.
– Typeset by FoilTEX – – May 5th 2006
2
Simple Harmonic Oscillator
Simple harmonic oscillator in 1D: f (x) = −kx,
M = m, (a scalar)
k
f (x)
2
= − x = −ω x
m
m
ω=
k/m is the natural frequency, so:
ẋ = v
v̇
= −ω 2x,
(2)
solution:
v(0)
sin(ωt).
ω
Because any physical force is smooth, we can always expand:
x(t) = x(0) cos(ωt) +
f (x) = a + b(x − x0 ) + . . .
Linear case is easy to study and gives good insight.
– Typeset by FoilTEX – – May 5th 2006
3
Integration
Balance:
. accuracy
. stability
. speed
Accuracy can also impliy preservation of energy and other physical invariants
(symmetry).
We want to integrate indefinitely, without having to change the time step all the time
so we need:
. fixed time step
. unconditional stability
. ease of implementation
Investigate simple methods (first order) on spring systems without damping to get
some good idea of what to expect.
– Typeset by FoilTEX – – May 5th 2006
4
Integrating the Simple Harmonic Oscillator
Ignore the literature and follow basic intuition.
Discretize time : t = nh where h is small and n = 1, 2, . . ., x(t) = x(nh) = x n ,
and similarly for v(t) = v(nh) = vn .
Simple formulae for derivatives:
ẋn+1
v̇n+1
'
'
xn+1 −xn
h
vn+1 −vn
h
so using the most naive integration formula, we have:
ẋn+1
v̇n+1
'
'
xn+1 −xn
h
vn+1 −vn
h
' vn
' −ω 2xn
and therefore:
xn+1
= xn + hvn
vn+1 = −hω 2xn + vn
(if you stay tuned, we’ll explain that this formula is bad for your health; it is called
the Explicit Euler integration formula, applied here to the harmonic oscillator)
– Typeset by FoilTEX – – May 5th 2006
5
Simple Harmonic Oscillator – cont
In matrix form, this is:
xn+1
1
=
vn+1
−hω 2
h
1
xn
vn
which we can also write as zn+1 = Azn with:
zn =
xn
;
vn
A=
1
−hω 2
h
.
1
and so:
zn = Azn−1 = . . . = Anz0.
This is a form of matrix iteration procedure (more on this later). We have stability
if kzn k stays bounded for all n. So, our first interesting matrix problem of the day:
what is
n
lim A ?
n→∞
Note that for this case, the energy is
E=
1
2
2
2
[mv + kx ] ≤ αkzk for some scalar α.
2
– Typeset by FoilTEX – – May 5th 2006
6
Intermission: the spectral radius
Think of a matrix A and its eigenvalues and eigenvectors: λi, vi . For any of these,
we have:
n
n
z n = A vi = λi vi
so that kznk is bounded iff |λi| < 1.
The spectral radius states that for any consistent matrix norm k · k, the following
holds:
ρ(A) = max(|λi|) = lim kAnk1/n .
i
n→∞
Using that theorem, we can state the following:
Theorem 1. Given a matrix A over the complex numbers, the iterations zn =
Anz0 are bounded if and only if ρ(A) ≤ 1.
Of course, an interesting limit case is when ρ(A) = 1 which leads to oscillations
which are bounded forever. For the case where ρ(A) is strictly less than one, the
iterates just decay to 0. To build good integrators, you need cases which have
ρ(A) = 1.
– Typeset by FoilTEX – – May 5th 2006
7
Back to Integration
What are the eigenvalues of our iteration matrix?
det(A − λI) =
1−λ
−hω 2
h
= (1 − λ)2 + h2ω 2 = 0
1−λ
which means that λ± = 1 ∓ ihω and so,
ρ(A) =
1 + h2ω 2 > 1.
This is always unstable!
It is easy to show that the energy increases at each step by the factor 1 + h 2ω 2 .
A naı̈ve analysis would suggest that keeping hω < 1 would keep things stable but
that is wrong.
Adding some damping force of the form −γ ẋ can make the system stable but too
much damping makes it unstable again (it is easy to do the analysis).
– Typeset by FoilTEX – – May 5th 2006
8
Energy as a function of time, hω = 0.015
1.8
γ = 2.005m/h
γ = 0.1mω
1.6
Energy
1.4
1.2
1
0.8
0.6
0.4
0.2
0
100 200 300 400 500 600 700 800 900 1000
discretized time (units of hω )
Figure 1: Energy of simple harmonic oscillator integrated explicitly. We either bet big
oscillations when the oscillator is damped or a blowup. But the energy should strictly
decrease at the rate γ .
– Typeset by FoilTEX – – May 5th 2006
9
Intermission: standard stability analysis (simplified)
Given ODE ż = f (z), integrator produces zn+1 = φ(h, f, zn, zn−1, ·).
For linear stability, look at the Dahlqvist (another famous Swede) test equation:
ż = λz,
→ z(t) = exp[λt]z(0),
where λ, z ∈ C. For this system, any integration method leads to a linear recurrence
relation of the form:
wn+1 = Awn
where A is a matrix function which depends on τ = λh and w is a generalized state
vector of the form: wn = [zn, zn−1, . . .]T .
– Typeset by FoilTEX – – May 5th 2006
10
Intermission: standard stability analysis (simplified)
To use this analysis for our case, we need to redifine:
s = ωt,
dt
1
=
ds
ω
y(s) = x(t),
w(s) = ω −1v(t)
and so:
dx(t) dt
1
dy
=
= v = w(s)
ds
dt ds
ω
dw
1
dv
dt
w0 =
=
= −ω 2x(t) = −x(t) = −y(s)
ds
ω dt ds
0
y =
(3)
If we then set z(s) = y(s) + iw(s) we find that this corresponds to:
z 0 = iz
where i is the imaginary unit. The analysis is straight forward but it does not
apply easily to all the integration cases we want to cover. A matrix format is more
convenient.
– Typeset by FoilTEX – – May 5th 2006
11
Discretizing Differently
Actually, we had some choice in discretizing. Using Taylor expansions:
xn+1 = xn + ẋn h + O(h2 )
(4)
xn = xn+1 − ẋn+1 h + O(h2 )
(5)
2
2
⇒(xn+1 − xn)/h = ẋn + O(h ) = ẋn+1 + O(h )
xn+1 −xn
h
vn+1 −vn
h
(6)
' vn0 = αvn + (1 − α)vn+1
' −ω 2xn00 = −ω 2 (βxn + (1 − β)xn+1 )
and we can chose 0 ≤ α ≤ 1, 0 ≤ β ≤ 1 independently.
– Typeset by FoilTEX – – May 5th 2006
12
Discretizing Differently
The general stepping equation is then:
1
hω (1 − β)
−h(1 − α)
1
2
xn+1
1
=
vn+1
−hω 2β
hα
1
xn
vn
The most interesting cases are the ones where α, β are either 0, 1 or 1/2. Three new
stepping formulae:
1
hω 2
−h
1
1
0
hω 2
2
−h
1
1
xn+1
1
=
0
vn+1
− h2
xn+1
1
=
vn+1
−hω 2
1
0
1
xn
,
vn
0
1
1
xn+1
=
vn+1
– Typeset by FoilTEX – – May 5th 2006
2
− hω2
fully implicit
v implicit: the Verlet method
xn
,
vn
implicit midpoint rule
h
2
1
xn
,
vn
(7)
13
Discretizing Differently
All of these are of the general form: Bzn+1 = Czn and with a little algebra, we
convert them to the form zn+1 = Azn.
Computing these matrices is nothing complicated (but tedious).
Once you get matrix A, you compute the eigenvalues, carefully check all cases to find
out what values of h, ω produce ρ(A) ≤ 1. This is the stability regime.
Some simple dimensional analysis shows that ρ(A) will always be a function of
τ = hω . This is the natural time scale of the system. The following cases will occur:
. ρ(A) > 1 for all values of τ : unstable, no good (explicit Euler)
. ρ(A) = 1 for some values of 0 < τ ≤ τ̄ : conditionally satble
. ρ(A) < 1 for all values of τ : unconditionally stable but everything is damped:
this is L-stability (implicit Euler)
. ρ(A) = 1 for all values of τ : unconditionally stable but nothing is damped: this
is A-stability (implicit midpoint)
– Typeset by FoilTEX – – May 5th 2006
14
Discretizing Differently: Special Cases
We cover the different cases and we use: τ = hω . Implicit stepper:
Aie =
1
1
2
2
2
1 + h ω −hω
h
,
1
λ± = √
1
1 + τ2
,
ρ(A) < 1.
Verlet stepper:
Avv =
1 − h2 ω 2
−hω 2
h
,
1
λ± = (1 −
τ2
−1
4
τ
)±τ
2
In this case, as long as hω < 2, we have |λ±| = 1 and ρ(Avv ) = 1. Otherwise,
ρ(Av v) > 1.
The implicit stepper is even more interesting:
Amid =
1
1+
τ2
4
1−
τ2
4
2
−hω
h
1−
τ2
4
,
λ± =
τ2
∓ iτ
1−
2
1
1+
τ2
2
and in this case, |λ±| = ρ(Amid) = 1 for any value of τ = hω .
– Typeset by FoilTEX – – May 5th 2006
15
Some Results
2.5 0
9
amplitude
2
1.5
1
0.5
0
-0.5
-1
-1.5
-2
-2.5 0
Damped Simple Harmonic Oscillator, 1D, c = 1E-1 , d = 0
time in units of natural period
1
2
3
4
5
6
7
8
2
4
Variational
Implicit midpoint
6
8
10
time
Vortex
Implicit Euler
12
14
16
18
Exact
Figure 2: Simple harmonic oscillator in one dimension using different integration
methods. At low frequency, they all perform reasonably well as expected but both the
Vortex and the implicit Euler method, which produce identical output in this case,
dissipate energy. Note that both the variational and the midpoint method produce
identical output in this case.
– Typeset by FoilTEX – – May 5th 2006
30
16
Damped Simple Harmonic Oscillator, 1D, c = 1E-4 , d = 0
time in units of natural period
2
4
6
8
10
12
14
amplitude
2
1
0
-1
-2
-3 0
0.1
Variational
Implicit midpoint
0.2
0.3
0.4
0.5
time
Vortex
Implicit Euler
0.6
0.7
0.8
0.9
Exact
Figure 3: Simple harmonic oscillator in one dimension using different integration
methods. At higher frequencies, the Vortex and implicit methods dissipate energy
very quickly. The midpoint and variational methods are producing the correct output
qualitatively but of course, the phase of the signal is erratic.
– Typeset by FoilTEX – – May 5th 2006
17
Damped Simple Harmonic Oscillator, 1D, c = 1E-6 , d = 0
time in units of natural period
10
20
30
40
50
20 0
60
15
amplitude
10
5
0
-5
-10
-15
-20 0
0.05
Variational
Implicit midpoint
0.1
0.15
0.2
time
Vortex
Implicit Euler
0.25
0.3
0.35
Exact
Figure 4: Simple harmonic oscillator in one dimension using different integration
methods. At even higher frequencies, the variational and midpoint methods show
signs of instability. A simple computation shows that both integration methods
produce an oscillatory signal which diverges linearly to infinity. Vortex and implicit
Euler are intrinsically damped and just decay to 0.
– Typeset by FoilTEX – – May 5th 2006
18
Summary of reasonably good methods
Derivations can be found elsewhere but here is the summary of the one and only
explicit method you should bother with at first order:
Velocity Verlet:
xn+1 = xn + hvn +
vn+1 = vn +
h2 −1
M f (xn )
2
h
(f (xn ) + f (xn+1 ))
2
(8)
(9)
Verlet or Leap Frog (don’t bother with vn )
xn+1 = xn + hvn+1/2
vn+1/2 = vn−1/2 + hf (xn )
– Typeset by FoilTEX – – May 5th 2006
(10)
(11)
19
Summary of reasonably good methods
Original Verlet: don’t even track velocity:
xn+1 = 2xn − xn−1 + M −1h2f (xn )
Position Verlet, partially implicit:
xn+1 = 2xn − xn−1 + M −1 h2 ((1 − α)f (xn+1 ) + αf (xn ))
Note that the half implicit Verlet method with α = 1/2 is of second order.
– Typeset by FoilTEX – – May 5th 2006
20
Summary of reasonably good methods
Two implicit methods that come in handy:
Implicit Euler (dissipative):
xn+1 = xn + hvn+1
vn+1 = vn + hf (xn+1 ) = vn + hf (xn + hvn+1)
(12)
Implicit midpoint:
h
[vn+1 + vn]
2
h
xn+1 + xn
) = vn + hf (xn + [vn+1 + vn])
= vn + hf (
2
4
xn+1 = xn +
vn+1
(13)
Must do root finding on the velocity equation: more about this later.
– Typeset by FoilTEX – – May 5th 2006
21
Conclusion of part I
It is possible to find simple methods to integrate physical systems in a stable way but
careful analysis is necessary.
There is a multitude of integration method to choose from and there is no real clear
cut method that will work in all cases, satisfying all requirements.
It is possible to integrate high frequency problems without resolving them at the
time scale of the simulation, and at least capturing some of the overall physics (the
midpoint rule). However, there are problems and this is discussed later.
– Typeset by FoilTEX – – May 5th 2006
22
Constraints Introduction
Simplify the machinery of Erleblen:
. only holonomic case for now: no velocity or time dependence, no friction
constraints
. ignore rigid rotation details and think of point particles
– Typeset by FoilTEX – – May 5th 2006
23
Constraints Introduction
Physical constraints Φ(x) are limits of strong forces. Think of a spring f = −kx.
As k becomes large, we get the constraint Φ(x) = x = 0
k
m
mg
Figure 5: A stretchable pendulum
– Typeset by FoilTEX – – May 5th 2006
24
Constraints Introduction
Geometric constraints produce forces. How much? Just enough to keep Φ(x) = 0.
We are going to have to compute the constraint forces somehow.
Since motion is tangential to Φ(x) = 0, a force in the direction normal to Φ does
not work. This means that fc = J T λ.
∂ Φ/∂xT = J T
fe (x) External force
Resulting force is tangent to the curve Φ(x) = 0
and motion satisfies constraints
Φ(x) = 0
Constraint force
fc = J T λ
Cancel component of f
that would violate the constraint
Figure 6: A bead on a wire
– Typeset by FoilTEX – – May 5th 2006
25
Constraints Introduction
The equation of motion is now:
T
M ẍ = fe + fc = fe + J λ
Φ(x) = 0
(14)
This is called a differential algebraic equation (DAE). The variable λ is implicitly
defined now as we can rewrite these equations in the form F (ẍ, ẋ, x, λ) = 0, i.e.,
as a nonlinear equations in the varaibles ẍ, ẋ, x, λ.
Since there is no differential equation for λ, we cannot use the stepping methods that
work for ODEs.
– Typeset by FoilTEX – – May 5th 2006
26
How can we solve for λ?
Let us try to find if there is a linear expression hiding here.
Since Φ(x) = 0, then, Φ = const so that Φ̇ = Φ̈ = 0.
∂Φ
d
Φ(x) =
ẋ = Jv = 0.
dt
∂x
d2
d
Φ̈ = 2 Φ(x) = [J ẋ] = J ẍ + J˙ẋ = J ẍ + c = 0.
dt
dt
Formally, the equation Φ(x) = 0 is mathematically equivalent to Φ̈ = J ẍ = 0. So:
Φ̇ =
M ẍ
J ẍ
−J T λ
= fe
= −c
(15)
This is linear in ẍ and λ:
M
J
– Typeset by FoilTEX – – May 5th 2006
−J T
0
ẍ
fe
=
−c
λ
27
How can we solve for λ?
Solve first equation for ẍ and sub in second equation:
ẍ = M −1 J T λ + M −1 fe
−1
T
−1
J ẍ = JM J λ + JM fe = −c
−1 T
−1
JM J λ = −JM fe − c
Put S = JM −1 J T , the Schur complement, and substitute back in:
ẍ = −M
−1
T
J S
−1
JM
−1
fe + c + M
−1
fe
Then use an ODE method, right?
No. Numerically, enforcing Φ̈(x) = 0 is not the same as Φ(x) = 0. Though we do
follow the tangent of the constraint, we take finite steps and move off the surface.
Hard problem but, fortunately, there are good simple methods (and many, many
simple looking BAD methods: be careful!)
– Typeset by FoilTEX – – May 5th 2006
28
Solve for λ numerically the Rattle way.
We can approximate ẍ numerically directly with:
xn+1 − 2xn + xn−1
h2
ẍ ≈
and put this in the equation in the equation of motion so we can write:
M xn+1
Φ(xn+1 )
−h2J T λ
= M [2xn − xn−1] + h2fe
= 0 ≈ Φ(xn ) + Jn(xn+1 − xn )
Now, this is linear in xn+1 . Remains to chose at which step to evaluate J in the first
equation and we can at least try Jn . Putting λ̄ = h2λ, we have:
M
Jn
−JnT
0
– Typeset by FoilTEX – – May 5th 2006
xn+1
M [2xn − xn−1 ] + h2fe (xn )
=
λ̄
−Φ(xn ) + Jnxn
29
Solve for λ numerically the Rattle way.
This will still drift though because we are not solving exactly for Φ(x n+1 = 0 but it
is still better than discretizing the ODE reduction directly. Curiously, Erleben insists
on calling this a velocity based formulation as opposed to the previous format which
he (and others) call an acceleration based formulation.
To make this stable, we need to solve the nonlinear equations Φ(x n+1 ) = 0. To see
how this is done, first remember that:
2
xn+1 = zn + h M
−1
T
Jn λ
where zn is the sum of all the terms that are evaluated at time n.
Then:
2
−1 T
Φ(xn+1 ) = Φ(zn + h M Jn λ) = 0
This is m nonlinear equations in m unknowns. Solve this with Newton iterations say:
Jν M −1JnT λ(ν+1) − λ(ν) = −
1
Φ(zn + h2M −1 JnT λ(ν))
2
h
Start with λ(0) = 0, say, and iterate over ν a few times. You could update Jν so
that it is reevaluated at the new coordinate xn+1 = zn + h2M −1 J T λ(ν) but you
can also keep it at Jn. This last choice corresponds to a modified Newton method.
(I’m not going into details because you will not implement this).
– Typeset by FoilTEX – – May 5th 2006
30
Is there another way?
Constraints are the limit of strong forces. Can we use strong forces numerically
instead of constraints?
Potential: V (x) = k2 ΦT Φ
T
Force: fc (x) = − ∂V
∂x = −kJ Φ
Maybe λ ≈ −kΦ?
Remember that for implicit integration, we need to evaluate f (αx n + (1 − α)xn+1 ).
Approximate this linearly in xn+1 so that:
f (αxn + (1 − α)xn+1 ) = f (xn ) + (1 − α)
∂f
(xn − xn+1 )
∂x
and then, using f = −kJ T Φ:
1 ∂fc
∂J T
∂ 2Φ
T ∂Φ
T
K=−
=J
+
Φ=J J+
k ∂x
∂x
∂x
∂x∂xT
– Typeset by FoilTEX – – May 5th 2006
31
Is there another way?
Substituting this in the implicit Verlet stepper:
xn+1 = 2xn − xn−1 + M −1 h2f (αxn + (1 − α)xn+1 )
and accounting for all the terms, this becomes:
2
I + h k(1 − α)M
−1
Kn xn+1 =
(16)
2xn − xn−1 + M −1h2fn + h2k(1 − α)M −1 Knxn
(17)
This is essentially the Baraff [1] cloth method when α = 0. In general, the case with
α = 1/2 is best as it corresponds to the midpont rule but one must explictly add
some damping.
The matrix multiplying xn+1 is symmetric but not necessarily positive definite. In
addition, the factor of k is problematic as the problem is ill-posed when this gets too
large.
Matrix K contains second derivative terms
and not well behaved.
∂2Φ
∂x∂xT
– Typeset by FoilTEX – – May 5th 2006
which are expensive to compute
32
To do
. relax Rattle using λ = −kΦ
. show the difference between the two matrices
. show what happens in the limit k → ∞.
Next time: spring-damper systems, cable example, Gauss-Seidel iterations, direct
methods.
References
[1] David Baraff and Andrew Witkin. Large steps in cloth simulation. In Michael F.
Cohen, editor, Computer graphics: proceedings: SIGGRAPH 98 Conference
proceedings, July 19–24, 1998, Computer Graphics -proceedings- 1998, pages
43–54, New York, NY 10036, USA and Reading, MA, USA, 1998. ACM Press
and Addison-Wesley. ACM order number: 428980.
– Typeset by FoilTEX – – May 5th 2006
33