Download Full-scale-PIC-simulations

Full-scale particle simulations of highenergy density science experiments
W.B.Mori , W.Lu, M.Tzoufras, B.Winjum, J.Fahlen,F.S.Tsung, C.Huang,J.Tonge
M.Zhou, V.K.Decyk, C. Joshi (UCLA)
L.O.Silva, R.A.Fonseca (IST Portugal)
C.Ren (U. Rochester)
T. Katsouleas (USC)
Directed high-energy density
Lasers
• Pressure=Energy/Volume
– Pressure=Power/Area/c
• PetaWatt with 10mm spot
– 3x1010 J/cm3
– 300 GBar
• Electric field in laser:
Particle beams
• At SLAC:
– N=2x1010 e- or e+
sr=1mm, sz =60mm
– E=50GeV
• Pressure:
– 15x1010J/cm3
– 1.5TBar
– TeV/cm
• Electric field of beam:
– 1.6TeV/cm
Radiation pressure and space forces of intense
lasers and beams expel plasma electrons
Particle Accelerators
Why Plasmas?
Conventional Accelerators
• Limited by peak power and
breakdown
Plasma
• No breakdown limit
• 20-100 MeV/m
• 10-100 GeV/m
Why lasers?
Radiation pressure can excite longitudinal wakes
a0 
eA
10
2

8.5x10
I(W
/cm
)mm
2
mc
Concepts For Plasma Based
Accelerators*

Plasma Wake Field Accelerator(PWFA)
A high energy electron bunch

Laser Wake Field Accelerator(LWFA, SMLWFA, PBWA)
A single short-pulse of photons

Drive beam
1. Wake excitation
2. Evolution of driver and wake
3. Loading the wake with particles
*Tajima and Dawson PRL 1979

Trailing beam
Plasma Accelerator Progress and
the “Accelerator Moore’s Law”
Slide 2
LOA,RAL
LBL
,RAL
Osaka
Courtesy of Tom Katsouleas
The blowout
and
Rosenzwieg et al. 1990
bubble regimes
Puhkov and Meyer-te-vehn 2002
Ion column provides ideal accelerating and focusing forces
Full scale 3D particle-in-cell modeling is
now possible:OSIRIS
Typical simulation
parameters:
~109 particles
~105 time steps
Other codes:VLPL, Vorpal, TurboWAVE, Z3 etc., but no all the same!
Progress in computer hardware
The “Dawson” cluster at
UCLA: <$1,000,000
$50,000,000
Progress in lasers
Courtesy of G.Mourou
Progress in hardware and software
• Era
Memory
particles
speed
max energy (full PIC)
•80’s
16MByte
105-106
5ms/part-step
100 MeV (2D)
~109
1x10-3ms/part-step 1-10GeV (3D
(~7.5 Tflops/3)
•Local
~500GByte
Clusters
(e.g., DAWSON)
~109
2x10-3 ms/part-step 1-10GeV (3D)
(2.3Tflops)
1 TeV (3D)
•Future 25-1000TByte
>1011
5x10-5ms/part-step 500 GeV (3D)
150Tflops - 10Pflops?
•Today ~6TByte/3
(e.g., NERSC)
The simulations of Tajima and Dawson
would take ~1 second on my laptop!
Computational challenges for modeling
plasma-based acceleration
(1 GeV Stage)
Beam-driven wake*
z
y, x
t
# grids in z
# grids in x , y
# steps
Nparticles
Particles x steps
Fully Explicit
 .05 c/p
 .05 c/p
 .02 c/p
350
150
2 x 10 5
~.25 x 1 08 (3D)
~1 x 10 6 (2D)
~.5 x 1013 (3D) -  10,000 hrs
~1 x 10 11 (2D) -  75 hrs
*Laser-driven GeV stage requires on the o rder of (o/p)2=1000 x l onger,
however, the the resolution ca n usually be relaxed.
Full-scale modeling:
Challenges and expectations
Challenges:
What is excellent agreement?
• As a laser propagates through• Don’t know exact plasma
the plasma it encounters
profile.
~1013-1014 electrons
• Don’t know laser intensity or
spot size.
• There are ~106-109 selftrapped electrons
• Don’t know laser transverse,
longitudinal, or frequency
• Need to model accuracy of 1
profile (not a diffraction
6
part in O(10 )
limited Gaussian beam).
Convergence of advances
in laser technology and
computer simulation
Full scale 3D LWFA simulation using OSIRIS:
6TW, 50fs
State-of- the- art ultrashort laser
pulse
0 = 800 nm, t = 50 fs
•Simulation Parameters
I = 2.5x1018 W/cm-2, W =12.5 mm
512 cells
100 mm
512 cells
100 mm
Plasma Background
ne = 2x1019 cm-3
2340 cells
56.18 mm
Simulation ran for 6400 hours on DAWSON
(~4 Rayleigh lengths)
–Laser:
• a0 = 1.1
• W0=15.6 12.5mm
• l/p = 10
–Particles
• 2x1x1 particles/cell
• 500 million total
–Plasma length
• L=.2cm
• 50,000 timesteps
Simulations: no fitting parameters!
Nature papers, agreement with experiment
3D Simulations for: Nature
V431, 541 (S.P.D Mangles et
al)
Imperial Data #( /((dE/E) )
3D OSIRIS
12
1.4 10
1.2 1012
1 1012
•
•
In experiments, the # of
electrons in the spike is 1.4
108.
In our 3D simulations, we
estimate
of
2.4
108
electrons in the bunch.
8 1011
6 1011
4 1011
2 1011
0
-2 1011
-50
0
50
100
Energy (MeV)
150
200
Movie of Imperial Run
Plasma density and laser envelope
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
3D PIC simulations:
Tweak parameters
Propagation: 2 mm
Scenario:
• self-focusing (intensity increases by 10)
• longitudinal compression
Excite highly nonlinear wakefield
with cavitation: bubble formation
• trapping at the Experiment
X point
• electrons dephase and self-bunch
• monoenergetic electrons are behind
PIC
the laser field
Parameters: E=1 J, 30 fs, 18 µm waist, 6×1018 cm-3
Full scale 3D LWFA simulation using OSIRIS
Predict the future: 200TW, 40fs
State-of- the- art ultrashort laser
pulse
0 = 800 nm, t = 30 fs
•Simulation Parameters
I = 3.4x1019 W/cm-2, W =19.5 mm
256 cells
80.9 mm
256 cells
80.9 mm
Plasma Background
ne = 1.5x1018 cm-3
4000 cells
101.9 mm
Simulation ran for 75,000 hours on DAWSON
(~5 Rayleigh lengths)
–Laser:
• a0 = 4
• W0=24.4 19.5mm
• l/p = 33
–Particles
• 2x1x1 particles/cell
• 500 million total
–Plasma length
• L=.7cm
• 300,000 timesteps
OSIRIS 200 TW simulation:
Run on DAWSON Cluster
A 1.3 GeV
beam!
The trapped particles form a
beam.
•
•
Beam loading
Normalized emittance:The
divergence of the beam is
about 10mrad.
Energy spread:
Physical picture
Evolution of the nonlinear structure
• The blowout radius remains
nearly constant as long as the
laser intensity doesn’t vary much.
Small oscillations due to the slow
laser envelope evolution have
been observed.
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
• Beam loading eventually shuts
down the self injection.
• The laser energy is depleted as
the accelerating bunch dephases.
The laser can be chosen long
enough so that the pump
depletion length is longer than
the dephasing length.
QuickPIC loop:
2-D plasma slab
Wake (3-D)
Beam (3-D):
Laser or particles
1. initialize
beam
2. solve  2  ,  2   e  Fp , 
3. push plasma, store 
4. step slab and repeat 2.
5. use  to giant step beam
QuickPIC: Basic concepts
Maxell’s equations
in Lorentz gauge
Full PIC
(no approximation)
Let :
s  z,   ct  z
1
( 2
c
1
( 2
c
dPe
1
q
(E

v e  B)
e
 dt
c

plasma e : 
dXe  v
e
 dt
 dPb
1

q
(E

v b  B)
b
 dt
c

beam e : 
 dXb  v
b
 dt
2
4
2


)A

j
t 2
c
2
2
2   )  4
t
Assume :
(1) s  
(quasi  static
approximation)
(2) v b  c
4
 A 
j
c
2  4
2


Let :     A//
QuickPIC
 2  4 
   4  (  

2

j //
c
Particle pusher(relativistic)
)
 dPe
qe
1
d( /c)  1 v /c (E  c v e  B)

ez
plasma e : 
 dXe  v e

d( /c) 1 v ez /c
 dPb
1

q
(E

v b  B)

d(s /c) b
c

beam e : 
 dXb  v
b

d(s /c)
Solved by 2D
field solver
QuickPIC: Code structure
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickPIC Benchmark: Full PIC vs. Quasi-static PIC
Benchmark for different drivers
0.1
Longitudinal Wakefield (mcp/e)
Longitudinal Wakefield (mcp/e)
3
Osiris
QuickPIC (l=2)
QuickPIC (l=4)
2
1
0
-1
-2
e- driver
-3
-8
-6
-4
-2
0
2
4
6
Osiris
QuickPIC (l=2)
0.05
0
-0.05
-0.1
-10
8
e+ driver
-5
0
10
1
2
OSIRIS
QuickPIC
Longitudinal Wakefield (mcp/e)
Longitudinal wakefield(mcp/e)
5
 (c/p)
 (c/p)
1
0
-1
e- driver with
ionization
-2
Osiris
QuickPIC (l=2)
0.5
0
-0.5
-1
-3
-5
0
5
(c/p)
10
laser driver
-6
-4
-2
0
2
4
6
 Excellent agreement with full
PIC code.
 More than 100 times timesavings.
 Successfully modeled current
experiments.
 Explore possible designs for
future experiments.
 Guide development on
theory.
 (c/p)
100+ CPU savings with “no” loss in accuracy
A Plasma Afterburner (Energy Doubler) Could be
Demonstrated at SLAC
0-50GeV in 3 km
50-100GeV in 10 m!
3 km
30 m
Afterburners
S. Lee et al., Phys. Rev. STAB, 2001
Excellent agreement between simulation and experiment
of a 28.5 GeV positron beam which has passed through a 1.4 m PWFA
OSIRIS Simulation Prediction:
Experimental Measurement:
OSIRIS
Head
Peak Energy Loss
64 MeV
65±10 MeV
Peak Energy Gain
78 MeV
79±15 MeV
E162 Experiment
Tail
Head
Tail
5x108 e+ in 1 ps bin at +4 ps
• Identical parameters to experiment
ionization: Agreement is excellent!
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
including
self-
Relative Energy (GeV)
Full-scale simulationof E-164xx is
possible using a new code QuickPIC
+4
+2
0
-2
-4
-5
0
+5
X (mm)
Full-scale simulationof E-164xx is
possible using a new code QuickPIC
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
QuickTime™ and a
YUV420 codec decompressor
are needed to see this picture.
Full-scale simulation of a 1TeV
afterburner possible using QuickPIC
5000 instead of 5,000,000 node hours
• We use parameters consistent
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
with the International Linear
Collider “design”
•We have modeled the beam
propagating through ~25 meters
of plasma.
I see a day where the
world is fueled
by fusion energy.
I see a day
when high energy
accelerators
will fit on a
tabletop.
I see a day where
particle simulations
will use
1 trillion particles
Wakefield equations:
“2D-electro and magneto-statics
Maxwell equations in Lorentz gauge
1
( 2
c
1
( 2
c
Reduced Maxwell equations
4
2

4

2
  A 
j


)
A

j
c
2
t
c
2

Quasi-static
approx.
   4 
2

2
2



4

(


j
/c)


)


4


z
2
t
 (   j/c)
2
We define
,A  (ct  z  ,x ), A(,x )

    Az
    j  0
Antonsen and Mora 1997

Whittum 1997
Huang et al., 2005 (QuickPIC)
Quasi-static Model including a laser driver
Maxwell equations in Lorentz gauge
1 2
4
( 2
  2 )A 
j
2
quasi - static : s  0
c t
c
1 2
2
s  z,   ct  z
( 2
2   )  4
c t

Laser envelope equation:
dPb

qb
c
3D loop
begin
2D loop
begin
Initialize beam
Initialize plasma
Call 2D routine
Field Solver
Push beam particles
Push plasma particles
Deposition
Deposition
3D loop
end
4
j
c
 2   4
 2 A 
 2p


2
2
2
2 ik0   a  a  k0  pa = k0
a
s 
 
02 p
V


E  b  B

ds
c


dP
qe
V
For plasma electrons : e  
[E   ( e  B )  ]
d
c  Ve //
c
For beam electrons :
Reduced Maxwell equations
2D loop
end
Iteration
Pipelining: scaling quasi-static PIC to 10,000+ processors
beam
1
Initial plasma slab
2
3
4
Initial plasma slab
1
solve plasma
response
2
3
4
solve plasma
response
solve plasma
response
solve plasma
response
solve plasma
response
update beam
update beam
update beam
update beam
update beam
beam
Without pipelining: Beam is not advanced
until entire plasma response is determined
With pipelining: Each section is updated when its
input is ready, the plasma slab flows in the pipeline.
LWFA - Accelerating Field
•Isosurface
values:
512 cells
40.95 mm
•Blue :
-0.9
•Cyan:
-0.6
•Green:
-0.3
•Red:
+0.3
•Yellow:
+0.6
•Electric Field in
normalized units
me c p e-1
Simulations
The 200 TW run: Dephasing ~ Pump depletion
Given a Laser we pick the plasma density and we evaluate from our formulas:
w0  20mm
  30 fs
P  200TW
  0.8mm
a0  4

n p 1.5 1018 cm3
After 5 Zr / 7.5 mm
P
 10
Pc
Total charge = 1.1 nC
f(E) (a.u.)
2.5
f(E)

LT  1.0cm
Ldp  1.3cm
E  1.5GeV
2
1.5
1
0.5
0
800
1200
1600
Energy (MeV)
2000
Physical picture of an “optimal” regime
Geometry - fields
• The ponderomotive force of the
laser pushes the electrons out of
the laser’s way.
• The particles return on axis
after the laser has passed.
• The region immediately behind
the pulse is void of electrons but
full of ions.
• The result is a sphere (bubble)
moving with the speed of (laser)
light,
supporting
huge
accelerating fields.
Physical picture
Evolution of the nonlinear structure
• The front of the laser pulse
interacts with the plasma and loses
energy. As a result the front etches
back.
• The shape
accelerating
change.
and size
structure
of the
slightly
• Electrons are self-injected in the
plasma bubble due to the
accelerating and focusing fields.
• The trapped electrons make the
bubble elongate.
PIC Simulations of beam loading in blowout regime:
Used the new code QuickPIC
(UCLA,USC,U.Maryland)
Bi-Gaussian shape
sz= 1.2 c/p, nb/np= 26
Wedge shape w/ beam load beam
length = 6 c/p, nb/np= 8.4, Ndrive
= 3x1010, Ntrailing = 0.5x1010