Download Helsinki-nanoportal-02 - Indico

EGI-InSPIRE
High Performance Computing
for Nanoscale Simulations
Ladislav Hluchý
Slovak NGI International Liaison
Viet Tran, Jaroslav Tóbik, Róbert Andok, Ladislav Hluchý, Giang Nguyen, Miroslav Dobrucký
Institute of Informatics
&
Institute of Electrical Engineering,
http://www.slovakgrid.sk
Slovak Academy of Sciences
Bratislava, Slovakia
EGI-InSPIRE RI-261323
www.egi.eu
The wide use of nanotechnology
Nanotechnology and
advanced materials
research
• offered as a
transformative
technology
• with the potential to
improve every
aspect of social,
physical, and
economic well-being
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
The wide use of nanotechnology
Magnetism at nanoscale - applications
•
•
•
•
Information storage
Medical applications
Spintronics
Many others…
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
The wide use of nanotechnology
What does the research community offer
• material research - search for new materials with
some specific property
• investigation of fundamental processes which defines
magnetic properties
• design of new devices
• manufacturing of device prototypes and optimization
of techological processes
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
The wide use of nanotechnology
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
The wide use of nanotechnology
Typical physical models and used SW
•
•
•
•
•
•
•
•
•
•
•
•
•
Quantum-mechanical models
Pros.: the most precise tools
Cons.: can treat few atoms, electrons, time and resources consuming
Use: unavoidable in new material research, adequate for devices where few
electrons are responsible for observed phenomena, or "truly quantum" nature of
effect is expected
Examples of SW: Quantum Espresso, Abinit, Siesta, ...
Users: prof. Stich (SAS), prof. Delin (KTH), prof. Fabian? (Uni. Regensburg)
Classical models
Pros.: simple models, with ability to simulate realistic sizes of devices
Cons.: true word is quantum, so results can be misleading in case that quantum
nature of the device is important
Use: devices with many constitutes and high-enough temperature. Typically
used to simulate devices of sizes from few nm up to microns, even more.
Examples of SW: OOMMF, MagPar, Nmag, ....
Users: Dr. Cambel (SAS), Dr. Chubykalo-Fesenko (CSIC), prof. Delin (KTH),
Dr. Junquera (CITIMAC), Dr. Glezos (NCRS)
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Slovak Grid NGI &
nanotechnology research
• Slovak Grid NGI will set up NGI
virtual center of excellence
– for application development and
consultancy
– co-operate with similar centers in other
NGI's
– the aim is to build human networks of
experts on grid and cloud infrastructures
as well as experts from application
domains
• Our domain interest of
nanotechnology subfield is
NanoElectronics
• The micro-magnetism case study
will be presented with more
detaills in next slides
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Micromagnetism and HPC simulations (1)
•
•
Micromagnetism: new phenomena & exotic magnetic arrangements like
vortices or skyrmions are investigated for practical applications e.g. information
storage (memory medium).
Computer simulations help to understand underlying processes in physics of
magnetic devices on sub-micrometer scale:
– Solving magnetic dynamics described by non-linear Landau-Lifshitz-Gilbert partial
differential equation,
– Searching for (meta)stable magnetic configurations minimizing total energy
functional for given systems (steepest descent, conjugated gradients, simulated
annealing, meta-dynamics algorithm)
•
Simulations of dynamical processes in magnetic devices, which
– are CPU time consuming and
– require algorithms’ involvements and HPC supports to increase simulation speeds
and efficiency
•
Used software packages for simulations:
– OOMMF: Object Oriented Micro Magnetic Framework
– Magpar: implementation of finite elements method solver for micro-magnetics
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Micromagnetism and HPC simulations (2)
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Scientific software for
nano-materials and nano-structures
PAGASOS
Parallel Genetic Algorithm Structure Optimization Simulator
A new kind of massive parallel structure optimization simulator
based on Genetic Algorithm
LAMMPS
Classical molecular dynamics code, and an acronym for
Large-scale Atomic/Molecular Massively Parallel Simulator
A package whose based on Density Functional Theory (DFT),
using pseudopotentials and a planewave or wavelet basis.
PAS
MagPar
OOMMF
The electron–positron annihilation process is the physical
phenomenon relied on Positron Annihilation Spectroscopy.
It is also used as a method of measuring the Fermi surface,
band structure and defects in metals.
A package for simulation of static and dynamic micromagnetic problems
including uniaxial anisotropy, exchange, magnetostatic interactions and
external fields implementation of finite elements method solver for micromagnetics. It is highly scalable due to efficient parallelization via MPI library
Object Oriented Micro Magnetic Framework (math.nist.org/oommf/), widely
used software in the field of simulations in nanomagnetism, which is designed so
that other users can easily contribute to the software core and modify the
software according their needs
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Case study: workflow (1)
Scientists develop new
magnetic device
Scientists prepare idealized
structure for computer simulation
Electron Beam Litography
(EBL)
&Scanning Electron
Microscope (SEM)
Simulation runs on supercomputer
and predicts new interesting features
Scientists manufacture
a number of real devices
Scientists measure real device
properties
Atomic Force
Microscope (AFM)
Magnetic Force
Microscope (MFM)
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Case study: workflow (2)
Manufactured devices have not
expected properties due to
experimental
conditions (geometrical
imperfections,
temperature, impurities, etc...)
Scientists prepare simulation input
templates for various expected
scenarios (+adding temperature effect,
+taking geometry from experiment, etc...)
Simulations with various suspected
scenarios run on super-computer.
The best match with experiment
observation
indicates suitable model.
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Simulation workflows
Theoretical and Real devices
Scientist manufacture
a number of REAL DEVICES
Cloud
automated determination of device
geometry
trying various models
for simulated devices
run
run
run
... of the simulation
post-processing, check if the results
coincide with the experiments
NO
YES
Interpret results
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Computing resources demands for
nanoscale simulations
• Modelling system of nano-magnetic device:
– 70nm wide in diameter, which is usual for practical applications
– Simulation requires cca. 20 min on PC with 2GHz processor with discretization mesh
of 1nm
• Real ability is to fabricate devices of the size 500nm in diameter
– Behaviors of the 500nm large device is qualitatively different
– Simulations for real 500nm wide device were above nanotechnology scientists
time limit on their computational capacity 
• Current research is focused on effect of the temperature on magnetic
structure. The temperature is introduced to simulations via stochastic
noise added to magnetization dynamics. To get reliable results
nanoscientists need to:
– Repeat simulations with various different realizations of stochastic noise to acquire
some reliable statistics
– Run simultaneously multiple instances of simulations with various random noise
to accelerate process
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
OOMMF experiments on PC
• Hardware: 4-core 3GHz AMD Athlon
processor, 4GB RAM
• Physical model:
Typical simulation of the angular dependence
of the vortex nucleation field
– simulation cell 90x90x40 nanometers,
– grid size 1nm, zero temperature, decreasing
external field from 100mT (miliTesla) to zero
by 2mT step = 51 simulations
– angular dependence was simulated for
angles from 0 to 90 degrees - the rest of the
graph was recovered due to symmetry
properties of the object
• Simulations’ total time: 4.5 days with
the above-mentioned hardware
• Total data produced: 232 MB (only final
magnetization structures were saved)
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
OOMMF experiments on IISAS cluster
• Physical model:
– simulation cell 90x90x40 nanometers,
– grid size 2nm, non-zero temperature, simulated
time 1 nano-second, fixed external field,
– reasonable statistics needs about 100 different
simulations of the stochastic noise
Typical simulation of the
temperature dependence of the
vortex nucleation field
• Simulations’ total time : 8 hours on 100
cores of IISAS HPC cluster
• Total data produced:
– approx. 3.5 GB of data for all 100 simulations,
– post-processing for presented video took about
4 hours on PC (one trajectory only).
• Advantages of HPC approach:
– More and bigger simulations, bigger amount of
produced data
– Much more faster simulations, which can not be
realized without HPC
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Ongoing nanotechnology evaluations
•
•
•
Physical model: increasing size of simulated
devices increase computational cost by second
power. Realized devices are typically 200-500nm
large(factor 4-25 in computational cost). More efficient
and parallel code is needed - we plan to use MagPar.
Input created automatically from experimental
data: fabrication technology produces order of
thousands devices at once. It is not necessary so
simulate every device, but it is desirable to make a
reasonable statistics over fabricated samples. We
estimate to run simulations for approximately 100
devices "overnight".
Having simulation feedback in short time
(overnight) would be great advance for micromagnetic
project, because experiment devices can work in
continuation without changing samples for other
projects meanwhile crucial questions are answered.
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
IISAS HPC supports
for nanoscale simulations
•
•
•
•
Access to IISAS cluster computational capacity
Access to Grid computational capacity
Access to IISAS experimental Cloud capacity
Computational environment and application
scalability supports to enhance nanoscale
simulation performance
• User-centric nanoscale gateway to HPC
environment
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
IISAS user-centric nanoscale gateway (1)
• User-friendly gateway to HPC/Grid/Cloud computational power
• Suitable for repeated simulations: sending simulations request and
receiving simulations results
• Do not requires from end-users deep IT knowledge on HPC
computing nor Grid/Cloud computing
• Customizable for generic HPC/Grid/Cloud jobs
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
IISAS user-centric nanoscale gateway (2)
• Monitoring simulations’ statuses
• Monitoring IISAS HPC cluster workload
• Unified statuses for HPC and Grid/Cloud jobs: simplified and unified job
states for end-users
• Hide the complexity of the Grid middleware and makes the access to
Grid resources transparent and comfortable
• Gateway approach helps scientists in other research areas to
concentrate better on their work
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Nanoscale gateway
Specify simulation request with parameters
n
Other
parameters
Send request
Successful
image transfer
Successful
simulation
specification
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
Nanoscale gateway
Simulation results and history
Unique request’s
indicator
Check request
input parameters
Simulation
statuses
Check input
nano-image
Cancel request
at any time
Simulation
results and logs
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
EGI-InSPIRE
This work was supported by projects
EGI-InSPIRE EU FP7-261323 RI
and
VEGA No. 2/0054/12.
Nanoscale simulations and gateway approach
presented in this paper were realized in the
hardware equipment obtained within
SIVVP ITMS 26230120002
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu
EGI-InSPIRE
Thank your for your attention
[email protected]
www.ui.sav.sk
[email protected]
www.elu.sav.sk
05/2013 Helsinki
EGI-InSPIRE RI-261323
www.egi.eu