Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
EGI-InSPIRE High Performance Computing for Nanoscale Simulations Ladislav Hluchý Slovak NGI International Liaison Viet Tran, Jaroslav Tóbik, Róbert Andok, Ladislav Hluchý, Giang Nguyen, Miroslav Dobrucký Institute of Informatics & Institute of Electrical Engineering, http://www.slovakgrid.sk Slovak Academy of Sciences Bratislava, Slovakia EGI-InSPIRE RI-261323 www.egi.eu The wide use of nanotechnology Nanotechnology and advanced materials research • offered as a transformative technology • with the potential to improve every aspect of social, physical, and economic well-being 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu The wide use of nanotechnology Magnetism at nanoscale - applications • • • • Information storage Medical applications Spintronics Many others… 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu The wide use of nanotechnology What does the research community offer • material research - search for new materials with some specific property • investigation of fundamental processes which defines magnetic properties • design of new devices • manufacturing of device prototypes and optimization of techological processes 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu The wide use of nanotechnology 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu The wide use of nanotechnology Typical physical models and used SW • • • • • • • • • • • • • Quantum-mechanical models Pros.: the most precise tools Cons.: can treat few atoms, electrons, time and resources consuming Use: unavoidable in new material research, adequate for devices where few electrons are responsible for observed phenomena, or "truly quantum" nature of effect is expected Examples of SW: Quantum Espresso, Abinit, Siesta, ... Users: prof. Stich (SAS), prof. Delin (KTH), prof. Fabian? (Uni. Regensburg) Classical models Pros.: simple models, with ability to simulate realistic sizes of devices Cons.: true word is quantum, so results can be misleading in case that quantum nature of the device is important Use: devices with many constitutes and high-enough temperature. Typically used to simulate devices of sizes from few nm up to microns, even more. Examples of SW: OOMMF, MagPar, Nmag, .... Users: Dr. Cambel (SAS), Dr. Chubykalo-Fesenko (CSIC), prof. Delin (KTH), Dr. Junquera (CITIMAC), Dr. Glezos (NCRS) 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Slovak Grid NGI & nanotechnology research • Slovak Grid NGI will set up NGI virtual center of excellence – for application development and consultancy – co-operate with similar centers in other NGI's – the aim is to build human networks of experts on grid and cloud infrastructures as well as experts from application domains • Our domain interest of nanotechnology subfield is NanoElectronics • The micro-magnetism case study will be presented with more detaills in next slides 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Micromagnetism and HPC simulations (1) • • Micromagnetism: new phenomena & exotic magnetic arrangements like vortices or skyrmions are investigated for practical applications e.g. information storage (memory medium). Computer simulations help to understand underlying processes in physics of magnetic devices on sub-micrometer scale: – Solving magnetic dynamics described by non-linear Landau-Lifshitz-Gilbert partial differential equation, – Searching for (meta)stable magnetic configurations minimizing total energy functional for given systems (steepest descent, conjugated gradients, simulated annealing, meta-dynamics algorithm) • Simulations of dynamical processes in magnetic devices, which – are CPU time consuming and – require algorithms’ involvements and HPC supports to increase simulation speeds and efficiency • Used software packages for simulations: – OOMMF: Object Oriented Micro Magnetic Framework – Magpar: implementation of finite elements method solver for micro-magnetics 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Micromagnetism and HPC simulations (2) 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Scientific software for nano-materials and nano-structures PAGASOS Parallel Genetic Algorithm Structure Optimization Simulator A new kind of massive parallel structure optimization simulator based on Genetic Algorithm LAMMPS Classical molecular dynamics code, and an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator A package whose based on Density Functional Theory (DFT), using pseudopotentials and a planewave or wavelet basis. PAS MagPar OOMMF The electron–positron annihilation process is the physical phenomenon relied on Positron Annihilation Spectroscopy. It is also used as a method of measuring the Fermi surface, band structure and defects in metals. A package for simulation of static and dynamic micromagnetic problems including uniaxial anisotropy, exchange, magnetostatic interactions and external fields implementation of finite elements method solver for micromagnetics. It is highly scalable due to efficient parallelization via MPI library Object Oriented Micro Magnetic Framework (math.nist.org/oommf/), widely used software in the field of simulations in nanomagnetism, which is designed so that other users can easily contribute to the software core and modify the software according their needs 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Case study: workflow (1) Scientists develop new magnetic device Scientists prepare idealized structure for computer simulation Electron Beam Litography (EBL) &Scanning Electron Microscope (SEM) Simulation runs on supercomputer and predicts new interesting features Scientists manufacture a number of real devices Scientists measure real device properties Atomic Force Microscope (AFM) Magnetic Force Microscope (MFM) 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Case study: workflow (2) Manufactured devices have not expected properties due to experimental conditions (geometrical imperfections, temperature, impurities, etc...) Scientists prepare simulation input templates for various expected scenarios (+adding temperature effect, +taking geometry from experiment, etc...) Simulations with various suspected scenarios run on super-computer. The best match with experiment observation indicates suitable model. 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Simulation workflows Theoretical and Real devices Scientist manufacture a number of REAL DEVICES Cloud automated determination of device geometry trying various models for simulated devices run run run ... of the simulation post-processing, check if the results coincide with the experiments NO YES Interpret results 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Computing resources demands for nanoscale simulations • Modelling system of nano-magnetic device: – 70nm wide in diameter, which is usual for practical applications – Simulation requires cca. 20 min on PC with 2GHz processor with discretization mesh of 1nm • Real ability is to fabricate devices of the size 500nm in diameter – Behaviors of the 500nm large device is qualitatively different – Simulations for real 500nm wide device were above nanotechnology scientists time limit on their computational capacity • Current research is focused on effect of the temperature on magnetic structure. The temperature is introduced to simulations via stochastic noise added to magnetization dynamics. To get reliable results nanoscientists need to: – Repeat simulations with various different realizations of stochastic noise to acquire some reliable statistics – Run simultaneously multiple instances of simulations with various random noise to accelerate process 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu OOMMF experiments on PC • Hardware: 4-core 3GHz AMD Athlon processor, 4GB RAM • Physical model: Typical simulation of the angular dependence of the vortex nucleation field – simulation cell 90x90x40 nanometers, – grid size 1nm, zero temperature, decreasing external field from 100mT (miliTesla) to zero by 2mT step = 51 simulations – angular dependence was simulated for angles from 0 to 90 degrees - the rest of the graph was recovered due to symmetry properties of the object • Simulations’ total time: 4.5 days with the above-mentioned hardware • Total data produced: 232 MB (only final magnetization structures were saved) 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu OOMMF experiments on IISAS cluster • Physical model: – simulation cell 90x90x40 nanometers, – grid size 2nm, non-zero temperature, simulated time 1 nano-second, fixed external field, – reasonable statistics needs about 100 different simulations of the stochastic noise Typical simulation of the temperature dependence of the vortex nucleation field • Simulations’ total time : 8 hours on 100 cores of IISAS HPC cluster • Total data produced: – approx. 3.5 GB of data for all 100 simulations, – post-processing for presented video took about 4 hours on PC (one trajectory only). • Advantages of HPC approach: – More and bigger simulations, bigger amount of produced data – Much more faster simulations, which can not be realized without HPC 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Ongoing nanotechnology evaluations • • • Physical model: increasing size of simulated devices increase computational cost by second power. Realized devices are typically 200-500nm large(factor 4-25 in computational cost). More efficient and parallel code is needed - we plan to use MagPar. Input created automatically from experimental data: fabrication technology produces order of thousands devices at once. It is not necessary so simulate every device, but it is desirable to make a reasonable statistics over fabricated samples. We estimate to run simulations for approximately 100 devices "overnight". Having simulation feedback in short time (overnight) would be great advance for micromagnetic project, because experiment devices can work in continuation without changing samples for other projects meanwhile crucial questions are answered. 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu IISAS HPC supports for nanoscale simulations • • • • Access to IISAS cluster computational capacity Access to Grid computational capacity Access to IISAS experimental Cloud capacity Computational environment and application scalability supports to enhance nanoscale simulation performance • User-centric nanoscale gateway to HPC environment 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu IISAS user-centric nanoscale gateway (1) • User-friendly gateway to HPC/Grid/Cloud computational power • Suitable for repeated simulations: sending simulations request and receiving simulations results • Do not requires from end-users deep IT knowledge on HPC computing nor Grid/Cloud computing • Customizable for generic HPC/Grid/Cloud jobs 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu IISAS user-centric nanoscale gateway (2) • Monitoring simulations’ statuses • Monitoring IISAS HPC cluster workload • Unified statuses for HPC and Grid/Cloud jobs: simplified and unified job states for end-users • Hide the complexity of the Grid middleware and makes the access to Grid resources transparent and comfortable • Gateway approach helps scientists in other research areas to concentrate better on their work 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Nanoscale gateway Specify simulation request with parameters n Other parameters Send request Successful image transfer Successful simulation specification 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu Nanoscale gateway Simulation results and history Unique request’s indicator Check request input parameters Simulation statuses Check input nano-image Cancel request at any time Simulation results and logs 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu EGI-InSPIRE This work was supported by projects EGI-InSPIRE EU FP7-261323 RI and VEGA No. 2/0054/12. Nanoscale simulations and gateway approach presented in this paper were realized in the hardware equipment obtained within SIVVP ITMS 26230120002 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu EGI-InSPIRE Thank your for your attention [email protected] www.ui.sav.sk [email protected] www.elu.sav.sk 05/2013 Helsinki EGI-InSPIRE RI-261323 www.egi.eu