Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Mon. Not. R. Astron. Soc. 425, 3024–3057 (2012) doi:10.1111/j.1365-2966.2012.21590.x Moving mesh cosmology: numerical techniques and global statistics Mark Vogelsberger,1 Debora Sijacki,1 Dušan Kereš,2,3 Volker Springel4,5 and Lars Hernquist1 1 Harvard–Smithsonian Center for Astrophysics, 60 Garden Street, Cambridge, MA 02138, USA of Astronomy and Theoretical Astrophysics Center, University of California, Berkeley, CA 94720-3411, USA 3 Department of Physics, Center for Astrophysics and Space Sciences, University of California at San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA 4 Heidelberg Institute for Theoretical Studies, Schloss-Wolfsbrunnenweg 35, 69118 Heidelberg, Germany 5 Zentrum für Astronomie der Universität Heidelberg, ARI, Mönchhofstr. 12-14, 69120 Heidelberg, Germany 2 Department Accepted 2012 June 25. Received 2012 May 29; in original form 2011 September 7 ABSTRACT We present the first hydrodynamical simulations of structure formation using the new moving mesh code AREPO and compare the results with GADGET simulations based on a traditional smoothed particle hydrodynamics (SPH) technique. The two codes share the same Tree-PM gravity solver and include identical subresolution physics for the treatment of star formation, but employ different methods to solve the equations of hydrodynamics. This allows us to assess the impact of hydro-solver uncertainties on the results of cosmological studies of galaxy formation. In this paper, we focus on predictions for global baryon statistics, such as the cosmic star formation rate density, after we introduce our simulation suite and numerical methods. Properties of individual galaxies and haloes are examined by Keres et al., while a third paper by Sijacki et al. uses idealized simulations to analyse in more detail the differences between the hydrodynamical schemes. We find that the global baryon statistics differ significantly between the two simulation approaches. AREPO shows systematically higher star formation rates at late times, lower mean temperatures averaged over the simulation volume and different gas mass fractions in characteristic phases of the intergalactic medium, in particular a reduced amount of hot gas. Although both hydrodynamical codes use the same implementation of cooling and yield an identical dark matter halo mass function, more gas cools out of haloes in AREPO compared with GADGET towards low redshifts, which results in corresponding differences in the late-time star formation rates of galaxies. We show that this difference is caused by a higher heating rate with SPH in the outer parts of haloes, owing to viscous dissipation of SPH’s inherent sonic velocity noise and SPH’s efficient damping of subsonic turbulence injected in the halo infall region, and because of a higher efficiency of gas stripping in AREPO. As a result of such differences, AREPO leads also to more disc-like morphologies in the moving mesh calculation compared to GADGET. Our results hence indicate that inaccuracies in hydrodynamic solvers can lead to comparatively large systematic differences even at the level of predictions for the global state of baryons in the universe. Key words: methods: numerical – galaxies: formation – dark matter. 1 I N T RO D U C T I O N Current theories of galaxy formation are based on the view that the dominant mass contribution to the Universe is in the form of cold dark matter (DM), which clusters under gravity and builds up the backbone of all cosmic structure. Together with the discovery of a large dark energy (DE) component (Perlmutter et al. 1999; E-mail: [email protected] Riess et al. 1999), this led to the emergence of the concordance cold dark matter (CDM) cosmogony. The third component to the energy density in this scenario, the baryons, condenses via radiative cooling at the centres of a population of hierarchically merging DM haloes, forming galaxies (Rees & Ostriker 1977; Silk 1977; White & Rees 1978; Blumenthal et al. 1984). Although the precise physical nature of DM and DE is not yet known (but see Bertone, Hooper & Silk 2005, for a review of possible DM candidates), large-scale predictions of this CDM theory show good agreement with a wide range of observations, among them cosmic C 2012 The Authors C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology microwave background (CMB) fluctuations (Dunkley et al. 2009), large-scale clustering of galaxies in the low-redshift universe (Percival et al. 2010) and the redshift z ∼ 2–3 power spectrum probed by the Lyα forest (Viel, Bolton & Haehnelt 2009). It is however crucial to further constrain and test the CDM paradigm, especially on the smaller scales of galaxies, which remains a theoretical and observational frontier. Structure formation is a highly non-linear process and only the early stages can be described analytically using linear theory (Zel’Dovich 1970). However, some statistical properties of the evolved DM field, in particular the halo mass function, can also be predicted analytically by combining linear theory and spherical collapse models, through the Press–Schechter formalism (Press & Schechter 1974) and subsequent extensions (e.g. Bond et al. 1991). These predictions ultimately rely on calibration through more accurate calculations, which typically can only be done numerically. Indeed, while Press–Schechter theory has been quite successful in describing the abundances of haloes, once computer models reached high precision it became clear that the original analytic treatment was not sufficiently accurate for the quantitative work required in today’s era of precision cosmology (e.g. Jenkins et al. 2001). Over the last two decades, numerical simulations have played a key role in guiding our knowledge of structure and galaxy formation. This is especially evident for the CDM component, where the relevant calculations have reached a high level of sophistication (e.g. Springel et al. 2005b, 2008; Diemand et al. 2008; Boylan-Kolchin et al. 2010; Klypin, Trujillo-Gomez & Primack 2011). Many important insights have been gained from purely collisionless simulations, e.g. the nearly universal density profiles of DM haloes (Navarro, Frenk & White 1996, 1997; Navarro et al. 2010), assembly bias (Gao, Springel & White 2005) and the internal structure of haloes on small scales (Vogelsberger et al. 2009; Vogelsberger & White 2011). An important reason for today’s reliability of CDM predictions is that the initial conditions are unambiguously specified in the CDM model, with parameters that are tightly constrained by CMB observations (Komatsu et al. 2011). Moreover, the computational problem is well posed and comparatively simple, with equations of motion that for DM involve only gravity. Efficient new algorithms and the rapid growth of computing power over the few last decades have allowed ever more detailed theoretical predictions for the DM distribution. It is worthwhile, however, to recall that initially such collisionless DM simulations suffered from numerical artefacts, leading to issues like the infamous ‘overmerging problem’, which resulted in featureless and smooth DM haloes primarily due to insufficient mass-, force- and time resolution (Moore et al. 1998, 1999). To understand properties of the observable universe one needs to relate the DM distribution to that of the baryonic material. This can be done in a number of different ways: (i) using so-called semianalytical modelling of galaxy formation (e.g. White & Frenk 1991; Kauffmann, White & Guiderdoni 1993; Baugh 2006; Croton et al. 2006; Benson 2010, 2012; Guo et al. 2011) or the closely related approach of halo occupation distributions (Benson et al. 2000), or (ii) using direct hydrodynamical simulations (e.g. Hernquist & Katz 1989; Katz, Hernquist & Weinberg 1992; Bryan & Norman 1998; Davé et al. 1999; Springel, Di Matteo & Hernquist 2005a; Governato et al. 2010). Approach (i) allows a fast exploration of a large parameter space of the underlying coarse description of galaxy formation physics, whereas (ii) makes possible the calculation of a self-consistent model that requires far fewer assumptions about the gas dynamics. Nevertheless, both methods share the common problem that complicated physics on very small scales, such as star C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3025 formation and associated feedback processes, need to be accounted for in a rather crude way. This propagates into uncertainties in the interpretation of simulation results for the hydrodynamic sector. One possible strategy to get better control over this problem is to calibrate subresolution treatments in semi-analytic models and hydrodynamical simulations observationally (e.g. Schaye et al. 2010; Guo et al. 2011). In addition, numerical artefacts in hydrodynamical simulation techniques need to be better understood, hopefully allowing one to reduce or completely eliminate them. State-of-the-art cosmological hydrodynamical simulations include a prescription for star formation (e.g. Springel & Hernquist 2003a), radiative cooling (Katz, Weinberg & Hernquist 1996b; Smith, Sigurdsson & Abel 2008), chemical enrichment (e.g. Wiersma et al. 2009) and various feedback processes relevant for low- and high-mass systems (e.g. Sijacki et al. 2007; Scannapieco et al. 2008). Furthermore, some simulation codes can also include magnetic fields (Dolag et al. 2005a; Collins et al. 2010), radiative transfer (Petkova & Springel 2011; Cantalupo & Porciani 2011), cosmic ray physics (Jubelgas et al. 2008), thermal conduction (Dolag et al. 2004; Jubelgas, Springel & Dolag 2004) or black hole physics (Di Matteo, Springel & Hernquist 2005; Springel et al. 2005a; Di Matteo et al. 2008). Hydrodynamical simulations have been successfully used to study the Lyα forest (e.g. Hernquist et al. 1996; Katz et al. 1996a; Viel et al. 2005) and the detailed physics of the intracluster medium, where cluster scaling relations were obtained that agreed with X-ray observations (Puchwein, Sijacki & Springel 2008). There have also been numerous studies that focused on the formation of a representative galaxy population (e.g. Pearce et al. 1999; Murali et al. 2002; Weinberg et al. 2004; Nagamine et al. 2005; Crain et al. 2009). While some successes have been achieved here as well, a general finding of these simulations is that it appears to be difficult to reproduce the observed shallow slope of the faint-end of the galaxy luminosity function and the observed morphological mix of galaxies, with its high abundance of disc-like galaxies. Although there has been some considerable progress over the last few years on the latter issue (e.g. Robertson et al. 2004; Governato et al. 2007; Scannapieco et al. 2008; Governato et al. 2010; Agertz, Teyssier & Moore 2011; Guedes et al. 2011), one of the outstanding problems is to produce a statistical sample of galaxies that agrees reasonably well with observations and reproduces the Hubble sequence. But modifications in the detailed feedback and star formation prescriptions proved to be important and successful in forming more realistic late-type spiral galaxies. We note that cosmological hydrodynamic simulations have also made it clear that a proper inclusion of the baryonic physics can lead to interesting back reactions on the DM distribution (D’Onghia et al. 2010a; Duffy et al. 2010), potentially eliminating or reducing the central DM cusp (Governato et al. 2010), or forming a dark disc (Read et al. 2008). It is hence clear that it is ultimately not sufficient to work with DM-only simulations. Most hydrodynamical simulations of galaxy formation carried out thus far have used the smoothed particle hydrodynamics (SPH) technique (Gingold & Monaghan 1977; Lucy 1977; Monaghan 1992, 2005), where the gas is discretized into a set of particles for which appropriate equations of motion can be derived. The SPH method is well suited for cosmological applications owing to its pseudo-Lagrangian character, which automatically brings resolution elements to regions where they are needed most, i.e. collapsing objects such as clusters and galaxies. Also, the conservation properties of SPH in terms of simultaneous conservation of energy, momentum, mass, entropy and angular momentum are excellent. A further convenient feature is that a particle-based gravity solver 3026 M. Vogelsberger et al. (needed for the DM component anyway) can be readily applied to SPH. These properties have made the method also popular for applications other than cosmic structure growth, e.g. for isolated merger simulations (Barnes & Hernquist 1991, 1992, 1996; Mihos & Hernquist 1994, 1996; Naab & Burkert 2003; Cox et al. 2006). However, different methods for solving the equations of hydrodynamics in a cosmological context have been employed as well, most of which are descendants of classic Eulerian methods (Stone & Norman 1992) on regular meshes. The fixed Cartesian grids originally available in the corresponding codes are insufficient to capture the large dynamic ranges encountered in galaxy formation, but finite-volume schemes incorporated in modern adaptive mesh refinement (AMR) codes (Berger & Colella 1989; Teyssier 2002; O’Shea et al. 2004) can alleviate this problem. Interestingly, these mesh-based methods have led in some cases to significantly different results compared with SPH. A prominent example of these discrepancies is highlighted in the Santa Barbara cluster comparison project (Frenk et al. 1999), where it was found that the entropy profiles of clusters are significantly and systematically different between AMR and SPH codes, the former yielding a large entropy core which is absent in the SPH calculations. Only recently has some progress been made in identifying the cause for this difference (Mitchell et al. 2009), but the issue is still not fully understood; hence we return to it in one of our companion papers (Sijacki et al. 2011). It thus appears that there are at least two important challenges in the field of cosmological hydrodynamics simulations. One lies in an adequate treatment of all relevant physics of galaxy formation, in particular with respect to the reliability of the parametrizations of subresolution physics, and the other having to do with the accuracy and efficiency of the underlying hydrodynamics solver. It is of course crucial to strive for a more faithful treatment of the physics in the simulations, but it should also be evident that a proper modelling of the physics relies on a correct solution of the basic hydrodynamical equations in the first place. It is therefore problematic to tune the subresolution physics without first verifying in detail the quality of the hydro solver, because this risks incorrectly absorbing deficiencies of a numerical technique into distorted physics models. Here, in this initial study we shall primarily be concerned with an assessment of the extent to which uncertainties in numerical methodology are reflected in the predicted galaxy properties, for a fixed physics model. In future works, we will explore the consequences of various treatments of feedback effects, which are known to be crucial for the galaxy formation problem (Scannapieco et al. 2012). We remark that it is not obvious whether SPH or AMR is more accurate in all simulation regimes, as both methods are known to have shortcomings. For example, SPH suffers from relatively poor shock resolution, noise on the scale of the smoothing kernel and low-order accuracy for the treatment of contact discontinuities. Furthermore, some hydrodynamic instabilities like the Kelvin–Helmholtz instability can be suppressed in SPH (Agertz et al. 2007). Recently, Bauer & Springel (2012) also showed that conventional implementations of SPH do not properly subsonic turbulence, which is generically present in cosmological haloes. Astrophysical Eulerian methods on the other hand (usually realized as AMR finite-volume schemes) may suffer from overmixing due to advection errors in the presence of bulk flows. One of the principal differences between SPH and AMR lies in their handling of mixing at the level of individual fluid elements. This is absent in SPH by construction (unless added in crudely by hand somehow), while in AMR it occurs implicitly through the averaging of the evolved Riemann solutions over the scale of the grid cells at the end of each time step. It is not always clear which scheme yields a more accurate result (Trac, Sills & Pen 2007; Mitchell et al. 2009). Another issue with classical AMR codes is that their handling of the discrete equations of motion can lead to errors with the fluid is moving rapidly across the mesh (Springel 2010a, hereafter S10). Because the truncation error of Eulerian codes depends on the fluid velocity relative to the grid, the results can thus be sensitive to the presence of bulk velocities (see also Tasker et al. 2008; Wadsley, Veeravalli & Couchman 2008). Finally, a further challenge for AMR schemes in cosmic structure formation arises from the need to accurately follow the gravitational growth of even very small structures. The mesh-based Poisson solvers typically employed by AMR codes have been shown to lack sufficient small-scale force accuracy and to produce too few low-mass haloes (O’Shea et al. 2005; Heitmann et al. 2008) potentially corrupting the solution on even well-resolved scales. While this can be addressed with more aggressive refinement strategies, the discontinuous changes in gravity resolution brought about by such refinements introduce non-Hamiltonian perturbations into the DM dynamics which are in principle undesirable. Recently, S10 introduced a new moving mesh approach as embodied in the AREPO code. The principal goal is similar to the earlier implementations of Gnedin (1995) and Pen (1998), but these moving mesh algorithms suffered from grid distortions which limited their applicability. AREPO does not use coordinate transformations like previous moving mesh codes in cosmology, but instead employs an unstructured Voronoi tessellation of the computational domain. The mesh-generating points of this tessellation are allowed to move freely, offering significant flexibility for representing the geometry of the flow. As discussed in detail in S10, this technique avoids several of the weaknesses of SPH and AMR schemes while it retains their most important advantages. For example, if the mesh motion is tied to the gas flow, the results are Galilean invariant (like in SPH), while at the same time a high accuracy for shocks and contact discontinuities is achieved (like in Eulerian schemes). The aim of this paper is to compare the novel cosmological hydrodynamics code AREPO with the widely used SPH code GADGET (Springel 2005). In this first paper of a series we give an overview of the numerical techniques used for our galaxy formation simulations, introduce our simulation set, present an analysis of the global baryon characteristics, and evaluate the performance and efficiency of the new simulation scheme. In one companion paper (Keres et al. 2011, hereafter Paper II), we discuss the properties and statistics of individual galaxies and haloes in the simulations. A further companion paper (Sijacki et al. 2011, hereafter Paper III) analyses idealized test problems to highlight various differences of the two simulation schemes and to elucidate how they affect galaxy formation. The outline of this paper is as follows. In Section 2, we provide a discussion of the initial conditions and numerical details which are particularly important for the goal of our comparison project. There we also discuss some modifications of the AREPO code that were adopted during the course of this work. Section 3 presents first results and discusses the large-scale structure and global statistics of the baryon content in the simulations at different resolutions. We turn to an analysis of the origin of the cooling difference we find between the codes in Section 4. In Section 5 we discuss some generic issues and problems of SPH. Finally, we give a summary of our results and our conclusions in Section 6. In an Appendix, we provide data on the mesh geometry, analyse the performance of the codes in different parts of the calculations and present scaling tests. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 2 N U M E R I C A L M E T H O D S A N D S I M U L AT I O N SET 2.1 Implemented physics Our GADGET and AREPO simulations follow the same physics, consisting of a collisionless DM fluid and an ideal baryonic gas. Both components act as sources of gravity and are evolved in a Newtonian approximation on top of an expanding Friedman–Lemaitre background model. We describe the gas dynamics with the ordinary inviscid Euler equations, augmented with additional terms that account for ‘extra physics’ related to radiative processes and star formation. In particular, we account for optically thin radiative cooling of a primordial mixture of helium and hydrogen, with a hydrogen mass fraction of 0.76. We also assume a spatially uniform, timedependent ionizing ultraviolet background with an amplitude and time evolution according to Faucher-Giguère et al. (2009). As a result of cooling, gas can collapse to high density and turn into stars. Both simulation codes describe this process with the simple star formation and supernova feedback model introduced in Springel & Hernquist (2003a), which has been used in many SPH simulations in recent years. In this approach, every sufficiently dense gas particle/cell is treated as a representative region of the interstellar medium (ISM) and is assumed to exhibit a multiphase structure consisting of cold and hot gas in pressure equilibrium. Stars form out of the cold gas and return energy back into the medium though supernova explosions. This energy feedback pressurizes the multiphase medium, the effect of which is described in terms of an effective equation of state that is imposed on to the gas once it has overcome the density threshold for star formation. The value of the density threshold we use in this work is nH = 0.13 cm−3 , set such that isolated disc galaxies at z = 0 are characterized with a relation between disc surface gas density and star formation density that agrees with the observed Kennicutt law (Kennicutt 1998). To reach these densities gas needs to fall into DM haloes and dissipate energy via radiative cooling to eventually form galaxies (White & Rees 1978; White & Frenk 1991). Once the gas density is higher than the threshold value, particles/cells are eligible to form stars and can be converted into collisionless stellar particles. We treat this conversion as a stochastic process that generates one generation of stellar particles per hydrodynamic resolution element (SPH particle or Voronoi cell, respectively) at an average rate equal to the star formation rate (SFR) predicted by the subresolution multiphase model. We stress again that for the purposes of our study it is crucial to have an identical implementation of this extra physics in both codes in order to allow a straightforward comparison of the two hydrodynamical schemes. Fortunately, the quasi-Lagrangian nature of AREPO typically allows an easy adoption of methods originally developed for particle-based schemes like SPH. We could hence adopt most of the ‘extra physics’ implemented in GADGET in a largely unmodified form in AREPO, helping to ensure that possible implementation differences for this physics do not affect our conclusions. We also emphasize that in our present context it is preferable to employ a relatively simple subresolution model like that of Springel & Hernquist (2003a) in order to facilitate a clear assessment of the differences between hydro solvers. In particular, for now we have not included the effects of strong feedback capable of inducing galactic winds and outflows. We are thus not attempting to solve the galaxy formation problem at this stage, but are instead concerned with identifying and understanding issues related to the treatment of the hydrodynamics. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3027 2.2 Simulation codes In the following two subsections we highlight the most relevant features and parameter settings of the codes we used, as well as any particular changes we made for this project. We note that full details are presented in substantial depth in the corresponding code papers, so in the interest of brevity we here restrict ourselves only to those aspects directly pertinent to our study. We are also aided by the fact that the AREPO and GADGET codes share the same gravity solver, something that was not the case in previous comparison studies of SPH and AMR (e.g. Regan, Haehnelt & Viel 2007). Consequently, the evolution of the collisionless DM component is identical in both codes in the limit of vanishing hydrodynamical forces. This similar handling of self-gravity is important for isolating differences caused primarily by the hydro solvers. 2.2.1 GADGET GADGET is a widely used and well-tested SPH code which employs a formulation of SPH that simultaneously conserves energy and entropy despite the use of fully adaptive smoothing lengths (Springel & Hernquist 2002). The gravity calculation is split into shortand long-range components, where short-range forces are calculated with an hierarchical oct-tree algorithm (Barnes & Hut 1986; Hernquist 1987) and the long-range forces are evaluated with a particle mesh (PM) method (e.g. Hockney & Eastwood 1981). Our GADGET runs are based on a default setting of the numerical SPH parameters, using 32 neighbours in smoothed estimates and an artificial viscosity parameter of α = 1.0, combined with Balsara’s switch (Balsara 1995) to reduce the viscosity in the presence of strong shear. 2.2.2 AREPO AREPO is a second-order accurate finite-volume method that solves the Euler equations using piecewise linear reconstruction and a calculation of hydrodynamical fluxes at each cell interface with an exact Riemann solver. The basic solution strategy of the code is that of the well-known MUSCL-Hancock scheme. What makes AREPO unusual is that it employs an unstructured mesh based on a Voronoi tessellation using a set of mesh-generating points. Furthermore, the mesh is allowed to move freely as a function of time. In our runs, we tie the motion of the mesh-generating points to the hydrodynamical flow, which gives the scheme a quasi-Lagrangian character and makes the mesh automatically adaptive. As S10 demonstrated, such an approach offers a number of advantages compared with ordinary Eulerian or Lagrangian codes, including a reduction of numerical diffusivity and advection errors and an improved handling of contact discontinuities. In the following, we discuss some details and modifications of AREPO relevant for our study. (i) Mesh regularization. It is desirable to have a Voronoi mesh geometry where the geometric centre of each Voronoi cell lies reasonably close to the mesh-generating point of that cell. This reduces the size of errors in the linear reconstruction step of the MUSCL-Hancock scheme and also limits the rate at which mesh faces turn their orientation during the mesh motion. AREPO therefore incorporates a method to regularize the mesh, as described in S10. This scheme uses a modified Lloyd algorithm to drift the mesh a bit towards its centroidal configuration at each time step. To this end the motion of the mesh-generating points contains an additional velocity component for certain cells, designed to move a mesh-generating point closer towards the geometric centre of its cell. The default set-up of 3028 M. Vogelsberger et al. uses the spatial offset between the actual location of a vertex and its cell’s geometric centre in units of the cell’s size to decide whether or not this corrective motion should be added to the cell’s vertex velocity. However, a number of test simulations revealed that this regularization scheme can introduce some unwanted side effects. Specifically, systematic differences in the average mass of gaseous cells can develop between the star-forming gas and the lower density material in the surrounding halo, with the former tending to become systematically more massive than the latter. That such a tendency exists in principle is to be expected because the Lloyd algorithm would ultimately converge to a homogeneous Voronoi mesh with cells of equal volume if it applied to every cell centre with full intensity all the time. The mesh regularization motions in the default scheme have hence the tendency to move the mesh away from the densest regions (but leaving the gas there – in AREPO, the gas motion and the mesh motion are independent), such that the mass per cell will tend to increase in the densest regions. This is somewhat undesirable because ideally we want to have the highest mass resolution in the central parts of galaxies and not in the surrounding low-density gas. There are at least two solutions to this problem (besides disabling the regularization scheme altogether). One is to simply make the settings of the standard regularization less aggressive, thereby reducing the systematic effects described above. Another possibility is to change the criterion that decides when such a mesh-correction motion should be applied. We have found that the simple criterion originally implemented in AREPO, based on the displacement of a mesh-generating point from the geometric centre of its cell, is prone to invoke mesh regularization drifts even when they are not needed. This happens especially in regions with strong density gradients, where such displacements naturally occur even if the mesh geometry is acceptable. As an alternative criterion we have therefore considered the maximum opening angle under which a cell face is seen from the mesh-generating point. This identifies problematic cell geometries more directly, which are in fact those where a point lies close to a wall, or equivalently, where this opening angle is large. We find that this opening angle criterion is more tolerant to rapid changes in spatial resolution, and is hence less prone to triggering unwanted mesh-correction motions in the presence of large density gradients. Because of this it virtually eliminates the mass segregation effect mentioned above. For definiteness, in the simulations presented here, we invoke mesh-regularization motions if the maximum √ face angle of a Voronoi cell (defined as the maximum of Ai /π/hi , where Ai is the area of a face and hi its distance to the vertex) is larger than 1.68. In this case we add a velocity component up to half the sound speed of the corresponding cell to the vertex velocity, which moves the mesh-generating point closer towards the geometric centre of the Voronoi cell. Ideally, this scheme should then guarantee reasonably regular cells where the bulk of all cells will have a maximum face angle less than 1.68. Our tests confirm that this is the case, while at the same time the mass bias between star-forming highdensity cells and the lower density cells in the surrounding halo is significantly reduced. The face-angle scheme applies the meshcorrection motions more rarely than the original approach but still frequently enough to maintain a regular mesh. In the Appendix, we discuss some measurements of the mesh statistics, which also show that this new regularization scheme still ensures that the offsets between mesh-generating points and the geometric centres of the corresponding cells remain small, as is desired to minimize errors in the linear reconstruction step. AREPO (ii) Refinement and derefinement. AREPO is quasi-Lagrangian in the sense that the vertices of the Voronoi mesh follow the flow of the fluid such that an approximately constant mass resolution is automatically achieved. The code is not purely Lagrangian, however, because mass can be exchanged between cells in a manner consistent with the equations of motion, in order to prevent the mesh from becoming highly distorted. Moreover, the code can also make use of re- and derefinement of individual cells in order to improve the local resolution beyond that offered by the dynamical motion of the mesh or to improve efficiency when high resolution is no longer required in certain regions. The method employed by AREPO is more general than that used by standard AMR methods, which typically employ subgrids to refine a certain section of the parent grid, thereby creating a hierarchy of overlapping grid patches. In AREPO, we are not required to use subgrids, but can split or merge individual cells, such that a single global mesh with a smooth transition from lowto high-resolution parts results. The criteria for when a change of the local resolution should be introduced are very flexible, just as in the AMR technique. Indeed, because the initial distribution of mesh-generating points and the manner in which they are updated in time are arbitrary, AREPO offers the capability of running in an AMR mode. In our cosmological galaxy formation simulations it is desirable to maintain a roughly constant mass resolution. This is because in runs with star formation whole gas cells can turn into collisionless stellar particles, and it is best to create them with approximately the same mass to avoid potential two-body heating effects among these star particles. A relatively homogeneous mass resolution in the gas phase is also warranted to guarantee a more proper comparison to the SPH runs, where a fixed mass resolution is present by construction. (We note, however, that this fixed mass resolution in SPH is a consequence of this method’s inability to correctly represent the flow on small scales, as we discuss in Section 5.3.) We have therefore introduced a new re-/derefinement scheme in AREPO that ensures that the mass resolution in the gas and the spawned star particles never deviates too much from the value of the SPH particle mass in the corresponding GADGET simulation. Specifically, we ‘force’ cells to have a mass in the range of 1/2 mcell /mtarget 2, where we set the target gas mass mtarget equal to the mean cell mass assuming a homogeneous gas distribution in the simulation volume with equal volume cells. This corresponds then exactly to the mass of an individual SPH particle in the GADGET calculation at the same particle/cell number. We allow only quite ‘roundish’ cells to be refined, specifically we only refine a cell if the maximum face angle is smaller than 3.38. This protects against introducing large local errors by splitting very irregular cells (which are usually cells that have just been split in the previous time step). As a result of this constraint our simulations may contain at any given time a few cells which are more massive than 2 × mtarget . We have also done runs where only cells with a sufficiently shallow temperature gradient across them were allowed to be derefined, but this did not make any significant difference to our results. We also slightly modified the star formation implementation to take this re-/derefinement scheme into account. Specifically, if a star particle is formed in a cell with mass less than 2 × mtarget , it inherits the full mass and the cell is removed. Otherwise, a star particle is created with a mass equal to mtarget , leaving the rest of the mass in the (surviving) cell. Cells that do not contain at least 1/4 × mtarget do not produce stellar particles, which prevents very low-mass collisionless particles from being formed. We note that such lowmass cells are exceedingly rare, because cells with less than 1/2 × mtarget are normally all removed during the derefinement process. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology However, since two neighbouring cells will never be dissolved in the same time step (see the discussion in S10), a cell can in rare cases briefly scatter below 1/2 × mtarget and potentially also attempt to spawn a stellar particle during this time. This modification of the star formation implementation guarantees that star particle masses are bounded by the interval 1/2 m /mtarget < 2. Our GADGET and AREPO simulations both use only one generation of stellar particles, which implies that the total number of baryonic resolution elements in the GADGET simulations is strictly conserved, while in AREPO it can fluctuate slightly around the initial value. (iii) Dual entropy formalism. As discussed in S10, finite-volume hydro solvers can suffer from spurious heating in highly supersonic, cold parts of the flow, where the kinetic energy dominates over the thermal energy. Especially at high redshifts in cosmological simulations, this can lead to an incorrect temperature evolution in low-density gas as a result of discretization errors in the advection of the kinetic energy. Owing to its quasi-Lagrangian character AREPO is somewhat less prone to this effect than fixed mesh techniques, but it can still be present at some levels. To circumvent this problem, AREPO can optionally force the entropy to be conserved instead of the energy during a time step. In this case, no spurious heating will be introduced. However, entropy is conserved only outside of shocks in adiabatic parts of the flow, so a criterion is required to decide on a cell-by-cell basis whether energy or entropy conservation should be given precedence for a cell’s evolution over the current time step. As S10 showed, this dual entropy formalism is especially useful for non-radiative simulations, because here a spurious heating cannot be compensated by radiative cooling. Since our simulations include radiative cooling due to a primordial gas mixture we expect this to be less problematic for our applications. Indeed, extensive test simulations showed that our results are not sensitive to whether or not the dual entropy formalism is used. We have therefore decided to exclusively use strict energy conservation in all our production calculations, without employing the dual entropy formalism. (iv) Gravitational softening length. GADGET uses a global gravitational softening length for collisional and collisionless particles, i.e. for gas and stellar/DM particles. AREPO also uses a global gravitational softening length for collisionless particles (which we set equal to that used in our GADGET simulations). However, as the code is a finite-volume method which uses gas cells of variable size to represent the fluid, we have used individual Plummer-equivalent gravitational softening lengths according to cell = fac × rcell for each gaseous cell, where rcell = (3Vcell /4π)1/3 is the cell’s fiducial radius assuming that the cell volume is spread over a sphere, and fac = 2.5 is an overall scaling factor. In addition, we imposed a lower floor on the gravitational softening lengths equal to that of the SPH particles in the corresponding GADGET simulation. This was done in order to safeguard against a potentially higher gravitational resolution in AREPO in the densest gas, which may have distorted our comparison. However, we note that we do not expect any significant difference due to the slightly different treatment of the gravitational softening length of the gas even if such a floor was not used. Indeed, as described in Appendix B, we have repeated lower resolution AREPO simulations with a fixed softening length for all gas cells and obtained nearly identical results. Also, we have repeated simulations with adaptive gravitational softening without a lower floor and again obtained equivalent results. We note that the presence of an effective equation of state for highly dense gas is ultimately responsible for this insensitivity. Without it, we would get fragmentation of very dense gas where the adaptive softening could make a difference (Bate & Burkert 1997). C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3029 2.3 Code comparison strategy In any comparison between codes as different as GADGET and AREPO, a number of different benchmarks can be used to gauge their relative performance, and various criteria can be defined to align runs that are deemed comparable with each other. In our study, we have chosen to compare GADGET and AREPO at matching mass resolution, using the same initial number of baryonic and DM resolution elements. This provides a well-defined comparison strategy and is advantageous for a number of reasons. (1) First, we can use the same initial conditions for the simulations with both codes. This would not be possible if we performed the runs with different initial mass resolutions. (2) Secondly, our choice makes it straightforward to relate the same objects between the different runs (and they are resolved in DM in the same way). (3) Thirdly, as we discuss below, for the same initial mass resolution, the CPU time requirements for GADGET and AREPO are similar, at least for cosmological simulations, with the moving mesh runs being only some ∼30 per cent more costly. In many cases, the limiting factor for the simulation size that can be achieved is the amount of computing time required, so our choice effectively enables a comparison for fixed computational resources. Of course, in principle one may also attempt a comparison at equivalent spatial resolution in the gas between GADGET and AREPO, or at the same accuracy in the solution obtained. We have not attempted this for the following reasons. First, while the spatial resolution that can be achieved for a given number of resolution elements is expected to be worse in SPH compared to AREPO, the degree to which this matters is likely very problem dependent. For example, for our treatment of star formation, the mass resolution is much more important than a precise treatment of fluid discontinuities. Aligning the spatial resolutions such that discontinuities are broadened over the same scale in both codes would lead to drastically different mass resolutions in the star-forming phases and hence likely to large systematic resolution effects in the smallest galaxies. Secondly, and even more important in our view, is that the convergence properties of SPH are not well understood, making a comparison at fixed accuracy a highly problematic undertaking from the start. As we discuss at length in Section 5, formal convergence in SPH ultimately requires that the number of neighbouring particles used in smoothed estimates around a particular location be increased as the total number of particles is made larger. This has typically not been done in cosmological applications, and is also in practice not readily possible due to clumping instabilities (e.g. Schuessler & Schmitt 1981). Moreover, while SPH is often described as being ‘Lagrangian,’ this is not formally correct because individual fluid elements, as represented by SPH particles, are not allowed to deform arbitrarily in response to e.g. shearing motions, as we illustrate in Section 5. SPH should, therefore, properly be referred to as ‘pseudo-Lagrangian.’ The errors that this feature of SPH entails depend on the detailed properties of the flow, in addition to the local spatial resolution, and consequently cannot be assessed in general. In its most basic formulation, SPH may simply lack a formal convergence condition, making it unclear how a comparison at fixed accuracy should be defined. Finally, in our study, we have chosen a particular implementation of ‘standard SPH’, as implemented by the GADGET code. Moreover, by choosing GADGET for our SPH simulations, we are able to use the same gravity solver and the same physics in our comparisons between SPH and AREPO, which might not be possible otherwise. This enables us to isolate differences in runs that owe primarily to 3030 M. Vogelsberger et al. Table 1. Names and basic set-up of our different simulations. All calculations were performed in a periodic box with side length of 20 h−1 Mpc. The number of baryonic resolution elements (SPH particles/Voronoi cells, and star particles) is fixed in the GADGET runs, but can vary due to reand derefinement operations in the AREPO simulations. We enforce a refinement scheme that keeps cell masses close to a target gas mass mtarget , which we set to the SPH particle mass of the GADGET run at the same particle/cell number. The comoving Plummer-equivalent gravitational softening length is constant in GADGET, but adaptive (with a floor) for gas cells in AREPO. Additional AREPO simulations with a fixed gravitational softening length yielded essentially equivalent results, as described in Appendix B. Name Code A_L20n512 G_L20n512 GADGET A_L20n256 G_L20n256 GADGET A_L20n128 G_L20n128 GADGET AREPO AREPO AREPO Box size (h−3 Mpc3 ) Hydro elements DM particles mtarget/SPH (h−1 M ) mDM (h−1 M ) (h−1 kpc) 203 203 5123 5123 5123 5123 7.444 × 105 7.444 × 105 3.722 × 106 3.722 × 106 1 1 203 203 2563 2563 2563 2563 5.955 × 106 5.955 × 106 2.977 × 107 2.977 × 107 2 2 203 203 1283 1283 1283 1283 4.764 × 107 4.764 × 107 2.382 × 108 2.382 × 108 4 4 the differences in the hydro solvers between GADGET and AREPO. As we discuss in Section 5, various modifications have been proposed to the basic structure of SPH in order to improve its reliability under some circumstances. It is, of course, possible that we would have obtained somewhat different results had we employed such formulations of SPH. However, we believe that many limitations of SPH are generic and are not specific to GADGET. Indeed, as we discuss in Section 5, it is difficult to see how SPH could be modified to entirely eliminate, e.g. the sources of error that owe to its pseudoLagrangian nature without radical modifications to the underlying algorithm. We also stress again that the primary goal of our comparison is not to arrive at the best fit to observational data, or to achieve the highest possible resolution for single galaxy models. While we acknowledge that important strides have been made recently in studying disc formation numerically using zoom-in procedures in cosmological SPH simulations (e.g. Agertz et al. 2011; Guedes et al. 2011), we do not think that these successes provide much evidence for the numerical reliability of SPH, although this misconception appears widespread. Our ultimate aim is not merely to model single objects, but entire populations of galaxies so that we can statistically constrain uncertain processes associated with star formation and feedback by using, e.g. observations of the distribution of galaxies with respect to their Hubble type. This requires the identification of accurate and computationally efficient techniques that are best suitable for the task. In particular, our strategy is designed to detect systematic inaccuracies in current simulation techniques that may easily be concealed by the adjustment of heuristic feedback parameters to match a certain observation, as it is commonly done, while at the same time these effects may have serious consequences in other places. tively, is used for the DM, and for the gas particles in the GADGET runs. For AREPO, we start initially with the same number of cells as there are particles in the corresponding SPH run, but we use an adaptive gravitational softening with a floor as described earlier. Note that due to re- and derefinement the number of baryonic resolution elements (cells plus star particles) is not strictly conserved during the course of a simulation but can fluctuate slightly around the initial number. Initial conditions were generated at z = 99 based on the power spectrum fit of Eisenstein & Hu (1999), with gas particles/cells added to the initial conditions by splitting each original particle into a DM and gas particle/cell pair, displacing them with respect to each other such that two interleaved grids are formed, keeping the centre-of-mass of each pair fixed. All our simulations have been evolved until redshift z = 0 even though the fundamental mode of our small box becomes mildly nonlinear at low redshift. However, our primary goal here is not to obtain quantitatively accurate large-scale statistics, but rather to compare the properties of galaxies formed by two different hydrodynamical schemes. As we start all simulations with the same phases, this relative comparison is unaffected by box-size effects down to z = 0. To easily refer to the different runs in our simulation suite we use the following naming convention. A_L20nX denotes AREPO runs (indicated by the leading ‘A’), where X = 128, 256 or 512 encodes the numerical resolution and the tag ‘L20’ stands for the box size of L = 20 h−1 Mpc. Likewise, G_L20nX refers to our GADGET runs (indicated by the ‘G’). In Table 1, we provide an overview of our simulation suite and list its most important parameters. 3 R E S U LT S F O R G L O BA L BA RYO N S TAT I S T I C S 3.1 Density and temperature maps 2.4 Simulation suite For our simulation set we adopt the cosmological parameters m0 = 0.27, 0 = 0.73, b0 = 0.045, σ 8 = 0.8, ns = 0.95 and H 0 = 100 h km s−1 Mpc−1 = 70 km s−1 Mpc−1 (h = 0.7). These parameters are consistent with the most recent 7-year Wilkinson Microwave Anisotropy Probe (WMAP7) measurements (Komatsu et al. 2011) as well as with a host of other cosmological constraints. We create a realization of this cosmology in a periodic box with a side length of 20 h−1 Mpc, at three different mass resolutions corresponding to 2 × 1283 , 2 × 2563 and 2 × 5123 particles/cells. A comoving gravitational softening length of 4, 2 or 1 h−1 kpc, respec- To obtain a first impression of the simulations, it is instructive to examine maps of the density and temperature fields. In Fig. 1, we show projections of the gas density and temperature distributions for our highest resolution simulations A_L20n512 and G_L20n512 at z = 2. The left-hand panel gives the AREPO results and the righthand panel those obtained with GADGET. In both cases, the image on top depicts a large part of the full box, while the zoom images below show two successive zooms on to a disc galaxy (as indicated by the white squares). In these enlarged images, the left half depicts the temperature field while the right half gives the density field. All slices have a thickness of 2 h−1 Mpc and their extent in the image C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3031 Figure 1. Projected gas density and temperature maps for the highest resolution AREPO (left) and GADGET (right) simulations (2 × 5123 particles/cells) at redshift z = 2. All slices have a thickness of 2 h−1 Mpc (comoving). The upper two panels show a large part of the full simulation volume. This region extends 20 h−1 Mpc in the x-direction and contains one of the largest filaments in the box. The other eight panels form a zoom-in on to one object. Each zoomed region is indicated by a white square in the previous step, and the corresponding comoving length scales are indicated by the small white bars in the lower left of each image. On the left-hand side of the zoom-images we show the temperature, and on the right-hand side we show the density fields. The colour maps at the different zoom levels change, but they are the same for both codes. At the highest zoom-in factor in the lower panels, a disc galaxy is visible. The disc is very extended in AREPO, and also features a bar. The SPH calculation produces a significantly smaller gas disc in the same halo, and the stellar bar is smaller compared to the moving mesh simulation. plane is specified by the scale bars included in the images. Note that we did not centre the two image sequences individually on to the galaxies, so the zoom sequence also illustrates the typical magnitude of coordinate offsets that can develop between the different simulations. Inspection of the maps in Fig. 1 demonstrates that both codes produce essentially identical results on large scales. This is reasonable since we do not expect large-scale changes induced by a different hydrodynamical scheme. However, already at the intermediate zoom level one can identify some significant differences. The distribution of hot gas is clearly quite different, with GADGET showing a more extended hot atmosphere compared to AREPO. On still smaller scales, the differences between the two hydrodynamical codes become even more pronounced: whereas AREPO produces an extended disc, GADGET forms a significantly smaller and clumpier disc. The difference can be best appreciated in the temperature map, where the cold gas disc stands out clearly as a large blue region in the moving mesh calculation. AREPO forms a spiral disc with a central bar, which is visible in the density map. For a more detailed analysis of the structural properties of this galaxy we refer the reader to Paper II of this series. We note that although some spiral features C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society are visible in the disc, these are partly seeded by Poisson noise in the potential due to the limited number of DM particles in the halo [see D’Onghia, Vogelsberger & Hernquist (2012) for details of this process], and have hence only limited physical significance. In order to demonstrate that differences like those seen in Fig. 1 do not appear only in unusual cases, we show in Fig. 2 gas density projections of the central galaxies of the 24 most massive haloes at z = 2 of A_L20n512 (top panels) and G_L20n512 (bottom panels), along random projection directions. We did not impose any constraint on the galaxy selection other than halo mass; as a result, we can also have galaxies in our sample that are disturbed or undergo a merger, such as the system labelled ‘galaxy-id 5’. The object ‘galaxy-id 8’ of Fig. 2 corresponds to the galaxy shown in Fig. 1, but happens to be oriented edge-on here. The random orientations of the galaxy set give an impression of how a population of galaxies would look like in these simulations. Clearly, the gas distributions of most of the galaxies in Fig. 2 are more extended and more reminiscent of spiral galaxies in AREPO than in GADGET. Inspection of animated sequences from the AREPO simulations reveals a population of interacting and merging systems. Many of them show thin bridges and tails, as expected if the galaxies are 3032 M. Vogelsberger et al. Figure 2. Gas density projections of the central galaxies in the 24 most massive haloes in A_L20n512 at z = 2, together with the corresponding galaxies in the G_L20n512 simulation (top four rows: AREPO; bottom four rows: GADGET). The small white bars indicate 5 kpc (physical). We project along random directions to show various viewing angles, similar to a real galaxy field of view. The galaxy with number 8 is the object shown in the zoomed-in region of Fig. 1, but here now shown more edge-on. Essentially in all of the 24 cases the AREPO gas distributions are more extended than those of the equivalent GADGET galaxies. This demonstrates that the galaxy shown in Fig. 1 is not an exception, but rather a typical case in our simulations. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology rotationally supported discs (e.g. Toomre & Toomre 1972; D’Onghia et al. 2010b).1 The differences in the gas properties are also reflected in the corresponding stellar distributions of these galaxies, which we show in Fig. 3. Although the stellar distributions are more similar between the two numerical schemes, in most cases one still notices that AREPO’s discs are slightly larger and appear typically more discy, an impression that is also supported by the quantitative analysis of galaxy properties that we present in Paper II. Furthermore, the stellar populations of the AREPO galaxies are bluer, i.e. younger, than those found in the GADGET run. In Figs 4 and 5 we show face-on projections of 32 other galaxies at z = 0 with Milky Way-like DM halo masses in the mass range from ∼1 × 1012 to ∼6 × 1012 h−1 M . The gas disc structure shown in Fig. 4 follows the trend already seen at z = 2, and demonstrates that AREPO produces significantly larger discs at all times and at all halo masses compared to the GADGET simulations. Fig. 5 demonstrates that the stellar discs are also slightly larger, but not as significant as the gas discs. But similar to the z = 2 result, the AREPO stellar disc populations are typically younger than those found in GADGET. A self-evident conclusion from these visual comparisons of Figs 2–5 is that the morphology of forming galaxies in cosmological hydrodynamics simulations can be strongly affected by the underlying hydrodynamics solver. SPH has significantly more problems producing disc-like galaxies than the moving mesh approach for a similar computational cost. Note however that we do not expect our simulations to produce a completely realistic z = 2 galaxy population, because we here refrained from including strong feedback capable of producing galactic winds and outflows. However, since both simulations followed exactly the same physics, we expect this qualitative difference to be present in simulations with stronger feedback as well. 3.2 Mass functions As we emphasized earlier, AREPO and GADGET use the same gravity solver based on a high-resolution Tree-PM scheme. Because the gravitational field is largely dominated by the collisionless DM component, we expect that the DM distribution in both simulations should hence be similar, at least on scales where baryonic effects do not change the structure of DM haloes. We explicitly verify this expectation in the left-hand column of Fig. 6, where we compare the DM halo mass function of friends-of-friends (FoF) groups with a linking length of 0.2 of the mean particle separation down to a particle number limit of 32 particles at z = 0 and 2. The codes show excellent agreement in the DM halo mass functions at both redshifts. Also, the convergence for the different resolutions is in both cases very good. We note that previous comparisons of hydrodynamical mesh codes and high-resolution tree codes (O’Shea et al. 2005; Heitmann et al. 2008) have shown a deficit of low-mass haloes in the mesh-based hydrodynamical codes, an effect that has its origin in their AMR-based gravity solver, which typically does not refine small DM haloes sufficiently early. As evidenced here, AREPO does not have this problem. The other panels of Fig. 6 count haloes in terms of their baryonic content, separately for gas and stars. In order to relate baryons to DM haloes, we determine for each stellar and gaseous cell/particle the 1 Volume-rendering movies of AREPO galaxies (at redshift z = 1 and 2) and high-resolution images are available for download at the website http://www.cfa.harvard.edu/itc/research/movingmeshcosmology. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3033 closest DM particle, and then associate the baryonic mass element with the FoF group this DM particle resides in (see Dolag et al. 2009, for more details). The middle column of Fig. 6 shows the cumulative halo gas mass functions at redshifts z = 2 and 0. While both codes converge equally well for the gas mass functions as a function of resolution, they do not agree on the result. At z = 2, AREPO has more objects of a given gas mass than GADGET over nearly the full range of halo masses. This behaviour changes towards lower redshifts, where fewer very gas-rich objects are found in AREPO. We will show below that this has to do with more efficient cooling and a higher SFR in the mesh code in massive haloes at late times. The right-hand column in Fig. 6 gives the mass functions for the stellar component of haloes, and here the situation is slightly different. At z = 2, we find fewer objects with low stellar mass in AREPO compared to GADGET, but the situation reverses at the highmass end where we find that AREPO has more haloes at a given stellar mass. This behaviour changes somewhat towards z = 0, where the low-mass end agrees now very well between the two codes. However, there are still more stellar systems of high mass in AREPO, and this difference becomes more pronounced and extends over a larger range of masses. Again, this can be attributed to the more efficient star formation at late times in massive systems in AREPO that we analyse in detail below. Since the baryonic mass contributes only a small amount to the total mass of the entire box, these differences in the baryonic mass functions are neither imprinted in the DM halo mass functions nor visible in the large-scale structure shown in Fig. 1. 3.3 Gas phases Because we use two completely different hydro solvers, we may expect some deviations in the breakdown of various gas phases. A first simple way to obtain an overview of the global state of the gas in the simulation volume is the construction of density–temperature phase-space diagrams. We show in Fig. 7 mass-weighted ρ–T phase-space diagrams of the simulations at the highest resolution for z = 2 (top row) and z = 0 (bottom row). The left-hand and righthand columns show results for AREPO and GADGET, respectively. We can readily identify a number of well-known features in the gas phase-space (Davé et al. 2001), which show up in both runs. The narrow ridge of gas with an upward slope and ρ/ρ bar < 10 and T < 105 K consists of low-density, highly photoionized gas in the intergalactic medium (IGM). The tight relation between temperature and density in this regime is maintained by the competition between adiabatic expansion cooling and photoionization heating. The plume of gas with ρ/ρbar ∼ 10−104 and T ∼ 105 –107 K is comprised on the other hand of shock-heated gas in virialized haloes and, at the lower density end, in and around filaments. The narrow, downward sloping locus of gas at ρ/ρ bar > 103 and T ∼ = 104 K represents radiatively cooled, dense gas in galaxies. The cooling times at these densities are short, so that gas remains close to its equilibrium temperature where photoionization heating balances radiative cooling. This temperature is a slowly decreasing function of density and lies close to 104 K. Finally, once the gas becomes very dense and reaches the star formation threshold, we describe the gas by an effective equation of state representing the mean thermal energy density of a two-phase medium of hot and cold gas. This effective equation of state results in the upward sloping high-density line of gas, which represents the ISM in galaxies. The pressurization by supernova feedback in our subresolution model prevents this gas from fragmenting under self-gravity. 3034 M. Vogelsberger et al. Figure 3. Stellar density projections of the central galaxies in the 24 most massive haloes in A_L20n512 at z = 2, together with the corresponding galaxies in the G_L20n512 simulation (top four rows: AREPO; bottom four rows: GADGET). The small white bars indicate 2.5 kpc (physical). Density is encoded as intensity, whereas the colour scale (blue–red) indicates the stellar age (young–old). Galaxies are shown from the same viewing angle as in Fig. 2, but with reduced side length. The differences in stellar radii are not as large as those found for the gas distribution. In most cases the stellar distribution is more discy in AREPO compared to the galaxies in the SPH runs, and the stellar population in the AREPO simulations is younger making the galaxies on average bluer. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3035 Figure 4. Gas density projections of 32 galaxies at z = 0 randomly selected within the halo mass range from ∼1 × 1011 to ∼6 × 1011 h−1 M . The top four rows show the AREPO galaxies and the bottom four rows show the matched GADGET galaxies. The galaxies are shown face-on for better visual inspection of the disc structure. The small white bars indicate 10 kpc (physical). The difference in the disc morphology is very striking and follows the same trend found at z = 2 in Fig. 2: AREPO gas discs are significantly more extended than those in the SPH simulations. As Fig. 7 demonstrates, AREPO and GADGET lead to a qualitatively very similar ρ−T phase-space diagram. This demonstrates that the properties of the gas distributions are overall quite comparable in both codes. However, although there is general broad agreement between the simulations, there are also some striking differences. First of all, the distribution of hot gas extends to higher densities in the AREPO runs compared to the GADGET simulations, i.e. there is more hot gas at higher densities in the AREPO simulations. We note however that in terms of total mass this is not very significant. More important, GADGET appears to have more hot gas in general, which C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society can be inferred from the slightly more extended yellow region in the phase diagrams. This is consistent with the temperature structure in the surroundings of the galaxy shown in Fig. 1. The AREPO runs also exhibit a more pronounced cooling feature around T ∼ 104.7 K, which corresponds to a local minimum of the cooling curve between the two line peaks of hydrogen and helium. Although this feature is readily visible in the phase-space diagrams of the AREPO runs, the actual gas mass populating that feature is very small. The faint stripe-like features seen in the top left-hand and bottom left-hand panels of Fig. 7 below the effective equation of state arise from 3036 M. Vogelsberger et al. Figure 5. Stellar density projections of the 32 galaxies shown in Fig. 4. The small white bars indicate 10 kpc (physical). As in Fig. 3 the density is encoded as intensity, whereas the colour scale (blue–red) encodes the stellar age (young–old). The viewing angle is the same as in Fig. 4. Stellar radii differ not as much as the gas disc scale length found in Fig. 4. As in Fig. 3 the GADGET galaxies are slightly redder than the galaxies found in the AREPO simulations. numerical inaccuracies in the temperature evolution of some cells around star formation sites. Here the temperature can sometimes scatter to very low values, which is one incarnation of the accuracy problems associated with supersonic cold fluid motions. In our multiphase model, the temperature of these cells is then relaxed back to the effective equation-of-state temperature T eff on a timescale τ h given by equation 12 of Springel & Hernquist (2003a). The simulation time step t of these cells is small against τ h , such that the temperature of the cell at the end of a relaxation step is T T eff t/τ h . Because T eff /τ h ∝ ρ 0.5 (see Springel & Hernquist 2003a) and due to the power-of-two hierarchy of possible values for t, a set of parallel stripes of cells spaced by a factor of 2 results. The mass in these cells is however negligibly small, so that this effect has no influence on the dynamics. To quantify the gas mass content in the different regions of the ρ– T phase-space diagrams in Fig. 7, we introduce the following cuts in the ρ–T plane: (1) diffuse cool gas with ρ/ρ bar < 103 and T < 105 K, (2) diffuse warm gas with ρ/ρ bar < 103 and 105 K < T < 107 K, (3) diffuse hot gas with ρ/ρ bar < 103 and T > 107 K, and finally (4) dense gas with ρ/ρ bar > 103 . In Fig. 8, we first show the different gas mass fractions of each of these four phases. All runs show the behaviour we anticipated earlier, i.e. there is the general trend C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3037 Figure 6. Mass functions of FoF groups at z = 2 (upper panels) and z = 0 (lower panels). Left-hand column: dark matter, middle column: gas and right-hand column: stars. Vertical lines indicate the 32 dark matter particle mass limit. The dark matter mass functions agree and converge very well, demonstrating that the gravitational evolution is indeed treated on an equal footing, whereas gas and stellar mass functions deviate due to the different hydro solvers. Towards lower redshifts there are typically fewer massive gas systems and more massive stellar systems in AREPO compared to GADGET. that the cold gas fraction decreases and the warm/hot gas fraction increases with time. The hot gas fraction (lower left-hand panel) increases due to shock heating in haloes. Interestingly, the details vary quite strongly between the different numerical schemes. The GADGET runs typically show more hot and warm gas, and accordingly produce less dense gas. Especially the mass fractions in the hot phase are very different between the two schemes: for most of the time the mass difference here is larger than one order of magnitude. We will show below that these two findings are also consistent with the results for the star formation histories (SFHs) of the different runs. We note that the same trends are also found for the lower resolution simulations, but the quantitative numbers differ slightly. Although not fully converged, there is more very dense gas (lower right-hand panel) in AREPO than in GADGET at all resolutions, and also the amount of cold gas (upper left-hand panel) is larger in AREPO. These results confirm the visual impression obtained from Fig. 1, where we also saw that the hot atmosphere is smaller in AREPO, but the cold gas disc is larger. Next, we consider in Fig. 9 the corresponding volume fractions of these gas phases. The volume fractions of the GADGET runs are based on the specific volume of individual SPH particles: V SPH = mSPH /ρ SPH . However, SPH is not volume conserving in the sense that the sum of the specific volumes of all SPH particles will not sum up to the total simulation volume. In finite-volume methods like the one used by AREPO, this does not occur; all the cell volumes add up to the correct simulation volume. For the comparisons of C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society the volume fractions we therefore do not calculate volume fractions with respect to the total simulation box volume, but with respect to the sum of all individual particle/cell volumes. This does not influence the volume fractions of AREPO, but changes the SPH volume fractions of GADGET, which would otherwise not add up to unity. The general trends we find for the volume fractions in Fig. 9 are similar to those for the mass fractions in Fig. 8. However, we can now clearly see that the hot atmospheres are not only strongly reduced in mass in AREPO, but also that their volume fraction is significantly smaller compared to GADGET. This is in agreement with the qualitative results shown in Figs 1 and 2. The same is true for the volume fraction of the cold gas, which is however not as well converged. To focus more on the density structure, we marginalize Fig. 7 over temperature and express the densities as physical hydrogen number densities. We hence derive the hydrogen number density probability distribution functions for the full simulation volume, which are shown in Fig. 10. The left-hand panel accounts for all gas particles/cells, whereas the right-hand panel shows only those with active star formation. In our star formation model, the density threshold for the onset of star formation is set to nH = 0.13 cm−3 , and therefore the right-hand panel occupies only this high-density region. Interestingly, GADGET shows at all resolutions a significantly reduced gas fraction slightly below the star formation density threshold. We checked that the reduction of the gas mass in this region is caused by the more extended discs in AREPO, i.e. the additional 3038 M. Vogelsberger et al. Figure 7. Gas density–temperature phase diagrams of AREPO and GADGET at the highest resolution (top: z = 2, bottom: z = 0, left: AREPO and right: GADGET). Each bin of the maps contains the total gas mass at the corresponding density and temperature. The different characteristic phases are populated similarly in the different runs, showing that the overall global gas properties agree. But there are also some visible differences between the SPH and moving mesh calculations. AREPO typically produces more high-temperature gas at large densities, but the overall amount of hot gas is reduced, which can be seen by the more extended yellow region in GADGET at high temperatures. The dashed lines and numbers indicate the gas cuts used for quantifying the mass and volume fractions in Figs 8 and 9, respectively. gas found in AREPO at these densities stems from disc material. Presumably, the ‘gap’ phenomenon found in SPH across strong contact discontinuities (Agertz et al. 2007), such as the one we have here, also contributes to the different breakdown in that density range. The other parts of the density distributions look rather similar for the two simulation methods, however. We verified that the gas slightly below the threshold is predominantly responsible for the more extended discs in AREPO. This gas effectively fills the gap which is present in SPH. The galaxies in GADGET typically lead to a peak in the density PDF above the star formation threshold instead of a more plateau-like feature due to the extended gas discs in AREPO. Therefore, the difference in the density PDF is a reflection of sig- nificantly different galaxy radii for the different hydro schemes (see also Paper II). 3.4 Cosmic star formation history An important prediction of our simulations is the global SFH, which encodes key information about the overall efficiency of the galaxy formation process. The SFHs of our different simulation runs are presented in Fig. 11. The top left-hand panel shows the SFH as a function of lookback time, whereas the lower left-hand panel plots it on a logarithmic scale as a function of redshift. The corresponding integrated SFHs are shown in the right-hand panels. The most C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3039 Figure 8. Fraction of gas mass in various phases, i.e. the different panels correspond to the mass fraction in the parts of the phase diagrams in Fig. 7. We do not include stellar mass in this plot. The four different gas selections add up to the total gas mass in the simulation volume. Neither code shows very good convergence in all of these fractions. This is true both for the SPH and the moving mesh calculations. Nevertheless, the different mass fractions do not vary dramatically between the various resolutions, and there are some characteristic and systematic differences between GADGET and AREPO. GADGET produces more low-density hot gas and AREPO shows a larger fraction of high-density gas (ρ/ρ bar > 105 K). This can already be seen from Fig. 7 by comparing the plume of hot particles/cells. Also, the fraction of mass in the warm to hot intergalactic medium (top right) is lower in AREPO compared to GADGET. We note that the redshift range in the lower left-hand panel is reduced compared to the other panels. The inset in the upper right-hand panel shows the fraction of all four main panels for the highest resolution runs together. The numbers indicate the cut they belong to. These cuts and numbers are also shown in Fig. 7. obvious trend with increasing resolution is that more high-z star formation in small haloes is resolved (Springel & Hernquist 2003b). This is simply due to the better mass and spatial resolution for smaller haloes, which are in part not even seen in the coarser simulations. This trend with resolution also shifts the peak in the SFR density to higher redshift with increasing resolution. Note however that there is little cosmic time at these high redshifts, so that the mass of stars formed there is small compared to the present stellar density. Once the simulations resolve all haloes with virial temperature of 104 K and above (necessitating a DM particle resolution of about 106 M ), we expect however that the trend with resolution C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society will stop and all ordinary star formation will be resolved (apart from additional Pop III star formation, which is not included in our models). After the high-redshift peak of star formation, the SFRs decline and converge particularly well at lower redshifts for the SPH-based GADGET simulations. In this regime, most of the star formation is contributed by higher mass haloes (Springel & Hernquist 2003b), for which our subresolution model produces converged results already at moderate resolution. This excellent convergence does not quite carry over to the moving mesh calculations with AREPO, where all resolutions slightly vary in their late-time SFRs (for further 3040 M. Vogelsberger et al. Figure 9. Fraction of gas volume in the same gas phases as in Fig. 8. For AREPO we naturally use the volumes of Voronoi cells to calculate these fractions, and for the GADGET (SPH) simulations we assigned the specific volume mSPH /ρ SPH to each SPH particle. We note that gas volume fractions are not very well defined in SPH, because the scheme is not volume conserving, i.e. the sum over mSPH /ρ SPH does not yield in the correct total gas volume. We therefore take as the total volume always the sum of all specific volumes. The differences between the two hydro schemes are qualitatively similar to those found in Fig. 8, especially the hot gas phase shows a very different volume fraction. We note that the redshift range in the lower left-hand panel is reduced compared to the other panels. The inset in the upper right-hand panel shows the fraction of all four main panels for the highest resolution runs together. The numbers indicate the cut they belong to. These cuts and numbers are also shown in Fig. 7. explanation see Section 4.3 and Paper III). However, the mesh-based results are still very close to one another, and more importantly, they lie significantly higher than the SPH simulations, which is clearly visible from the inset in the upper left-hand panel of Fig. 11 showing the ratio of the individual SFHs to the mean. We emphasize once more that this systematic difference is a result only of the different hydrodynamical schemes, since both codes use the same star formation model and calculate identical SFRs for a given gas density. There is a minor technical difference in how stellar particles are created, but this does not affect the outcomes, as we have explicitly checked in numerous test calculations of isolated haloes and galaxies (see Paper III). The latter do show very good agreement in the SFRs between GADGET and AREPO, so the difference we find in Fig. 11 must have a cosmological origin. To understand better whether this increased SFR affects all halo masses or whether it is dominated by a certain halo population, we plot in Fig. 12 the SFRs as a function of FoF group mass for our highest resolution simulations. Solid lines in Fig. 12 represent the median values and shaded regions show the 1−99 per cent range of the distribution. The insets show the specific SFRs. In both cases, and for all redshifts shown, the scatter around the median relation is rather small. At high redshift (z = 4), the differences between the SPH and moving mesh calculations mainly occur at the intermediate- and low-mass end, where GADGET produces more C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3041 Figure 10. Density probability distribution function of the gas in the simulation volume in units of hydrogen number density, assuming a hydrogen mass fraction of 0.76. The left-hand panel shows all gas particles/cells, whereas the right-hand panel shows only those with active star formation. In our star formation model the density threshold for the onset of star formation is set to nH = 0.13 cm−3 , and therefore the right-hand panel occupies only this high-density region. Interestingly, GADGET shows at all resolutions a significantly reduced gas population slightly below the star formation density threshold. This gas is about to overcome the density threshold for star formation if it contracts a bit more via cooling. The rest of the hydrogen density distributions look rather similar for the two simulation schemes. The reason for the difference slightly below the threshold lies in the more extended disc in AREPO. This gas fills the ‘gap’ present in the SPH calculations. stars than AREPO. This is in agreement with the general trend found for the global SFH, where GADGET shows a slightly higher SFR at high redshift. We note that the higher specific SFRs in AREPO at late times are due to a significantly more efficient cooling. We point out that one reason why GADGET shows at high-z a larger SFR at a given resolution than AREPO is related to the fact that already at this early time the discs in AREPO are slightly more extended and hence have a lower gas surface density (as demonstrated in Paper II). Therefore, the SFR is lower compared to the denser, more blob-like galaxies formed in the SPH calculations. We note that this discrepancy between GADGET and AREPO decreases once the resolution in SPH becomes high enough, but it still induces a noticeable delay in AREPO’s SFR peak compared to GADGET, although the peak amplitudes agree quite well. Towards lower redshifts, a gradual trend of a different nature appears: while AREPO and GADGET start to agree better at low masses, the schemes deviate more and more strongly towards the high-mass end. At z = 0, there is a nearly perfect overlap of the SFRs below ∼1011 h−1 M , but FoF groups above this mass form significantly more stars in the AREPO simulations compared to GADGET runs. Interestingly, this difference in large haloes first starts to show up at around z = 3 (upper right-hand panel), which roughly coincides with the time when the global SFRs start to deviate, as can be seen from the lower left-hand panel of Fig. 11. This implies that the massive end of the halo population is mainly responsible for the higher SFR in the moving mesh calculations at late times. These massive systems form at late times and systematically create more stars in AREPO than the corresponding GADGET calculations, driving the global SFH to a different behaviour in the two codes. The only way such a difference in the SFH can occur is a more efficient cooling behaviour in AREPO in haloes above ∼1011 h−1 M , allowing more gas to accumulate at the bottom of halo potentials and above the density threshold for star formation. This interpreta C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society tion also agrees with our findings for the mass fractions and volume fractions in the different gas phases, where GADGET shows at all resolutions significantly more hot gas than AREPO. Furthermore it agrees with the gas density PDFs in Fig. 10, which show that AREPO has a larger fraction of dense gas slightly below the star formation density threshold. As we will discuss in the next section, the culprit of the reduced cooling in GADGET lies in different hydrodynamical dissipation rates in the outer parts of haloes compared with AREPO. In Paper III of this series, we furthermore show that AREPO’s more accurate treatment of hydrodynamical fluid instabilities strongly affects the stripping of gas out of gas clumps that fall into haloes. This in turn leads to significant differences in how the thermodynamic structure of haloes is affected by the hierarchical merging process, contributing to the cooling differences. We note that the different SFHs also result in a different distribution of stellar ages in individual galaxies. This can already be seen in Figs 3 and 5, where not only the morphology of the stellar distribution is different, but also the average age as can be inferred from the colour scale. The GADGET galaxies look clearly redder overall than the AREPO galaxies, which are on average bluer. To quantify this in more detail, we show in Fig. 13 distribution functions of the formation redshifts of the stellar populations at z = 0, 2 and 3, for our two highest resolution simulations. This figure demonstrates that at high redshift (z = 3) the distributions agree reasonably well in shape. But at z = 2, a clear trend is already visible, because the AREPO distribution becomes biased towards slightly younger stars. This has to do with the slight shift of the peak of the SFH and the different late-time behaviour of the SFH found in Fig. 11. The difference in the distributions becomes even more pronounced towards lower redshift and is clearly evident at z = 0. At that time the AREPO distribution is quite distinct from GADGET and shows a significantly larger fraction of young stars. The reason for this lies in the larger SFR at late times, as discussed above. 3042 M. Vogelsberger et al. Figure 11. Star formation rates per unit volume as a function of lookback time (top left) and redshift (bottom left). The panels on the right-hand side show the corresponding integrated star formation rates as a function of lookback time (top right) and redshift (bottom right). The inset in the upper left-hand panel shows the relative deviations from the inferred mean SFR. Although the AREPO star formation rates do not converge as well as the SPH results obtained with GADGET, AREPO shows at all resolution levels a significantly higher star formation rate at late times. The high-redshift star formation peak agrees in amplitude quite well between the different codes if we focus only on the highest resolution (L20n512). 4 GAS COOLING 4.1 Cooling emission and temperature evolution The differences between AREPO and GADGET discussed so far point towards a different cooling behaviour between the two codes. To demonstrate this more clearly, Fig. 14 shows the total net cooling rate [ mi (Ci − H i )], where Ci and H i are the cooling and heating rates per unit mass as a function of redshift, accounting for all gas in our simulations. In the case of net heating we set this rate to zero. The figure demonstrates that at late times, the cooling rates of all AREPO runs are higher than those of corresponding GADGET runs. We note that this is true for all three resolutions, yielding a clear separation between the set of blue curves (representing the SPH simulations) and the red curves (representing the moving mesh calculations). Interestingly, the difference partially reverses at high redshift, where the cooling rates of AREPO at the intermediate- and the low resolution are lower than those of GADGET. This is a consequence of the more compact galaxies in GADGET at these high redshifts, which result in larger densities and therefore more cooling radiation. This high-z cooling rate difference decreases and nearly disappears once we reach the highest mass resolution. In that case the cooling rates of GADGET and AREPO agree reasonably well at high redshifts, as shown in Fig. 14. In contrast, the quite large and systematic difference at low redshifts persists independent of resolution. Further insight into the origin of the cooling difference is provided by Fig. 15, which shows the mean mass-weighted net cooling C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3043 Figure 12. Star formation rates as a function of FoF group mass for the highest resolution simulations. Solid lines show the median values and the shaded regions represent the 1−99 per cent range of the distribution. For both cases and for all redshifts shown here the scatter is rather small. Towards lower redshifts a gradual trend shows up: both schemes start to deviate strongly towards the high-mass end. At z = 0 there is nearly perfect overlap of the star formation rates below ∼1011 h−1 M , but FoF groups above this mass form significantly more stars in the AREPO simulation compared to the GADGET run. Interestingly, this difference at the high-mass end starts to show up first at around z = 3. This coincides with the time that the global star formation rates deviate, as can be seen from the lower left-hand panel of Fig. 11. The massive end of the halo population is hence mainly responsible for the higher star formation rate in the moving mesh calculations at late times. The insets show specific star formations rates, where we divided the median curve of the main panel by the stellar mass M of the group. rates [( mi (Ci − H i ))/( mi )] at redshifts z = 2 (left-hand panel) and z = 0 (right-hand panel) as a function of hydrogen number density. If the temperature structure of the gas at fixed density would agree between all simulations, then the mean mass-weighted cooling rates should also agree for a given hydrogen number density. But as Fig. 15 shows, this is not the case and some systematic differences between GADGET and AREPO are present. Interestingly, the curves in Fig. 15 differ however only over a limited density range corresponding to the diffuse gas in haloes, beginning at the star formation density threshold (marked by the vertical green line) and extending to lower densities. In that density range, AREPO shows a significantly larger mean cooling rate at all resolutions, which must C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society be due to a slightly lower gas temperature in the mean at these densities. We note that the magnitude of the difference is however redshift dependent and becomes smaller as we go to higher redshifts, as seen by comparing the z = 0 and z = 2 panels in Fig. 15. At other hydrogen densities, including the range not shown in Fig. 15, the SPH and moving mesh cooling rates agree well for all our runs. Above the threshold for star formation this is however not surprising, because here the effective equation of state produces a tight correlation between density and temperature (as seen in Fig. 7). This leads to a nearly one-to-one mapping of density to temperature in this regime, and explains the two-peak structure of the curves in Fig. 15 at high densities, which simply reflects the 3044 M. Vogelsberger et al. Figure 13. Distribution of star formation times (redshifts) for the highest resolution AREPO and GADGET simulations. The lines show the distribution at z = 0, 2 and 3 as indicated. At higher redshift (z = 3) the two curves agree reasonably well in shape. But at z = 2 there is already a trend visible, because the AREPO distribution is biased towards slightly younger stars. This effect becomes even more pronounced towards lower redshift and is clearly visible in the distribution at z = 0. At that time the AREPO result is significantly different from GADGET and shows a substantially larger fraction of young stars. The reason for this is the larger overall star formation rate at late times, as shown in Fig. 11. The different stellar age distributions are also responsible for the bluer appearance of the stellar populations of the AREPO galaxies shown in Figs 3 and 5. primordial cooling curve with its characteristic hydrogen and helium peaks. Further evidence for a different temperature structure in the two simulation runs is provided by Fig. 16, which plots the mean massweighted temperature in the whole simulation volume as a function of time. In the inset we divide each curve by the mean of all the curves to emphasize deviations. We find that the mean temperature of GADGET simulations is ∼15 per cent higher than that of the AREPO runs. The higher temperature can explain the reduction in cooling emission of the SPH simulations relative to our moving mesh calculations, and it is consistent with our above findings for the SFR evolution and the cooling emission. It thus appears that the origin of the low-redshift discrepancy must lie in a different efficiency of non-adiabatic heating processes in the gas, as this is required to explain differences in the temperature distribution at a given density. 4.2 Dissipative heating in haloes It is widely appreciated that the intracluster medium of galaxy clusters is partially supported by subsonic turbulence (Schuecker et al. 2004), which is created by curved accretion shocks around the haloes, and more importantly, by the hierarchical infall of structures. Indeed, a number of numerical studies have analysed turbulence in the intracluster medium (Dolag et al. 2005b; Vazza et al. 2009; Iapichino et al. 2011; Valdarnini 2011). A similar level of turbulence can also be expected in smaller haloes that are large enough Figure 14. Summed net cooling rate [ mi (Ci − H i )] of the gas for the different simulations, where Ci and H i are the cooling and heating rates per unit mass. We set the net cooling to zero for particles/cells that are subject to net heating. The rates of all AREPO runs are higher than those of the GADGET simulations at late times. This is directly related to the higher SFRs of all AREPO runs compared to GADGET, as seen at late times in Fig. 11. At high redshift, the cooling rates of AREPO are lower than those of GADGET for the intermediate and the lowest resolution. This is a consequence of the more compact discs in GADGET at these high redshifts. This cooling rate difference decreases once we reach the highest resolution. In that case the cooling rates of GADGET and AREPO agree better at high redshifts, but still show quite large differences at low redshifts, which also produces the different SFR history. to support quasi-hydrostatic atmospheres. It has further been shown that shock heating is not only important in strong shocks at the outer accretion radius, but is also significant in weaker flow shocks that occur in large parts of the halo volume (Pfrommer et al. 2006). Physically, the dissipation associated with shocks and with the decay of turbulence occurs through microphysics on very small scales. In our numerical approach, we neglect the physical viscosity [which is assumed to be effectively zero on all resolved scales; this is why we employ the Euler and not the Navier–Stokes equations; see e.g. Muñoz et al. (2012) for a Navier–Stokes implementation of AREPO] and account for the necessary dissipation through numerical viscosity. In SPH this is provided explicitly in terms of an artificial viscosity, whereas in AREPO it is introduced implicitly through the solutions of the Riemann solver and cell averaging. The good conservation properties of SPH allow it to capture 1D shock waves quite well, even though the post-shock velocity field typically shows substantial noise (Springel 2010b; Abel 2011). This already hints that SPH may not be particularly accurate for subsonic flow phenomena, such as the turbulence we expect in the virialized gas of newly formed haloes. Indeed, Bauer & Springel (2012) have systematically compared driven isothermal subsonic turbulence for GADGET and AREPO, using the same versions of the codes we employ here. They find that SPH fails quite badly to account for a turbulent cascade in the subsonic case, whereas a Kolmogorov scaling is obtained for the moving mesh code. This happens despite identical driving fields in both cases, and is ultimately caused by inaccurate gradient estimates in SPH and different dissipation as a function of scale. The SPH simulations dissipate most of the energy already close to the driving scale in the subsonic case, whereas in AREPO C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3045 Figure 15. The two panels show the mean mass-weighted net cooling rate [ mi (Ci − H i )/( mi )], where Ci and H i are the cooling and heating rates per unit mass, as a function of hydrogen number density at z = 2 (left) and z = 0 (right). We set the net cooling to zero for particles/cells that experience net heating. At high redshift the cooling rates agree well between different codes, but towards lower redshifts SPH shows systematic differences compared to the moving mesh approach. At z = 0, AREPO shows a significantly higher mean cooling rate directly below the star formation density threshold. We note that the cooling rates agree well outside of the shown density range, i.e. the discrepancies in cooling occur primarily just below the threshold. Figure 16. Mean mass-weighted temperature of the gas in the different simulations. In the inset we divide each curve by the mean of all curves. In agreement with the findings for the SFR and the cooling rates, the mean temperature of all GADGET simulations is higher than that of the AREPO runs at nearly all redshifts. efficient dissipation happens only on much smaller scales, so that a self-similar turbulent cascade can develop over some inertial range. In addition, Bauer & Springel (2012) find that SPH exhibits a second maximum in the dissipation on very small scales of the order of the mean interparticle separation. Here the subsonic noise of SPH is dissipated by the artificial viscosity. Interestingly, the total amount of dissipation on these scales is quite independent of the artificial viscosity value itself. While a higher viscosity reduces the amplitude of the small-scale SPH noise, the dissipation rate on these scales C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society still remains roughly the same, suggesting that the cause of the small-scale noise, which originates in errors in the pressure gradient estimates, cannot be reduced effectively with a different viscosity setting. There are good reasons to expect that these differences in the dissipation properties of the hydrodynamical schemes may also induce important effects for the thermodynamic structure of cosmological haloes, which in turn can easily give rise to variations of the amount of gas that cools out of these haloes. To examine this further, we have carried out two auxiliary non-radiative simulations at resolutions of 2 × 1283 and 2 × 2563 particles/cells. These simulations use the same box size and parameters as our galaxy formation runs, except that cooling and star formation were not included. The use of non-radiative simulations allows us to cleanly employ the method of Bauer & Springel (2012) for measuring the instantaneous dissipation rate of particles and cells, respectively. In the case of SPH, this can simply be done by measuring the work per unit time done by the artificial viscosity forces, which represents the sole source of entropy generation in this method. For the moving mesh code, we instead also advect the thermodynamic entropy between the cells, and then measure the rate of entropy generation by comparing the state of a cell at the end of a time step computed adopting energy conservation with the state expected if the entropy would have remained constant. The inferred rate of entropy production can then also be converted into a fiducial heating rate. In Fig. 17, we compare stacked profiles of the spherically averaged dissipative heating rate of haloes obtained in this way. For the plots, we have selected all haloes in the virial mass range 3 × 1011 h−1 M < M 200 < 1 × 1012 h−1 M , where M 200 refers to the mass inside a radius R200 that encloses 200 times the critical density. We have then placed 20 logarithmic bins on to the radial range 0.01 × R200 to 4.0 × R200 , and produced mass-weighted averages of the dissipation rate per unit mass, du/dt, that we measured for each particle/cell. In order to allow a calculation of an average profile by stacking the haloes, we express the dissipation rate in dimensionless units by multiplying it with the Hubble time at the 3046 M. Vogelsberger et al. Figure 17. Dissipation rate profiles for haloes formed in non-radiative simulations. Each panel is for a given redshift between z = 4 and 0 (as labelled), and stacks the results for all haloes contained in the mass range 3.0 × 1011 h−1 M ≤ M200 ≤ 1.0 × 1012 h−1 M . The number of haloes that are averaged over is indicated in each panel. The red lines show results for the 2 × 2563 non-radiative simulation with AREPO, the blue lines show the corresponding results for the GADGET simulation. Dashed lines give the equivalent 2 × 1283 results. 2 given redshift, and √ by normalizing it to V200 of the particular halo, where V200 = GM200 /R200 is the circular velocity at the virial radius. In the individual panels of Fig. 17, we show results for different redshifts, ranging from z = 4 to 0, and we compare AREPO with GADGET at the two resolutions considered here. We see that there is a clear systematic difference in the dissipative heating rates as a function of halo-centric distance. AREPO produces more heating in the infall region directly outside the virial radius whereas GADGET shows more heating in the outer parts of virialized haloes, where most of the gas mass is located. This systematic difference is present at all redshifts in large haloes. We also note that the nature of the difference is robustly preserved at different resolutions, even though there seem to be small residual trends with spatial resolution. Interestingly, at fixed halo mass, the relative level of dissipation in the innermost parts of haloes increases in the mesh code with time, which may be related to the build-up of a turbulent entropy core in these haloes. In Fig. 18, we consider dissipation profiles as a function of halo mass at z = 0, which however does not reveal any clear trend with halo mass. Our interpretation of the systematic difference revealed by Figs 17 and 18 is that (1) shocks are captured more efficiently by AREPO in the accretion regime of haloes, and (2) AREPO strips gas out of infalling objects more efficiently there; both effects contribute to a stronger dissipation in this region. In contrast, SPH stops infalling gas less efficiently, leading to more dissipation at smaller radii. A further contributor to the heating there lies in the dissipation of the subsonic noise in SPH, which is constantly recreated in this region by tapping free energy from the disturbances that impinge on the haloes from the outside. As a net result of this dissipation of subsonic noise in the outer parts of haloes, the SPH simulations end up hotter overall (see also Fig. 16). Furthermore, this difference in the heating occurs in a region where one expects the cooling radius of many haloes in cosmological radiative simulations, thereby directly affecting the amount of cold gas produced in the quasi-hydrostatic cooling flows in large haloes. We note that the measurement of the dissipation rate is difficult due to its high time variability. We use above stacked profiles of small sets of haloes to mitigate this problem, but even then the inner logarithmic bins average over much smaller gas mass than the outer parts. This implies that the profiles in the inner 10 per cent of the virial radius are relatively noisy. But this inner part is largely irrelevant as the impact on the cooling rates comes from larger radii as we discussed above. 4.3 Mixing in haloes Another reason why differences in the global SFR occur between and GADGET lies in the different accuracy with which hydrodynamical fluid instabilities are treated. In particular, we expect that for the moving mesh code cold gas can be stripped more efficiently from galaxies as they fall into larger haloes, and this gas is then mixed with the halo’s diffuse gas, which also lowers its temperature. AREPO C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3047 Figure 19. Test simulation of a 10 h−1 Mpc box with 1283 SPH particles/cells. The y-axis shows the total amount of star-forming gas and stellar mass in the simulation volume. Solid lines show the result of a calculation with our standard star formation prescription. For the dashed lines we turned off the creation of stellar particles, i.e. there are no stars formed but the gas still cools and is supported by the multiphase equation of state. The solid and dashed curves overlap well for GADGET, which shows that once gas overcomes the density threshold it either forms stars or stays above the threshold. There is no loss of star-forming gas mass. For AREPO, the amount of stellar mass and star-forming gas mass is larger than the amount of starforming gas if stellar particle creation is turned off. This is because AREPO resolves fluid instabilities better and therefore allows significant gas mass loss from infalling satellites. Here gas that once has been above the threshold for star formation may well be mixed with halo gas and reach much lower densities again. On the other hand, once stellar particles are created from dense gas, the total amount of baryonic material that can be lost from subhaloes is reduced, which explains the difference seen in the two moving mesh calculations. Figure 18. Dissipation rate profiles for haloes formed in non-radiative simulations. The different panels are for z = 0 haloes contained in different mass ranges, from more massive (top) to less massive (bottom), as labelled. Note that the bottom panel is identical to the last panel in Fig. 17, which is here repeated for ease of comparison. The solid lines show results for the 2 × 2563 non-radiative simulations, while the dashed lines give the equivalent 2 × 1283 results. To demonstrate this difference more explicitly in our cosmological runs, we performed special test simulations of a 10 h−1 Mpc box with 1283 SPH particles/cells. In Fig. 19, we show the sum of the total amount of star-forming gas mass and stellar mass in this C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society box as a function of redshift. We note that an SPH particle/cell is star forming if its density is higher than the threshold of the subresolution star formation model. The solid lines show the result of calculations with our standard star formation prescription, i.e. these simulations are equivalent to the main runs presented in the paper. However, for the dashed lines we turned off the creation of stellar particles, keeping everything else the same. In this case, the cold gas accumulates in the galaxies and is supported by the effective equation of state expected for gas at these densities, except that the gas is not depleted and does not turn into stars at the normal rate. For these runs, the mass in the figure therefore refers to the total amount of star-forming gas in these simulations (no stars are present). Interestingly, the solid and dashed curves overlap well for GADGET (blue) in Fig. 19, showing that once gas has overcome the density threshold it will never return to lower density. Stripping of gas out of galaxies does not appear to happen in any significant way, otherwise we would expect that the run without star formation should end up with a smaller amount of collapsed baryons. Instead, the SPH result is consistent with no stripping at all, i.e. once a gas particle has cooled, it never returns to lower density even though the galaxy may be subject to substantial shear flows upon halo infall. This behaviour has also been found by Heß & Springel (2011) 3048 M. Vogelsberger et al. in a comparison of galaxy–wind interactions in SPH and the VPH technique (Heß & Springel 2010). For AREPO the situation is very different. As Fig. 19 shows, the sum of stellar mass and star-forming gas mass is larger than the amount of star-forming gas in the run without star particle creation. Here the relevant fluid instabilities for stripping (like the Kelvin– Helmholtz instability) are significantly better resolved and facilitate a substantial mass loss of infalling overdense blobs/subhaloes. Because of this, the sum of the mass of stars and of star-forming gas is higher in AREPO for the run with star particle creation than for the fiducial run where this is suppressed. In the latter simulation, there is simply more gas around that can be stripped again, while in the run with star particle creation, baryons that have been converted to stars do of course not suffer from fluid instabilities anymore. Apart from affecting the cooling in haloes, this difference in the stripping and mixing efficiency also modifies the dynamical friction time-scale of infalling gas clumps, as we examine in more detail in Paper III of this series. 5 GENERIC ISSUES WITH SPH Various modifications have been proposed to the conventional implementation of SPH as in GADGET to improve its reliability in certain cases. For example, changes to the computation of smoothed pressure gradients (Abel 2011) or the addition of an artificial thermal conductivity to the equations of motion (e.g. Price 2008) enable SPH to better handle shearing interfaces and the onset of Kelvin– Helmholtz instabilities. Also, a number of studies focused on the development of improved artificial viscosity parametrizations in SPH codes (Morris 1997; Cullen & Dehnen 2010), with the goal of reducing the numerical viscosity away from shocks. Given the results presented in this paper, it is tempting to argue that one or several of these modifications of ‘standard SPH’ may lead to much better agreement with the moving mesh results or even resolve the discrepancies. While we cannot exclude this possibility, we note that the above refinements are ad hoc in the sense that they are designed to mitigate against particular problems in some circumstances, and may have unwanted side effects in other situations. While these side effects may often be benign, the proposed modifications of SPH do not address other, more generic problems with this technique. In this section, we summarize these issues in order to put our findings into context with the recent literature on SPH techniques. One issue that has rarely been discussed in the cosmological community concerns the convergence properties of SPH. This is manifested in at least two ways, the first having to do with local smoothed estimates in SPH, and the second is related to the fact that SPH is not formally Lagrangian. Below, we first address the question of under what conditions convergence is expected in SPH and how this influences the relation between spatial resolution and the total particle number. Then, we discuss the pseudo-Lagrangian character of SPH and its implications for defining convergence with this method. Finally, we briefly suggest ways in which these issues might be dealt with. Incidentally, they all would have the effect of making SPH resemble a moving mesh scheme like AREPO. 5.1 Convergence in SPH and nearest neighbour number For a real fluid, we define the density, ρ(x), to be a continuous function of space, determined by averaging over many molecules locally around x. In order to formulate a discrete SPH counterpart to this (and this applies to all fluid variables, as well as the equations of motion) a two-step procedure is applied. First, we introduce a smoothed version of the original density field, ρ(x), by convolving ρ(x) with a smoothing kernel W (x, h) according to ρ(x) = ρ(x )W (x − x , h) dV , (1) where the integral is over all space and h is the smoothing length. Secondly, the continuous smoothed field ρ(x) is replaced by a discrete quantity, ρd (x), so that it can be represented computationally. This is done by regarding ρ(x ) dV as a mass element dm and partitioning the fluid into a set of N discrete mass elements so that the above integral can be approximated by a discrete sum: ρ(x) → ρd (x) = Nngb mj W [x − x j , h(x)], (2) j =1 where mj is the mass of fluid element j, x j is its spatial location and the expression allows different fluid locations/elements to have different smoothing lengths. In principle, the sum extends over all N fluid elements. However, in order that the smoothed density ρ(x) approaches the continuum limit represented by ρ(x), it is necessary that W (x, h) is spatially localized. So in practice a localized kernel with compact support is adopted, implying that the discrete sum extends not over all N fluid elements but over a subset N ngb of those neighbouring a certain point in space. Now, we can ask about the conditions that must be satisfied for the smoothed, discrete version of the density field to asymptote to the actual continuum limit. That is, under what circumstances is the following true? ρd (x) → ρ(x) → ρ(x) . (3) Consider the second of these requirements first. In order for ρ(x) → ρ(x) we must have W (x, h) → δ(x) as h → 0, as implied by the definition of ρ(x) in the above integral. This also requires that N → ∞, otherwise there exist not enough particles around a given spatial location to form smoothed averages. Turning to the first requirement above, ρd (x) → ρ(x) will be satisfied only if at the same time we enforce the condition N ngb → ∞ . This has not been emphasized in the cosmological literature, but from the defining relation above for ρd (x) it is clear that the discrete approximation will not asymptote to the smoothed density field unless this limit is taken. Or, to put it another way, if the number of neighbours is held fixed as h → 0, there will be a constant source of error present on the most well-resolved scales that will not vanish as the resolution and number of particles are increased indefinitely (see also Rasio 2000; Read, Hayfield & Agertz 2010). (For a discussion of this requirement in the context of elasticity, see e.g. Belytschko et al.) To the best of our knowledge, the optimal way to enforce this requirement has not been established. However, there is little doubt that this must be addressed in order that SPH provides properly convergent solutions, as argued recently by Robinson & Monaghan (2011): ‘When performing a convergence study using SPH, it is important to vary both the number of particles... as well as the ratio of smoothing length to particle spacing [to increase the number of neighbours]. [Otherwise] the error due to the summation interpolant [remains] constant [as the total particle number is increased].’ We emphasize that this requirement is not fundamentally an issue with SPH, but simply reflects the fact that a numerical approximation to an integral in the form of a discrete sum will not converge to the true C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology answer unless the number of points where the integrand is sampled is made larger. An increase in the number of neighbours is also warranted to reduce the gradient errors in SPH, which give in fact rise to a sizable ‘zeroth-order error’ (Read et al. 2010). These gradient errors seriously degrade the accuracy of SPH in subsonic flow problems (Springel 2010b) and are a primary cause for problems in the representation of subsonic turbulence (Bauer & Springel 2012). However, in practice, simply increasing the number of neighbours is met by a serious obstacle, because it invokes the so-called clumping instability in which particles located in the inner parts of the kernel are pushed together by the pressure forces of the surrounding particles, forming little clumps that frustrate attempts to reduce the error of the kernel sums in this way. Partially circumventing this problem and allowing a higher neighbour number require different kernel shapes than are commonly employed (Read et al. 2010; Price 2012), but such kernels merely have different stability bands that still impose severe restrictions on the number of neighbours that may be used. In essence, due to the clumping instability, establishing formally convergent SPH solutions is an unsolved problem because the path of increasing the neighbour number is blocked in practice. 5.2 Implications for SPH convergence rates and efficiency The requirement that the number of neighbours should increase to reduce errors as the total number of particles is made larger has also serious implications for the efficiency of SPH. As an example, consider a uniform fluid in a cube of side length L represented by N SPH particles. (For more general circumstances the argument generalizes if we think about small scales on which the fluid properties are nearly uniform.) The mean separation between the particles, r0 , is 1/3 3 N −1/3 . (4) r0 = L 4π If we take h ∝ r0 , as has been done conventionally in SPH codes used in cosmology, then the smoothing length will shrink in proportion to r0 and, so, the number of neighbours will be nearly constant. In a sense, this maximizes the spatial resolution, but leaves a fixed source of error in the discrete sums (Read et al. 2010; Robinson & Monaghan 2011), so this path is in principle not appropriate for achieving proper convergence with SPH. β If, instead, we set h ∝ r0 then the fundamental relation between −3/β . What are the conditions on β so that we N and h becomes N ∝ h can meaningfully construct a procedure that converges numerically? Clearly, β < 1, otherwise h will not decrease more slowly than r0 as N increases. Also, we must have β > 0, as the choice β = 0 would mean that the spatial resolution is constant, independent of N. The optimal value for β is undetermined. One physically appealing choice would be β = 1/2, because then h would be the geometric mean between the mean interparticle separation, r0 , and the size of the system, L. In that case, the relation between N and h becomes N ∝ h−6 , implying a steeply rising computational cost with resolution. It is possible that a slightly larger value of β could be preferred in terms of CPU costs, but it is clear, however, that in order to extend the dynamic range in spatial scales resolved by SPH in a way that guarantees eventual convergence, it is absolutely necessary to increase the number of particles more aggressively than has been advocated previously. The dilemma, of course, is that the clumping instability makes this highly problematic. Ignoring the clumping issue for the moment, the implications for the convergence rate of SPH are in any case important. When C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3049 the common practice in the field is followed and the number of neighbours is held constant, Springel (2010b) reported a global L1 error for a vortex flow (the Gresho test) that scales as h0.7 with resolution, as opposed to L1 ∝ h1.4 for AREPO. We can then ask how the ratio of the CPU-time cost of simulations with the two schemes varies with the size of the error. For simulations in 3D, the computational cost scales for both codes as h−4 , where three powers of h come from the spatial dimensions, and one additional power enters due to the reduction of time steps for better resolution. This then implies that the cost ratio scales as CPUGADGET /CPUAREPO ∝ L1−2.86 with the error of the calculation. Reducing the error by a factor of 10 requires therefore about a ∼700 times higher effort in SPH than in AREPO. This conclusion is problem dependent and can also be affected by specific details of the SPH algorithm, like the kernel shape. However, the question of computational efficiency of different numerical schemes is best phrased in terms of such convergence rate comparisons. Here the Monte Carlo character in the approximation of the SPH kernel sum invariably introduces a serious disadvantage for SPH compared with the moving mesh approach, as outlined above. A related conclusion, but focusing on the efficiency impact of the artificial viscosity parametrization in SPH, was reached by Bauer & Springel (2012). If the goal is a representation of subsonic turbulence with a certain Reynolds number Re, they showed that the computational cost of SPH scales at least as CPU ∝ Re4 . In contrast, in the mesh-based treatment of AREPO, the dissipation scale is tied to the mesh resolution, implying a computational cost for reaching a certain Reynolds number that scales as ∝ Re3 . Given that our cosmological simulations with AREPO are only about 30 per cent slower than GADGET for simulations involving self-gravity, for the same number of resolution elements, this means that at a given computational cost, AREPO resolves much larger Reynolds numbers than SPH. In fact, to reach the same Reynolds numbers as in our present moving mesh simulations, it appears plausible that a factor ∼100−1000 more SPH particles than AREPO cells would be required. 5.3 The pseudo-Lagrangian nature of SPH SPH is often referred to as a Lagrangian algorithm, but this is not formally correct and we suggest that it is perhaps better to characterize it as a ‘pseudo-Lagrangian’ technique. To appreciate the reasons for this, consider the following example, which also highlights that this distinction is intimately related to issues of (suppressed) mixing in SPH. Suppose gas is revolving in a thin disc on circular orbits, and imagine that the gas is partitioned at some instant into regions that are then represented by SPH particles, with the smoothing done over a circular area. Each SPH particle will comprise gas from an area around its centre, extending outwards in radius of the order of the local smoothing length, as shown for one such SPH particle in the left-hand panels of Fig. 20 at time zero. Consider now how the system will look like a short time later, t > 0, after the disc has rotated by a small angle. If the disc is in solid body rotation, as in the upper panels of Fig. 20, the fluid initially contributing to the SPH particle will occupy an area identical to the smoothing region of the SPH particle at time t > 0, as indicated by comparing the ‘true’ situation in the upper middle panel with the SPH version in the upper righthand panel. In this case, the SPH representation is faithful to the actual equations of motion because the fluid contributing to the SPH particle at time zero is the same as that at t, and the region 3050 M. Vogelsberger et al. Figure 20. Schematic representation of a thin disc with gas evolving on circular orbits. The dark blue region shows the gas that is within the smoothing region of one SPH particle in this disc. The upper panels show the situation for solid body rotation, whereas bottom panels depict the situation for differential rotation. The leftmost panels give the initial set-up at time t = 0. The middle panels show the true gas distribution of the disc a short amount of time later. For solid body rotation, the initial blue region retains its shape. This is different for the differentially rotating disc, where the blue gas patch experiences the shear in the disc and gets distorted. The panels on the righthand side show the corresponding SPH representations of the disc at this later time. Clearly, the distortion in the differentially rotating case is not captured correctly by SPH since the smoothing region does not change in shape. SPH is therefore only pseudo-Lagrangian, as opposed to AREPO, which implements a quasi-Lagrangian scheme that allows for mass exchange between cells in a manner consistent with the hydrodynamic equations of motion. The dashed circle in the bottom middle panel illustrates a re-partitioning of the gas into SPH particles at the updated time and demonstrates why mixing is suppressed in SPH at the resolution scale (see text). bounded initially by the SPH smoothing volume has not changed its shape. However, suppose instead that the disc is rotating differentially, turning around more rapidly in the inner parts than in the outer ones (lower panels of Fig. 20). The ‘true’ situation shown in the middle panel tells us that the gas contributing to the area bounded by the SPH smoothing volume at time zero will be stretched out owing to shear. However, in the SPH formulation, shown in the lower right-hand panel, the material initially in the SPH smoothing volume is forced to remain tied to this area and is not allowed to shear, inconsistent with the equations of motion of the real fluid. A true Lagrangian picture would allow fluid elements to be deformed in the presence of shear, but this is inhibited by the SPH approach on scales smaller than those set by the smoothing procedure. Hence, while SPH retains some characteristics of a Lagrangian method, it does not evolve the fluid entirely in a manner consistent with the equations of motion and should, in this sense, be termed ‘pseudo-Lagrangian.’ Unfortunately, the error incurred by the approximations inherent to the SPH formalism is highly problem dependent and cannot be estimated based solely on the discretization procedure. For example, there is no error made in the case of a disc in solid body rotation, or for a uniform disc rotating differentially, or for gas motions that are limited to expansion or contraction in three dimensions. However, this is not true for shearing flows involving fluids with distinct internal properties; here the implicit ‘remapping’ involved in enforcing a spherical shape for the SPH smoothing volume, which would change the local composition of the internal properties, is ignored. This limitation is ultimately the reason why SPH does not handle mixing accurately, as we illustrate in the bottom, middle panel of Fig. 20. Suppose at the updated time we were to re-partition the gas into new SPH particles, as indicated by the dashed circle in this frame. The gas associated with this particle should include gas from the original particle, but also gas from the surrounding flow. This would allow mixing to occur between the gas in the original particle and nearby parts of the flow, but is not allowed to occur in SPH, by construction. We note that AREPO does not suffer from this defect. In this code, cells are not allowed to become arbitrarily distorted, in the interests of efficiency and accuracy, and so AREPO is also not formally Lagrangian. However, it still evolves fluids correctly when deviations from strict Lagrangian behaviour occur by allowing for mass exchange between cells, in a manner consistent with the equations of motion. Therefore, this method should be termed ‘quasiLagrangian.’ A related issue arises in the context of N-body simulations of collisionless systems. There, a 6D phase fluid is partitioned into particles of fixed size that move through space in a manner determined by the equations of motion. Because the phase fluid initially associated with a particular particle is always tied to that particle, the small-scale dynamics of the system will not be represented properly. However, in this case, unlike for the example discussed above, forces are strictly long range and are less prone to inaccuracies in the small-scale distribution of the material than pressure gradients or viscosity. In that sense, the local distortions in the phase-space fluid have fewer dynamical consequences than for a hydrodynamical fluid in three dimensions. Nevertheless, if an accurate description of the small-scale structure of collisionless systems is essential, these effects must be accounted for at some levels (e.g. Vogelsberger et al. 2008; Abel et al. 2011; Vogelsberger & White 2011). 5.4 Discussion We should emphasize that whether the limitations inherent to SPH lead to significant inaccuracies in the solution depend in detail on the circumstances. SPH can be expected to provide reliable results for flows that are kinetically dominated, or if the motions are controlled mainly by long-range forces. In these situations, errors in the local quantities are subdominant. For example, Bauer & Springel (2012) show that while SPH does not accurately describe subsonic turbulence, in the supersonic regime, when the flow energy is mainly kinetic, GADGET and AREPO give similar answers. Also, comparisons between SPH and AMR codes have yielded compatible results for the structure of the IGM, as reflected in the properties of the Lyα forest (e.g. Regan et al. 2007). At these low densities, gravity dominates over internal energy and, moreover, the fluid is not subject to strong shear, so the above considerations are not critical. Even in some cases where shear is present, the approximations underlying SPH will not be significant, provided that gravitational forces are more important than local ones and the relevant evolution in flow properties occurs rapidly. Hayward et al. (in preparation) have demonstrate this explicitly by simulating galaxy mergers using both GADGET and AREPO. They find that when the gas is treated using an effective equation of state, SFRs during the course of a merger are nearly identical between these two codes. Thus, conclusions drawn from studies of galaxy collisions with SPH, such as the stellar profiles of merger remnants and their evolution (e.g. Hopkins et al. 2008, 2009a,b,c) or the survivability of discs during mergers (Springel & Hernquist 2005), are likely robust to variations in the hydro solver. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology Unfortunately, this good level of agreement does not extend to the more complex flows associated with haloes in cosmological environments, as we highlight in this paper. There, various phases of gas will be present in close proximity, often shearing relative to one another, leading to errors in simulations done with SPH like those we have identified here. Also, in the hydrostatic atmospheres of haloes the hydrodynamic forces are comparable to gravity forces, and the subsonic turbulence present in these regions is affected by gradient errors in SPH. Without a formal convergence criterion, the consequences of these errors are difficult to assess, because even if a solution may plateau as the number of SPH particles is increased, this by no means guarantees that the correct answer is being obtained. In particular, the solution may exhibit ‘false convergence,’ appearing to reach a converged solution, while fixed sources of error from e.g. summation interpolant are still present (Robinson & Monaghan 2011). A case in point is the longstanding discrepancy identified for the central cluster entropy predicted by non-radiative SPH and mesh-based simulations, first identified in the Santa Barbara cluster comparison project (Frenk et al. 1999). The above considerations motivate thinking about ways to cure the defects inherent to SPH so that it can provide solutions of comparable accuracy and resolution to those obtained with a moving mesh approach like AREPO. For example, the artificial viscosity typically used in SPH codes to handle shocks could, in principle, be eliminated by incorporating a Riemann solver into SPH. Pioneering efforts along these lines have been made by Inutsuka (2002) and Murante et al. (2011). However, these implementations solve the Riemann problem for each particle-neighbour pair, and this approach will become prohibitively expensive as long as the smoothing procedure requires averaging over an increasingly large number of neighbours to achieve convergence as the total particle number is increased. We note that alternatively a grid could be introduced into SPH just for the purposes of solving the Riemann problem around each particle. This grid would necessarily have to be adaptive, and hence resemble the Voronoi tessellation used in AREPO. In order to eliminate the pseudo-Lagrangian character of SPH and allow for fluid elements to become distorted on scales small compared to a few smoothing lengths, as should occur in e.g. shearing flows, it is necessary to allow for mass exchange between particles in a manner consistent with the equations of motion. This would have the added benefit of eliminating the mixing problem in SPH. First suggestions in this direction have recently been made (Price 2008; Wadsley et al. 2008; Read et al. 2010; Read & Hayfield 2012), but the best way to formulate such mixing terms is not clear. One possibility to unambiguously calculate the required mass flux between particles would be to utilize some kind of grid. Again, this grid would need to be adaptive, like the unstructured mesh in AREPO, reinforcing the notion that our new moving mesh approach quite naturally addresses several of the generic issues with SPH. 6 CONCLUSIONS In this study, we have presented the first cosmological hydrodynamical simulations of structure formation carried out with the novel AREPO moving mesh code. Compared to the widely employed SPH technique, this method promises an important gain in accuracy in the numerical treatment of gas dynamics for similar computational costs. In order to understand the implications of these differences for galaxy formation simulations, we have evolved the same initial conditions both with AREPO and the well-established SPH code C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3051 GADGET. Both codes share the same high-resolution gravity solver, and incorporate an identical radiative cooling and star formation model. Differences in the outcome are hence tied to systematics of the hydrodynamical solvers that are used. It is the primary goal of our paper to identify the main differences and their magnitude, as well as providing evidence for the primary sources of potential discrepancies. To achieve this goal, we have considered a primary set of simulations in boxes of 20 h−1 Mpc that include a basic treatment of galaxy formation physics, at various resolutions ranging from 2 × 1283 to 2 × 5123 particles/cells. In addition, we have carried out a few auxiliary simulations either in smaller boxes, or of non-radiative type, in order to more clearly measure particular numerical effects. For reasons discussed in Section 2.3, we have chosen to perform the comparisons between the codes at the same initial mass resolution, because then the computational costs are similar. Our principle findings may be summarized as follows. (i) The overall distribution of gas in density and temperature is broadly in agreement between SPH and AREPO, but there are some subtle and important differences. In particular, the mean massweighted temperature in AREPO is slightly smaller than in GADGET. Also, the hot gas extends to somewhat lower density in GADGET and hence occupies a larger volume fraction. (ii) The cosmic SFR density peaks at similar values, but at slightly lower redshifts in AREPO. At high redshift and at high resolution, the SFRs between the two codes are in good agreement, but towards lower redshift there are significantly stronger cooling flows in AREPO, causing a correspondingly larger SFR. This difference occurs primarily in haloes with mass larger than ∼1011 h−1 M . (iii) We find that the two codes differ significantly in the dissipative heating rates within haloes, with AREPO producing more dissipation in the halo infall regions, whereas SPH produces higher dissipative heating throughout most of the outer regions of virialized haloes. This difference is mainly responsible for the higher temperatures found in the SPH simulations, and the correspondingly weaker cooling. We argue that this dissipation in SPH is likely to be of spurious nature, and is a combination of viscous damping of SPH’s inherent noise and the unphysical damping of subsonic turbulence injected into haloes in the infall regions. (iv) Visual comparison of simulated galaxies shows considerably larger stellar discs and more extended and discy stellar distributions in AREPO compared with GADGET. Also, the halo gas surrounding the galaxies looks smoother and less clumpy in AREPO. Fig. 21 shows gas surface density disc scale length radii obtained from exponential surface density fits of more than 50 galaxies at z = 0 in the halo mass range ∼4 × 1011 to ∼8 × 1012 h−1 M . This clearly demonstrates that the gas discs forming in the AREPO simulations are typically significantly larger than those in the corresponding SPH runs. We further note that the gas surface density of disc galaxies in the AREPO simulations follow very closely exponential profiles, which is typically not the case for the SPH runs. A more detailed analysis is presented in Torrey et al. (2011). We note that the large discs in the AREPO simulations result even without strong feedback in the form of galactic winds or modifications of the star formation prescription. This is also demonstrated in simplified test simulations in Paper III. The larger extent of disc galaxies is also demonstrated in Paper II, where we do not assume exponential profiles, but rather use a cut in surface density to determine a characteristic size. We also independently calculated half-mass radii, which show the same trends. All these results confirm the findings reported in Fig. 21, i.e. that AREPO produces in general larger discs than GAGDET. 3052 M. Vogelsberger et al. at the Texas Advanced Computing Center (TACC). We thank Paul Torrey for allowing us to show Fig. 21 and Elena D’Onghia, Patrik Jonsson, Andrew MacFadyen, Diego J. Munoz for useful discussions. We also thank the anonymous referee for helpful comments. DS acknowledges NASA Hubble Fellowship through grant HSTHF-51282.01-A. DK acknowledges support from NASA through Hubble Fellowship grant HSTHF-51276.01-A. REFERENCES Figure 21. Gas disc scale lengths obtained from exponential surface density fits of more than 50 galaxies at z = 0 in the halo mass range ∼4 × 1011 to ∼8 × 1012 h−1 M . Clearly, the gas discs forming in the AREPO simulations are typically significantly larger than those in the corresponding SPH runs. The above findings clearly suggest that there are significant quantitative differences caused in galaxy formation simulations by the choice of hydrodynamical technique. It appears that the limited accuracy of SPH for subsonic flow phenomena such as Kelvin– Helmholtz instabilities or subsonic turbulence induces important differences in the predicted properties of galaxies at low redshift, affecting both their morphology and stellar mass. Because of the accuracy problems of SPH for certain subsonic test problems (Agertz et al. 2007; Springel 2010b, Bauer & Springel 2012), it appears that SPH cannot be expected to reach highly accurate results in all regimes relevant for cosmic structure formation. At the same time, AREPO performs significantly better on these same problems, and yields generally a smaller error norm and higher convergence rate when scrutinized against problems with known analytic solutions (Springel 2010a,b). We are hence confident that the AREPO results presented in this paper entail a more faithful treatment of cosmological hydrodynamics, especially in view of the discussion in Section 5.3. It is also reassuring that the AREPO runs improve the predicted morphologies of simulated galaxies, as our preliminary analysis suggests. In Paper II of this series we will back up this finding with a detailed analysis of the galaxy properties. Finally, in Paper III we provide a more detailed analysis of the relevant numerical effects responsible for the differences in mixing and cooling. The body of these results makes it clear that AREPO simulations provide a significant opportunity for the development of next-generation models of galaxy formation that promise to achieve a so-far unknown combination of accuracy, dynamic range and faithfulness to the relevant physics. AC K N OW L E D G M E N T S The simulations in this paper were run on the Odyssey cluster supported by the FAS Science Division Research Computing Group at Harvard University. Scaling tests were done on the Ranger cluster Abel T., 2011, MNRAS, 413, 271 Abel T., Hahn O., Kaehler R., 2011, preprint (arXiv e-prints) Agertz O. et al., 2007, MNRAS, 380, 963 Agertz O., Teyssier R., Moore B., 2011, MNRAS, 410, 1391 Balsara D. S., 1995, J. Comput. Phys., 121, 357 Barnes J. E., Hernquist L. E., 1991, ApJ, 370, L65 Barnes J. E., Hernquist L., 1992, ARA&A, 30, 705 Barnes J. E., Hernquist L., 1996, ApJ, 471, 115 Barnes J., Hut P., 1986, Nat, 324, 446 Bate M. R., Burkert A., 1997, MNRAS, 288, 1060 Bauer A., Springel V., 2012, MNRAS, 423, 2558 Baugh C. M., 2006, Rep. Progress Phys., 69, 3101 Belytschko T., Krongauz Y., Dolbow J., Gerlach C., 1998, Int. J. Numer. Methods Eng., 43, 785 Benson A. J., 2012, New Astron., 17, 175 Benson A. J., 2010, Phys. Rep., 495, 33 Benson A. J., Cole S., Frenk C. S., Baugh C. M., Lacey C. G., 2000, MNRAS, 311, 793 Berger M. J., Colella P., 1989, J. Comput. Phys., 82, 64 Bertone G., Hooper D., Silk J., 2005, Phys. Rep., 405, 279 Blumenthal G. R., Faber S. M., Primack J. R., Rees M. J., 1984, Nat, 311, 517 Bond J. R., Cole S., Efstathiou G., Kaiser N., 1991, ApJ, 379, 440 Boylan-Kolchin M., Springel V., White S. D. M., Jenkins A., 2010, MNRAS, 406, 896 Bryan G. L., Norman M. L., 1998, ApJ, 495, 80 Cantalupo S., Porciani C., 2011, MNRAS, 411, 1678 Collins D. C., Xu H., Norman M. L., Li H., Li S., 2010, ApJS, 186, 308 Cox T. J., Dutta S. N., Di Matteo T., Hernquist L., Hopkins P. F., Robertson B., Springel V., 2006, ApJ, 650, 791 Crain R. A. et al., 2009, MNRAS, 399, 1773 Croton D. J. et al., 2006, MNRAS, 365, 11 Cullen L., Dehnen W., 2010, MNRAS, 408, 669 D’Onghia E., Springel V., Hernquist L., Keres D., 2010a, ApJ, 709, 1138 D’Onghia E., Vogelsberger M., Faucher-Giguere C.-A., Hernquist L., 2010b, ApJ, 725, 353 D’Onghia E., Vogelsberger M., Hernquist L., 2012, ApJ, submitted (arXiv:1204.0513) Davé R., Hernquist L., Katz N., Weinberg D. H., 1999, ApJ, 511, 521 Davé R. et al., 2001, ApJ, 552, 473 Di Matteo T., Springel V., Hernquist L., 2005, Nat, 433, 604 Di Matteo T., Colberg J., Springel V., Hernquist L., Sijacki D., 2008, ApJ, 676, 33 Diemand J., Kuhlen M., Madau P., Zemp M., Moore B., Potter D., Stadel J., 2008, Nat, 454, 735 Dolag K., Jubelgas M., Springel V., Borgani S., Rasia E., 2004, ApJ, 606, L97 Dolag K., Grasso D., Springel V., Tkachev I., 2005a, J. Cosmol. Astropart. Phys., 1, 9 Dolag K., Vazza F., Brunetti G., Tormen G., 2005b, MNRAS, 364, 753 Dolag K., Borgani S., Murante G., Springel V., 2009, MNRAS, 399, 497 Duffy A. R., Schaye J., Kay S. T., Dalla Vecchia C., Battye R. A., Booth C. M., 2010, MNRAS, 405, 2161 Dunkley J. et al., 2009, ApJS, 180, 306 Eisenstein D. J., Hu W., 1999, ApJ, 511, 5 Faucher-Giguère C., Lidz A., Zaldarriaga M., Hernquist L., 2009, ApJ, 703, 1416 C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology Frenk C. S. et al., 1999, ApJ, 525, 554 Gao L., Springel V., White S. D. M., 2005, MNRAS, 363, L66 Gingold R. A., Monaghan J. J., 1977, MNRAS, 181, 375 Gnedin N. Y., 1995, ApJS, 97, 231 Governato F., Willman B., Mayer L., Brooks A., Stinson G., Valenzuela O., Wadsley J., Quinn T., 2007, MNRAS, 374, 1479 Governato F. et al., 2010, Nat, 463, 203 Guedes J., Callegari S., Madau P., Mayer L., 2011, ApJ, 742, 76 Guo Q. et al., 2011, MNRAS, 413, 101 Heitmann K. et al., 2008, Comput. Sci. Discovery, 1, 015003 Hernquist L., 1987, ApJS, 64, 715 Hernquist L., Katz N., 1989, ApJS, 70, 419 Hernquist L., Katz N., Weinberg D. H., Miralda-Escudé J., 1996, ApJ, 457, L51 Heß S., Springel V., 2010, MNRAS, 406, 2289 Heß S., Springel V., 2011, submitted Hockney R. W., Eastwood J. W., 1981, Computer Simulation Using Particles. McGraw-Hill, New York Hopkins P. F., Hernquist L., Cox T. J., Dutta S. N., Rothberg B., 2008, ApJ, 679, 156 Hopkins P. F., Cox T. J., Dutta S. N., Hernquist L., Kormendy J., Lauer T. R., 2009a, ApJS, 181, 135 Hopkins P. F., Lauer T. R., Cox T. J., Hernquist L., Kormendy J., 2009b, ApJS, 181, 486 Hopkins P. F., Hernquist L., Cox T. J., Keres D., Wuyts S., 2009c, ApJ, 691, 1424 Iapichino L., Schmidt W., Niemeyer J. C., Merklein J., 2011, MNRAS, 414, 2297 Inutsuka S.-I., 2002, J. Comput. Phys., 179, 238 Jenkins A., Frenk C. S., White S. D. M., Colberg J. M., Cole S., Evrard A. E., Couchman H. M. P., Yoshida N., 2001, MNRAS, 321, 372 Jubelgas M., Springel V., Dolag K., 2004, MNRAS, 351, 423 Jubelgas M., Springel V., Enßlin T., Pfrommer C., 2008, A&A, 481, 33 Katz N., Hernquist L., Weinberg D. H., 1992, ApJ, 399, L109 Katz N., Weinberg D. H., Hernquist L., Miralda-Escude J., 1996a, ApJ, 457, L57 Katz N., Weinberg D. H., Hernquist L., 1996b, ApJS, 105, 19 Kauffmann G., White S. D. M., Guiderdoni B., 1993, MNRAS, 264, 201 Kennicutt R. C., Jr, 1998, ApJ, 498, 541 Keres D., Vogelsberger M., Sijacki D., Springel V., Hernquist L., 2011, preprint (arXiv e-prints) (Paper II) Klypin A. A., Trujillo-Gomez S., Primack J., 2011, ApJ, 740, 102 Komatsu E. et al., 2011, ApJS, 192, 18 Lucy L. B., 1977, AJ, 82, 1013 Mihos J. C., Hernquist L., 1994, ApJ, 437, 611 Mihos J. C., Hernquist L., 1996, ApJ, 464, 641 Mitchell N. L., McCarthy I. G., Bower R. G., Theuns T., Crain R. A., 2009, MNRAS, 395, 180 Monaghan J. J., 1992, ARA&A, 30, 543 Monaghan J. J., 2005, Rep. Progress Phys., 68, 1703 Moore B., Governato F., Quinn T., Stadel J., Lake G., 1998, ApJ, 499, L5 Moore B., Ghigna S., Governato F., Lake G., Quinn T., Stadel J., Tozzi P., 1999, ApJ, 524, L19 Morris J., 1997, J. Comput. Phys., 136, 41 Muñoz D., Springel V., Marcus R., Vogelsberger M., Hernquist L., 2012, preprint (arXiv e-prints) Murali C., Katz N., Hernquist L., Weinberg D. H., Davé R., 2002, ApJ, 571, 1 Murante G., Borgani S., Brunino R., Cha S.-H., 2011, MNRAS, 417, 136 Naab T., Burkert A., 2003, ApJ, 597, 893 Nagamine K., Cen R., Hernquist L., Ostriker J. P., Springel V., 2005, ApJ, 627, 608 Navarro J. F., Frenk C. S., White S. D. M., 1996, ApJ, 462, 563 Navarro J. F., Frenk C. S., White S. D. M., 1997, ApJ, 490, 493 Navarro J. F. et al., 2010, MNRAS, 402, 21 O’Shea B. W., Bryan G., Bordner J., Norman M. L., Abel T., Harkness R., Kritsuk A., 2004, preprint (arXiv e-prints) C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3053 O’Shea B. W., Nagamine K., Springel V., Hernquist L., Norman M. L., 2005, ApJS, 160, 1 Pearce F. R. et al., 1999, ApJ, 521, L99 Pen U., 1998, ApJS, 115, 19 Percival W. J. et al., 2010, MNRAS, 401, 2148 Perlmutter S. et al., 1999, ApJ, 517, 565 Petkova M., Springel V., 2011, MNRAS, 412, 935 Pfrommer C., Springel V., Enßlin T. A., Jubelgas M., 2006, MNRAS, 367, 113 Press W. H., Schechter P., 1974, ApJ, 187, 425 Price D. J., 2008, J. Comput. Phys., 227, 10040 Price D. J., 2012, J. Comput. Phys., 231, 759 Puchwein E., Sijacki D., Springel V., 2008, ApJ, 687, L53 Rasio F. A., 2000, Progress Theor. Phys. Suppl., 138, 609 Read J. I., Hayfield T., 2012, MNRAS, 422, 3037 Read J. I., Lake G., Agertz O., Debattista V. P., 2008, MNRAS, 389, 1041 Read J. I., Hayfield T., Agertz O., 2010, MNRAS, 405, 1513 Rees M. J., Ostriker J. P., 1977, MNRAS, 179, 541 Regan J. A., Haehnelt M. G., Viel M., 2007, MNRAS, 374, 196 Riess A. G. et al., 1999, AJ, 118, 2675 Robertson B., Yoshida N., Springel V., Hernquist L., 2004, ApJ, 606, 32 Robinson M., Monaghan J. J., 2011, preprint (arXiv e-prints) Scannapieco C., Tissera P. B., White S. D. M., Springel V., 2008, MNRAS, 389, 1137 Scannapieco C. et al., 2012, MNRAS, 423, 1726 Schaye J. et al., 2010, MNRAS, 402, 1536 Schuecker P., Finoguenov A., Miniati F., Böhringer H., Briel U. G., 2004, A&A, 426, 387 Schuessler I., Schmitt D., 1981, A&A, 97, 373 Sijacki D., Springel V., Di Matteo T., Hernquist L., 2007, MNRAS, 380, 877 Sijacki D., Vogelsberger M., Keres D., Springel V., Hernquist L., 2011, preprint (arXiv e-prints) (Paper III) Silk J., 1977, ApJ, 211, 638 Smith B., Sigurdsson S., Abel T., 2008, MNRAS, 385, 1443 Springel V., 2005, MNRAS, 364, 1105 Springel V., 2010a, MNRAS, 401, 791 (S10) Springel V., 2010b, ARA&A, 48, 391 Springel V., Hernquist L., 2002, MNRAS, 333, 649 Springel V., Hernquist L., 2003a, MNRAS, 339, 289 Springel V., Hernquist L., 2003b, MNRAS, 339, 312 Springel V., Hernquist L., 2005, ApJ, 622, L9 Springel V., Di Matteo T., Hernquist L., 2005a, MNRAS, 361, 776 Springel V. et al., 2005b, Nat, 435, 629 Springel V. et al., 2008, MNRAS, 391, 1685 Stone J. M., Norman M. L., 1992, ApJS, 80, 753 Tasker E. J., Brunino R., Mitchell N. L., Michielsen D., Hopton S., Pearce F. R., Bryan G. L., Theuns T., 2008, MNRAS, 390, 1267 Teyssier R., 2002, A&A, 385, 337 Toomre A., Toomre J., 1972, ApJ, 178, 623 Torrey P., Vogelsberger M., Sijacki D., Springel V., Hernquist L., 2011, preprint (arXiv e-prints) Trac H., Sills A., Pen U., 2007, MNRAS, 377, 997 Valdarnini R., 2011, A&A, 526, A158 Vazza F., Brunetti G., Kritsuk A., Wagner R., Gheller C., Norman M., 2009, A&A, 504, 33 Viel M., Lesgourgues J., Haehnelt M. G., Matarrese S., Riotto A., 2005, Phys. Rev. D, 71, 063534 Viel M., Bolton J. S., Haehnelt M. G., 2009, MNRAS, 399, L39 Vogelsberger M., White S. D. M., 2011, MNRAS, 413, 1419 Vogelsberger M., White S. D. M., Helmi A., Springel V., 2008, MNRAS, 385, 236 Vogelsberger M. et al., 2009, MNRAS, 395, 797 Wadsley J. W., Veeravalli G., Couchman H. M. P., 2008, MNRAS, 387, 427 Weinberg D. H., Davé R., Katz N., Hernquist L., 2004, ApJ, 601, 1 White S. D. M., Frenk C. S., 1991, ApJ, 379, 52 3054 M. Vogelsberger et al. White S. D. M., Rees M. J., 1978, MNRAS, 183, 341 Wiersma R. P. C., Schaye J., Theuns T., Dalla Vecchia C., Tornatore L., 2009, MNRAS, 399, 574 Zel’Dovich Y. B., 1970, A&A, 5, 84 A P P E N D I X A : VO RO N O I M E S H S TAT I S T I C S Here we present some basic geometric properties of the Voronoi mesh in our cosmological simulations. As discussed in Section 2, it is beneficial for the accuracy of AREPO for the Voronoi mesh to remain as regular as possible throughout the simulation. To this end, the motion of the mesh vertices is slightly modified for highly distorted cells in order to obtain a mesh that has more ‘roundish’ cells and is closer to a centroidal Voronoi tessellation where geometrical the centres of the cells coincide with the vertex points of the individual Voronoi cells. We can quantify the quality of a mesh in a similar way as was done in S10. There, mesh quality indicators were calculated for a non-radiative simulation of a massive cluster (the ‘Santa Barbara Cluster’, Frenk et al. 1999), and it was demonstrated that the mesh indeed preserved its desired properties of reasonably round cells during the simulation. However, in this paper we have used a modified regularization scheme based on the maximum face angle, and we include cooling, star formation and feedback, which drastically increases the dynamic range the simulations need to address. It is therefore important to verify whether our regularization scheme performs sufficiently well in these more demanding simulations. In Fig. A1, we first show in the top left-hand panel the maximum face-angle distribution at z = 0 for all AREPO simulations. Our new regularization scheme should guarantee that these angles do not get too large. We have set up the regularization such that the mesh is steered towards face angles smaller than 1.68. As the upper left-hand panel of Fig. A1 demonstrates, the mesh indeed avoids very large face angles with a maximum in the distribution around the desired value, i.e. the regularization works as expected. The remaining three panels of Fig. A1 show distribution functions for the number of faces of the Voronoi cells, for the distance d of mesh-generating points to the geometric centres of their cell in units of the fiducial 1/3 cell radius √ R = (3V /4π) of each cell, and finally, for the η = S 3/2 /(6 πV ) parameter which measures how ‘roundish’ each cell is (here S is the total surface area of a cell). The corresponding distributions for a random Poisson sample of 1283 vertex points are shown as black lines in all panels, for comparison. Note that we do not explicitly control the offset d in our regularization scheme, but the face-angle criterion is of course correlated with this quantity. This allows us to obtain also quite small values for the distance of the geometric centre of the cell from the location of the meshgenerating points. This is important for the numerical accuracy of the linear reconstruction. Overall, our Voronoi mesh is significantly ‘rounder’ and much closer to a centroidal Voronoi tessellation than the mesh of a random point distribution. The goal of our re- and derefinement strategy is to keep all cell masses close to a pre-defined target cell mass, which we have chosen to be the total baryonic mass in the box divided by the total number of cells at the initial time. In this way, a narrow mass spectrum for the formed stellar particles is obtained, and a straightforward and even-handed comparison to the SPH calculations done with GADGET becomes possible. As we have described in Section 2, we guarantee the approximately constant mass resolution by splitting cells that reach a mass larger than 2 × mtarget into two cells, while cells dropping in mass below 0.5 × mtarget are dissolved. We note that thanks to the Lagrangian mesh motion in AREPO, these re-/derefinement Figure A1. Properties of the Voronoi mesh at z = 0 for the different AREPO runs. The black line shows the characteristics for a mesh where 1283 vertex points are randomly distributed. Our implemented regularization scheme tries to keep the maximum face angle small to avoid very distorted Voronoi cells. This can clearly be seen in the upper left-hand panel, where the random distribution deviates significantly from the regularized simulation distributions. The regularization also affects the offset between geometrical centre of the cell and the vertex location shown in the bottom left-hand panel. It is important to avoid too large offsets here since they introduce inaccuracies in the linear reconstruction step of the MUSCL-Hancock scheme. The ‘roundness’ of the cells is measured by η, which is closer to 1 (the value of a sphere) for the regularized simulations. Again, this is a measure of the regularity of the Voronoi mesh. For each face of a Voronoi cell, AREPO needs to solve a Riemann problem. Since each face is only solved once, the average number of Riemann problems per cell is given by half the average number of its faces. The distribution of Voronoi faces per cell is shown in the top right-hand panel. operations are invoked only rarely. In the top panel of Fig. A2 we demonstrate that the re- and derefinement scheme successfully keeps the cell masses in the desired mass range, yielding approximately a log -normal distribution around the target mass within the desired bounds. The mass distribution of stellar particles at z = 0 is shown in the bottom panel of Fig. A2. Since our star formation implementation does not allow star particles to be formed with a mass larger than 2 × mtarget , the stellar particle mass distribution is cut off at that value. On the other hand, the derefinement operations are constrained by the requirement that neighbouring cells cannot both be derefined in the same time step. As a result, it is possible that some cells can temporarily have masses below 1/2 × mtarget for a few time steps, and hence also some star particles with masses smaller than this value may form in principle. To protect against the formation of unreasonably low-mass star particles, which may be prone to two-body effects, we however do not allow cells with mass less than 1/4 × mtarget to form any star particles, imposing a lower limit in the star particle mass distribution. As the bottom panel of Fig. A2 shows, star particles with masses between 1/4 × mtarget and 1/2 × mtarget make up only a tiny fraction, as desired, such that the suppression of the formation of extremely small star particles does C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology 3055 A P P E N D I X B : G R AV I TAT I O N A L S O F T E N I N G A N D R E G U L A R I Z AT I O N S C H E M E The AREPO simulations presented in this paper use an adaptive gravitational softening length with a lower floor for the gas, whereas the SPH simulations done with GADGET employ a fixed comoving softening length equal to the lower floor. We checked that this does not bias our results in any way. To demonstrate this point we show in Fig. B1 the SFH of the standard A_L20n128 simulation together with a run where we held the gravitational softening of the cells fixed at the same value as in the SPH calculations. Clearly this leads only to very minor differences in the SFH. We also checked that disc half-mass radii are not affected by this. As stated above this is due to the fact that star-forming gas is pressurized by an effective equation of state of the ISM and therefore gravitational softening effects do not play an important role in that regime. The mesh regularization in AREPO can be done in different ways and we have used a scheme that is based on the maximum face angle as described above. We have also done simulations with the original regularization scheme presented in Springel (2010a), which is based on the displacement of the geometric centre of the cell. The resulting SFH for the A_L20n128 is also shown in Fig. B1. Again this leads only to minor changes. We note that this difference becomes smaller with resolution and essentially vanishes. Regularization scheme differences are only relevant for very low resolution simulations. Disc sizes are also not significantly affected by the details of the regularization scheme. In Fig. B2, we show gas density maps of a matched object in A_L20n128 and the A_L20n128 run, which features an alternative regularization scheme. We stress again that L20n128 is our lowest resolution set-up, and therefore most sensitive to details of how exactly the mesh is treated. But even in this regime the differences in the gas density maps are very small, and the size and overall extent of the gas disc do not change. Note that due to the low resolution of L20n128, individual Voronoi cells are clearly visible in the map, highlighting the changes in Figure A2. Top panel: distribution of cell masses divided by the target gas mass for each AREPO simulation at z = 0. Our re-/derefinement scheme guarantees that the mass of a Voronoi cell never deviates by more than approximately a factor of 2 from the target gas mass, which is in our case chosen to be the mean baryon mass of a cell for a uniform distribution assuming that all cells have equal volume. Bottom panel: distribution of masses of the stellar particles divided by the imposed target gas mass for the Voronoi cells. Our constraints for the mass of cells and our scheme for creating star particles result in a quite narrow mass distribution of stellar particles, where each stellar particle deviates at most a factor of ∼2 from the desired target gas mass, except for a tiny number of lighter star particles, whose mass is however always larger than a quarter of the target gas mass. not lead to any appreciable error. Note that star-forming cells with a mass above 2 × mtarget will form stars with exactly mtarget , keeping the rest of the gas mass in the cell. This explains the small spike in the distribution at that mass value. Our analysis thus confirms that star particles with a narrow range of masses are formed, which helps to limit two-body relaxation effects and is well matched to the fixed gravitational softening we use for collisionless particles. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Figure B1. Star formation rates per unit volume as a function of lookback time (top left) and redshift (bottom left). The panels on the right-hand side show the corresponding integrated star formation rates as a function of lookback time (top right) and redshift (bottom right). We show the result of the A_L20n128 simulation along with a the same run with a fixed (comoving) gravitational softening length and a run with a different mesh regularization scheme. All runs show very similar star formation rates. 3056 M. Vogelsberger et al. Figure B2. Gas density maps of a matched galaxy of the simulation A_L20n128 and a similar simulation with a different mesh regularization scheme (based on the displacement of the geometric centre of the cell). As expected, the detailed mesh geometry and the shape of cells change if we employ a different regularization scheme. But the resulting density field looks similar, and especially the overall extent of the disc is not significantly affected by the way the mesh is regularized. We note that L20n128 is our lowest resolution and therefore most sensitive to details of how exactly the mesh is treated. These effects are significantly smaller for our intermediate resolution and nearly vanish for the highest resolution simulation. mesh geometry and cell shapes induced by different regularization schemes. APPENDIX C: CODE PERFORMANCE The total runtime of our AREPO simulations is in the worst case only about 30 per cent longer than the corresponding GADGET run at the same nominal resolution. Fig. C1 shows that in which code parts most of this CPU time is spent. Obviously, the rather complex Figure C1. Cumulative time spent in different parts of the code as a function of time step for AREPO (top panel) and GADGET (bottom panel). The Voronoi mesh contribution of AREPO includes mesh constructions and mesh updates. The total number of time steps in AREPO is somewhat smaller because a larger CFL factor can be used, which makes the smallest time step of AREPO larger than that of GADGET. The remaining fraction of computing time is spent in the time integration, star formation, cooling and other minor contributions. mesh construction and mesh update in AREPO take up a significant amount of time. However, in both codes the Tree-PM-based gravity calculation consumes a large amount of time, too, alleviating speed differences in the hydrodynamical solvers. In any case, we argue that AREPO is actually surprisingly fast given the extremely complex mesh management operations that are required. Given the small difference in raw speed, one can rightly describe the overall performance of both codes as similar. However, accounting for the fact that we here find that SPH leads to a systematic and significant offset in the cooling rates of haloes at late times, as well as in galaxy sizes, the discussion of CPU time requirements at the same nominal resolution is a bit moot. Ideally one would like to select the fastest method for a given desired accuracy, and as it appears, this can be reached with AREPO more efficiently, whereas standard SPH’s systematic bias may be difficult or even impossible to overcome. The white parts in Fig. C1 account for the time spent in the time integration, star formation, cooling and other minor contributions. We note that the I/O contribution is a bit larger than typical, because we wrote out snapshots at a very high frequency. A P P E N D I X D : S T RO N G A N D W E A K S C A L I N G In Fig. D1, we show a strong scaling test of AREPO, from 32 to 512 cores, carried out on Ranger at the Texas Advanced Computing Center (TACC). Here, the simulation size has been kept constant at 2 × 2563 , and only the number of compute cores has been increased. The reported times have been averaged for three full time Figure D1. Strong scaling test for the AREPO code on Ranger, using between 32 and 512 cores. In all runs an identical simulation size of 2 × 2563 in a 12.5 h−1 Mpc box was used, with a 5123 FFT for the long-range gravity calculation. The black solid line shows the total wall-clock time for a full step as a function of the number of cores used, averaged over three steps at high redshift. The code was run with all physics enabled, and all code overhead was included in these averages. The other measurements included in the figure give the times for the most important individual parts of the code, which are the Voronoi mesh construction, the short-range and longrange gravity calculations, the hydrodynamical flux calculation, the domain decomposition and the tree construction. The black dashed line indicates the ideal strong scaling. C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society Moving mesh cosmology steps at high redshift. In this regime, gravity accounts for ∼33 per cent of the computational time, mesh construction and mesh bookkeeping consume ∼40 per cent of the time, while the calculation of the hydrodynamical fluxes amounts to ∼10 per cent of the time. The remainder is needed for miscellaneous items such as domain decomposition, radiative cooling and star formation. In order to clearly show the scalability of the most important different parts of our code, we have included separate measurements in Fig. D1 for long-range gravity (by means of FFTs), short-range gravity (done through a tree walk), the necessary tree construction, the mesh construction and management, the hydrodynamic flux calculations, and the domain decomposition. We see that the code shows quite good strong scaling, despite the tightly coupled nature of the system. Some losses are in particular apparent for the mesh construction. These arise because the more cores are used, the larger the number of spatial domains in which the simulation volume is cut. This enlarges the surface area of domain boundaries, and as a result the cumulative ‘ghost’ region volume in which the mesh has to be constructed twice on two neighbouring processes to ensure seamless consistency across the domain boundary. For large-scale cosmological applications it is more important that the code parallelization shows good weak scaling behaviour. GADGET has been used for many large-scale simulations and we will demonstrate here that AREPO performs also well in weak scaling tests. For these tests the computational load per core is kept constant, and the simulation volume and particle/cell number is scaled accordingly. In Fig. D2, we show two different weak scaling series carried out on Ranger at the TACC. They differ in their particle and cell load per core. In the results shown in the top panel, this load was ‘maximal’, corresponding to ∼1 million cells and particles per core, where the current version of AREPO requires up to 1600 MB memory consumption per core in the peak. Because some memory is needed for the operating system and the MPI communication library, not all of the physical memory is available for the application code on the Ranger platform. Unfortunately, the amount of memory consumed by the MPI subsystem increases with increasing size of the MPI partition. This in fact prevents us from running the ‘full load’ configuration for partitions larger than 4096 cores. This also prompted us to create a second weak scaling series shown in the bottom panel of Fig. D2, where we reduced the load by almost a factor of 2. This allows us to scale AREPO up to 6912 cores with 6912 MPI tasks. It is apparent that the short-range gravitational tree calculation, the Voronoi mesh construction and the hydrodynamical flux calculation show excellent weak scaling (which runs horizontally in the plots of Fig. D2). However, there are deviations from perfect scalability for the FFT-based long-range gravitational force calculation, the domain decomposition and the tree construction. This is primarily because all three of these parts involve substantial all-toall communication. For larger processor counts, especially for 6912 cores, these communication costs start to affect the weak scalability of our code and lead to losses in efficiency. One reason for the suboptimal scaling in this regime lies in the FFT part of the code. The slab-based parallelization of the FFTs we use for the long-range gravity calculation does not scale to core numbers larger than the C 2012 The Authors, MNRAS 425, 3024–3057 C 2012 RAS Monthly Notices of the Royal Astronomical Society 3057 Figure D2. The two panels show weak scaling plots for the AREPO code on Ranger, using from 32 up to 6912 cores. In all runs of each of the two series, the load per core was held constant, and the simulation box size and FFT size used were scaled in proportion to the core number. For ideal weak scaling, the code speed in all of its parts should then be constant, despite the fact that the simulation size grows by a factor of more than 200 in these tests. We show in black the total times averaged for three full steps, and with different colours the most important code parts, as labelled. The top panel refers to a series with ‘fully loaded’ nodes, where we use a load that is close to the maximum we can fit into the memory available per Ranger core. In the bottom panel, we have reduced the load by a factor of 2, allowing us to extend the tests to a core count of 6912 (for such large MPI partition sizes, less application memory remains available on the specific machine). Here, however, the communication costs in three parts of our code become quite substantial and lead to a noticeable negative impact of the scalability. The particle load of the top panel goes from 2 × 2563 to 2 × 12803 , whereas for the bottom panel it goes from 2 × 2143 to 2 × 12803 . The black dashed line indicates the ideal weak scaling. size of the FFT, a regime we enter once more than 4096 cores are used (because the FFT size reaches 40963 at this point and cannot be grown further due to memory constraints). We note that this can easily be optimized by either using a block-structured FFT decomposition or by using a mixed approach for the parallelization with either threads or OpenMP. Overall we find that AREPO shows good weak scaling behaviour and can readily be applied to large-scale cosmological hydrodynamics simulations. This paper has been typeset from a TEX/LATEX file prepared by the author.