Download Upgrade for DiRAC Investment in e-infrastructure to upgrade the

Upgrade for DiRAC
Investment in e-infrastructure to upgrade the DiRAC facility will have a beneficial impact on the growth
agenda almost immediately, through the development of skilled people, cutting edge science attracting the
best researchers, inspiring students to study science and enabling new and existing industrial collaborations.
An analysis has already been undertaken of the science supported by the STFC DiRAC facility which will
require upgrades to its e-infrastructure over the next 5 years. A diversity of architectures is essential for the
DiRAC community because of their highly specialised and optimised codes. This will ensure that the UK
does not lose its lead in a number of key scientific areas with the resultant loss of scientific competiveness
and key research staff which could not be recovered. The optimum solution for DiRAC would be to provide
a variety of central compute architectures including involvement in a large national service facility coupled
with a virtual integrated distributed facility offering a range of architectures.
Tier 1: National Service. DiRAC has a requirement for a 1 PetaFlop Cluster facility (about 4000 two
processor servers and 10000 cores) in 2012. The current national service provision, HECToR did not have
the architecture required by the DiRAC community but, providing that the architecture of any future
national service was appropriately configured, the DiRAC requirement could form part of a future
consolidated national service facility. Due to the nature of calculations, sufficient dedicated time on the
national facility would need to be guaranteed and the timescale must be competitive.
The following requirements could not be incorporated into a national service facility in a cost effective
manner because of the diverse nature of the architectures required and would thus be a mix of specialist
Tier 1 and Tier 2 facilities:
Tier 1. Extension of current IBM BlueGene Q low latency 5-d toroidal system in 2012
Tier 2. 10 Terabyte RAM shared memory solution – large scale NUMA system in 2012.
Tier 2. 100 Tflops All-to-all switching InfiniBand Cluster in 2012
Tier 2. Data Mining and analysis and visualisation (200 Tflops) – required to analyse locally the data
produced by our central facilities (2011-2014).
Data federation – the UK must invest in data centres and the long term archiving of valuable research data.
A 30-50 PB data system would allow easy movement of data between all DiRAC components (2011-2015).
At the current time, very limited resources have been devoted to data federation and this increase will
improve the utility and impact of the data produced by STFC science and is in line with the clear goal of
Government policy.
Software Development and software optimisation – it is vital to ensure that codes are optimised and
developed to ensure that all computation is as time and cost efficient as possible. This would be carried out
by DiRAC's programmer team.
Software libraries will be required to work at petascale levels and for new architectures. This would utilise
existing Centres of Excellence such as that at the Daresbury Laboratory. All knowledge gained from such
library development would be spun out into industry. This shared expertise and knowledge between
academia and industry will place the UK at the cutting edge of computer development.
Skills development - training programmes will ensure the next generation of researchers are in a position to
take full advantage of advances in e-infrastructure.
Technical development - technical development of machines such as the DiRAC collaboration with IBM in
developing the memory prefetch engineering for the next Blue Gene architecture which is a half billion
dollar project for IBM.
Assuming a decision date of 1 January, given the lead time on capital hardware purchases, no expenditure
would be anticipated in 2011/12. An earlier decision date would allow expenditure of ~ £1-2 million spend
in 2011/12.