UK HPC Service: from HECToR to ARCHER
ARCHER is the next generation national high performance computing (HPC) service for academic research in the UK – a follow-on to the High-End Computing Terascale Resource (HECToR) project, which has been in existence since 2007.
The hardware for the new service consists of a Cray XC30 supercomputer and a Cray Sonexion storage system, part of a $30 million contract between Cray and the Engineering and Physical Sciences Research Council (EPSRC) at the University of Edinburgh in Scotland.
As ARCHER comes online, an article on the Edinburgh Parallel Computing Centre (EPCC) explores what it takes to bring an HPC service from concept to implementation, while ensuring a smooth transition for users between services.
The new Cray XC30 system, with 1.56 petaflops of theoretical peak performance, provides nearly four times the computational throughput of its predecessor HECToR, a Cray XE6 supercomputer. This performance improvement can be traced to two primary upgrades: the next-generation Aries interconnect and Intel’s latest Xeon Ivy Bridge multicore processors.
The Intel Xeon processors used in ARCHER provide best-in-class floating-point performance, memory bandwidth and energy efficiency, according to the article. Each ARCHER node sports 12-core 2.7 GHz Ivy Bridge multicore processors and a minimum of 64 GB of DDR3 main memory. Compute nodes are interconnected via an Aries Network Interface Card. ARCHER has 3,008 such nodes for a total of 72,192 cores, housed in just 16 cabinets. The system also includes 20 Cray Sonexion Scalable Storage Units, which contribute 4.4 petabytes of accessible space and sustained read-write bandwidth of over 100 GB per second.
A direct connection between ARCHER and the UK Research Data Facility makes it easier for users to transfer large data sets between high-performance scratch space and long-term archival storage. The Data Facility also supports the transition from HECToR to ARCHER.
The ARCHER project dates back to Autumn 2011, when the UK Minister for Science announced a new capital investment initiative that included £43 million for an upgraded national HPC facility. Work to expand the University of Edinburgh’s Advanced Computing Facility (ACF), where ARCHER would be housed, started in May 2012.
The ACF renovation included the addition of 500m2 of computer room floor space, and an additional 760m2 plant room to house the necessary electrical and mechanical systems. The plans called for new high-efficiency 4MW-capacity cooling system and an upgrade to the site’s private high-voltage network, increasing capacity to approximately 7MW.
The entire renovation project was completed by the end of 2012, two months ahead of schedule and under budget. In July 2013, Cray announced its role as hardware partner and the supercomputer and storage systems were delivered and installed in September 2013. Acceptance testing was successfully completed in late October and core users were brought online in November.
ARCHER opened for full user service on December 16th. Investigators who wish to transfer their allocation from HECToR to ARCHER are invited to contact the ARCHER Helpdesk.
Other parties who wish to gain access to ARCHER can find additional information on the project’s website. The HECToR Service will be turned off in March 2014.