January 14 marked the debut of HECToR, the UK’s most advanced HPC resource. HPCwire asked Jane Nicholson, head of High End Computing for the Engineering and Physical Sciences Research Council, how the big Cray XT4 system will serve the UK’s academic research community.

HPCwire: What is HECToR, and how important is it for the UK?

Jane Nicholson: HECToR stands for High End Computing Terascale Resource. It’s the flagship academic computing service for the UK and is available to all scientific and engineering researchers in the UK academic community. HECToR is the latest in a line of academic computing services in the UK. The prior service was called HPCx. HECToR is an important part of the UK research infrastructure. An up-to-date computing facility adds to the resources academics have access to in universities.

HPCwire: What role does your organization, the Engineering and Physical Sciences Research Council, play with respect to HECToR?

Nicholson: We’re one of seven research councils in the UK. EPSRC serves as the agent for procuring High End Computing academic services for three of the research councils: EPSRC, BBSRC and NERC. We run the procurement and provide a large share of the funding. EPSRC was formed in 1993 as an evolution of a previous research council called the Scientific and Engineering Research Council. Together, the seven UK research councils are roughly equivalent to America’s NSF and NIH. The EPSRC itself is governed by a council consisting of leading researchers and industrialists in the UK. They act like a board of directors. Officially, the EPSRC is a nondepartmental government body. In sum, we’re a government body that serves as a funding agency.

HPCwire: The initial system is a 63-peak-teraflop Cray XT4 that will morph in stages to a 250-teraflop system. Can you shed more light on that?

Nicholson: First, the Cray XT4 system is fully installed and working. It was installed in the August 2007 period, passed acceptance tests in September, became available to early users in that same month and to all users from 1 October 2007. Our agreement with Cray includes three phases. Phase 1 is the current 63-teraflop Cray XT4 system. Phase 2 is an upgrade to 250 peak teraflops in two years. Phase 3 is an optional upgrade two years after that to a substantially larger system.

HPCwire: Talk about the academic research community that will have access to the HECToR resource.

Nicholson: The prior HPCx service was based on a 15-teraflop computer that is still installed and has about 80 different research consortia working on it. Some have already moved to HECToR and we expect most to do so. Some of these groups are large, and others are small. There are just under 1,000 users on the current service. The vast majority of these are UK academics, but of course many of them are involved in collaborations with researchers all over the world.

Their disciplines range from the physical sciences and engineering to biology, biomedicine, environmental sciences and others. There is also research activity at and across the boundaries of disciplines, such as modelling blood flow in the heart. This brings together medical and biological researchers with physicists, and requires the use of CFD and structural analysis for the fluid-structures interactions.

The service focuses on larger, more complex problems needing access to a computer of this size. Some problems, such as turbulence studies, use 20-25 percent of the system.

HPCwire: What are the main challenges in serving a user community this broad and diverse?

Nicholson: The real challenges begin up front with specifying the service so that it will meet a wide range of user and job requirements and strike the right balance between compute, memory and other resources. Once the system is operational, we have rules for how the system is accessed: queue sizes, lengths of jobs, and so on. Prioritization is based on the science.

HPCwire: To what extent will HECToR be used for EU collaborations?

Nicholson: A small portion of HECToR will be allocated for DEISA, the Distributed European Infrastructure for Supercomputing Applications. European researchers outside the UK will have access to HECToR, and in return UK researchers will gain access to other DEISA resources in Europe. The DEISA goals are for each researcher to be able to use the most appropriate resource, no matter in which country it’s located, and to promote collaborations within the EU.

HPCwire: Collaborating with industry is part of the charter for many government-funded HPC centers around the world, but this doesn’t always happen. Does it in your case?

Nicholson: The main focus of our service is academia, but all the UK research councils are required to make an economic impact as well. We do see collaborations with industry in several areas on our current service. For example, academic researchers collaborate with the aerospace industry on engineering and modelling work related to turbulence, combustion, etc. There is also some collaboration with the pharmaceutical industry for drug delivery and drug modelling.

HPCwire: I assume the UK approach to providing academic researchers with HPC resources differs in some ways from the approaches used in the U.S., France or Germany. What’s distinctive about the UK approach?

Nicholson: Well, it may be that in our case the funding agency rather than a computing center procures the hardware service, oversees the competition for the host site, and also provides computational science and engineering support. We also ensure that there is a source of expertise for the users to help them develop their codes and solve any problems they may have. For the HECToR service, NAG [Numerical Algorithms Group] will provide this support. This could employ up to 20 people per year.

HPCwire: How did you gather user requirements for the HECToR procurement?

Nicholson: The process involved many discussions and meetings with the research community. We established a small working group to prepare the specs. We held community-wide workshops. We asked researchers what they would like, and what science would be enabled. When we move from one generation of the service to the next, our goal is to make the resource four times larger. The computer for the HPCx service was about 15 teraflops, and HECToR is over 60 teraflops, so we are meeting our goal. The requirements for the service are driven by the science we can enable.

HPCwire: What benchmarks did you run?

Nicholson: We collected and ran five user codes that we feel are representative of the workloads of our user base. We ran the HPC Challenge benchmarks. Our goal is to enable science, as opposed to focusing only on overall compute power.

HPCwire: Are you satisfied with the benchmarks?

Nicholson: They worked well.

HPCwire: Why did you choose Cray?

Nicholson: Cray’s proposal performed best in relation to our evaluation criteria, including the benchmark codes and value for the money. When we did the procurement, we looked for the whole package from the vendors we evaluated: phase 1, phase 2 and the potential phase 3 offerings, along with their approach to maintenance, their operations, etc., because we’re providing a comprehensive service.

HPCwire: Was Cray’s hybrid computing roadmap an attraction?

Nicholson: Yes. We’ll get some vector capability from Cray later this year. We have a diverse research community, and this will allow us to see which technologies are best on which problems. We can have more vector later if we wish.

HPCwire: Did you provide access for your user community all at once or in stages?

Nicholson: The Phase 1 kit was shipped in two batches during August. It was operating and ready for acceptance testing in the second week of September. Early users were able to access it in September, a couple of weeks ahead of schedule. And the system was fully available on 1 October 2007.

HPCwire: What do you see as the largest challenges the worldwide HPC community faces today?

Nicholson: I see two of them. First, the power costs of running increasingly large systems, along with related environmental issues. The cost of power went up 40 percent in the UK over the last few years. The second challenge is software development to exploit systems that are growing much larger and use multicores.

HPCwire: Final question. What do you hope will come out of the HECToR program?

Nicholson: We are investing in the HECToR facility so researchers can do exciting science and engineering — that means leading science publications that contribute to the advancement of knowledge. We also expect an economic impact that will benefit UK industry and the public sector.