Researchers working with the Oak Ridge Leadership Computing Facility’s (OLCF’s) Titan supercomputer are making the most of the system’s hybrid design, which pairs traditional CPUs with highly-parallel GPUs. As the fastest computer in the United States, Titan has a max theoretical speed of 27 petaflops, but as a recent OLCF article points out, “Titan is only as powerful as the applications that use its unique architecture to solve some of our greatest scientific challenges.”
“The real measure of a system like Titan is how it handles working scientific applications and critical scientific problems,” reports Buddy Bland, project director at the OLCF. “The purpose of Titan’s incredible power is to advance science, and the system has already shown its abilities on a range of important applications.”
To help users extract the highest benefit from the multi-million dollar machine, the OLCF understands that having the right application set is essential. With this in mind, they launched the Center for Accelerated Application Readiness (CAAR) two years before Titan’s scheduled arrival. The program brings together staff from Cray and NVIDIA, who had helped to build Titan, with application developers and OLCF’s scientific computing experts. The team identified a set of six target applications based on their benefit to science and predicted performance and/or fidelity gains on Titan. Because these application codes were originally developed for CPU-based machines, the team was tasked with modifying the codes so they could fully exploit the power of GPUs at scale. The goal was to enable productive work to begin as soon as Titan passed acceptance.
Winnowed to five codes, the final application set includes the following:
- S3D – provides insight into chemistry-turbulence interaction of combustion processes.
- LSMS – used to compute the magnetic structure and thermodynamics of magnetic structures.
- LAMMPS – a popular molecular dynamics application that provides understanding of molecular processes such as cellular membrane fusion.
- Denovo – used to model radiation transport for reactor safety and nuclear forensics.
- CAM-SE – used in climate climate change adaptation and mitigation scenarios; accurately represents regional-scale climate features of significant impact.
OLCF holds that Titan’s design will have a profound effect on simulation speed. Running the climate change application CAM-SE on the GPU-accelerated Cray XK7 system will enable 1 and 5 years per computing day, compared to just three months per computing day on Jaguar, Titan’s Cray XT5 predecessor. “This speed increase is needed to make ultra-high-resolution, full-chemistry simulations feasible over decades and centuries and will allow researchers to quantify uncertainties by running multiple simulations,” observes the lab’s science writer Scott Jones.
Going forward, OLCF expects the S3D code, in addition to modeling simple fuels, will be able to take on more complex, larger-molecule hydrocarbon fuels and biofuels, paving the way for advanced internal combustion engines with higher energy efficiency yields.
The molecular dynamics code LAMMPS has also benefited from the Oak Ridge supercomputer, especially the system’s GPU parts. As a code that simulates the movement of atoms through time, LAMMPS is a staple application in biology, materials science, and nanotechnology. When GPUs were added to Titan, LAMMPS achieved a seven-fold performance speedup compared with an earlier CPU-only version of the machine.
Another CAAR application, called WL-LSMS, has also proven to be a good match for the hybrid Titan supercomputer. Used to study candidate magnetic materials, WL-LSMS ran 3.8 times faster on a GPU-enabled Titan than its CPU-only predecessor. The run harnessed an astounding 18,600 of Titan’s compute nodes, out of a total 18,688. Equally as significant, the GPU-enabled version of Titan used 7.3 times less energy than the CPU-only system. The result of faster performance with reduced energy consumption was considered a big win since these tandem benefits were cited as the main motivators for the move to GPU-accelerated supercomputing.
OLCF reports that all of the CAAR applications have seen significant speedups using Titan’s GPUs. As the rest of Titan’s users begin the process of ramping up their codes to make use of Titan’s massive core count, they can rely on the guidelines and practices established by CAAR.