New system prepares for transformational science

The National Institute for Computational Sciences (NICS) is the newest member of an elite supercomputing community. Dedicated on April 3, the organization — formed through a National Science Foundation (NSF) grant to the University of Tennessee and its partners — is on its way to delivering a soon-to-be petascale system that promises substantial contributions in the effort to solve the world’s greatest scientific challenges, such as understanding the fundamentals of matter and unlocking the secrets to the origin of our universe.

The system, a Cray XT4 dubbed Kraken (after a gargantuan sea creature in Norse mythology), will come online in mid-summer and is expected to feature more than 18,000 2.3GHz AMD high-performance cores delivering 170 teraflops of performance. A new Cray-designed interconnect, featuring Cray SeaStar2 chips and high-speed links, will greatly increase reliability and provide for excellent scaling while eliminating the related cost and complications of external switches.

NICS is seeking “large, tightly coupled applications,” to take advantage of the newly-designed Cray interconnect, said NICS Project Director Phil Andrews. Currently a dozen large-scale applications are poised to run at NICS, spanning a diverse range of scientific fields including climate, fusion energy, biology, lattice QCD, and astrophysics. “ENZO cosmology simulations exhibit near-ideal scaling to 8,000 cores on the XT4,” said Michael Norman, a professor of physics at the University of California, San Diego. “Clearly even larger simulations are possible. This opens up all kinds of new frontiers in understanding cosmic evolution.”

Climate also figures to play a large role in Kraken’s research potential. As climate change continues to gain prominence both in the policy and scientific arenas, powerful systems such as Kraken will play an ever-increasing role in all types of climate simulations, from CO2 cycles to the role of ocean currents. Just as previous efforts in eastern Tennessee contributed substantially to the recent Nobel Prize given to the United Nations’ Intergovernmental Panel on Climate Change, Kraken also will greatly contribute to man’s understanding of his impact on the planet.

The Cray XT4 will ultimately evolve into a Baker system featuring more than 10,000 compute sockets, 100 trillion bytes of memory, and 2,300 trillion bytes of disk space. It will provide more than 700 million CPU hours per year and one petaflops of performance, making it the nation’s most powerful academic supercomputer.

Kraken is designed specifically for sustained application performance, scalability, and reliability and will incorporate key elements of the Cray Cascade system to prepare the user community for highly productive petascale science and engineering. The Cray XT4 will continue to operate in support of users until the Baker system is in full production.

The system, and the resulting NICS organization, are the result of an NSF Track II award of $65 million to the University of Tennessee and its partners to provide for next-generation high-performance computing. The award was won in an open competition among high-performance computing (HPC) resource institutions vying to facilitate America’s continued competitiveness via the next generation of supercomputers.

The NSF used a series of system performance-related benchmarks as a key factor in the selection process, setting the stage for the future of simulation research by employing a system that is usable and reliable and well-suited to computationally-intensive scientific issues, such as protein shape and function and climate modeling.

As the foundation for NICS, (a collaboration of universities, research institutions, and HPC industry leaders), the new system will be fully linked to the NSF-supported TeraGrid, a network of supercomputers across the country that is the world’s largest computational platform for open scientific research.

The NSF award places the University of Tennessee among a select group of supercomputing facilities, including the University of Illinois at Urbana-Champaign and the Texas Advanced Computing Center, likewise an NSF-funded facility. Due to the collaborative relationship between the University of Tennessee and Oak Ridge National Laboratory, NICS promises to deliver state-of-the-art scientific research.

For instance, a team led by Erik Schnetter of Louisiana State University is seeking to understand the merger of binary black hole systems through mesh refinement and multi-block methods and numerical and Einstein-based equations. In particular, these heavily computational simulations will focus on the spins, velocities, and masses of black holes in binary systems.

Another team, led by Carlos Simmerling of the State University of New York at Stony Brook, is seeking to increase our knowledge of biomolecular structure and dynamics. The project’s simulations are shedding light on areas such as possible drugs for the treatment of tuberculosis, the reasons for drug resistance in HIV/AIDS, and the biological role played by the anti-cancer drug Taxol.

Other projects explore galaxy formation and the properties of nanostructures, just to name a few.

“Combined with the more traditional approaches of theory and experiment, scientific computation is a profound tool for insight and solution, as researchers move their problems for modeling and simulation from existing terascale systems to petascale systems later this year and onward to exascale (quintillion calculations per second) systems in the next decade,” states Thomas Zacharia, vice-president for science and technology at UT and the associate lab director for computing and computational sciences at Oak Ridge National Laboratory.

Allocations on the NICS system may be requested via the TeraGrid proposal form. Details about the types and sizes of awards are found at Teragrid Allocations and Accounts (http://www.teragrid.org/userinfo/access/allocations.php), or by calling TeraGrid (toll-free at 1-866-907-2383). NICS is currently fielding requests for projects that will make effective use of more than 10,000 cores for capability jobs. Due to the fact that Kraken is an NSF-funded system, all open science research in the United States is valid for consideration. However, those overseas are also welcome to apply if they are currently working with a researcher based in the United States.

NICS offers researchers a great opportunity to begin to port and scale code on a system that will ultimately move to the petascale. For more information, visit the NICS Web site at www.nics.tennessee.edu.

—–

Source: University of Tennessee