Oak Ridge National Lab, which already hosts two petascale supercomputers, is planning to add another to its elite stable of HPC machines. According to a news report in the Knoxville News Sentinel, the DOE lab will begin installing a new 20 petaflop supercomputer, named “Titan,” in late 2011, with the complete system ready to boot up sometime in 2012.

According to Jeff Nichols, the associate lab director for ORNL’s Computing and Computational Sciences group, the initial 2011 installation will be used as a test-bed before the full system is in place. When installed, Titan will dwarf the lab’s current top-tier number-crunchers, the 2.3 petaflop Jaguar and the 1.0 petaflop Kraken, by a hefty margin. Both of those systems are Cray XT5 machines.

As you might expect from Oak Ridge, Titan will also be a Cray supercomputer, in this case, a yet-to-be-released GPU-accelerated machine that will use NVIDIA Tesla parts to deliver most of the FLOPS. Although the Knoxville News Sentinel report didn’t specify the actual system, it will likely be an XE6 variant with Tesla GPU-equipped blades, which Cray has said it will launch later this year.

Nichols told Knoxville News that entire system will cost about $100 million. That is probably going to be quite a bit less expensive than the DOE’s other 20 petaflop system: the IBM Blue Gene/Q “Sequoia” supercomputer. The price tag for that machine hasn’t been disclosed, although the 10 petaflop IBM Blue Waters system at NCSA will run more than $200 million.

Like Titan, Sequoia is slated for initial delivery later this year, with the full set-up completed in 2012. That should make for any interesting match-up. For one thing, unless the Chinese fund another big system next year (which is certainly not out of the question), the two DOE machines will vie for supercomputing supremacy in 2012.

Assuming both machines deliver 20 peak petaflops, it’s more likely that the Blue Gene/Q Sequoia will take the TOP500 (Linpack) title that year. Like most top-of-the-line CPU-based supers, Blue Genes yield something north of 80 percent of peak FLOPS for Linpack, while the GPU accelerated machines are only delivering 50 percent of peak on Linpack. Maybe yields will be better by 2012, but probably won’t match the CPU-based supers.

For a number of reasons, the Sequoia machine is also more likely to deliver a better performance per watt metric than Titan. In the latest Green500 list, a Blue Gene/Q prototype was about 75 percent more efficient than the third ranked TSUBAME 2.0 GPU-equipped super. (The number two Green500 system was a special-purpose GRAPE-DR supercomputer). Some of that has to do with better Linpack performance as mentioned above, but the Blue Gene technology, in general, tends to be quite energy efficient thanks to its custom integration and SoC PowerPC architecture.

The performance match-up will be for bragging rights only since the two DOE machines won’t really be competing for applications. Sequoia’s primary duty will be to run classified nuclear weapons simulations for the NNSA’s Stockpile Stewardship program, although it will be available part-time for science apps like astronomy, energy, genomics, and climatology.

Titan, on the other hand, will be dedicated to running a wide variety of open science application. Presumably Titan will also end up in the INCITE (Innovative and Novel Computational Impact on Theory and Experiment) program, which means a number of academic and commercial users will get a chance to play on the first multi-petaflop GPU supercomputer in the US.