Despite all the recent fanfare about the latest CPU wonderchips from Intel, AMD and IBM, not everyone has hopped aboard the multicore train. In a recent column in Forbes, NVIDIA chief scientist, Bill Dally, argues that the traditional multicore implementation of Moore's Law is a dead end. He sums it up thusly:

To continue scaling computer performance, it is essential that we build parallel machines using cores optimized for energy efficiency, not serial performance. Building a parallel computer by connecting two to 12 conventional CPUs optimized for serial performance, an approach often called multi-core, will not work.

The fact that Bill Dally is saying this should come as no surprise. He works for a GPU maker after all, so his view of the computing landscape is from a rather particular vantage point. In his commentary, he only mentions GPUs once, but the subtext of GPUs as the savior of Moore's Law is palpable enough.

In fact, his main point is valid, and one that been recognized for years: CPU power scaling, which enabled performance increases at a constant level of wattage, is over. The workaround is multiple cores, but since CPU cores are optimized for serial work, there is a built-in inefficiency when trying to mold highly-parallel codes around this architecture.

The reasoning is a little bit more subtle than that. Multicore CPUs are generally fine for traditional task parallelism, where each thread more or less can operate independently. CPUs, however, are less adept at data parallelism, and that's where GPUs really shine. The other side to this is that task parallelism usually doesn't scale well (or easily) as the size of the problem grows. Data parallelism, on the other hand, is relatively easy to scale.

To keep Moore's Law-type scaling viable for applications, Dally says that we need to build throughput computers made up of many simple cores. That just so happens to coincide with the GPU model, but other manycore processors from companies such as Tilera and Tensilica also fit this architectural style. The Larrabee architecture was Intel's first attempt to build a true throughput computer, with x86 as the starting point. That didn't quite work out as they planned, although you can bet the chipmaker will take another run at this.

Beyond the construction of throughput computers, Dally believes the real challenge will be converting the huge bulk of existing serial apps to run in parallel. Here's my take on this is: don't bother. Most serial programs are serial for a reason. For example, the text editor I'm using to compose this article is about as fast as I need it to be. Outside our particular HPC community, there are plenty of apps in this category.

Most of the killer apps for throughput processors have yet to be designed, much less implemented. A next-generation word processor that converts my English to German on the fly and simultaneously suggests Web references to what I'm writing about will be able to take advantage of throughput processors. And that's a fairly trivial example. Companies like Intel and NVIDIA are betting the "3D Web" will be one of the big playgrounds for these highly parallel applications.

Meanwhile, back in Fermiville...

Whether intentional or not, Dally's Forbes commentary last week served as an interesting precursor to NVIDIA's slow-motion rollout of the company's new Fermi Tesla 20-series hardware. NVIDIA quietly posted the specs for the new products on its Web site on Tuesday, even though volume production of the processors is not expected until late May. The GPU maker's fab partner, TSMC, is having problems with yields for the new 40nm chips -- not too surprising considering Fermi sports around 3 billion transistors for the high-end parts.

In fact, NVIDIA has scaled back the core count on the first batch of Tesla GPUs. Back in September the company was talking about 512-core Fermis, but the first Tesla silicon will come in with just 448 cores (not quite twice the 240 cores of the previous 10-series). They've also throttled the clock frequency a bit to keep the heat manageable. Even at that, the new Tesla chips suck plenty of power -- 225 watts TDP, to be precise.

But for that wattage, you get 515 gigaflops double precision and over a teraflop of single precision. EM Photonics benchmarked the new Fermi GPUs using DGETRF (a double precision LAPACK routine) and demonstrated a three-fold performance increase over the previous generation GPUs. In a real-world application, Artemis Capital Asset Management demonstrated a performance boost for certain financial analytics codes with the new Fermi GPUs. "The new cache structure in combination with the huge number of processor cores provides excellent resources for high-frequency trading," said Tobias Preis, managing director of Artemis Capital Asset Management.

Despite the late production start for the Fermi Tesla parts, Appro, AMAX, Supermicro and Tyan all announced new Fermi-based server gear this week. Tyan revealed two new platforms that stuff as many as 8 Tesla M2050 GPUs in a 4U chassis. Supermicro launched three Fermi-based offerings: a 1U server with two GPUs, a 4U with four GPUs, and 2U with two hot-plug GPU nodes. AMAX unveiled a GPU cluster using NVIDIA S2050/S2070 Tesla servers as well as a 4U server with 2 CPUs and up to 8 GPUs per chassis. Appro launched a couple of new Fermi-based product, which we covered in greater depth here.

The Fermi deluge is just beginning. Most of the major and minor HPC OEMs will come out with products using the new GPUs between now and ISC'10, and even beyond that. If all goes according to plan, I expect to see a smattering of Fermi-accelerated supers on the TOP500 list in November.