NVIDIA Works On CPU Co-Dependency Issues with Kepler GPU

By Michael Feldman

May 22, 2012

With Intel’s manycore MIC coprocessor looming on the horizon, NVIDIA’s is counting on its upcoming K20 Tesla to retain its dominance in the HPC accelerator marketplace. And while Intel has shared few technical details about its upcoming Knights Corner MIC, NVIDIA has conveniently provided a 24-page white paper (PDF) describing the inner workings of the GK110, the GPU that will power the K20 card for supercomputers.

If you’re a GPU programmer and like to get intimate with the silicon, or are just curious about where NVIDIA is heading with GPU computing, the GK110 paper should be on your summer reading list. It contains a nice description of the GK110 architecture and goes into some depth on the new features that this high-end Kepler brings to the HPC table.

As we described in our Kepler launch coverage last week at the GPU Technology Conference, the big new features in the architecture are Hyper-Q and Dynamic Parallelism. Both are changes that aim to relieve the CPU-GPU bottleneck, enabling the GPU to be better utilized for continuous processing, and freeing up the CPU for more mundane serial tasks. Those two features, however, are only available in the supercomputing-grade GK110, not the GK104 that powers the less powerful K10 card.

To recap, Hyper-Q allows the GPU to execute up to 32 MPI processes, CUDA streams, or threads at the same time. The Fermi GPU could only manage a single task at a time, which limited how much true parallelism the application could attain, and, in many cases, how much of the GPU could be utilized at any particular moment. Hyper-Q should automagically speed up a lot of existing CUDA applications without the need for any source code changes.

Dynamic parallelism, on the other hand, will require some source tweaking for existing GPU code, since it enables programmers to explicitly place more of the application on the graphics chip. It basically allows the GPU to generate work on its own, without having to rely on the CPU to keep feeding it. With dynamic parallelism, a kernel can now launch another kernel, enabling recursive and nested execution. For codes not yet ported to GPUs, this is good news, since this style of programming is a much more natural way to write applications.

Along those same lines is GPUDirect, a hardware/software-enabled feature that allows GPUs to talk to one another directly as peers, bypassing the CPU entirely. GPUDirect was present in Fermi, but the new Kepler has additional support that further lessens its reliance on the CPU. Using this feature, a GPU would be able to go through the NIC and exchange data with other GPUs on the network without CPU buffering in main memory. It also enables other PCIe attached devices, like SSDs, to directly access GPU data.

The NVIDIA engineers have also included some other tweaks to support greater application complexity. One of these is quadrupling the register count per thread compared to the Fermi architecture (from 63 to 255). Routines that do a lot of register spilling to memory because they have to deal with so many variables, like those in quantum chromodynamics, could see some pretty significant speed-ups, according to NVIDIA.

The GK110 also adds an extra 48K read-only data cache per multiprocessor for local functions. The new GPU also doubles the L2 cache capacity, to 1,536 KB, which helps data-dependent codes like physics solvers, ray tracing, and sparse matrix multiplication. This is all in addition to the 64 KB of multiprocessor memory (to divide between L1 and shared data) that Kepler inherited from Fermi, but which now supports more bandwidth for large reads.

All of this is geared to boost application performance in a big way. Here though, the paper hedges on the specifics, promising only “over 1 teraflop of double precision throughput.” NVIDIA is claiming the Kepler will deliver three times the performance per watt of the Fermi GPUs, but that doesn’t necessarily map to peak performance on a given chip. With regard to that metric, we should probably except a doubling or so of the Fermi Tesla’s 665 gigaflops for the top-of-the line Kepler hardware.

But application performance with the GK100 is expected to do much better that with the Fermi-generation GPUs. To a large degree that’s due to all the aforementioned CPU-offload features and other architectural tweaks. But a good chunk of the performance boost will be delivered via brute force, in the form of lots of cores.

The paper says the “full” GK110 implementation will have 15 streaming multiprocessor (SMX units), each of which has 192 cores. That would make the top Kepler a 2,880-core processor, which beats out the 512-core Fermi by a wide margin. But all those cores will be running at about half the clock speed as its predecessor. As the GK110 white paper explains:

For Kepler, our priority was performance per watt. While we made many optimizations that benefitted both area and power, we chose to optimize for power even at the expense of some added area cost, with a larger number of processing cores running at the lower, less power-hungry GPU clock.

The increased core count is enabled by a transistor shrink, in this case, TSMC’s 28nm process technology. In fact, the GK110 will be the largest processor ever built, at least the largest one that is not still sitting in a research lab somewhere. At 7.1 billion transistors, the GK110 is nearly twice the size of the new 4.3 billion transistor Radeon HD 7900 GPU from AMD. For some context, the new “Sandy Bridge” Xeon E5-2600 series CPUs are made up of less than 2.3 billion transistors.

There will also be two slightly smaller GK110 GPU parts, with 13 and 14 multiprocessors, respectively. Presumably the clock frequencies could be cranked up a bit on those if faster thread performance is desired, or down if lower wattage is the goal. In any case, the three GK110 variants suggests NVIDIA will offer a range of HPC products aimed at different price/performance/power points.

The first GK110 GPUs are expected to debut in the K20 Tesla cards in Q4. NVIDIA might be initially hard-pressed to ramp up volumes, especially since TSMC has a number of customers (AMD and Qualcomm, in particular) also vying for 28nm capacity. Supposedly though, NVIDIA chips are going to be priority at the foundry. Even so, such a big chip might still be a challenge for TSMC, from a yield perspective.

In any case, most, if not all of the early GK110s will likely end up in just two systems: the DOE’s Titan supercomputer at Oak Ridge National Lab and the NSF’s Blue Waters machine at NCSA. About 15,000 of the GK100s are expected to go into the Titan super, while the more conservative Blue Waters system will be equipped with around 3,000 of new GPUs.

NVIDIA expects to sell a lot more of them that over the next two or three years, until the “Maxwell” GPU kicks in. That architecture is expected to encompass CPU-GPU integration, the so-called “Project Denver” work that glues a 64-bit ARM CPU onto a CUDA GPU. As such, it will represent an architectural watershed for NVIDIA, but one that Kepler laid the groundwork for.

Kepler, and the GK110 in particular, is NVIDIA’s most general-purpose processor to date. By reducing the dependency of the GPU on the CPU, and making the GPU more capable of supporting complex types of processing, NVIDIA is not just trying to make the two architectures equal peers, but to make the GPU the star of the show. If NVIDIA continues to pursue this architectural trend line, the CPU, while necessary, could be reduced to the role of an OS microcontroller: fielding interrupts, managing I/O, and scheduling jobs. The GPU, meanwhile, would be able to encompass the high-value application processing, which not only conforms to NVIDIA’s philosophical bent, but also its business strategy.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion XL — were added to the benchmark suite as MLPerf continues Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing power it brings to artificial intelligence.  Nvidia's DGX Read more…

Call for Participation in Workshop on Potential NSF CISE Quantum Initiative

March 26, 2024

Editor’s Note: Next month there will be a workshop to discuss what a quantum initiative led by NSF’s Computer, Information Science and Engineering (CISE) directorate could entail. The details are posted below in a Ca Read more…

Waseda U. Researchers Reports New Quantum Algorithm for Speeding Optimization

March 25, 2024

Optimization problems cover a wide range of applications and are often cited as good candidates for quantum computing. However, the execution time for constrained combinatorial optimization applications on quantum device Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at the network layer threatens to make bigger and brawnier pro Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as the the ability to train 1 trillion pa Read more…

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HB Read more…

Nvidia Looks to Accelerate GenAI Adoption with NIM

March 19, 2024

Today at the GPU Technology Conference, Nvidia launched a new offering aimed at helping customers quickly deploy their generative AI applications in a secure, s Read more…

The Generative AI Future Is Now, Nvidia’s Huang Says

March 19, 2024

We are in the early days of a transformative shift in how business gets done thanks to the advent of generative AI, according to Nvidia CEO and cofounder Jensen Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

Intel Won’t Have a Xeon Max Chip with New Emerald Rapids CPU

December 14, 2023

As expected, Intel officially announced its 5th generation Xeon server chips codenamed Emerald Rapids at an event in New York City, where the focus was really o Read more…

IBM Quantum Summit: Two New QPUs, Upgraded Qiskit, 10-year Roadmap and More

December 4, 2023

IBM kicks off its annual Quantum Summit today and will announce a broad range of advances including its much-anticipated 1121-qubit Condor QPU, a smaller 133-qu Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire