NVIDIA Works On CPU Co-Dependency Issues with Kepler GPU

By Michael Feldman

May 22, 2012

With Intel’s manycore MIC coprocessor looming on the horizon, NVIDIA’s is counting on its upcoming K20 Tesla to retain its dominance in the HPC accelerator marketplace. And while Intel has shared few technical details about its upcoming Knights Corner MIC, NVIDIA has conveniently provided a 24-page white paper (PDF) describing the inner workings of the GK110, the GPU that will power the K20 card for supercomputers.

If you’re a GPU programmer and like to get intimate with the silicon, or are just curious about where NVIDIA is heading with GPU computing, the GK110 paper should be on your summer reading list. It contains a nice description of the GK110 architecture and goes into some depth on the new features that this high-end Kepler brings to the HPC table.

As we described in our Kepler launch coverage last week at the GPU Technology Conference, the big new features in the architecture are Hyper-Q and Dynamic Parallelism. Both are changes that aim to relieve the CPU-GPU bottleneck, enabling the GPU to be better utilized for continuous processing, and freeing up the CPU for more mundane serial tasks. Those two features, however, are only available in the supercomputing-grade GK110, not the GK104 that powers the less powerful K10 card.

To recap, Hyper-Q allows the GPU to execute up to 32 MPI processes, CUDA streams, or threads at the same time. The Fermi GPU could only manage a single task at a time, which limited how much true parallelism the application could attain, and, in many cases, how much of the GPU could be utilized at any particular moment. Hyper-Q should automagically speed up a lot of existing CUDA applications without the need for any source code changes.

Dynamic parallelism, on the other hand, will require some source tweaking for existing GPU code, since it enables programmers to explicitly place more of the application on the graphics chip. It basically allows the GPU to generate work on its own, without having to rely on the CPU to keep feeding it. With dynamic parallelism, a kernel can now launch another kernel, enabling recursive and nested execution. For codes not yet ported to GPUs, this is good news, since this style of programming is a much more natural way to write applications.

Along those same lines is GPUDirect, a hardware/software-enabled feature that allows GPUs to talk to one another directly as peers, bypassing the CPU entirely. GPUDirect was present in Fermi, but the new Kepler has additional support that further lessens its reliance on the CPU. Using this feature, a GPU would be able to go through the NIC and exchange data with other GPUs on the network without CPU buffering in main memory. It also enables other PCIe attached devices, like SSDs, to directly access GPU data.

The NVIDIA engineers have also included some other tweaks to support greater application complexity. One of these is quadrupling the register count per thread compared to the Fermi architecture (from 63 to 255). Routines that do a lot of register spilling to memory because they have to deal with so many variables, like those in quantum chromodynamics, could see some pretty significant speed-ups, according to NVIDIA.

The GK110 also adds an extra 48K read-only data cache per multiprocessor for local functions. The new GPU also doubles the L2 cache capacity, to 1,536 KB, which helps data-dependent codes like physics solvers, ray tracing, and sparse matrix multiplication. This is all in addition to the 64 KB of multiprocessor memory (to divide between L1 and shared data) that Kepler inherited from Fermi, but which now supports more bandwidth for large reads.

All of this is geared to boost application performance in a big way. Here though, the paper hedges on the specifics, promising only “over 1 teraflop of double precision throughput.” NVIDIA is claiming the Kepler will deliver three times the performance per watt of the Fermi GPUs, but that doesn’t necessarily map to peak performance on a given chip. With regard to that metric, we should probably except a doubling or so of the Fermi Tesla’s 665 gigaflops for the top-of-the line Kepler hardware.

But application performance with the GK100 is expected to do much better that with the Fermi-generation GPUs. To a large degree that’s due to all the aforementioned CPU-offload features and other architectural tweaks. But a good chunk of the performance boost will be delivered via brute force, in the form of lots of cores.

The paper says the “full” GK110 implementation will have 15 streaming multiprocessor (SMX units), each of which has 192 cores. That would make the top Kepler a 2,880-core processor, which beats out the 512-core Fermi by a wide margin. But all those cores will be running at about half the clock speed as its predecessor. As the GK110 white paper explains:

For Kepler, our priority was performance per watt. While we made many optimizations that benefitted both area and power, we chose to optimize for power even at the expense of some added area cost, with a larger number of processing cores running at the lower, less power-hungry GPU clock.

The increased core count is enabled by a transistor shrink, in this case, TSMC’s 28nm process technology. In fact, the GK110 will be the largest processor ever built, at least the largest one that is not still sitting in a research lab somewhere. At 7.1 billion transistors, the GK110 is nearly twice the size of the new 4.3 billion transistor Radeon HD 7900 GPU from AMD. For some context, the new “Sandy Bridge” Xeon E5-2600 series CPUs are made up of less than 2.3 billion transistors.

There will also be two slightly smaller GK110 GPU parts, with 13 and 14 multiprocessors, respectively. Presumably the clock frequencies could be cranked up a bit on those if faster thread performance is desired, or down if lower wattage is the goal. In any case, the three GK110 variants suggests NVIDIA will offer a range of HPC products aimed at different price/performance/power points.

The first GK110 GPUs are expected to debut in the K20 Tesla cards in Q4. NVIDIA might be initially hard-pressed to ramp up volumes, especially since TSMC has a number of customers (AMD and Qualcomm, in particular) also vying for 28nm capacity. Supposedly though, NVIDIA chips are going to be priority at the foundry. Even so, such a big chip might still be a challenge for TSMC, from a yield perspective.

In any case, most, if not all of the early GK110s will likely end up in just two systems: the DOE’s Titan supercomputer at Oak Ridge National Lab and the NSF’s Blue Waters machine at NCSA. About 15,000 of the GK100s are expected to go into the Titan super, while the more conservative Blue Waters system will be equipped with around 3,000 of new GPUs.

NVIDIA expects to sell a lot more of them that over the next two or three years, until the “Maxwell” GPU kicks in. That architecture is expected to encompass CPU-GPU integration, the so-called “Project Denver” work that glues a 64-bit ARM CPU onto a CUDA GPU. As such, it will represent an architectural watershed for NVIDIA, but one that Kepler laid the groundwork for.

Kepler, and the GK110 in particular, is NVIDIA’s most general-purpose processor to date. By reducing the dependency of the GPU on the CPU, and making the GPU more capable of supporting complex types of processing, NVIDIA is not just trying to make the two architectures equal peers, but to make the GPU the star of the show. If NVIDIA continues to pursue this architectural trend line, the CPU, while necessary, could be reduced to the role of an OS microcontroller: fielding interrupts, managing I/O, and scheduling jobs. The GPU, meanwhile, would be able to encompass the high-value application processing, which not only conforms to NVIDIA’s philosophical bent, but also its business strategy.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Pfizer HPC Engineer Aims to Automate Software Stack Testing

January 17, 2019

Seeking to reign in the tediousness of manual software testing, Pfizer HPC Engineer Shahzeb Siddiqui is developing an open source software tool called buildtest, aimed at automating software stack testing by providing the community with a central repository of tests for common HPC apps and the ability to automate execution of testing. Read more…

By Tiffany Trader

Senegal Prepares to Take Delivery of Atos Supercomputer

January 16, 2019

In just a few months time, Senegal will be operating the second largest HPC system in sub-Saharan Africa. The Minister of Higher Education, Research and Innovation Mary Teuw Niane made the announcement on Monday (Jan. 14 Read more…

By Tiffany Trader

Google Cloud Platform Extends GPU Instance Options

January 16, 2019

If it's Nvidia GPUs you're after to power your AI/HPC/visualization workload, Google Cloud has them, now claiming "broadest GPU availability." Each of the three big public cloud vendors has by turn touted the latest and Read more…

By Tiffany Trader

HPE Extreme Performance Solutions

HPE Systems With Intel Omni-Path: Architected for Value and Accessible High-Performance Computing

Today’s high-performance computing (HPC) and artificial intelligence (AI) users value high performing clusters. And the higher the performance that their system can deliver, the better. Read more…

IBM Accelerated Insights

Resource Management in the Age of Artificial Intelligence

New challenges demand fresh approaches

Fueled by GPUs, big data, and rapid advances in software, the AI revolution is upon us. Read more…

STAC Floats ML Benchmark for Financial Services Workloads

January 16, 2019

STAC (Securities Technology Analysis Center) recently released an ‘exploratory’ benchmark for machine learning which it hopes will evolve into a firm benchmark or suite of benchmarking tools to compare the performanc Read more…

By John Russell

Google Cloud Platform Extends GPU Instance Options

January 16, 2019

If it's Nvidia GPUs you're after to power your AI/HPC/visualization workload, Google Cloud has them, now claiming "broadest GPU availability." Each of the three Read more…

By Tiffany Trader

STAC Floats ML Benchmark for Financial Services Workloads

January 16, 2019

STAC (Securities Technology Analysis Center) recently released an ‘exploratory’ benchmark for machine learning which it hopes will evolve into a firm benchm Read more…

By John Russell

A Big Data Journey While Seeking to Catalog our Universe

January 16, 2019

It turns out, astronomers have lots of photos of the sky but seek knowledge about what the photos mean. Sound familiar? Big data problems are often characterize Read more…

By James Reinders

Intel Bets Big on 2-Track Quantum Strategy

January 15, 2019

Quantum computing has lived so long in the future it’s taken on a futuristic life of its own, with a Gartner-style hype cycle that includes triggers of innovation, inflated expectations and – though a useful quantum system is still years away – anticipatory troughs of disillusionment. Read more…

By Doug Black

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

IBM’s New Global Weather Forecasting System Runs on GPUs

January 9, 2019

Anyone who has checked a forecast to decide whether or not to pack an umbrella knows that weather prediction can be a mercurial endeavor. It is a Herculean task: the constant modeling of incredibly complex systems to a high degree of accuracy at a local level within very short spans of time. Read more…

By Oliver Peckham

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Summit Supercomputer is Already Making its Mark on Science

September 20, 2018

Summit, now the fastest supercomputer in the world, is quickly making its mark in science – five of the six finalists just announced for the prestigious 2018 Read more…

By John Russell

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

HPE No. 1, IBM Surges, in ‘Bucking Bronco’ High Performance Server Market

September 27, 2018

Riding healthy U.S. and global economies, strong demand for AI-capable hardware and other tailwind trends, the high performance computing server market jumped 28 percent in the second quarter 2018 to $3.7 billion, up from $2.9 billion for the same period last year, according to industry analyst firm Hyperion Research. Read more…

By Doug Black

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

Germany Celebrates Launch of Two Fastest Supercomputers

September 26, 2018

The new high-performance computer SuperMUC-NG at the Leibniz Supercomputing Center (LRZ) in Garching is the fastest computer in Germany and one of the fastest i Read more…

By Tiffany Trader

Houston to Field Massive, ‘Geophysically Configured’ Cloud Supercomputer

October 11, 2018

Based on some news stories out today, one might get the impression that the next system to crack number one on the Top500 would be an industrial oil and gas mon Read more…

By Tiffany Trader

Microsoft to Buy Mellanox?

December 20, 2018

Networking equipment powerhouse Mellanox could be an acquisition target by Microsoft, according to a published report in an Israeli financial publication. Microsoft has reportedly gone so far as to engage Goldman Sachs to handle negotiations with Mellanox. Read more…

By Doug Black

House Passes $1.275B National Quantum Initiative

September 17, 2018

Last Thursday the U.S. House of Representatives passed the National Quantum Initiative Act (NQIA) intended to accelerate quantum computing research and developm Read more…

By John Russell

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This